I was using Google Web Master tools and noticed the number of 404 pages found on my blog had skyrocketed over the past few weeks, so I did a little digging and found that when I had installed many of the wordpress plug-ins on dragonblogger.com that the Google Index bots no longer can find them and trigger the 404 pages.

This made me realize that no blogger should have Google or any other search engine indexing their wordpress admin stuff, so I decided to modify my robots.txt to prevent this from happening. It is easier than you think and you can follow these instructions to prevent the search engines from indexing your wp-admin, wp-content, and plug-ins for your wordpress blog.

All you do is have physical access to your server, (IE be able to upload a file or ssh into your box)
create a file called robots.txt in your root directory of your web server
Make your robots.txt include these statements:

User-Agent: *
Disallow: /wp-admin/
Disallow: /wp-content/
Allow: /

save the file

Now your wp-admin, wp-content and all subfolders including plug-ins will not be indexed by the search engines.
This should help increase your PR of your blog overall and prevent 404 errors from displaying in Google indexes when you remove plug-ins and other admin related content.

-Dragon Blogger

Justin Germino
Working in the IT Industry for over 17 years starting with OS, Networking and then moving into web based technologies including authentication and online security. Dragon Blogger has unique insights and opinions to how the internet and web technology works. An Avid movie fan, video game fan and fan of trying anything and everything new.
Justin Germino

@dragonblogger

https://t.co/iWP6afuOc9 is a technology, gadget, software and video game site that features articles, reviews and giveaways.
@onuro @GrowMap @kiokentheme I personally am not a fan of Gutenberg Editor and immediately use Classic, I don't eve… https://t.co/lJ1ZTSQEtv - 23 mins ago
Justin Germino