Finding And Removing Dead Links For Your Blog
I was using Google Web Master tools and noticed the number of 404 pages found on my blog had skyrocketed over the past few weeks, so I did a little digging and found that when I had installed many of the wordpress plug-ins on dragonblogger.com that the Google Index bots no longer can find them and trigger the 404 pages.
This made me realize that no blogger should have Google or any other search engine indexing their wordpress admin stuff, so I decided to modify my robots.txt to prevent this from happening. It is easier than you think and you can follow these instructions to prevent the search engines from indexing your wp-admin, wp-content, and plug-ins for your wordpress blog.
All you do is have physical access to your server, (IE be able to upload a file or ssh into your box)
create a file called robots.txt in your root directory of your web server
Make your robots.txt include these statements:
User-Agent: *
Disallow: /wp-admin/
Disallow: /wp-content/
Allow: /
save the file
Now your wp-admin, wp-content and all subfolders including plug-ins will not be indexed by the search engines.
This should help increase your PR of your blog overall and prevent 404 errors from displaying in Google indexes when you remove plug-ins and other admin related content.
-Dragon Blogger
Related posts:
- Building an XML Sitemap in WordPress There is a great plug-in for wordpress called “Google (XML) Sitemaps Generator for WordPress” this plug-in once downloaded and installed...
- Tracking your Blog or Website’s External Links with Google Webmaster Tools A friend of mine was mentioning to me that far more accurate than Blogsearch’s link:http://www.dragonblogger.com method of tracking links to...
- Blog Link Backs – Some Interesting Things I Discovered I was experimenting with wondering why I when my fellow blogger’s Linkback to my website, I don’t see the Linkback...
- Adding Previous or Next Post Links to your WordPress Blog By adding a simple statement into your single.php you can create links to your next and previous posts directly in...
- Add NoFollow To Your WordPress Archive and Category Links One of the ways to improve your Google PageRank is to limit the number of link juice you have flowing...
[...] the original post:Â Finding And Removing Dead Links For Your Blog | Dragon Blogger Posted in Blogger, Blogging, WordPress | Tags: advertising, editing-robots-txt-file, funny, [...]