I can not believe what I discovered today. This reminds my of James Schramko’s saying: “It’s all BS until you test!”.
I did a broken link test on my site today and for all pages I get a result that the link could not be checked because of an exclusion in robots.txt. This is a major disaster because this means that none of my resources and pages are being indexed and therefor not found via any search engine.
After a bit of research I found out why this is, there is a privacy setting in WordPress that allows you to block search engine spiders from crawling your site. Now there might be reasons that for certain sites this is a good idea however why the WordPress guys have set the default to blocking spiders is beyond me. Or at the very least a warning should appear on your site to ensure you are made aware of this fact. I assume their idea was that you should only open your blog once you have added content.
Anyhow here is how to fix it:
logon to wp-admin > settings > privacy and you’ll see to options for blog privacy:
and make sure you tick the first and save.
Follow this to find out about testing tools including for link checking
No related posts.
Related posts brought to you by Yet Another Related Posts Plugin.
{ 1 comment… read it below or add one }
No surprise. As soon as I had changed the above setting I ran the lick check test again and all is fine. Moral: Don’t for get to test!!!!