Wednesday, September 7, 2011

Robots.txt File in Blogspot ? How Robots.txt is Looking for Blogger ? Where is Robots.txt File in Blogger ?

Robots.txt File in Blogspot ?

Do you have a blog in blogger and you are familiar with robots.txt file and want to check where and how is your default robots.txt file is looking in blogger. By default each blogger have robots.txt file.

Default Robots.txt file for any Blogger


By default each blogger have the following robots.txt file

User-agent: Mediapartners-Google
Disallow:

User-agent: *
Disallow: /search
Allow: /

Sitemap: http://YourBlogName.blogspot.com/feeds/posts/default?orderby=updated

How can You Check Your Blogger's Robots.txt ?


If you have account in Google Webmaster Tools then you can check your blogger's default robots.txt file by visiting Crawler access Page.

The url of robots.txt file be as follows

http://YourBlogName.blogspot.com/robots.txt

Replace 'YourBlogName' with your 'blog's name'.

Want to ask any question related to robots.txt file ? then you can ask here by giving comment. I will give answers to your all queries.

1 comment:

  1. I know this is an old post, but I am a new blogger. Looks like all my pages affected by this, according to webmaster tools.

    Should I be concerned?

    ReplyDelete