28 April, 2012

Custom Robots.txt

Robots.txt file helps you allow or block the search engine Robots to Crawl/index the pages. Now you can control what content you want to crawl or ignore by search engine. 
 
Not only this now you can use the robots.txt file to block search engines from crawling different parts of your blog like pages, archives, tags.  You can also add sitemap URL to BlogSpot’s robots.txt file for indexing. 
 
You can access the robots.txt file of any blog by visiting this url (http://yourblog.blogspot.com/robot.txt)
Warning!! If you don’t have much knowledge about this, never try to changing or Modifiying this file because it can effect the search engine visibility.

2 comments:

  1. Good information about robot txt file....its very helpful

    ReplyDelete
  2. whoah this blog is excellent i really like reading your
    articles. Stay up the great work! You recognize, many persons are looking round for this info, you can help them greatly.


    my website - dieta antycellulitowa

    ReplyDelete

Your feedback is always appreciated. I will try to reply to your queries as soon as possible- Amol Ghuge

Note: Only a member of this blog may post a comment.