Robots.txt

robots-for-wordpress Robots.txt is a simple ASCII file, which gives an instruction to SEO Bots to visit webpage for indexing. Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol. It works likes this: a robot wants to vists a Web site URL, say... Read more »

March 8, 2014 at IST / How-to Guides, Web Development.