Don't Block Search Engine Crawlers
Revision as of 13:41, 23 January 2014 by Asad Mahmood Butt (talk | contribs)
By [[User:|]] on
Make sure your important pages can rank
A great page on your website will never rank well in search results - and be discovered by more people - if the search engines don't know it's there in the first place.
First, some background: Search engines use computer programs called "crawlers" (or “spiders” or “robots”) to discover web pages and add them to their indexes. Google calls its crawler “Googlebot,” and lots of SEO (search engine optimization) professionals call it that, too.