Thursday, July 24, 2008

Hindi Song | Google Spiders Got A Bit Lazy in April; Crawling Lesser Web Pages

googlebot-download

The Google Webmaster Tools dashboard has graphs indicating the activity of Google spiders on your websites. You get to know the number of pages that were crawled by Googlebots per day and some other details.

To see such graphs for your own website, open Google Webmaster Tools, go to Tools and then click Set Crawl rate.

Now if you look at the crawling stats for April in for your site, you may find that Google is spidering lesser number of web pages on your site. I checked this for two of my sites and the graphs followed a similar trend as the one above.

Since fresh content (or new web pages) almost immediately gets indexed in Google, this trend probably means that Google spiders are revisiting older web pages (pages already in Google’s index) at a much lower frequency.

Brett Tabke feels that Google was focused on some new ranking changes during April and a lot of their spidering and indexing was throttled during that period. Makes lot of sense.

0 comments: