Webmasters who are new at SEO will usually want to know how Google indexes the pages on their website. Google does this by using a software program which is known as a "bot" or "spider." The spider will travel through each page on your site, and will study it and then place it within an index. Google uses three different bots, and they are known as the Freshbot, Adsense bot, and DeepCrawl.



As the name implies, the Adsense bot is used to for websites which use Adsense code. When you create a new page on your site which has Adsense, the code within it sends a notice to the Adsense bot, and it will travel to your website in about 15 minutes. Once it arrives, it will index your page so that the Adsense displays specific ads which are related to the content. The Freshbot will crawl the pages on your website which are the most prominent. On some sites this may be one page, while on others it could be hundreds of pages. Large websites like CNN are visited by the Freshbot every few minutes.

The reason why Freshbot visits these pages so often is because they update so frequently. The average website will only receive a visit from Freshbot every 2 to 15 days.The times may vary depending on the popularity of a page. Freshbot will look for links which are deep in your site. It will place these links into an index so they can be used by DeepCrawl. DeepCrawl will visit your site every month or so. It will review the links which were detected by Freshbot. Because of this, it may take a month or longer for your entire website to become indexed by Google.

It is best to be patient and continue building content on your site. You will also want to focus on getting both outbound and inbound links. Once Google as indexed your site, they will begin rewarding your site. Understanding how Google indexes your site will allow you to perform SEO at a much higher level. The bots will be visiting you on a consistent basis.