Crawler-based Search Engines

These are search engines that will automatically list your website, with the best example being Google. With this type of search engine, if your web site changes then your listing will change. There are 3 parts to the way these search engines work and these are:

1.The Spider - also knows and the crawler or robot. The spider will travel around the internet reading web pages and the code that makes them up and also follow and links to other web pages. Once a site has been visited by a spider it is said that the site has been "spidered" or "crawled". On a monthly basis the spider will revisit all old sites and find any new ones through the links from these sites. All the results of its work are stored in:

2.The Index - this is a database that contains a copy of every web page that has been spidered. If a web page changes then the database will be updated. There can often be a delay between the site being spidered and indexed. If a site is not indexed then it will not be seen by a user of the search engine. The search results that are found by the user of the search engines are pulled out by:

3.The Software - this will go into the database and rank all of the sites in order of relevance for each search term. This rank depends on the rules that are set up for that particular search engine and will be different for each one.

Human-Powered Search Engines

These are more commonly called directories and sites must be submitted by the owner for them to get listed. There are various types and some are free and some are paid. All entries must be reviewed by the editor and approved before they are submitted. All searches of the site are done through the text in the title and the description given to the site. If your site changes then your listing will not change.

Hybrid Search Engines

These are a combination of both crawler-based and human-powered search engines. Most of these will have a bias to either submitted sites or crawled sites and so you may find that submitted sites appear higher in the search results even if they are less relevant.

Conclusion

Even if 2 different search engines have indexed all of the same sites then the results that visitors see may be different depending on how each engine's software ranks the sites. In order to rank highly in as many search engines as possible it is important to learn as much about their rules to ranking as possible and do the necessary things to your site.