The Deep Web, or Silent Web as it's sometimes called, is a hidden resource that every writer should know about. Whether you want information for an article, for an historical novel or even for a play or film script, the Deep Web may find it for you when Google can't.



It is estimated that there may be as many as 500 million web pages that are indexed by Google, MSN, Yahoo and others. The Deep Web is actually not included in these and is reckoned to be many times greater in size!

The 'surface' or 'visible' web is crawled by 'spiders'. These are small robot programs that travel from site to site picking up information which they relay back to the search engine to compile its index.

Unfortunately many websites can't be accessed by spiders. The spider can call at the door, index the address, but can't get in to record the contents. The reasons for this are varied, and include the fact that website owners can deliberately ban spiders from entering their sites if they wish. Another is that if the site comprises of databases, these are anathema to spiders which can't do anything with them. And it is sites like these, made up of databases full of fascinating information on an bewildering variety of subjects, that mainly comprise the Deep Web.

There are thousands, maybe even millions, of such databases out there on the web, but you need to know where they are and what their address is in order to get into them.

The Deep Web is recognised as producing a much higher quality of information than regular search engines due to the many universities and technical and scientific institutions that contribute.

Some databases, such as SJLibrary.org (a collaboration between San Jose State University and San Jose Public Libraries) restrict access to part or all of their content to their students and faculties, but there are plenty of others that do not.

So, it's time to dip your toes into this vast ocean and see what you can find. To start you off here are some sites to try:

Direct Search is a good one (http://www.freepint.com/gary/direct.htm)

Intute - I found this an excellent site http://www.intute.ac.uk/

Complete Planet http://aip.completeplanet.com/

InfoMine http://infomine.ucr.edu/

FindArticles is a database of - you guessed it - articles http://findarticles.com/

For more information on the subject of Deep Web go to deepweb.com http://www.deepweb.com/