https://www.webmaster-forums.net/crss/node/1038722 en https://www.webmaster-forums.net/webmasters-corner/how-google-gets-their-info#comment-1220497 <p>Google sends out a program called Googlebot that weiws websites and follows links much the same way as you would in a text based browser. It indexs these pages in it's databases which can be searched using google.com. Yes you can write your own bot that will search and index sites but there is little point as Google and Yahoo have the market prety much sewn up.</p> Thu, 14 Jun 2007 00:18:13 +0000 vibins comment 1220497 at https://www.webmaster-forums.net