Busy — Fri, 04 Mar 2005 22:27:27 +0000

if you are using cookies from the form, include a file on each page to see if the cookie is set, same for sessions, javascript is no good because you can just disable it

bja888 — Fri, 04 Mar 2005 10:34:59 +0000

Well... I password protected my blog site. oldbja888.com?Web-forums The mian page is a frame set. But you can get around the password by typeing in individual pages in the url. I can add ASP to all the pages and make sure they have thr framset open before they load but thats too much work. I just whan them to stop showing up when I search the net on google.

Busy — Fri, 04 Mar 2005 09:08:27 +0000

Just remember robots.txt shouldn't be used as the only means of protection, bad bots are bad bots for a reason, because they ignore the robots file.
The robots file is only a guideline for bots, the .htaccess is a command which can't be ignored

Depends what you want/need bja888

bja888 — Thu, 03 Mar 2005 22:20:53 +0000

wow! thankx for the information!!
anything else I can do to block personal content?

Re

james.thornton — Thu, 03 Mar 2005 21:45:37 +0000

For creating robots.txt files I use the free tool at:

http://www.1-hit.com/all-in-one/tool-robots.txt-generator.htm

Creates exactly what you want in your robots.txt file and has the useful capability of blocking 'nasty' robots (ones which don't index your site, just dowload your entire site to somebodies machine, wasting a large amount of your bandwidth)

Busy — Sun, 23 Jan 2005 21:10:17 +0000

If the robots reads the robots txt file it wont go into or follow the path

I take it you have the complete thing:

User-agent: *
Disallow: /gallery/

depending on your servers stats you may also want to throw up an .htacces file to stop people viewing your folders (if you dont have a default file like index.html in there)