PDA

View Full Version : robots.txt


Phunn Gye
03-25-2003, 07:24 AM
I have 2 domains,
www.edmousseau.net
and www.joyfulfatima.com
( aka www.edmousseau.net/fatima or http://fatima.edmousseau.net )
- this is an addon domain

I have a question about the robots.txt use...
If I put one in the main site, and disallow the robots access to fatima ( i don't want them to see fatima if they come to the main site ), and then put a seperate one in the fatima folder just for the fatima site, will this work?

-for the main site-

User-agent: *
Disallow: /cgi-bin/
Disallow: /misc/
Disallow: /fatima/


-for the fatima site -

User-agent: *
Disallow: /cgi-bin/
Disallow: /misc/


Thanks:confused:

marce
03-25-2003, 11:19 AM
you don't need the robots.txt in the fatima folder as you already are blocking the complete folder in the main site by having the line Disallow: /fatima/

Phunn Gye
03-25-2003, 01:41 PM
Thanks marce.
What happens if some robot comes looking for the fatima site?
It won't see it because it's blocked in the main folder?

Duh!!! :rolleyes < slaps self on side of head >:rolleyes

I have to block each 'fatima' file or folder from the main robots file. That file has to be in the root directory, won't work anywhere else.