X Nations - View Single Post

Cyndalie · 08-27-2003, 12:20 PM

yep, robots.txt file is the first thing and engine will request when indexing or spidering your site, it tells it what NOT to index. Unless you want to keep spiders OUT, you dont need it.

08-27-2003, 12:20 PM	#3
Cyndalie Cyndalie is not it. Marketing Director / SEO Join Date: Aug 2002 Location: Philadelphia Posts: 2,334 xBucks: 88,585	yep, robots.txt file is the first thing and engine will request when indexing or spidering your site, it tells it what NOT to index. Unless you want to keep spiders OUT, you dont need it. __________________ Hardcore, Blowjobs, Gay, Wives, Lesbians, Gangbang, Movies, Tranny, Interracial, Teen.... 16 Sites, $35 PPS/ 60% Revshare :: Capitalize on Experience Cyndee - Director of Marketing & SEO