|
07-04-2003, 11:02 AM
|
#1
|
|
Evil Chris
is drinking Heineken
Clone of myself
Join Date: Aug 2002
Location: Montreal
Posts: 12,984
xBucks: 406,129
|
Inktomi
It appears that Inktomi is spidering the hell out of xnations the past 24 hours or so.
I'm wondering... is this a good thing or a bad thing? Should I ban that IP or just let it go?
Anyone have any thoughts?
|
|
|
|
07-04-2003, 12:52 PM
|
#2
|
|
wsjb78
should edit this
Local Shadow Agent #1
Join Date: Nov 2002
Location: Home
Posts: 1,061
xBucks: 17,450
|
Our sites get often spidered by Inktomi but they never show up in any search results... but we still let it spider us!
|
|
|
|
|
07-04-2003, 01:11 PM
|
#3
|
|
StuartD
should edit this
Sofa King Band
Join Date: Sep 2002
Location: Outside the box
Posts: 1,053
xBucks: 10,851
|
nah, they're the good guys... they used to power a ton of search engines... msn, yahoo, alltheweb.... a bunch of others I think.
But lately most of them have been switching to google.
I'm sure there's still enough search engines using inktomi to make it worth having them spider you.
__________________
"If you are not going to heaven, why miss it by an inch?" - Sam Kinison
|
|
|
|
|
07-04-2003, 01:13 PM
|
#4
|
|
pornJester
should edit this
The one and only.
Join Date: Sep 2002
Location: Florida
Posts: 221
xBucks: 3,033
|
I wouldn't worry too much. Your site is run off the Vbulletin script and they probably would design it to be SE friendly.
|
|
|
|
|
07-04-2003, 01:14 PM
|
#5
|
|
Cyndalie
is not it.
Marketing Director / SEO
Join Date: Aug 2002
Location: Philadelphia
Posts: 2,334
xBucks: 76,811
|
I got hit by MSIE Crawler about 1500 times last minth. Google 316 .. these numbers are normal.
A search engine spidering your site is a good thing - is you want to be indexed and it's not overloading your system. (On average a spider may use 5-50 Megs of Bandwidth or more). The number of revisits means that it is coming from pages that have linked to Xnations, or it could be repeating efforts because Xnations is so dynamic.
I wouldn't really worry about it.
__________________
Hardcore, Blowjobs, Gay, Wives, Lesbians, Gangbang, Movies, Tranny, Interracial, Teen....
16 Sites, $35 PPS/ 60% Revshare :: Capitalize on Experience
Cyndee - Director of Marketing & SEO
Last edited by Cyndalie; 07-04-2003 at 01:27 PM.
|
|
|
|
|
07-04-2003, 01:14 PM
|
#6
|
|
Cathedra
should edit this
Back in the Biz
Join Date: Aug 2002
Location: Montreal, Quebec
Posts: 110
xBucks: 3,071
|
No no i'd say it's a good thing.. let it ride  |
|
|
|
|
07-04-2003, 01:28 PM
|
#7
|
|
Evil Chris
is drinking Heineken
Clone of myself
Join Date: Aug 2002
Location: Montreal
Posts: 12,984
xBucks: 406,129
|
It's freaking me out! LOL  |
|
|
|
|
07-04-2003, 07:50 PM
|
#8
|
|
NetRodent
should edit this
X Cuseme
Join Date: Jun 2003
Location: In the walls of your house.
Posts: 154
xBucks: 386
|
If its not causing a problem let it spider. The only time we ever had a problem was years ago when we had about 200 domains with about 10,000 pages each in a single P2 400 mhz. One day T-Rex from Lycos started hitting the site so hard the machine froze up (swapping memory for run away apache processes). Ah, the good old days.
__________________
<!--<font size="1">"Did you ever hear anyone say 'that work had better be banned because I might read it and it might be very damaging to me'?"
Joseph Henry Jackson,
American Journalist, 1894-1946</font>-->
<font size="1">"I know you believe you understand what you think I said, but I am not sure you realize that what you heard is not what I meant."<br>--Richard Nixon</font>
|
|
|
|
Posting Rules
|
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
HTML code is On
|
|
|
|
|
Linear Mode
