Loading Profile...
Wikipedia robots.txt and IP blocking
I'd like to bring everyone's attention to the following:
http://en.wikipedia.org/wiki/Wikipedi...
Some quotes:
"Robots.txt has a rate limit of one per second set using the Crawl-delay setting."
"Please don't try to circumvent it - we'll just block your whole IP range."
Wouldn't it be fun to get all of UCI barred from wikipedia? :D
Also, the robots.txt is interesting and has lots of comments:
http://en.wikipedia.org/robots.txt
http://en.wikipedia.org/wiki/Wikipedi...
Some quotes:
"Robots.txt has a rate limit of one per second set using the Crawl-delay setting."
"Please don't try to circumvent it - we'll just block your whole IP range."
Wouldn't it be fun to get all of UCI barred from wikipedia? :D
Also, the robots.txt is interesting and has lots of comments:
http://en.wikipedia.org/robots.txt
1
person has this problem
I have this problem, too!
Tell me when someone solves it.
The more people who report this problem, the more it gets noticed.
The more people who report this problem, the more it gets noticed.
Create a customer community for your own organization
Plans starting at $19/month
-
Inappropriate?I've never had anyone get blocked while doing this for me in a class.
I wouldn't worry too much about being blocked at our scale. -
Inappropriate?Maybe that could be a future assignment. Getting wikipedia to ban the whole UCI network!
I’m silly
EMPLOYEE
