silent.chris
2007-12-12 18:34:09 UTC
We run a service which crawls users' del.icio.us RSS feeds to display
updates. Lately, we've been getting lots of 503 errors. I assume
these indicate some kind of rate-limiting. Is there a way to be able
to crawl many users without getting these errors? We don't crawl any
single user more than once every half-hour.
updates. Lately, we've been getting lots of 503 errors. I assume
these indicate some kind of rate-limiting. Is there a way to be able
to crawl many users without getting these errors? We don't crawl any
single user more than once every half-hour.