I am getting a lot of 503 "Scraping failed" errors
Our API succeeds over 90% of the time, but occasional 503 errors, especially during peak hours, are unavoidable due to LinkedIn's erratic rate limits.

/ proxycurl

I am getting a lot of 503 "Scraping failed" errors

Steven Goh
Share:

Subscribe to our newsletter

Get the latest news from Proxycurl

Our API succeeds 90+% of the time. It turns out that our scraping infrastructure is limited and inconsistent because we are dealing with external targets such as LinkedIn which is very erratic when it comes to rate limits, particularly with their public profiles.

We notice 503s occur more during the peak period, which is around 3-5am Singapore time (GMT +8).

There are no workarounds to 503 or ways to reduce 503 errors other than to avoid calling the API during peak hours.

The recommended advice is to retry immediately after you get a 503 error.

Subscribe to our newsletter

Get the latest news from Proxycurl

Steven Goh
Share:

Featured Articles

Here’s what we’ve been up to recently.