I am getting a lot of 503 "Scraping failed" errors

Our API succeeds 90+% of the time. It turns out that our scraping infrastructure is limited and inconsistent because we are dealing with external targets such as LinkedIn which is very erratic when it comes to rate limits, particularly with their public profiles.

We notice 503s occur more during the peak period, which is around 3-5am Singapore time (GMT +8).

There are no workarounds to 503 or ways to reduce 503 errors other than to avoid calling the API during peak hours.

The recommended advice is to retry immediately after you get a 503 error.