I am getting a lot of 503 "Scraping failed" errors

Our API succeeds 90+% of the time. It turns out that our scraping infrastructure is limited and inconsistent because we are dealing with external targets such as Professional Social Network which is very erratic when it comes to rate limits, particularly with their public profiles.

We notice 503s occur more during the peak period, which is around 3-5am Singapore time (GMT +8).

There are no workarounds to 503 or ways to reduce 503 errors other than to avoid calling the API during peak hours.

The recommended advice is to retry immediately after you get a 503 error.

Featured Articles

Here's what we've been up to recently.

View all

profile api

Ultimate Guide To The Professional Social Network API: People Profile API (with Python Examples)

Learn all about the Professional Social Network API, with comprehensive Python code demos. Dive into the world of Professional Social Network APIs, official and third-party alternatives, and understand how to access and utilize Professional Social Network profile data through Python code examples.

I dismissed someone, and it was not because of COVID19

The cadence of delivery. Last month, I dismissed the employment of a software developer who oversold himself during the interview phase. He turned out to be on the lowest rung of the software engineers in my company. Not being good enough is not a reason to be dismissed. But not

sharedhere

I got blocked from posting on Facebook

I tried sharing some news on Facebook today, and I got blocked from posting in other groups. I had figured that I needed a better growth engine instead of over-sharing on Facebook, so I spent the morning planning the new growth engine. Growth Hacking I term what I do in