Introducing NinjaPear Learn more

NinjaPear Logo NinjaPear
proxycurl

We updated the status code for "scrape failed" event from 429 to 503

We updated the API today with a new 503 error status code.

Context -- there are two reasons why you will get an error with the status code of 429. And it has been confusing for users because users assume they made a mistake.

The first reason is that your account has hit an account-level rate limit of 300 requests/minute. However, most users do not hit that limit.

The second reason is that scraping failed. And this is a limitation on our end—a backend error. And backend errors should not return a 4xx error code, given that 4xx errors signify a client-side error.

Today, we deployed an update to the API, which will return a status code of 503 if a scrape failed instead of 429. The way you should handle it is the same as is with error 429. If you get a status code of 503, do retry the request immediately, and you're good to go.

Steven Goh | CEO
World's laziest CEO. Before starting the highly-successful Proxycurl and Sapiengraph, Steven founded 5 other startups: Gom VPN, Kloudsec, SilvrBullet, NuMoney, and SharedHere.

Featured Articles

Here's what we've been up to recently.

I dismissed someone, and it was not because of COVID19

The cadence of delivery. Last month, I dismissed the employment of a software developer who oversold himself during the interview phase. He turned out to be on the lowest rung of the software engineers in my company. Not being good enough is not a reason to be dismissed. But not

sharedhere

I got blocked from posting on Facebook

I tried sharing some news on Facebook today, and I got blocked from posting in other groups. I had figured that I needed a better growth engine instead of over-sharing on Facebook, so I spent the morning planning the new growth engine. Growth Hacking I term what I do in