How to maximize throughput on Proxycurl?

Steven Goh | CEO -

More often or not, Proxycurl is not the bottleneck for your application.

On a regular account, every account has a rate limit of 300 requests every minute. So, in theory, you can make 432,000 API requests per day, which amounts to $4320 per day at a minimum. This brings me to the point that most customers are bottlenecked by budget and not the API itself.

That said, should budget not be a problem for you, you can take the following steps to maximize throughput on Proxycurl API.

  1. Make API requests in parallel. It would help if you increased the concurrency until you are maxing out the rate limit.

For each thread that is making the API request:

  • Handle 503 errors by retrying immediately up to 3 times.
  • Handle 500 errors by retrying up to 2 times later. (Could be an intermittent database issue).
  • Handle 429 with exponential backoff.