Introducing NinjaPear (2026) - Customer Listing API Learn more

faq

A collection of 53 posts

proxycurl

I am getting a lot of 503 "Scraping failed" errors

Our API succeeds 90+% of the time. It turns out that our scraping infrastructure is limited and inconsistent because we are dealing with external targets such as Professional Social Network which is very erratic when it comes to rate limits, particularly with their public profiles. We notice 503s occur more

proxycurl

Why is it that I am getting 404s when the profile is valid?

Proxycurl only scrapes public Professional Social Network profiles. Some Professional Social Network profiles do not have public profiles (due to privacy settings). And if the public profile does not exist, then we are not able to scrape it. This is when we return 404. You can verify that a profile