Introducing NinjaPear Learn more

NinjaPear Logo NinjaPear
NinjaPear Logo

NinjaPear Blog

Welcome to the NinjaPear blog, where we will chronicle our journey in building the world's ultimate customer communication suite.

proxycurl

How often is LinkDB updated?

LinkDB is updated with every real-time (API) request to scrape a Professional Social Network profile. Essentially, this methodology keeps data on LinkDB constantly updated to the count of (up to) millions of profiles a day.

proxycurl

Is Professional Social Network Scraping GDPR compliant?

What is Proxycurl? Proxycurl is an API to scrape Professional Social Network profiles at scale. Proxycurl’s Professional Social Network API can scrape 1 million Professional Social Network profiles in a day. Not only does Proxycurl scrape the profiles of people, but it also scrapes profiles of companies and jobs.

proxycurl

The profile returned is missing experience data!

Proxycurl only scrapes public Professional Social Network profiles. Sometimes, the privacy settings of a profile keep certain portions of a private profile hidden publicly. We can only return what exists on the public profile. This is why the profile returned might be missing some data that is otherwise present on

proxycurl

I am getting a lot of 503 "Scraping failed" errors

Our API succeeds 90+% of the time. It turns out that our scraping infrastructure is limited and inconsistent because we are dealing with external targets such as Professional Social Network which is very erratic when it comes to rate limits, particularly with their public profiles. We notice 503s occur more

proxycurl

Why is it that I am getting 404s when the profile is valid?

Proxycurl only scrapes public Professional Social Network profiles. Some Professional Social Network profiles do not have public profiles (due to privacy settings). And if the public profile does not exist, then we are not able to scrape it. This is when we return 404. You can verify that a profile

proxycurl

Available now: LinkDB Snapshots

There is a certain power in owning things, particularly things that power the guts of your business's product. It is why we self-host GitLab, amongst a ton of other essential services. A data company requires big data. Big data that you can own. I am happy to share

proxycurl

Proxycurl VS PeopleDataLabs - Global Company Profile Dataset

Since we launched LinkDB, we have received a barrage of requests for a company profile dataset. We understand that pairing people with companies will help our customers understand questions like: 1. How many employees does a company have? 2. What is the makeup of roles in a company? It is

proxycurl

Introducing V2 of Job Listings Endpoint

In a blog post I written, I shared how I got a list of companies seeking to hire software engineers. The crux of the data scraping exercise was our Job Listing Endpoint. However, we found that the Job Listings Endpoint was inconsistent during testing. Job posts returned via our Job