Introducing NinjaPear Learn more
A collection of 196 posts
What is Proxycurl? Proxycurl is an API to scrape Professional Social Network profiles at scale. Proxycurl’s Professional Social Network API can scrape 1 million Professional Social Network profiles in a day. Not only does Proxycurl scrape the profiles of people, but it also scrapes profiles of companies and jobs.
We only scrape public profiles, and it turns out that Skills data is not a part of public profiles. We can only return what we can scrape, as such we do not scrape Skills data. However, we are working with external partners to help you enrich skills data via Person
Proxycurl only scrapes public Professional Social Network profiles. Sometimes, the privacy settings of a profile keep certain portions of a private profile hidden publicly. We can only return what exists on the public profile. This is why the profile returned might be missing some data that is otherwise present on
Our API succeeds 90+% of the time. It turns out that our scraping infrastructure is limited and inconsistent because we are dealing with external targets such as Professional Social Network which is very erratic when it comes to rate limits, particularly with their public profiles. We notice 503s occur more
Proxycurl only scrapes public Professional Social Network profiles. Some Professional Social Network profiles do not have public profiles (due to privacy settings). And if the public profile does not exist, then we are not able to scrape it. This is when we return 404. You can verify that a profile
No, we are not able to resolve Sales Navigator URLs into regular profile URLs. As such, we are not able to scrape Sales Navigator URLs. The reason is that Sales Navigator links require an active Professional Social Network login session to resolve into profiles. And we do not scrape Professional
Clearbit Connect I was so amazed when Clearbit Connect launched. It was a game-changing product. No other products existed then that made it so easy to find emails of decision-makers. On top of that, the emails were accurate for the most part. I was curious how did they do it,
There is a certain power in owning things, particularly things that power the guts of your business's product. It is why we self-host GitLab, amongst a ton of other essential services. A data company requires big data. Big data that you can own. I am happy to share
I am happy to share that we performed an exhaustive crawl of all public Professional Social Network Company Profiles, and we have added 17M global companies into LinkDB. It is available right now in LinkDB. Proxycurl LinkDB Global Company Profile VS People Data Labs (PDL) Free Company Dataset We are
Since we launched LinkDB, we have received a barrage of requests for a company profile dataset. We understand that pairing people with companies will help our customers understand questions like: 1. How many employees does a company have? 2. What is the makeup of roles in a company? It is
A few weeks ago, a student at the University of Hong Kong University of Science and Technology (HKUST) contacted me. The student represented a group of academics who needed to enrich a list of people with work experience and education history. The following article will share how I did this
Last week, we push out an update to the Company Profile API. In this update: * We updated the endpoint to return 404 status code when a request is made to scrape an invalid Professional Social Network company. * Added follower_count to the API endpoint * Revamped the parsing method to improve
In a blog post I written, I shared how I got a list of companies seeking to hire software engineers. The crux of the data scraping exercise was our Job Listing Endpoint. However, we found that the Job Listings Endpoint was inconsistent during testing. Job posts returned via our Job
Problem I got on a call with the CEO of a company that helps businesses recruit tech talents. Let's call this company TechTalentRecruitment. The perfect customer profile for TechTalentRecruitment is small and medium-sized companies that are hiring for software engineers. Getting a list of small and medium companies
So you are trying to scrape Professional Social Network profiles in bulk. The chances are that you are crawling the public profiles with a proxy (in a very inefficient manner, as discussed in my other post on building your own Professional Social Network Profile Scraper). If so, then you will
I want to close bigger deals by reaching decision-makers directly I am the founder of Nubela. I lead sales for Proxycurl, and I do this alone. So it is vital that whatever I do is leveraged and effective. Luckily, I can code. I figured that general emails do not get
Steven, the Professional Social Network API you guys offer is excellent. But how do I begin to use the API if I do not even have the profiles? I get this question so often, so we decided to crawl every profile and put it into a Postgresql database. We call
Having built the early prototype for [Proxycurl API](https://nubela.co/proxycurl/Professional Social Network) which turns Professional Social Network profiles into JSON, I learnt a little bit about how one might be able to scrape public Professional Social Network profiles in scale. In this tutorial, I will share my
We launched an update to Proxycurl today, and it includes two new endpoints for Professional Social Network Jobs. The first endpoint is the Job Profile API endpoint. Like the existing People and Company Professional Social Network API endpoints, the Job Profile endpoint returns you structured data of a job given
Disclaimer: This article does not, by any means, constitute legal advice. Views expressed are purely opinion and are not legally binding. Consult with your local legal counsel for advice specific to your project, country or application. In our previous article, Is Professional Social Network Scraping Legal?, we discussed two influential
With Proxycurl API, you can enrich Professional Social Network Profiles at one cent per profile, at scale. Since we introduced Proxycurl, the feedback has revolved around two issues: 1. It is too hard to scrape public Professional Social Network profiles 2. And that our corporate plans are too expensive I
