Introducing NinjaPear Learn more
Welcome to the NinjaPear blog, where we will chronicle our journey in building the world's ultimate customer communication suite.
Hello, For a few months now, Postgresql has been a performance bottleneck for us as we pushed past 1B rows of data. Today, we performed a smooth and uneventful migration to CockroachDB, which I hope will get us to 10B rows of data. Along with this under-the-bonnet infrastructure upgrade, I
When we scrape a public Professional Social Network profile for example with our Person Profile API, we are served with a temporary CDN URL by Professional Social Network which usually begins with https://media-exp... Some of our customers store the temporary CDN URL which expires in an unpredictable manner. To
LinkDB is updated with every real-time (API) request to scrape a Professional Social Network profile. Essentially, this methodology keeps data on LinkDB constantly updated to the count of (up to) millions of profiles a day.
Proxycurl is a pay-as-you-go service. Credits cost US$0.01 / credit. Credit cost varies with API endpoints. You can check out the API documentation [https://nubela.co/proxycurl/docs#enrichment-api] for the price (in credits) of various API endpoints. We offer volume pricing for our Enterprise plans You have to
Your Professional Social Network Public Profile is a profile of you shown to any user browsing your profile without being logged into . If you are on Professional Social Network, chances are that you do have a Professional Social Network public profile. By default, all Professional Social Network accounts have public
Get 10,000 credits every month. Here is how.
How to get a list of all employees with Proxycurl's Employee Listing API Endpoint?
What is Proxycurl? Proxycurl is an API to scrape Professional Social Network profiles at scale. Proxycurl’s Professional Social Network API can scrape 1 million Professional Social Network profiles in a day. Not only does Proxycurl scrape the profiles of people, but it also scrapes profiles of companies and jobs.
We only scrape public profiles, and it turns out that Skills data is not a part of public profiles. We can only return what we can scrape, as such we do not scrape Skills data. However, we are working with external partners to help you enrich skills data via Person
Proxycurl only scrapes public Professional Social Network profiles. Sometimes, the privacy settings of a profile keep certain portions of a private profile hidden publicly. We can only return what exists on the public profile. This is why the profile returned might be missing some data that is otherwise present on
Our API succeeds 90+% of the time. It turns out that our scraping infrastructure is limited and inconsistent because we are dealing with external targets such as Professional Social Network which is very erratic when it comes to rate limits, particularly with their public profiles. We notice 503s occur more
Proxycurl only scrapes public Professional Social Network profiles. Some Professional Social Network profiles do not have public profiles (due to privacy settings). And if the public profile does not exist, then we are not able to scrape it. This is when we return 404. You can verify that a profile
No, we are not able to resolve Sales Navigator URLs into regular profile URLs. As such, we are not able to scrape Sales Navigator URLs. The reason is that Sales Navigator links require an active Professional Social Network login session to resolve into profiles. And we do not scrape Professional
Clearbit Connect I was so amazed when Clearbit Connect launched. It was a game-changing product. No other products existed then that made it so easy to find emails of decision-makers. On top of that, the emails were accurate for the most part. I was curious how did they do it,
There is a certain power in owning things, particularly things that power the guts of your business's product. It is why we self-host GitLab, amongst a ton of other essential services. A data company requires big data. Big data that you can own. I am happy to share
I am happy to share that we performed an exhaustive crawl of all public Professional Social Network Company Profiles, and we have added 17M global companies into LinkDB. It is available right now in LinkDB. Proxycurl LinkDB Global Company Profile VS People Data Labs (PDL) Free Company Dataset We are
Since we launched LinkDB, we have received a barrage of requests for a company profile dataset. We understand that pairing people with companies will help our customers understand questions like: 1. How many employees does a company have? 2. What is the makeup of roles in a company? It is
A few weeks ago, a student at the University of Hong Kong University of Science and Technology (HKUST) contacted me. The student represented a group of academics who needed to enrich a list of people with work experience and education history. The following article will share how I did this
Last week, we push out an update to the Company Profile API. In this update: * We updated the endpoint to return 404 status code when a request is made to scrape an invalid Professional Social Network company. * Added follower_count to the API endpoint * Revamped the parsing method to improve
I am very happy to share that we are scraping more fields in our Person Profile Endpoint in our Professional Social Network API. These fields are: 1. recommendations 2. similarly_named_profiles 3. activities 4. connection (count) 5. articles (posted by user) This update is now live and you can
In a blog post I written, I shared how I got a list of companies seeking to hire software engineers. The crux of the data scraping exercise was our Job Listing Endpoint. However, we found that the Job Listings Endpoint was inconsistent during testing. Job posts returned via our Job
Proxycurl API People Profile endpoint crawls public Professional Social Network Profiles. Not all Professional Social Network profiles, even if valid, have public profiles. In instances as such, our Professional Social Network People Profile Endpoint will return with a 404 status code. Today, we launched an update to our service to
Problem I got on a call with the CEO of a company that helps businesses recruit tech talents. Let's call this company TechTalentRecruitment. The perfect customer profile for TechTalentRecruitment is small and medium-sized companies that are hiring for software engineers. Getting a list of small and medium companies
So you are trying to scrape Professional Social Network profiles in bulk. The chances are that you are crawling the public profiles with a proxy (in a very inefficient manner, as discussed in my other post on building your own Professional Social Network Profile Scraper). If so, then you will
I want to close bigger deals by reaching decision-makers directly I am the founder of Nubela. I lead sales for Proxycurl, and I do this alone. So it is vital that whatever I do is leveraged and effective. Luckily, I can code. I figured that general emails do not get