Proxycurl Blog | Read our stories on data, scraping, APIs (Page 8)

Welcome to the Proxycurl blog, your new hub for all things relating to B2B data. Especially data enrichment and scraping! You can find our product and company updates here too.

We updated our pricing and made Email Lookup API generally available

We updated our pricing and made Email Lookup API generally available

Hello, For a few months now, Postgresql has been a performance bottleneck for us as we pushed past 1B rows of data. Today, we performed a smooth and uneventful migration to CockroachDB, which I hope will get us to 10B rows of data. Along with this under-the-bonnet infrastructure upgrade, I

We updated our pricing and made Email Lookup API generally available

Why is the API returning S3 links for profile pictures scraped from Linkedin profiles?

When we scrape a public Linkedin profile for example with our Person Profile API [https://nubela.co/proxycurl/linkedin], we are served with a temporary CDN URL by Linkedin which usually begins with https://media-exp... Some of our customers store the temporary CDN URL which expires in an unpredictable manner.

How often is LinkDB updated?

How often is LinkDB updated?

LinkDB [https://nubela.co/proxycurl/linkdb] is updated with every real-time (API) request to scrape a Linkedin profile. Essentially, this methodology keeps data on LinkDB constantly updated to the count of (up to) millions of profiles a day.

How often is LinkDB updated?

How much does Proxycurl cost? How do I get on an Enterprise plan? Do you have volume pricing?

How much does Proxycurl cost? How do I get on an Enterprise plan? Do you have volume pricing?

Proxycurl is a pay-as-you-go service. Credits cost US$0.01 / credit. Credit cost varies with API endpoints. You can check out the API documentation [https://nubela.co/proxycurl/docs#enrichment-api] for the price (in credits) of various API endpoints. We offer volume pricing for our Enterprise plans You have to

How much does Proxycurl cost? How do I get on an Enterprise plan? Do you have volume pricing?

The Ultimate Guide To LinkedIn Public Profile Visibility

Your LinkedIn Public Profile is a profile of you shown to any user browsing your profile without being logged into . If you are on LinkedIn, chances are that you do have a LinkedIn public profile. By default, all LinkedIn accounts have public profiles generated for them automatically. However, your public

How to list all employees of any company

How to list all employees of any company

How to get a list of all employees with Proxycurl's Employee Listing API Endpoint?

How to list all employees of any company

Does Proxycurl's LinkedIn API return "Skills" data?

Does Proxycurl's LinkedIn API return "Skills" data?

We only scrape public profiles, and it turns out that Skills data is not a part of public profiles. We can only return what we can scrape, as such we do not scrape Skills data. However, we are working with external partners to help you enrich skills data via Person

Does Proxycurl's LinkedIn API return "Skills" data?

The profile returned is missing experience data!

Proxycurl only scrapes public LinkedIn profiles. Sometimes, the privacy settings of a profile keep certain portions of a private profile hidden publicly. We can only return what exists on the public profile. This is why the profile returned might be missing some data that is otherwise present on a private

I am getting a lot of 503 "Scraping failed" errors

I am getting a lot of 503 "Scraping failed" errors

Our API succeeds 90+% of the time. It turns out that our scraping infrastructure is limited and inconsistent because we are dealing with external targets such as LinkedIn which is very erratic when it comes to rate limits, particularly with their public profiles. We notice 503s occur more during the

I am getting a lot of 503 "Scraping failed" errors

We are serious about purchasing credits. However, I need more trial credits for evaluation. Are you able to provide us with more trial credits?

We are serious about purchasing credits. However, I need more trial credits for evaluation. Are you able to provide us with more trial credits?

Do you have a budget of $1.99 for testing the service? If so, are you able to head over to the dashboard and make a top-up? $1.99 will yield you 100 credits, which I believe should be plenty for you to evaluate the service. In our early days,

We are serious about purchasing credits. However, I need more trial credits for evaluation. Are you able to provide us with more trial credits?

Why is it that I am getting 404s when the profile is valid?

Proxycurl only scrapes public Linkedin profiles. Some Linkedin profiles do not have public profiles (due to privacy settings [https://nubela.co/blog/the-ultimate-guide-to-linkedin-public-profile-visibility/]). And if the public profile does not exist, then we are not able to scrape it. This is when we return 404. You can verify that a

Can Proxycurl scrape Linkedin's Sales Navigator Links

Can Proxycurl scrape Linkedin's Sales Navigator Links

No, we are not able to resolve Sales Navigator URLs into regular profile URLs. As such, we are not able to scrape Sales Navigator URLs. The reason is that Sales Navigator links require an active Linkedin login session to resolve into profiles. And we do not scrape Linkedin profiles with

Can Proxycurl scrape Linkedin's Sales Navigator Links

Proxycurl's (LinkedIn Profile) Work Email Extraction API Endpoint is coming soon

Clearbit Connect I was so amazed when Clearbit Connect launched. It was a game-changing product. No other products existed then that made it so easy to find emails of decision-makers. On top of that, the emails were accurate for the most part. I was curious how did they do it,

Available now: LinkDB Snapshots

Available now: LinkDB Snapshots

There is a certain power in owning things, particularly things that power the guts of your business's product. It is why we self-host GitLab, amongst a ton of other essential services. A data company requires big data. Big data that you can own. I am happy to share

Available now: LinkDB Snapshots

Available now: 17M Linkedin Company Profiles in LinkDB

I am happy to share that we performed an exhaustive crawl of all public Linkedin Company Profiles, and we have added 17M global companies into LinkDB. It is available right now in LinkDB [https://nubela.co/proxycurl/docs#linkdb]. Proxycurl LinkDB Global Company Profile VS People Data Labs (PDL) Free

Proxycurl VS PeopleDataLabs - Global Company Profile Dataset

Since we launched LinkDB, we have received a barrage of requests for a company profile dataset. We understand that pairing people with companies will help our customers understand questions like: 1. How many employees does a company have? 2. What is the makeup of roles in a company? It is

How we enriched 7084 records for two professors and a pHD student of University of Hong Kong University of Science and Technology (HKUST)

A few weeks ago, a student at the University of Hong Kong University of Science and Technology (HKUST) contacted me. The student represented a group of academics who needed to enrich a list of people with work experience and education history. The following article will share how I did this

Our LinkedIn Company Profile API Endpoint just got faster and better

Last week, we push out an update to the Company Profile API [https://nubela.co/proxycurl/linkedin/company]. In this update: * We updated the endpoint to return 404 status code when a request is made to scrape an invalid LinkedIn company. * Added follower_count to the API endpoint * Revamped the

We added activity, articles, recommendations and MORE to our Linkedin Profile API endpoint

I am very happy to share that we are scraping more fields in our Person Profile Endpoint [https://nubela.co/proxycurl/linkedin] in our Linkedin API. These fields are: 1. recommendations 2. similarly_named_profiles 3. activities 4. connection (count) 5. articles (posted by user) This update is now live

Introducing V2 of Job Listings Endpoint

Introducing V2 of Job Listings Endpoint

In a blog post I written [https://nubela.co/blog/how-to-find-companies-that-are-hiring-for-a-specific-role-with-linkedin-api/] , I shared how I got a list of companies seeking to hire software engineers. The crux of the data scraping exercise was our Job Listing Endpoint. However, we found that the Job Listings Endpoint was inconsistent during testing. Job

Introducing V2 of Job Listings Endpoint

Credits are now used for scraping missing LinkedIn profiles (404).

Credits will now be consumed when you scrape a missing Linkedin profile (404). This is why.

Proxycurl API People Profile endpoint crawls public Linkedin Profiles. Not all Linkedin profiles, even if valid, have public profiles. In instances as such, our Linkedin People Profile Endpoint [https://nubela.co/proxycurl/docs#linkedin-person-profile-endpoint] will return with a 404 status code. Today, we launched an update to our service to

Credits are now used for scraping missing LinkedIn profiles (404).

How to find companies that are hiring for a specific role with Linkedin API

How to find companies that are hiring for a specific role with Linkedin API

Problem I got on a call with the CEO of a company that helps businesses recruit tech talents. Let's call this company TechTalentRecruitment. The perfect customer profile for TechTalentRecruitment is small and medium-sized companies that are hiring for software engineers. Getting a list of small and medium companies

How to find companies that are hiring for a specific role with Linkedin API

Handling Linkedin Public Profiles that exist but cannot be scraped

Handling Linkedin Public Profiles that exist but cannot be scraped

So you are trying to scrape Linkedin profiles in bulk. The chances are that you are crawling the public profiles with a proxy (in a very inefficient manner, as discussed in my other post on building your own Linkedin Profile Scraper). If so, then you will come across a weird

Handling Linkedin Public Profiles that exist but cannot be scraped

Programmatically send personalized emails

How to programmatically send personalized emails to 200 decision makers of Fortune 500 companies (with code samples)

I want to close bigger deals by reaching decision-makers directly I am the founder of Nubela. I lead sales for Proxycurl, and I do this alone. So it is vital that whatever I do is leveraged and effective. Luckily, I can code. I figured that general emails do not get

Programmatically send personalized emails

Subscribe to our newsletter

Get the latest news from Proxycurl