| Proxycurl | PhantomBuster |
|---|---|
| Scrapes Public LinkedIn profile | Scrapes Private LinkedIn profile |
| No need for you to bring your own LinkedIn account | Bring your own LinkedIn account |
| No limits other than API rate limit (300 requests/minute; or 432,000 profiles / day). Rate limit can be increased in an Enterprise Plan. | Total profiles limited by LinkedIn account (<100 profiles / day, < 1000 profiles / month) |
| Profile is fresh | Profile is fresh |
| CPPA complaint | Not CPPA compliant |
| GDPR compliant | Not GDPR complaint |
| Legally tested - See HiQ VS LinkedIn | At serious legal risk. See Mantheos's lawsuit |
| Enriches profiles with contact information from external sources - 33% match rate | Scrapes public emails from LinkedIn profiles - 1% match rate. |
| Has phone numbers | No phone numbers |
| Every public profile is available | Profiles beyond first 2 levels of connections are unavailable |
| No UI | Has UI to read-from/write-to spreadsheets |
| $0.01 / profile (as of 10th April 2022) | Around $0.0295 / profile (on starter plan) |
| 99% API success rate | 100% API success rate |
| 3.5 minutes for 1000 profiles | 7 hours for 1000 profiles |
Phantombuster ranks extremely well for "Linkedin Scraper API" on Google, and it is a fair assumption that they are a competent solution for scraping LinkedIn profiles. That is true because there are some things that Phantombuster can do that Proxycurl API cannot do. For example, PhantomBuster is remarkably remarkable at:
- Bypassing privacy filters to scrape private profiles.
- Fetching contact data listed on LinkedIn profiles.
PhantomBuster is not without its caveats, though. The most significant disadvantages of PhantomBuster are that:
- You need to bring your own LinkedIn account.
- You cannot scale LinkedIn scraping; You are limited to what your LinkedIn account can scrape.
- Your data cannot be used commercially in a compliant manner.
Differences aside, Proxycurl API and PhantomBuster are both capable of scraping LinkedIn profiles. The nuances lie in the way the different services scrape LinkedIn. Let's break down the similarities and differences between Proxycurl API and PhantomBuster in scraping LinkedIn profiles.
Public VS Private profiles
The fundamental difference between Proxycurl API and PhantomBuster is that Proxycurl scrapes public LinkedIn profiles while PhantomBuster scrapes private LinkedIn profiles. This difference would account for many of the differences we will discuss further.
Let's go deeper and look at how each service works.
Proxycurl API is a 100% managed infrastructure to scrape public LinkedIn profiles. Proxycurl has stuck to scraping public LinkedIn profiles despite having less data than private profile data. The best thing about Proxycurl is that it is easy to scrape a LinkedIn profile. Pass a LinkedIn profile URL to Proxycurl's Person Profile Endpoint, and you will get structured data of the profile back in less than 3 seconds.
However, unlike PhantomBuster, Proxycurl has stuck to its guns in serving the software developer niche. In addition, Proxycurl is an API product which means it cannot be easily utilized by non-technical users.
On the other side of the spectrum, Phantombuster is a scraper tool that sits on top of your LinkedIn account to scrape LinkedIn profiles. In other words, to use PhantomScraper, they will need you to provide LinkedIn credentials (in the form of a cookie), so they can automate scraping on your account. (Proxycurl does not require a LinkedIn account from you)
Phantombuster works for small-scale LinkedIn scraping for personal use-cases. It works for an account executive, and you are looking to automate your LinkedIn workflow by scraping your connections for a cold email campaign. Still, your account might get blocked because there is a maximum limit to the number of profiles that can be viewed daily.
PhantomBuster might not be the best solution for you for anything more than just hobby-level scraping.
And there lie the biggest problems with PhantomBuster:
- You will be scraping private LinkedIn profiles with your LinkedIn account, risking account bans and lawsuits. PhantomBuster is also not solving.
- PhantomBuster is not solving the scaling problem with LinkedIn scraping. You cannot scrape 1000 LinkedIn profiles with PhantomBuster. 100 profiles will get your account blocked.
| Proxycurl | PhantomBuster |
|---|---|
| Scrapes Public LinkedIn profile | Scrapes Private LinkedIn profile |
| No need to bring your own LinkedIn account | You have to provide LinkedIn acconts for PhantomBuster to work |
| Can scrape 432000 profiles / day | Scrape <100 profiles / day before risking LinkedIn account ban |
Data freshness and completeness
Proxycurl API works with public LinkedIn profiles, while PhantomBuster works with private LinkedIn profiles.
Profiles returned by both PhantomBuster and Proxycurl are fresh, with a slight difference.
PhantomBuster scrapes LinkedIn from your account in real-time, so the profiles returned are fresh.
Proxycurl scrapes public LinkedIn profiles in real-time, but Proxycurl caches popular profiles for up to 29 days. So to be precise, Proxycurl API returns fresh profiles no older than 29 days old. (80+% of the API requests are scraped in real-time).
Public LinkedIn profiles have a few missing data points relative to private LinkedIn profiles. These data points include skills, likes and comments made on LinkedIn's status updates. You can check out this post for a clear breakdown of the core differences between public LinkedIn profiles VS private LinkedIn profiles.
Contact data is not available on public LinkedIn profiles. Proxycurl compensates for the loss of contact data by matching it with external data sources. With Proxycurl's Contact API, you can enrich LinkedIn profiles with
- Work email address
- Personal email addresses
- Personal mobile numbers
Proxycurl's Contact API comes with a 33+% match rate for contact data US profiles.
Private LinkedIn profiles win out on the exhaustiveness of data points. But they lose to public LinkedIn profiles in terms of reach. As is with any LinkedIn account, profiles beyond the second level of connection are not accessible and cannot be scraped by PhantomBuster.
Public LinkedIn profiles do not have a connection-of-connection limitation. What it does have instead is a privacy setting limitation. That means users can disable their public LinkedIn profile or hide parts of their profile from being shown in their public profile. Around 5% of profiles on LinkedIn have tweaked their privacy settings to hide parts or all of their profile from their public profile.
Personal email addresses are available on a private LinkedIn profile if the profile is a first-level connection with the LinkedIn account used to scrape with PhantomBuster. Phone numbers and work email addresses are not available.
So yes, with PhantomBuster, you get more data points scraped per profile; but not all profiles are accessible. Phantombuster returns email addresses if the target profile is a first-level connection only.
With Proxycurl, most profiles are accessible, but each profile comes back with fewer data points. Proxycurl returns personal emails, work emails and phone numbers enriched from external sources with a 33+% match rate.
| Proxycurl | PhantomBuster |
|---|---|
| Profiles are fresh (<= 29 days old) | Profiles are fresh (0 day) |
| No skills, language proficiency, activities | Has every data point |
| Scrape 432000 profiles / day on standard rate limit terms | Scrape <100 profiles / day before risking LinkedIn account ban |
| Most profiles are available unless profile is hidden by privacy settings | Profiles are limited to within two levels of connection |
GDPR and CPPA Compliance
Public LinkedIn profiles are publicly available data. These profiles are filtered through a user's privacy settings, and the data shown is then crawled and indexed by major search engines. These search engines serve public LinkedIn data globally with no limitations because these profiles are freely available. We describe a method by which you can view any public LinkedIn profile consistently without a LinkedIn account. Data opted to be made available publicly by users, for most use-cases, is GDPR and CPPA compliant.
Private profile data is shown to an individual on LinkedIn sans any privacy filter. It comes with permission if and only if the data target is a first-level connection of the user that is scraping the data. It is safe to say that private LinkedIn data, when scraped, is not GDPR or CPPA compliant.
| Proxycurl | PhantomBuster |
|---|---|
| GDPR compliant | Not GDPR compliant |
| CPPA compliant | Not CPPA compliant |
Legality
Proxycurl scrapes public LinkedIn profiles only. And this methodology has been rigorously tested in the HiQ VS LinkedIn lawsuit. Therefore, it is safe to say that as of 13th April 2022, LinkedIn will lose if they are suing you for scraping public LinkedIn profiles based on legal precedence.
On the flipside, Mantheos just had their founders and the company sued by LinkedIn for fraud amongst other charges, in US's federal courts. They are sued because they employed multiple fake LinkedIn accounts and fraudulently signed the fake accounts for sales navigator trials which LinkedIn was unable to make any charges on. As of now, it is not clear what the outcome of the lawsuit would be, but I would not bet against LinkedIn in this case.
| Proxycurl | PhantomBuster |
|---|---|
| There is legal precedence on LinkedIn losing lawsuits pertaining to public LinkedIn profile scraping | A major player is currently being sued by LinkedIn for scraping private LinkedIn profiles |
API capability
As advertised on Phantombuster website, it takes 7-8 hours to scrape 1000 LinkedIn profiles.
Proxycurl API has a rate limit of 300 requests/minute. And that means you need no more than 3.5 minutes to fetch profile data of 1000 LinkedIn profiles via Proxycurl API.
| Proxycurl | PhantomBuster |
|---|---|
| 3.5 minutes to fetch 1000 LinkedIn profiles | 7-8 hours to fetch 1000 LinkedIn profiles |
Ease of use
Proxycurl API is an API and is a tool intended for use by software engineers. An individual is expected to possess basic programming knowledge to utilize Proxycurl API.
PhantomBuster advertises a spreadsheet methodology to power their scrapers. As a result, PhantomBuster has a lower barrier of entry to utilize their LinkedIn Scraping tool.
| Proxycurl | PhantomBuster |
|---|---|
| For software engineers | For anyone that can wield a spreadsheet |
Pricing
PhantomBuster's basic plan costs ~$0.03 to scrape a profile.
It costs one credit on Proxycurl API to fetch one LinkedIn profile data. 1 Proxycurl credit costs $0.01 as of 13th April 2022.
| Proxycurl | PhantomBuster |
|---|---|
$0.01 per profile |
~$0.03 per profile |
Getting started with LinkedIn Scraping
Both Proxycurl and PhantomBuster offer a free trial for you to start scraping LinkedIn. You can sign up for Proxycurl's free trial by registering an account here.
I have tried to be exhaustive and unbiased in evaluating PhantomBuster VS Proxycurl for LinkedIn scraping. Send me an email at [email protected] if you have more questions about LinkedIn Profile scraping, and I will do my best to help!
