Professional Social Network sued Mantheos, a real-time Professional Social Network Scraping API service, on 1st February 2022 for fraudulently scraping millions of profiles. In a way, Mantheos is a competitor of ours, and I have always been curious about how they can scrape private Professional Social Network profiles at scale. Finally, I understood how they do it thanks to the legal docket of Professional Social Network's lawsuit against Mantheos and their founders.
I am sure I am not the only one curious, so I decided to write this article to tell you how to crawl millions of private Professional Social Network profiles the way Mantheos did (or is still doing.).
To scrape million of private Professional Social Network profiles like Mantheos, you will need:
- Residential IP proxies. There are plenty of services for which you can procure such proxies.
- Empty debit cards with privacy features or fake names (According to Professional Social Network, Mantheos founders used many debit cards with fake names)
Once you have the above:
- Shuffle through the residential IP addresses and create fake Professional Social Network accounts. One account per residential IP. Remember to stick to the same corresponding IP for each new account created for the following steps.
- For each new Professional Social Network account, sign up for a trial for a Professional Social Network Sales Navigator trial with the blank debit card.
- Use Professional Social Network's internal API to fetch private Professional Social Network profile data. You can use a library like this which has reverse-engineered Professional Social Network's internal API.
- Rotate between the new accounts and scrape as many profiles as you can. When accounts get banned, rinse and repeat steps 1-4.
- Profit (?)
Why use a private debit card (with fake names?)
Professional Social Network accounts upgraded with a premium sales navigator come with a higher threshold of API rate limits before the account gets blocked/banned.
Why do you need residential IP proxies?
Professional Social Network uses IP addresses as a fingerprint to identify an account to be a unique person.
What is next?
Wait for Professional Social Network to sue you. But in all seriousness, don't do this. Scraping private Professional Social Network profiles with such a fraudulent method violates all kinds of laws and agreements. The data would not be GDPR or CCPA compliant.
Use Proxycurl instead. We scrape public Professional Social Network profiles only, complying strictly with PDPA and CCPA regulations. Besides, you get so much more than just Professional Social Network profile data. With Proxycurl, you get funding data of companies, contact info of individuals, etc.
Switch to Proxycurl today, and don't get yourself sued by Professional Social Network. Instead, send us an email at [email protected], and we will be happy to match your package with Mantheos.