I really should be doing more marketing instead of iterating on the product. But here I am, with a huge performance improvement to the time it takes for the Person Profile Endpoint and the Company Details endpoint to perform a live enrichment. Here are the new speeds:
| API Endpoint | Average Request Duration Previously | Average Request Duration Currently | Speedup % |
|---|---|---|---|
| Person Profile Endpoint (v2) | 38.7s | 10.5s | 3685% |
| Company Details Endpoint | 42.4s | 10.4s | 4080% |
Just for context, as the CEO of Proxycurl, which was by far the largest Enterprise-grade LinkedIn scraper before we shut it down, it took about 5-6s to perform a live LinkedIn enrichment. This is a huge feat given that we are performing real-time web research with LLMs in order to fetch this data. Don't take my word for it; try using the NinjaPear API and watch it fetch data for companies and people that do not have any social media presence.
I think I can cut out maybe another 2s with a significantly faster LLM like the Xiaomi Mimo Ultraspeed, which I'm supposed to have access to but for some reason can't query the model yet. So, more improvements are to come, and you can bet that I will be squeezing every bit of juice out of our API because I know latency matters. It's really quite off-putting to have to wait 40s for an API query to complete. At 10s, it's actually palatable.
Person Profile Endpoint Version Bumped To v2, with Breaking Changes
In order to eke out this performance improvement for the Person Profile Endpoint, I had to make a tough compromise and mildly tweak the API behavior of said endpoint. Hence the bump of this endpoint to v2 and a new API route of /api/v2/employee/profile.
v2 of the Person Profile Endpoint introduces a new input parameter called enrichment, which takes the value of either fast or detailed. The enrichment parameter defaults to the enrichment=fast value.
In the fast path of enrichment, the Person Profile Endpoint will strive to quickly perform research, structure the important bits of a person's profile data, and return a response quickly with these data points. These prioritized data bits include:
- bio
- work history
- education history
That said, you are not missing out on the rest of the good parts. While the fast enrichment is happening, and even after the initial response is given, the backend is still working on the enrichment. What that means is that you can continue to poll the same endpoint with the same input parameter until the X-NinjaPear-Enrichment-Status response header turns from pending to complete, which is when you get the complete person profile data with the rest of the good bits. The completion usually happens within 20-30s after the initial response.
And if you find this too cumbersome, you can call v2 of the Person Profile Endpoint with the enrichment=detailed parameter and simply wait for every bit of data to be available without needing to poll for the complete data. In fact, v1 of the Person Profile Endpoint is now syntactic sugar for calling the v2 endpoint with the enrichment=detailed parameter, and it remains supported to preserve backward compatibility.
Company Profile Endpoint Remains at v1
I was able to make the performance improvements for the Company Profile Endpoint without any breaking changes; hence, the Company Profile Endpoint remains at v1, and all of you will get the performance improvement for free!
Pricing Changes
NinjaPear is a real-time B2B data enrichment service. We are not reading off a database like our competitors are.
To squeeze out these massive performance gains, we are using more resources to make a lot more concurrent web crawls and employing premium and speedy LLM agents/models. Our margins are slashed, and the pragmatic thing to do is to raise prices. (Glares at Anthropic).
This is why I'm going to announce that:
- I will not raise the cost of the Person Profile Endpoint with
enrichment=fastto6credits, up from3credits. - I will not raise the cost of the Company Details Endpoint with
enrichment=fastto6credits, up from3credits.
We are too small an outfit to be doing this now. That said, we might raise our prices when our margins become untenable.
So no, we are not raising prices.
Profile Data in Gmail
If you do not know already, NinjaPear offers a Firefox and Chrome browser extension known as Supportive that adds the profile data of the people you are emailing in Gmail. It's pretty sick, and I use it all the time. I'll be like:
Hey, nice meeting a fellow Singaporean.
And I know he/she is from Singapore because Supportive tells me so! The only problem is that profiles take so long to load. I don't want to be waiting 40s after opening a thread before having to reply to an email with good context. It was Supportive that provided the impetus to dramatically improve the UX of our API endpoints.
If you are not using Supportive yet, do download it and give it a try! The first 50 profile loads are free!
Negative Caching
One more thing. Another annoying thing that surfaced when I was dogfooding Supportive was that emails/profiles that previously failed to surface any data kept loading every time I opened a thread. And I had to wait 40s again and again for it to go away, knowing that there was no real person behind that email (a notification email address, for example).
This is why the Person Profile and Company Details endpoints now include negative caching. If you make a request that fails and you are charged for it, you will not be charged again when you make another request with the same parameters because the failure is now cached. The failure response will be returned immediately without any further live enrichment.
Of course, you can disable this caching behavior with the use_cache=never parameter. NinjaPear is made by developers, for developers. I care about these things.
Anyway, I hope you enjoyed this slightly more technical write-up with no LLMs involved. I really should be doing more marketing; hence, I'm veering towards more personal writing now.
As usual, if you have any questions, shoot me an email at [email protected]!
