Updates (so far)
- As of 17th May 2022, 6.22PM SGT, we pushed out an update that should bring Person Profile Endpoint back to normalcy.
- As of 19th May 2022, 5.12PM SGT, we pushed out an update that should bring Company Profile Endpoint back to normalcy.
- As of 25th May 2022, 6.00PM SGT, we pushed out an update that should bring Job Listing Endpoint back to normalcy.
- As of 27th May 2022, 10.30AM SGT, we pushed out an update that should bring Job Profile Endpoint back to normalcy.
- Everything is back to normal.
Service Disruption
There is a severe service disruption as of now. These are the services that are disrupted:
(Temporary issue) Person Profile Endpoint is not returning all profiles correctly or returning at all. Some valid profiles are returning with a Profile-not-found (404) error.(Temporary issue) Company Profile Endpoint is not returning all profiles correctly or returning at all. Some valid profiles are returning with a Profile-not-found (404) error.(Permanent issue with workaround) Company Profile Endpoint can no longer accept Profile URLs with numerical IDs.(Temporary issue) Job Listings Endpoint is not working.(Temporary issue) Job Profile Endpoint is not working.
Why is this happening?
The crux of the problem is that LinkedIn is a moving target. Unfortunately, a few things broke at the same time:
- Our primary scraping infrastructure is experiencing an outage.
- LinkedIn changed a few things:
- The jobs listing page is changed.
- They removed the functionality to fetch a publicly accessible company profile with a numerical ID.
What are we doing to bring things back to normalcy
We are bringing up our redundancy methods of scraping LinkedIn profiles to power the entire stack, one endpoint at a time. We are doing so with the following priority:
- Person Profile Endpoint
- Company Profile Endpoint (ETA 18th May 2022)
- Jobs Listing Endpoint
- Job Profile Endpoint
However, there are permanent side effects. Given that LinkedIn removed the ability to fetch a publicly accessible company profile with a numerical ID, we can no longer support that as a core competency.
We are working on a workaround for the numerical ID defect with Company Profile Endpoint. This workaround is made possible with LinkDB for which we have an internal database of most, if not all LinkedIn companies and their numeric IDs.
This workaround will come in the form of a parameter named resolve_numeric_id. This new parameter, when enabled, will lookup LinkDB for the company's vanity ID.
For example, if you have send a request to our Company Profile Endpoint with the url of https://www.linkedin.com/company/162479 and with the parameter of resolve_numeric_id=true; then our API will resolve 162479 to apple and fetch data for https://www.linkedin.com/company/apple instead.
Effectively, this workaround will bring back support for Company URLs with numeric IDs. We consider this a value-added feature, and we will be charging an extra 2 credits on top of the base credit cost for this parameter.
The ETA of the fixes for the Company Profile Endpoint and this workaround to land in production tomorrow (18th May 2022).
As for the other two job-related API endpoints will take a bit longer, but we hope to bring them back to normalcy within a week or two.
I will be sharing updates over at our Twitter @proxycurl account. Alternatively, you can send me an email at [email protected], and I will do my best to respond to you personally as soon as I can.
