How to Get a Professional Social Network Profile’s Contact Information Programmatically
Many businesses use Professional Social Network for prospecting and outreach because of the massive amounts of B2B data available on it, but it can be rather challenging and expensive.
On one hand, Professional Social Network's native solutions like InMail messages are designed to keep you on their platform, rather than natively integrating with your workflow because it's more expensive for you, and more profitable for them.
And then, on the other hand, [third-party solutions like Professional Social Network messaging/outreach automation tools and browser extensions](https://nubela.co/blog/best-Professional Social Network-automation-tools/) don't scale well and include the risk of getting banned from the platform.
The next option is scraping data from Professional Social Network yourself, but that comes with its share of headaches too like constantly rotating proxies, Professional Social Network accounts, and always having to update your scraper to keep it working.
So, you're basically, stuck between a rock and a hard place.
Unless, of course, you had a way to programmatically access all of the B2B data you could ever need without having to scrape a single thing yourself or download any browser extensions...
Meet Proxycurl, a B2B enrichment API
Hi! That's us.
We've already done the hard part for you by scraping over 472 million Professional Social Network profiles ([we call our Professional Social Network dataset LinkDB](https://nubela.co/blog/linkdb-Professional Social Network-dataset-with-people-and-company-profiles/)) and enriching it with external sources. We make B2B data easy.
All you need to do is request the data from our API (application programming interface, ELI5 definition: a fast food menu for data), and you're free to do whatever you want with it. You can even natively integrate our API into your outreach systems and automate the whole process.
When it comes to contact information enrichment specifically, our Contact API provides a programmatic and cost-effective way to enrich a profile/prospect with verified work emails, personal emails, contact numbers, and more.
By integrating our API into your existing workflow, you can:
- Access verified personal/work emails and contact numbers
- Enrich your existing leads with accurate and up-to-date information
- Scale your outreach efforts without the need for manual data entry
- Save time and money compared to using Professional Social Network's native tools
How to request B2B data from our Professional Social Network API
There are many different ways you can request data from an API, like cURL, Python or JavaScript, but regardless all of these different ways revolve around making HTTP requests.
Continuing on from the ELI5 fast food menu example from earlier, HTTP requests are like the different ways you can order your food or ask about it.
You can GET (ask for the menu), POST (order something), PUT (change your order), DELETE (cancel your order), and a few more. These requests are the standard way the internet talks; it's the language of web browsers and servers.
cURL is a tool available on Linux, macOS, and Windows used to transfer data to or from a server, and and is one of the easiest ways to request data, so we'll start with that.
In the case of cURL, it's like a messenger you can send to the restaurant (server) with a specific note (request).
The note says things like "Please tell me what pizzas you have" or "I want one large pepperoni pizza". cURL knows how to talk to the restaurant no matter what language they speak or what dishes they serve. It's versatile and can send messages to any restaurant (server) in the world as long as you know the address (URL) and what you want to order (the request method and data).
How to get personal email addreses from Professional Social Network profiles
Using cURL, we can request the personal email address from a given Professional Social Network profile with our Personal Email Lookup Endpoint.
Here's how:
curl \
-G \
-H "Authorization: Bearer ${YOUR_API_KEY}" \
'https://nubela.co/proxycurl/api/contact-api/personal-email' \
--data-urlencode 'Professional Social Network_profile_url=https://professionalsocialnetwork.com/in/williamhgates' \
--data-urlencode 'email_validation=include' \
--data-urlencode 'page_size=0'
Which would return a JSON result such as this:
{"emails":[],"invalid_emails":["[email protected]""]}
In this case, the email is invalid because it's not a personal email but rather charity/business.
You could sign up for a Proxycurl account here and test out this cURL request yourself. After creating your account you'll receive 10 free credits.
By the way, the above cURL command is using GET which is declared by the -G.
How to get personal phone numbers from Professional Social Network profiles
Similar to our Personal Email Lookup Endpoint, a simple cURL request to our Personal Contact Number Lookup Endpoint with a given Professional Social Network profile URL will allow you to receive a phone number back in return.
Here's how:
curl \
-G \
-H "Authorization: Bearer ${YOUR_API_KEY}" \
'https://nubela.co/proxycurl/api/contact-api/personal-contact' \
--data-urlencode 'Professional Social Network_profile_url=https://professionalsocialnetwork.com/in/williamhgates' \
--data-urlencode 'page_size=0'
Which would return a JSON result such as this:
{"numbers": [+13334248181]}%
How to get work emails from Professional Social Network profiles
Finally, last, but not least, our Work Email Lookup Endpoint is our API endpoint that allows you to retrieve the work email address associated with a given Professional Social Network profile URL.
It works a bit differently from the other two because it will only return the data to a webhook. For testing, there are free services out there like webhook.site. Other than that, it's the same.
Here's how we could request a work email with cURL:
curl \
-G \
-H "Authorization: Bearer ${YOUR_API_KEY}" \
'https://nubela.co/proxycurl/api/Professional Social Network/profile/email' \
--data-urlencode 'Professional Social Network_profile_url=https://professionalsocialnetwork.com/in/williamhgates' \
--data-urlencode 'callback_url= https://webhook.site/0a0a6db2-6593-4bdc-bfa5-cc85dc9303ca'
Which will immediately return JSON back with your work email queue count:
{"email_queue_count": 1}%
And then send the work email to your webhook:
We even built a demo to show you how to enrich contacts
To demonstrate the capabilities of our Contact API (because we believe in showing and not just telling), I've developed a simple B2B contact enrichment demo application using Django and Python:
It allows you to:
- Enter a Professional Social Network profile URL and enrich it with verified [work email addresses](https://skrapp.io/Professional Social Network), personal email addresses, and phone numbers.
- Import Professional Social Network profile URLs in bulk using a CSV file, making it easy to process large numbers of profiles simultaneously.
- Store enriched contact information in a SQLite database, providing a persistent and easily accessible storage solution.
- Export enriched contact data as a CSV file, allowing you to easily import the data into your existing CRM or marketing automation tools.
- View and search enriched contact information through a user-friendly web interface, making it simple to find and manage the data you need.
While this application is intended as a demo, it could indeed be used locally in production or slightly altered (such as implementing authentication logic so random individuals couldn't use it publicly and changing out the development Django web server) and used between an entire team.
Once adding things like authentication, you could further integrate something like Stripe's API and build a full-blown SaaS tool. Accomplishing this is pretty easy with Django's framework.
Download the contact enrichment demo
The application is entirely open source, and you can download it below:
(Just check your email shortly after submitting the above form.)
How to integrate our API into your application
The rest of the article will be spent explaining how you can use our API in the background for your contact enrichment/outreach using the application as a demonstration.
That said, I've intentionally built the demo with Python as it's a very high level programming language that's easy to pick up, and easy to get help with from ChatGPT or Claude.
Regardless if you know zero code, you're somewhat experienced, or your a coding guru, you'll be able to find value out of this article.
We'll cover the following steps:
- [Setting up your development environment](https://nubela.co/blog/get-a-Professional Social Network-profiles-contact-information/#setting-up-your-development-environment)
- [Configuring ngrok for handling webhooks in local development](https://nubela.co/blog/get-a-Professional Social Network-profiles-contact-information/#setting-up-ngrok-for-webhooks)
- [Defining your data models for persistent storage](https://nubela.co/blog/get-a-Professional Social Network-profiles-contact-information/#modelspy)
- [Implementing your view functions](https://nubela.co/blog/get-a-Professional Social Network-profiles-contact-information/#viewspy)
- [Setting up URL patterns](https://nubela.co/blog/get-a-Professional Social Network-profiles-contact-information/#urlspy)
- [Creating HTML templates](https://nubela.co/blog/get-a-Professional Social Network-profiles-contact-information/#styling-your-django-application)
- [Running the application](https://nubela.co/blog/get-a-Professional Social Network-profiles-contact-information/#running-your-new-application)
Let's dive in.
Setting up your development environment
First things first, you'll need a Python IDE
IDE stands for integrated development environment, which is a software suite that includes a code editor, compiler or interpreter, debugger, and build automation tools, streamlining the entire development process.
For Python, my favorite IDE is PyCharm, which advertises itself as, "the Python IDE for data science and web development," and has a community edition available for free here.
If you don't have Python installed already, you can install it during the process of installing PyCharm as well.
Create a new project
Once PyCharm is installed, open it and select "File" > "New Project". Choose a location for your project and give it a name (such as contact_demo_app).
In the new project window, ensure that the "Virtualenv" option is selected under the interpreter section. PyCharm will automatically use the bundled Python interpreter:
Building your contact enrichment application
Installing Django and needed dependencies
Next, open the terminal in PyCharm and run the following command to install Django and the requests library:
pip3 install django requests
Creating a new Django project
After installing the required packages, navigate to the project directory you created earlier in the PyCharm terminal and run the following command to create a new Django app, replacing contact_app with the name of whatever you'd like to name your application:
django-admin startproject contact_app
If successful, you'll see a new folder appear named after your project name, including another folder with your project name and a file named manage.py within. This is your Django project.
In the same folder as manage.py within a terminal run the following command:
python3 manage.py startapp contact_app
This will generate all of the necessary files and directories for your application.
Setting up ngrok for webhooks
As mentioned above, webhooks will play a crucial role in our contact enrichment application, as our Work Email Lookup Endpoint only returns results via webhook.
So, for development purposes, we'll use ngrok HostedHooks for easy local webhooks that can communicate with both our API and Django locally (keyword here is locally, which is why we can't use Webhook.site).
You can create your ngrok account for free here, and then download the agent here.
After that, navigate to the directory where you've extracted the ngrok executable and run the following command to authenticate your ngrok account:
ngrok authtoken <your-auth-token>
Of course replacing the authentication token with your own from within the ngrok dashboard.
Then run the following command in the same directory as your ngrok executable to create a secure tunnel to your local Django server:
ngrok http 8000
If successful, ngrok will display a public URL (such as https://your-url.ngrok-free.app) that you can use to access your local server from the internet:
Copy that URL. You'll need to keep ngrok running while using the contact enrichment application to receive the results back for work emails.
By the way, ngrok is for development uses only in this case. You'd need a different webhook process for production environments like shared team use or a SaaS.
Settings.py
Next you'll need to open the settings.py file in your project directory.
By default, SQLite should already be set to be used as your database for Django. But you'll need to add your application name under INSTALLED_APPS as well as including your ngrok URL in your ALLOWED_HOSTS. This will be necessary everytime your ngrok address changes (when you restart it).
Here's an example of my settings.py file, with my contact application being named contact_app:
from pathlib import Path
# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent
# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/5.0/howto/deployment/checklist/
# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = 'django-key'
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True
ALLOWED_HOSTS = ['your-url.ngrok-free.app', 'localhost', '127.0.0.1']
# Application definition
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'contact_app',
]
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
ROOT_URLCONF = 'contact_demo_app.urls'
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
WSGI_APPLICATION = 'contact_demo_app.wsgi.application'
# Database
# https://docs.djangoproject.com/en/5.0/ref/settings/#databases
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': BASE_DIR / 'db.sqlite3',
}
}
# Password validation
# https://docs.djangoproject.com/en/5.0/ref/settings/#auth-password-validators
AUTH_PASSWORD_VALIDATORS = [
{
'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
},
]
# Internationalization
# https://docs.djangoproject.com/en/5.0/topics/i18n/
LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_TZ = True
# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/5.0/howto/static-files/
STATIC_URL = 'static/'
# Default primary key field type
# https://docs.djangoproject.com/en/5.0/ref/settings/#default-auto-field
DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'
Models.py
Next up is the models.py file, where you define the data models for your application.
Models are Python classes that represent database tables and define the structure and relationships of your application's data.
In the contact enrichment application, we'll define a ProfileResult model to store the enriched contact information retrieved from Proxycurl's API.
Go ahead and copy and paste the following into models.py:
from django.db import models
class ProfileResult(models.Model):
Professional Social Network_url = models.URLField(unique=True)
work_emails = models.TextField(blank=True, null=True)
personal_emails = models.JSONField(blank=True, null=True, default=None) # Updated
contact_info = models.JSONField(blank=True, null=True, default=None) # Updated
status = models.CharField(max_length=36, default='pending')
batch_id = models.CharField(max_length=36, blank=True, null=True) # Add this line
def __str__(self):
return self.Professional Social Network_url
Then go back to your PyCharm terminal in the same directory and run the following:
python3 manage.py makemigrations
python3 manage.py migrate
That will reflect the changes for your SQLite database. The ProfileResult model will include fields such as:
Professional Social Network_url: A URLField to store the Professional Social Network profile URL.work_emails: A TextField to store the work email addresses associated with the profile.personal_emails: A JSONField to store the personal email addresses as a JSON object.contact_info: A JSONField to store additional contact information (e.g., phone numbers, social media profiles) as a JSON object.status: A CharField to indicate the status of the enrichment process (e.g., "pending", "completed", "failed").batch_id: A CharField to store the batch ID if the profile was processed as part of a bulk import.
By defining the ProfileResult model, you're creating a way for the enriched contact information to be stored.
Django uses this model to generate the necessary database schema and provides an ORM (object-relational mapping) layer to interact with the database using Python code.
Urls.py
The urls.py file is where you define the URL patterns for your application. URL patterns map specific URLs or URL patterns to corresponding view functions.
from django.urls import path
from contact_app import views
urlpatterns = [
path('', views.home, name='home'),
path('start-lookup/', views.start_lookup, name='start_lookup'),
path('webhook/', views.webhook, name='webhook'),
path('check-webhook-status/<int:task_id>/', views.check_webhook_status, name='check_webhook_status'),
path('check_batch_status/<str:batch_id>/', views.check_batch_status, name='check_batch_status'),
path('upload-csv/', views.upload_csv, name='upload_csv'),
path('export-csv/', views.export_csv, name='export_csv'),
]
You'll need to make sure you update the application name to yours again.
In this urls.py file, we define:
/: The root URL pattern that maps to the main dashboard or homepage view./enrich/: A URL pattern for initiating the enrichment process for a single Professional Social Network profile URL./import/: A URL pattern for handling bulk imports of Professional Social Network profile URLs from a CSV file./webhook/: A URL pattern for receiving webhook notifications from Proxycurl's API./export/: A URL pattern for exporting the enriched contact data as a CSV file.
By defining these URL patterns, you're specifying how the application should route incoming requests to the appropriate views based on the requested URLs.
Views.py
Now time for the most important part of the application, views.py, which controls what our Python application does.
In this file, you'll need to replace the ngrok URL with your own, as well as the Proxycurl API key.
Additionally, you'll need to change the return render line to the correct folder name per your project name from contact_app.
If you didn't already create your Proxycurl account above, you can create one for free here to get your key and you'll start with 10 credits.
Here's my views.py file:
from django.shortcuts import render, redirect, get_object_or_404
from django.http import JsonResponse, HttpResponse
from django.views.decorators.http import require_http_methods
from django.views.decorators.csrf import csrf_exempt
from django.core.paginator import Paginator
import json
import requests
import logging
import csv
import uuid
import io
from .models import ProfileResult
import threading
logger = logging.getLogger(__name__)
API_KEY = "Proxycurl_API_Key_Here"
HEADERS = {'Authorization': f"Bearer {API_KEY}"}
WEBHOOK_URL = 'https://your-url.ngrok-free.app/webhook/'
def get_personal_emails(Professional Social Network_profile_url):
api_endpoint = 'https://nubela.co/proxycurl/api/contact-api/personal-email'
params = {'Professional Social Network_profile_url': Professional Social Network_profile_url, 'email_validation': 'include'}
response = requests.get(api_endpoint, headers=HEADERS, params=params)
return response.json() if response.status_code == 200 else None
def get_personal_numbers(Professional Social Network_profile_url):
api_endpoint = 'https://nubela.co/proxycurl/api/contact-api/personal-contact'
params = {'Professional Social Network_profile_url': Professional Social Network_profile_url}
response = requests.get(api_endpoint, headers=HEADERS, params=params)
return response.json() if response.status_code == 200 else None
def lookup_work_email(Professional Social Network_profile_url):
api_endpoint = 'https://nubela.co/proxycurl/api/Professional Social Network/profile/email'
params = {'Professional Social Network_profile_url': Professional Social Network_profile_url, 'callback_url': WEBHOOK_URL}
requests.get(api_endpoint, headers=HEADERS, params=params)
@csrf_exempt
def webhook(request):
if request.method == 'POST':
try:
data = json.loads(request.body)
Professional Social Network_profile_url = data.get('profile_url')
work_email = data.get('email')
status = 'success' if work_email else 'email_not_found'
logger.info(f"Received webhook data: {data}")
logger.info(f"Setting status to: {status}")
profile, _ = ProfileResult.objects.update_or_create(
Professional Social Network_url=Professional Social Network_profile_url,
defaults={
'work_emails': work_email if work_email else '',
'status': status
}
)
logger.info(f"Returning status: {status}")
return JsonResponse({'status': status, 'task_id': profile.id, 'message': 'Email updated/created'})
except Exception as e:
logger.error(f"Error processing webhook: {e}")
return JsonResponse({'status': 'error', 'message': 'Error processing webhook'}, status=500)
else:
logger.error("Invalid request method")
return JsonResponse({'status': 'error', 'message': 'Invalid request method'}, status=405)
@require_http_methods(["POST"])
def start_lookup(request):
Professional Social Network_url = json.loads(request.body).get('Professional Social Network_profile_url')
Professional Social Network_url = Professional Social Network_url.rstrip('/')
try:
profile = ProfileResult.objects.get(Professional Social Network_url=Professional Social Network_url)
except ProfileResult.DoesNotExist:
profile = ProfileResult.objects.create(Professional Social Network_url=Professional Social Network_url)
emails = get_personal_emails(Professional Social Network_url)
numbers = get_personal_numbers(Professional Social Network_url)
profile.personal_emails = emails if emails else {}
profile.contact_info = numbers if numbers else {}
profile.save()
lookup_work_email(Professional Social Network_url)
return JsonResponse({'status': 'success', 'task_id': profile.id, 'message': 'Lookup initiated.'})
def check_webhook_status(request, task_id):
profile = get_object_or_404(ProfileResult, id=task_id)
status_value = profile.status
print(f"Status value for task_id {task_id}: {status_value}")
return JsonResponse({'status': status_value})
def check_batch_status(request, batch_id):
incomplete_profiles = ProfileResult.objects.filter(batch_id=batch_id, work_emails__isnull=True)
if incomplete_profiles.exists():
return JsonResponse({'status': 'pending'})
else:
return JsonResponse({'status': 'complete'})
def home(request):
search_query = request.GET.get('search', '')
if search_query:
all_profiles = ProfileResult.objects.filter(
Professional Social Network_url__icontains=search_query
).order_by('-id')
else:
all_profiles = ProfileResult.objects.all().order_by('-id')
paginator = Paginator(all_profiles, 10)
page_number = request.GET.get('page')
page_obj = paginator.get_page(page_number)
if request.method == 'POST':
action = request.POST.get('action')
if action == 'enrich':
Professional Social Network_url = request.POST.get('Professional Social Network_profile_url')
if Professional Social Network_url:
Professional Social Network_url = Professional Social Network_url.rstrip('/')
try:
profile = ProfileResult.objects.get(Professional Social Network_url=Professional Social Network_url)
except ProfileResult.DoesNotExist:
profile = ProfileResult.objects.create(Professional Social Network_url=Professional Social Network_url)
emails = get_personal_emails(Professional Social Network_url)
numbers = get_personal_numbers(Professional Social Network_url)
profile.personal_emails = emails if emails else {}
profile.contact_info = numbers if numbers else {}
profile.save()
# Start the enrichment process in a separate thread
enrichment_thread = threading.Thread(target=start_enrichment, args=(Professional Social Network_url,))
enrichment_thread.start()
return JsonResponse({'status': 'success', 'task_id': profile.id, 'message': 'Lookup initiated.'})
elif action == 'search':
search_query = request.POST.get('Professional Social Network_profile_url')
return redirect(f'/?search={search_query}')
return render(request, 'contact_app/home.html', {'page_obj': page_obj, 'search_query': search_query})
def start_enrichment(Professional Social Network_url):
lookup_work_email(Professional Social Network_url)
def upload_csv(request):
if request.method == 'POST' and request.FILES.get('csv_file'):
batch_id = str(uuid.uuid4()) # Generate a unique batch ID
csv_file = request.FILES['csv_file']
decoded_file = csv_file.read().decode('utf-8')
io_string = io.StringIO(decoded_file)
reader = csv.reader(io_string)
next(reader, None) # Skip the header row
profiles_to_process = []
for row in reader:
Professional Social Network_url = row[0].strip()
if Professional Social Network_url:
profile, created = ProfileResult.objects.get_or_create(
Professional Social Network_url=Professional Social Network_url,
defaults={'batch_id': batch_id}
)
if created or not profile.work_emails:
profiles_to_process.append(profile)
for profile in profiles_to_process:
lookup_work_email(profile.Professional Social Network_url)
return JsonResponse({'status': 'success', 'batch_id': batch_id, 'profiles_to_process': len(profiles_to_process)})
else:
return JsonResponse({'status': 'error', 'message': 'Invalid request'}, status=400)
def export_csv(request):
response = HttpResponse(content_type='text/csv')
response['Content-Disposition'] = 'attachment; filename="profile_results.csv"'
writer = csv.writer(response)
writer.writerow(['Professional Social Network URL', 'Work Email', 'Personal Email', 'Invalid Emails', 'Contact Info'])
profiles = ProfileResult.objects.all().order_by('-id')
for profile in profiles:
# Process personal emails
if isinstance(profile.personal_emails, dict):
personal_emails_list = profile.personal_emails.get('emails', [])
personal_emails = ', '.join(personal_emails_list) if personal_emails_list else 'N/A'
invalid_emails_list = profile.personal_emails.get('invalid_emails', [])
invalid_emails = ', '.join(invalid_emails_list) if invalid_emails_list else 'N/A'
elif isinstance(profile.personal_emails, list):
personal_emails = ', '.join(profile.personal_emails) if profile.personal_emails else 'N/A'
invalid_emails = 'N/A'
else:
personal_emails = 'N/A'
invalid_emails = 'N/A'
# Process contact info
if isinstance(profile.contact_info, dict):
contact_numbers_list = profile.contact_info.get('numbers', [])
contact_numbers = ', '.join(contact_numbers_list) if contact_numbers_list else 'N/A'
elif isinstance(profile.contact_info, list):
contact_numbers = ', '.join(profile.contact_info) if profile.contact_info else 'N/A'
else:
contact_numbers = 'N/A'
# Process work emails
work_email = profile.work_emails if profile.work_emails else 'N/A'
# Write the data into the CSV
writer.writerow([
profile.Professional Social Network_url or 'N/A',
work_email,
personal_emails,
invalid_emails,
contact_numbers,
])
return response
In a production environment, you would want to store your API key with an environment variable.
This views.py file includes the majority of our application logic, such as:
Lookup initiation
start_lookup processes a POST request to start the enrichment process for a given Professional Social Network profile URL, fetching personal emails, contact numbers, and initiating a work email lookup.
The results are stored in a ProfileResult object.
Email and contact number fetching
get_personal_emails and get_personal_numbers functions make GET requests to Proxycurl's API endpoints to fetch personal emails and contact numbers, respectively, using a Professional Social Network profile URL.
The responses are returned as JSON.
Work email lookup
lookup_work_email initiates a work email lookup by making a GET request to our Work Email Lookup Endpoint.
This function includes a callback_url parameter, to send the results to our webhook URL.
Webhook handling
The webhook view is designed to handle POST requests, which sends back the work email lookup results.
The function processes the received data, updates or creates a ProfileResult object with the result, and logs the operation.
Status checks
check_webhook_status and check_batch_status provide endpoints for checking the status of individual and batch enrichment tasks, respectively.
Web interface and batch processing
The home view renders a simple web interface for interacting with the system, including initiating enrichments and searching through enriched profiles.
upload_csv allows for batch uploading of Professional Social Network URLs for enrichment via CSV file.
export_csv provides functionality to export enriched profile data as a CSV file.
Styling your Django application
Django uses a templating language that allows you to embed Python-like code and variables within HTML, allowing dynamic rendering of the content.
In addition to the templating language, Django templates can also include JavaScript code and perform client-side operations.
In the contact enrichment application, there are two main templates:
-
base.html: This is the base template that defines the common structure and layout of the application's pages. It includes the necessary HTML boilerplate, such as the<!DOCTYPE html>declaration,<html>,<head>, and<body>tags. Thebase.htmltemplate also contains blocks({% block %})that can be overridden by child templates to provide page-specific content. -
home.html: This is the main template for the application's homepage or dashboard. It extends thebase.htmltemplate and provides the specific content for the homepage. Thehome.htmltemplate includes the necessary JavaScript, HTML, and Django template tags to display the enriched contact information, search functionality, pagination, and forms for initiating enrichment and importing/exporting data.
The home.html template is responsible for rendering the main user interface of the contact enrichment application. It displays the list of enriched profiles, allows users to search and navigate through the results, and provides options to initiate enrichment for individual profiles or import profiles in bulk using a CSV file.
The view function associated with the homepage (views.home) renders the home.html template and passes the necessary data (such as page_obj, search_query) to populate the template with dynamic content.
Base.html
You'll need to first create a templates folder within the same folder as your settings.py file.
Then within that templates folder, create a file named base.html with the following inside of it:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Contact App</title>
<!-- Bootstrap CSS -->
<link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css">
</head>
<body>
{% block content %}{% endblock %}
<!-- Bootstrap JS, Popper.js, and jQuery -->
<script src="https://code.jquery.com/jquery-3.5.1.slim.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/umd/popper.min.js"></script>
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/js/bootstrap.min.js"></script>
</body>
</html>
As you can see, it's pretty simple and relies on Bootstrap for styling.
Then within your templates folder, create a new folder named after your application, such as contact_app.
Home.html
Next within templates/contact_app, we'll need to create our home file, which will control the majority of our application.
Go ahead and create a new file named home.html, and paste the following inside:
{% extends 'base.html' %}
{% block content %}
<div class="container mt-4">
<h1 class="text-center text-gray mb-4">Contact enrichment with Proxycurl</h1>
<div class="row">
<div class="col-md-12">
<table class="table table-striped table-bordered">
<thead class="thead-dark">
<tr>
<th scope="col">Professional Social Network URL</th>
<th scope="col">Work Email</th>
<th scope="col">Valid Personal Email</th>
<th scope="col">Invalid Personal Email</th>
<th scope="col">Contact Number</th>
</tr>
</thead>
<tbody id="profile-table-body">
{% for profile in page_obj %}
<tr>
<td><a href="{{ profile.Professional Social Network_url }}" target="_blank">{{ profile.Professional Social Network_url }}</a></td>
<td>{{ profile.work_emails|default:'None' }}</td>
<td>
{% if profile.personal_emails and profile.personal_emails.emails %}
{{ profile.personal_emails.emails|join:', ' }}
{% else %}
None
{% endif %}
</td>
<td>
{% if profile.personal_emails and profile.personal_emails.invalid_emails %}
{{ profile.personal_emails.invalid_emails|join:', ' }}
{% else %}
None
{% endif %}
</td>
<td>
{% if profile.contact_info and profile.contact_info.numbers %}
{{ profile.contact_info.numbers|join:', ' }}
{% else %}
None
{% endif %}
</td>
</tr>
{% endfor %}
</tbody>
</table>
{% if page_obj.has_other_pages %}
<nav aria-label="Page navigation example">
<ul class="pagination">
{% if page_obj.has_previous %}
<li class="page-item"><a class="page-link" href="?page=1{% if search_query %}&search={{ search_query }}{% endif %}">First</a></li>
<li class="page-item"><a class="page-link" href="?page={{ page_obj.previous_page_number }}{% if search_query %}&search={{ search_query }}{% endif %}">Previous</a></li>
{% endif %}
{% for num in page_obj.paginator.page_range %}
{% if page_obj.number == num %}
<li class="page-item active"><a class="page-link" href="?page={{ num }}{% if search_query %}&search={{ search_query }}{% endif %}">{{ num }}</a></li>
{% elif num > page_obj.number|add:'-3' and num < page_obj.number|add:'3' %}
<li class="page-item"><a class="page-link" href="?page={{ num }}{% if search_query %}&search={{ search_query }}{% endif %}">{{ num }}</a></li>
{% endif %}
{% endfor %}
{% if page_obj.has_next %}
<li class="page-item"><a class="page-link" href="?page={{ page_obj.next_page_number }}{% if search_query %}&search={{ search_query }}{% endif %}">Next</a></li>
<li class="page-item"><a class="page-link" href="?page={{ page_obj.paginator.num_pages }}{% if search_query %}&search={{ search_query }}{% endif %}">Last</a></li>
{% endif %}
</ul>
</nav>
{% endif %}
</div>
</div>
<div class="my-4">
<div id="enrichment-message" class="alert alert-secondary" style="display:none;">
<p>Enriching contact information... please wait.</p>
</div>
<form method="post" class="form-inline" id="lookupForm">
{% csrf_token %}
<input class="form-control mr-sm-2" name="Professional Social Network_profile_url" type="text" placeholder="Enter Professional Social Network Profile URL" value="{{ search_query }}" required>
<button class="btn btn-primary mr-2 my-2 my-sm-0" type="submit" name="action" value="enrich">Enrich</button>
<button class="btn btn-primary my-2 my-sm-0" type="submit" name="action" value="search">Search</button>
</form>
<div class="mt-4">
<form method="post" enctype="multipart/form-data" action="{% url 'upload_csv' %}" class="form-inline" id="csvForm">
{% csrf_token %}
<div class="input-group mr-2">
<div class="custom-file">
<input type="file" name="csv_file" id="csv_file" class="custom-file-input">
<label class="custom-file-label" for="csv_file">Choose CSV file</label>
</div>
</div>
<button type="submit" class="btn btn-primary">Enrich CSV</button>
</form>
</div>
<div class="mt-4">
<a href="{% url 'export_csv' %}" class="btn btn-primary">Download contacts as CSV</a>
</div>
</div>
</div>
<script>
let webhookPollInterval;
document.getElementById('lookupForm').addEventListener('submit', function(event) {
event.preventDefault();
const Professional Social NetworkURLInput = document.querySelector('[name="Professional Social Network_profile_url"]');
const Professional Social NetworkURL = Professional Social NetworkURLInput.value.trim();
if (Professional Social NetworkURL === '') {
alert('Please enter a Professional Social Network Profile URL.');
return;
}
const action = document.querySelector('button[type="submit"][name="action"]:focus').value;
if (action === 'enrich') {
showEnrichmentMessage();
fetch('/start-lookup/', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-CSRFToken': '{{ csrf_token }}',
},
body: JSON.stringify({ 'Professional Social Network_profile_url': Professional Social NetworkURL }),
})
.then(response => response.json())
.then(data => {
if (data.status === 'success') {
start_work_email_lookup(Professional Social NetworkURL, data.task_id);
pollWebhookStatus(data.task_id);
} else {
console.error('Error initiating lookup:', data.message);
}
})
.catch(error => {
console.error('Error initiating lookup:', error);
});
} else if (action === 'search') {
window.location.href = `/?search=${encodeURIComponent(Professional Social NetworkURL)}`;
}
});
document.getElementById('csvForm').addEventListener('submit', function(event) {
event.preventDefault();
const formData = new FormData(this);
showEnrichmentMessage();
fetch('{% url 'upload_csv' %}', {
method: 'POST',
body: formData,
headers: {
'X-CSRFToken': '{{ csrf_token }}',
},
})
.then(response => response.json())
.then(data => {
console.log('CSV upload response:', data);
if (data.status === 'success' && data.batch_id && data.check_batch_url) {
pollBatchStatus(data.batch_id, data.check_batch_url);
} else {
alert('Failed to start batch processing.');
}
})
.catch(error => {
console.error('Error uploading CSV:', error);
});
});
function showEnrichmentMessage() {
const enrichmentMessage = document.getElementById('enrichment-message');
enrichmentMessage.style.display = 'block';
}
function pollWebhookStatus(taskId) {
clearInterval(webhookPollInterval); // Clear any existing interval
webhookPollInterval = setInterval(() => {
fetch(`/check-webhook-status/${taskId}/`)
.then(response => response.json())
.then(data => {
if (data.status === 'success' || data.status === 'email_not_found') {
clearInterval(webhookPollInterval);
window.location.reload(); // Redirect on success or email_not_found
} else {
console.log(`Received status for ${taskId}:`, data);
}
})
.catch(error => {
console.error('Error checking webhook status:', error);
clearInterval(webhookPollInterval);
});
}, 2000);
}
function pollBatchStatus(batchId) {
const checkBatchUrl = `/check_batch_status/${batchId}/`;
console.log('Starting to poll for batch status, batch ID:', batchId);
const pollInterval = setInterval(() => {
fetch(checkBatchUrl)
.then(response => response.json())
.then(data => {
if (data.status === 'redirect') {
// Handle the redirect
console.log('Batch processing complete. Redirecting...');
clearInterval(pollInterval);
window.location.href = data.redirect_url;
} else if (data.status === 'pending') {
console.log('Batch processing pending...');
} else {
console.log('Unexpected status:', data.status);
}
})
.catch(error => {
console.error('Error during batch status polling:', error);
clearInterval(pollInterval);
});
}, 5000); // Adjust polling interval as needed
}
function start_work_email_lookup(Professional Social Network_url, task_id) {
const api_endpoint = 'https://nubela.co/proxycurl/api/Professional Social Network/profile/email';
const params = {
'Professional Social Network_profile_url': Professional Social Network_url,
'callback_url': '{{ WEBHOOK_URL }}',
'task_id': task_id
};
fetch(api_endpoint, {
headers: {
'Authorization': 'Bearer {{ API_KEY }}',
},
method: 'GET',
params: params,
})
.then(response => {
console.log('Lookup work email response:', response);
})
.catch(error => {
console.error('Error looking up work email:', error);
});
}
</script>
{% endblock %}
Running your new application
The last step is pretty simple.
All you'll need to do to run your contact enrichment application is type in the following within a terminal that's in the same folder as manage.py:
python3 manage.py runserver
That'll run the Django development web server, and your application will be available in any browser through 127.0.0.1:8000:
Understanding the contract enrichment application
Application and file structure
Your application structure should look like this, only differing when it comes to the application/directory names:
<project_name>/
│
├── __init__.py
├── asgi.py
├── settings.py
├── urls.py
├── wsgi.py
│
├── <app_name>/
│ │
│ ├── migrations/
│ │
│ ├── templates/
│ │ ├── base.html
│ │ └── <app_name>/
│ │ └── home.html
│ │
│ ├── __init__.py
│ ├── admin.py
│ ├── apps.py
│ ├── models.py
│ ├── tests.py
│ └── views.py
│
├── db.sqlite3
└── manage.py
The important things here being:
<app_name>/: This is the main Django application directory containing the core functionality of the contact enrichment application.<project_name>/: This is the project-level directory containing project-wide configuration files such assettings.pyandurls.py.templates/: This is the project-level templates directory where the base.html template is located (as well as the<app_name>folder withintemplates/andhome.htmlinside of that).db.sqlite3: This is the SQLite database file used by the application.manage.py: This is the command-line utility for managing the Django project.
How the application works
Th home.html file along with our views.py file handle the vast majority of our application.
Specifically:
1) Form submission
The JavaScript code on our home.html file adds an event listener to the form with the ID lookupForm.
When the form is submitted, it prevents the default form submission behavior and retrieves the entered Professional Social Network profile URL.
Based on the selected action (enrich or search), it either sends a POST request to initiate the enrichment process or redirects the user to the search results page.
2) API requests (pulling data)
The Python code in the views.py file makes requests to Proxycurl's API using the requests library.
The get_personal_emails and get_personal_numbers functions send GET requests to the Proxycurl API endpoints https://nubela.co/proxycurl/api/contact-api/personal-email and https://nubela.co/proxycurl/api/contact-api/personal-contact, respectively, to retrieve personal emails and contact information.
The lookup_work_email function sends a GET request to the Proxycurl API endpoint https://nubela.co/proxycurl/api/Professional Social Network/profile/email to initiate the work email enrichment process. This request includes the Professional Social Network profile URL and the callback URL for receiving the webhook response.
The server-side views handle the responses from Proxycurl's API and perform the necessary operations, such as storing the retrieved data in the ProfileResult model instances.
The JavaScript code on the home.html file interacts with the server-side endpoints using the fetch function to trigger the enrichment process and handle the responses accordingly.
3) Server-side processing
The views.py file contains the server-side logic for handling requests and interacting with Proxycurl's API.
The start_lookup view function receives the POST request from the client-side JavaScript and initiates the work email enrichment process by making a GET request to Proxycurl's API endpoint https://nubela.co/proxycurl/api/Professional Social Network/profile/email.
The webhook view function handles the incoming webhook from Proxycurl's API, updating the corresponding ProfileResult model instance with the retrieved work email.
4) Storing data
The models.py file defines the ProfileResult model, which represents the database table for storing enriched contact information.
The model defines fields such as Professional Social Network_url, work_emails, personal_emails, contact_info, status, and batch_id to store the relevant data.
The server-side views interact with the ProfileResult model to create, update, and retrieve records as needed during the enrichment process.
5) UI updates, data export, batch processing, and webhook handling (backend logic)
The JavaScript code on the home.html file dynamically updates the user interface based on the responses received from the server-side and Proxycurl's API.
It shows enrichment messages, polls for webhook status updates, and updates the displayed data without requiring a full page reload.
The JavaScript code also handles functionalities like batch processing of CSV files and exporting enriched contact data as a CSV file.
Batch processing
For batch processing, the JavaScript code sends a POST request to the /upload-csv/ endpoint on the server-side to upload a CSV file containing a list of Professional Social Network profile URLs.
It then polls the /check_batch_status/ endpoint to monitor the batch processing status and redirects the user to the appropriate page once the batch is completed.
Webhook handling
For webhook handling, the JavaScript code sets up polling intervals to check the status of webhook events by sending requests to the /check-webhook-status/ endpoint.
It then updates the UI accordingly based on the webhook status (success, email_not_found, etc.).
In this example, I've used ngrok for local webhooks.
Server-side functions
The server-side views.py file contains functions for handling CSV file uploads, batch processing, exporting enriched data as a CSV file, and managing webhooks.
The upload_csv view function handles the CSV file upload, processes the Professional Social Network profile URLs, initiates the enrichment process for each profile, and generates a unique batch ID for tracking.
The check_batch_status view function checks the status of the batch processing and returns the appropriate response (pending or complete).
The webhook view function handles the incoming webhooks from Proxycurl's API, updating the corresponding ProfileResult records with the retrieved work email.
A summarization
Overall, the application combines client-side JavaScript, server-side Python code (Django views), interaction with Proxycurl's API, data persistence using Django's ORM, batch processing, and webhook handling to provide a complete contact enrichment solution.
In terms of pricing per profile enriched, the application costs 3 credits per profile enrichment base rate (because of the work emails), 1 credit per phone number successfully returned, and 2 credits per personal email successfully returned (because the script uses email verification, if you wanted you could remove this and save 1 credit per result).
Amounting to a grand total of 6 credits per result if every variable was successfully returned, and 3 credits base rate per use (you can learn more about what a credit costs here, it varies).
Not bad, huh?
It's time to put this into practice
If you'd like, you can take our contact enrichment app right now and use it commercially. It's open source, and I don't care what you do with it.
Or, taking the knowledge you've learned here about our Contact API, specifically our:
You can build your own custom application, tailored specifically to your needs and workflow. You now have access to all of the B2B data and contact information you could ever need.
What are you waiting for?
Create your Proxycurl account
You can click here to create your Proxycurl account for free. You'll start with 10 credits.
After that, you can opt for a pay-as-you-go or subscription plan. Our pricing information is available here.
Thanks for reading!
P.S. Have any questions about how we can help with contact enrichment? Feel free to reach out to us at "[email protected]" and we'll get back to you ASAP.