Entity resolution is the process of determining which data records refer to the same real-world entity. An “entity” in this context can refer to a number of things, but when talking about entity resolution, data practitioners are usually working with customer, product, patient, healthcare provider or location data. Simply put: people, places and things.

Entity resolution is often a necessary part of master data management, especially when dealing with large datasets or data from multiple different sources. Because it’s common for data about the same entities to get recorded in slightly different ways, become outdated or contain inaccuracies, trying to determine which records reference the same real-world entities can be a challenge — especially when you’re working on a scale of thousands or millions of records.

This is where entity resolution software can be a huge help.

Entity resolution (ER) software is a type of data management software that helps data professionals identify records that point to the same real-world object, or entity. It is usually sold as a standalone software application or as a feature within a larger data platform like a master data management (MDM) or unified data management solution.

How Does Entity Resolution Software Work?

Entity resolution software works by:

1. Integrating data from a number of different sources
2. Identifying potentially matching records among those datasets
3. Merging records that match
4. Removing, or deduplicating, duplicate records

From there, the cleaned and deduplicated data is usually loaded into a data warehouse or data lake, where it can then be used by other business systems like CRM, BI and analytics or ERP. This data is usually referred to as “consumable” or “business-ready” data.

How Does Corporate Entity Data Management Software Help Businesses?

Large organizations that work with vast quantities of data use ER tools to create a more comprehensive picture of the things they care about the most. This lets businesses better target their customers through advertising and product recommendations, operate more efficiently, reduce unnecessary spending and lower the risk of fraud, among other benefits.

How Do Entity Resolution Tools Handle Data?

Modern organizations generate data from an ever-growing number of sources, including:

• CRM systems that capture data like customer name, email address, shipping and billing addresses, payment information, products viewed and purchase history
• Product information management (PIM) systems that store product-related data like SKU numbers, product images, size and weight measurements, availability regions, price and inventory counts
• Internet of things (IoT) devices that track everything from personal health information like blood pressure, heart rate, step counts and blood oxygen levels to information about machinery on a factory floor like machine temperature, supply levels, vibration readings, humidity and more

Taken alone, each of these systems produce a lot of data, but when you consider that enterprise-sized organizations rarely only use one CRM or ERP system — not to mention any number of other data-producing systems — it’s easy to see how difficult it is to maintain data quality at scale.

Entity resolution software works with this data by integrating it from, say, all of an organization’s CRM systems. It then uses technology like fuzzy matching to spot records that look similar and appear to refer to the same entity. For example, one CRM might record a customer’s ZIP code as “30022” while another records it as “30022-8464,” or one system might format a customer’s name like “last name, first name,” while another does “first initial, last name.”

From there, the ER tool usually works with a data governance platform to standardize the data according to the organization’s data governance policies. Matching records are merged, eliminating duplicate records and preparing the data for loading into a target system, like a data warehouse or data lake.

Entity resolution is sometimes best understood by way of an example. Suppose you work for a large healthcare organization that wants to get a more holistic view of its patients. Among the several different hospitals and multiple clinics your organization operates, patient information is scattered across various systems like electronic health records (EHR), lab results, scheduling and billing systems.

Using a master data management solution, you conduct entity resolution to consolidate the data from every disconnected source and store it in one central location for patient data. The MDM system ensures that all records pertaining to a single patient are matched and merged, providing healthcare professionals with a comprehensive view of the patient’s medical history.

Now, instead of having to track down different types of data on one patient from several different sources, healthcare providers can access this single source of truth and see a patient’s full health history at once. This not only improves patient care but also streamlines operations by reducing redundancies and errors.

There are multiple benefits of entity resolution:

• Businesses can market their products and services to customers more effectively
• Eliminating redundant processes lowers costs and helps businesses be more nimble
• Customers enjoy a more seamless shopping experience and get better recommendations for products they might need or enjoy
• Healthcare providers can provide better, more holistic treatment to their patients
• Retailers can reduce the number of reshipments due to address errors and reduce customer service interactions

There are many other examples you can find across different organizations and industries, but these are some general benefits of using entity resolution.

Implementing entity resolution will look different across organizations depending on factors like company size, industry and the type of data you’re working with, but in general, you can follow these steps:

1. Develop a business case: You’re much more likely to get executive buy-in for an entity resolution project if you can tie it back to a specific business case. For instance, you might be able to show that the company could save money by consolidating supplier contracts and negotiating better rates. This will be very difficult without performing entity resolution on supplier data to identify where multiple contracts exist with the same suppliers.
2. Perform data discovery: Audit the various source systems your organization uses to identify which systems store the kind of data you’ll be performing entity resolution on. Make a list. Secure the appropriate level of access to those source systems so you can integrate data from them into your entity resolution software later.
3. Configure data governance policies: If you don’t already have a data governance program, now would be a good time to consider putting one in place. If your organization already has a data governance program, make sure policies are written and published in your data governance or data catalog tool that pertain to the specific type of data you’ll be working with for entity resolution.
4. Deploy entity resolution software: Choose and deploy an entity resolution software tool. Talk to peers in your field for recommendations, and check with consulting firms like Gartner for advice on what might work best for your specific organization.
5. Integrate, deduplicate and standardize: Using a purpose-built entity resolution tool or a master data management solution like Profisee, integrate the source systems you identified in step 2 and use the tool to identify matching records and remove duplicates. Look for a tool with survivorship capabilities to ensure you keep only the most accurate, up-to-date information about each entity, and integrate it with your data governance or data catalog tool to standardize records according to data governance policies.

Machine learning models can help speed up entity resolution processes, especially when they’re part of a more robust entity resolution tool. Machine learning models can enable automated matching, a powerful capability when dealing with large volumes of complex data that can save data practitioners hours on what would otherwise be highly manual work.

Tools like Profisee use machine learning to identify matching records with greater speed and accuracy, resulting in fewer instances of records requiring manual review by data stewards. Most datasets are bound to contain some ambiguous data, so it’s important to keep a human in the loop to make certain judgment calls that a software tool can’t.