The Difference Between Data Cleansing and Data Enrichment

Think of data cleansing like the work of a prep cook: separate the good from the bad; wash off the dirt; measure; cut to size; then put the pieces in labeled containers. Data enrichment, on the other hand, is the chef’s touch: combining different ingredients to create a dish that’s greater than the sum of its individual components.

What is data cleansing?

Broadly, data cleansing consists of inspecting and formatting the data to make it suitable for analysis. Problems in the data need to be rectified in order to make the data useful for data scientists – they can be simple or complex, quick & painless or time-consuming & tedious. Common data problems include: incorrect data types (such as text cells in a currency field, or integers in a name column); data that does not conform to requirements or patterns (such as dates and times, postal code formats, e-mails, phone numbers); inconsistencies within the data (such as conflicting addresses for the same business, duplicates of rows); and many more.

What is data enrichment?

After the data is cleansed, it can be enriched. Data enrichment is the process of enhancing data by appending additional information to an existing dataset. The result is a richer dataset that makes it easier to derive more and better insights. For example, imagine you have a dataset containing a business name, their phone number, and address. Now we can enrich that data by pulling records from the stock market indices to get the stock price and valuation for each publicly-traded business. Additionally, we can pull from an online Maps API to get the precise location or add additional geospatial information. The possibilities are almost endless, and it’s critical to fully understand the problem you’re trying to solve when sourcing and combining data.

The impact of data variety

Data coming from a multitude of sources, both inside and outside an organization, introduces data variety. Depending on who you are, data variety can sound like a blessing or a curse. While variety is vital to getting unbiased, corroborated data, it also means jumping over massive hurdles to achieve data parity before any data can be engineered efficiently, or used together in a meaningful way. Like anything, an ounce of prevention is worth a pound of cure – having a data variety solution from the start of a project is the easiest way to ensure consistent progress throughout.

Why do these things matter?

Why does data need to be cleansed? Why does it need to be enriched? And most importantly, why is data cleansing and prep so important to your organization?

The one-word answer is: “Money.” Insights-driven organizations are growing at a rate of 30% annually and are on track to earn $1.2 trillion by 2020 – looking at these numbers, it would be downright bad business to ignore the value in a data-driven strategy.

Data variety is an obstacle for everyone, no matter how large the team. Although, the majority of companies are worried about a talent shortage when it comes to data, the root of the problem is a strategy shortage. Finding and preparing data is the most time-consuming and least enjoyable part of a Data Scientist's job. The below graph highlights the reallocation of investments when teams leveraging data prep automation tools. 

As more and more organizations begin to adopt a data-driven approach to business, relying solely on internal data is no longer enough to be competitive. To stay ahead, organizations need to introduce external data sources into every piece of their data pipeline – and that means implementing adoption strategies and methodologies (like DataOps) to ensure high performance and consistent successful outcomes.

ThinkData offers a lot more data than that – over 250,000 thousand datasets from more than 75 countries around the world. Request a consultation with one of our data experts to talk about external data.