The 101 On The Underrated Topic Of Master Data

Data Scientists often do not consider master data as an important topic. Consideration of master data is often crucial when working with different data sources in the Big Data area. Hence, this article describes what Master Data is, how it’s so important and in which areas it matters most.

Assorted files on shelves.
What is master data and why does it matter? This article will shed light on this underrated topic.
Image Source: Viktor Talashuk

What is master data?

Organisations usually deal with a few main types of data, including unstructured data, transactional data, metadata, hierarchical data, reference data and, of course, master data.

Yes, master data is just one type of data inherent in organisations.

A commonly quoted definition of master data is Gartner’s definition.

“Master data is the consistent and uniform set of identifiers and extended attributes that describes the core entities of the enterprise including customers, prospects, citizens, suppliers, sites, hierarchies and chart of accounts.”

Gartner

According to Profisee, master data consists of four domains (which may vary across different organisations):

  • Customers: Within the customer’s domain, there are customer, employee and salesperson sub-domains.
  • Products: Within products domain, there are product, part, store and asset sub-domains.
  • Locations: Within the locations domain, there are office location and geographic division sub-domains.
  • Other: Within the other domain, there are things like contract, warranty and license sub-domains.

Whatever the domains of master data are, three characteristics about master data remain the same for every organisation, according to Semarchy:

  • Critical for business decision-making.
  • Scattered throughout the organisation.
  • Represents a “single source of truth”, in other words, the “core data”.

With that said, maybe Dataversity’s definition of master data is the clearer one.

“Master data can be described as an organization’s core data, containing the basic information needed to conduct business.

It is fairly stable information, changing only when something dramatic happens, such as a client moving to a new location.”

Dataversity

How does it differ from time-series data?

To explain how master data is considered as stable, here’s a comparison with time-series data. Time-series data are actions, events and/or indicators which occur at different times.

The primary factor of time-series data is the focus on the time dimension.

This means that the point in time of an event is considered as a new data record, which can be compared to past records as part of a time-series analysis.

Time series analytics airplane timelapse
Time Series Analysis is measuring and analysing how features and indicators change and relate over time. Imagine how sensors in an airplane measure pressure, distance, kerosene and so on throughout a flight.

Another crucial factor of time-series data is that its underlying concepts drift over time.

On the other hand, master data doesn’t experience such changes unless, as mentioned earlier, something dramatic happens.

If the master data changes, everything changes and this phenomenon of master data change is referred to as “slowly changing dimensions”.

The changing dimensions of master data has to be credited in the analysis for it to make sense.

In the case of a commuter bus, there are temporal concept drifts on weekdays and weekends.

On a Monday, which marks the start of a work week, many passengers use a particular bus route, causing a delay in the departure from each stop.

On a Saturday, less people use the bus, resulting in less delays.

Since the bus route is always the same and the delays differ from time to time, the bus route is the master data while the daily delay is the time-series data.

On rare occasions, the transport department decides to adjust, expand or shrink the bus route to better accommodate the commuters’ demand in that town.

Data scientists will have to take note of the adjustment in the bus route since it will shift the passenger numbers and delays.

Taking the bus at night in Seoul
A change in the bus route is an example of a master data’s changing dimensions because adjusting the bus route is a rare public transport policy unless there’s enough proof for the need to make the change.
Image Source: Niklas Herrmann

In another example, you use a Facebook account that has all the personal info associated with you, like your relationship status, photos, links, opinions, participation in groups, the pages you follow, etc.

The Facebook account acts as your master data because it’s the core of all the online data associated with you.

Meanwhile, your Facebook activities, which happen at different points in time, are considered your time-series data.

The same can be said in the case of Twitter.

You have the Twitter account acting as master data while the Tweets are time-series data.

Your social media activities (time-series data) may differ with time but your social media accounts (master data) remain as they are, unless you deactivate them.

What is master data management (MDM)?

In their 2007 publication Master Data Management and Customer Data Integration for a Global Enterprise, Alex Berson and Larry Dubov (quoted by Mike2.0) define MDM in the following way:

Master Data Management (MDM) is a framework of processes and technologies

aimed at creating and maintaining an authoritative, reliable, and sustainable, accurate, and secure data environment

that represents a ‘single version of truth,’

an accepted system of record used both intra- and interenterprise across a diverse set of application systems, lines of business, and user communities.”

Alex Berson and Larry Dubov’s definition of MDM

This basically means that MDM is the idea of using tools and/or setting procedures to maintain the qualities of the organisation’s core data for everyone’s benefit.

Referring to Dataversity’s explanation, MDM aims to achieve the following qualities:

  • All of its essential data in one master file for the entire organisation’s common reference.
  • Elimination of bad data governance.
  • Uniformity and accuracy of data.
  • Streamlined data sharing between different users and departments.

So, the basic idea of MDM is consolidating and maintaining this core data to make it easier for the relevant parties of an organisation to access it.

One box for all clipboards
Maintaining the qualities of master data is like arranging all the important clipboards in one box. The master box is accessed by everyone in the organisation who needs to access the clipboards.
Image Source: David Fintz

What’s the big deal with MDM?

A business is nothing without its domains of customers, products, employees and legal documents.

We mentioned earlier that master data consists of these domains, hence, it plays a big role in keeping the business alive if managed properly.

And because master data is used by various parts of an organisation for different applications, an error in one part of the master file can cause errors in all the applications that use that part.

Here’s a classic example of how an error in the master data can be detrimental to an organisation’s operations:

An e-commerce business sold a product to a customer. However, the business later harassed the customer with ads of the same product that the customer no longer needed.

Now, you might be thinking how the business could allow this to happen.

The answer is simple: Poor MDM.

In this scenario, the sales team and the marketing team did not integrate the customer’s information.

The marketing team was unaware of the sale, so they ended up wasting everyone’s time.

Therefore, the solution to such an annoyance is for the sales and marketing teams to coordinate by using the same master data.

Brainstorming over paper. Reflects how teams need to coordinate.
The sales and marketing teams’ lack of coordination is like a horrible marriage whereby the spouses are not communicating with each other about household needs (like who’s in charge of tonight’s dinner). That’s what a lack of coordination in an organisation looks like.
Image Source: Scott Graham

Okay, so how can master data be created and maintained?

This is why the power of master data and its management cannot be underestimated.

Creating master data involves cleaning and standardising data, and matching data from all the sources to consolidate duplicates.

Examples of data cleaning work are turning all the phone numbers, addresses and other details into common formats, replacing missing values, and converting all measurement units to the industry standard.

Meanwhile, consolidating duplicates requires accurate matches and needs to be done with extra care as false matches can lose data and missed matches defeats the purpose of the process.

It should be noted that there are several MDM approaches and techniques including:

  • Migration of historical data
  • Application co-existence
  • Single copy
  • Multiple copies with single maintenance
  • Continuous merge

Whatever approach the organisation decides to use, the organisation should be following these dimensions and disciplines of MDM strategy:

  • Governance and policy to set MDM rules and standards.
  • Measurement for progress tracking.
  • Organisation to assign the right team of master data owners, data stewards and governing parties in the strategy.
  • Process to determine the steps to manage MDM.
  • Quality assurance to uphold data quality.
  • Technology, of course, provides the tools needed to make the MDM strategy possible.

The best MDM approach for a particular organisation holds the secret sauce to an organisation’s success.

Therefore, as long as the disciplines above are applied, the master data is in good hands.

No Responses

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    Your free special webinar guest invitation: How to avoid the worst big data failures