Some define Big Data value by use cases. Others describe its advancements by the attributes of the Big Data. Ultimately, business capabilities and opportunities result from these Big Data Landscape maturity levels of an enterprise. In this article, describe here different maturity levels of Big Data and the assciated Big Data Science capabilities of enterprises.
Enterprise Big Data Maturity
Big Data by different maturity levels do not have a uniform standard.
Commonly, also the application of Big Data plays here a role and the focus is less on the data and more on the applications.
In the following, we simplify the Big Data landscape maturity of an enterprise to the basic foundations. Thereby, we follow our in practice learned patterns how Big Data is often introduced in enterprises.
Level 0: Big Data Landscape planning
Data production in various infrastucture systems is happening naturally and maybe the data is even stored in a non special defined way. The data of the various systems is only consolidated partially in a Data Warehouse.
First thoughts are existing which additional data can be stored. This might be data such as Time Series Data in transactional systems which did not get logged before.
Maybe IoT gateways to store sensor data are defined or machines are getting equipped with sensors to transmit data in the future.
Use cases are drafted which Big Data Scenarios can bring value in digization. First experiments with Big Data tools and exempary data in a small amount might happen.
A professional and productive Big Data Landscape and infrastucture is absent and no real Big Data analysis with enterprise value happens.
Level 1: Simple Big Data Analysis infrastucture
Often a value propotye, demonstrating first Big Data and Data Science capabilities and providing Big Data value at the same time is manufactured.
Such a Big Data value prototype is often realized in in form of a mixture between first case exploration and testing with the aspiration to gain first value out of Data Science.
The Big Data infrastructure gets used for first applications and analytics results are generated manually.
Different use cases get defined give insights that more investment into the Big Data Landscape can generate a potential return.
Typical for this Big Data Landscape maturity is that Data integration happens in most cases manually or without proper monitoring frameworks, generating reliability challenges on the Big Data platform.
Level 2: Big Data Analysis Insights
Big Data Analysis projects are delivering first value.
The maturity of Big Data Landscape leads to generate enterprise advancements by supporting with light data driven decisions.
Often the Big Data infrastructure offers some ad-hoc dashboards and more continous data integrations allow to gain structured overviews for semi-IT-personal when data updates happen.
Typical for this Big Data maturity level is that first manual applications of machine learning indicate clear AI value. For example, machine learning gets used to do half-automated classifications what shows new levels of outomation that save plenty of manual human labor.
Other first applications allow to gain Big Data analysis insights or to visualize data which was not possible before and gives new insights and overviews.
Out of the results first use cases to productize the first trial scenarios of this maturity level get identified.
Different challenges like data quality or the necessity for machine learning model management are defined as todo.
Bottleneck to gain more value out of the Big Data infrastucture is the effort to import data from legacy systems and the lack of use cases that can generate distinctive value at this point of time.
Level 3: Enterprise Big Data Service
Sole Big Data use cases are operationalized and deliver value added for internal or external customers.
The operationalized cases are in the area of machine learning, visualization and complex event processing.
Typical for such a service are recommendations, near-real-time reactions of the software, alerting, real-time Big Data dashboarding or automated labeling of document contents by AI.
The Big Data Services are delpoyed for mostly uncrucial areas to identify risks and establish business and technical processes for updates of AI model updates.
The use cases in operation generate new challenges that were not expected in interactions with end users. Processes within the enterprise need to be established how to handle these situations
For instance, users might ask chatbots things about suicide and an enterprise needs to decide how they react to such unforseen situations even if the chatbot was defined and operationalized for other cases.
Level 4 – Mastery of Big Data, reactive intelligence, continous progression
In the last step, optimisations of operationalized use cases are executed and more advanced learning techniques get applied.
Thereby, challanges such as model update pipelines and advanced training fine tunings are in focus.
The more actual and the more real time the data is the more also continous monitoring, investigations and advacements are important.
For example, natural phenomena like virus outbreaks, weekdays or seasons can lead to . Such changes in patterns needs therefore continous monitoring and improvement in the applied tools and techniques.
What are the different levels of Big Data Landscape maturity?
– Big Data Landscape planning
– Simple Big Data Analysis infrastucture
– Big Data Analysis Insights
– Enterprise Big Data Service
– Mastery of Big Data, reactive intelligence, continous progression
What are typical features of first Big Data analysis?
– Data integration happens manually
– The use case is a mixture between Big Data and Data science
– The first use case is supposed to be a prototype to assess value
– A proper and well maintained Big Data infrastucture is not installed, yet.
What are signs of Big Data Analysis Insights?
– First operative desissions get supported by Big Data Analysis results
– Ad hoc dashboards show insights and value
– Tests of AI show cost saving benefits or potential business speed ups
What is typical for enterprise Big Data services?
– semi automated alerting and reaction
– real-time Big Data dashboarding
– automated labeling of document contents by AI.
What is happening when the mastery of Big Data Services is reached?
Continous monitoring of the Big Data and AI Services is still needed to maintain valid results. Also cutting edge research results need to be more and more applied to ensure the best results. Ultimately, continous improvements is needed to ensure the enterprise stays on top.