What is Data Integration ?
- Data Integration is concerned with combining data from various Sources into one consistent stream.
- It provides an essential Single View of Data, for example, a Single View of a Customer.
- It also provides a natural point at which Data Quality can be addressed.
- At this Stage, Data Quality can be assessed and a Single View of a Customer can be achieved.
- When Data Quality is of a uniform good quality, it can be integrated and made available as a consistent View.
- This will be supported using a Glossary, as described in the Information Catalog Stage.
- The current incarnation of Data Integration is Master Data Management,(MDM).
- Data Integration combines related data from avrious source.
- This can be anything of importance to the organisation, such as Traders, Products and Movements.
- It provides a natural point at which data quality can be addressed.
- When Data is of uniform good quality it can be integrated and made available as a consistent View.
- This leads naturally to Master Data Management,(MDM).
- Details of the Integration, such as mapping specifications, are held in a Glossary, which is described in Stage 6.
Why is this Stage important ?
- It provides one view of the truth
- It offers a point at which Data Integrity can be measured and User involvement obtained to improve Quality until it meets User standards.
How do we get started ?
- Data Profiling is a good starting-point for determining the quality of the data in a Database.
- This involves drafting some validation and transformation that can be used to get started.
For example, replace LTD by LIMITED (or vice versa), and ‘&’ by AND.
- The Design Approach requires Data Models for the areas of the within Scope.
- It will also require Generic Data Models to support one view of the truth for major entities, such as Traders or Customers.
- This one view will be implemented as Master Data Management (MDM).
- Get a broad understanding of the data available
- Establish a common view of the Data Platform
- Get a broad understanding of Data Sources
- Determine the available Data
- Choose the MDM product
- Determine strategy for Clouds, for example to have Reference Data available globally
1. In 1 month, produce Generic Data Models
2. In 3 months, confirm GDM with sample data and Facilitated Workshops and choose MDM product.
3. In 6 months, implement MDM and publish GDM and CMI on the Intranet.
4. Adjust timescales in light of experience
- Data Integration covers a number of Steps, each of which can have its own Templates.
- Examples are included here for Data Profiling and Mapping Specifications.
Here is a Kick-Start Tutorial of Best Practice
These Steps define a Tutorial of Best Practice :-
Step 1. Define the Target which is usually a ‘Single View Data Model’.
Step 2. Define the Data Sources
Step 3. Define the Mapping Specifications from the Sources to the Target.
Step 4. Define the Data Platform
Step 5. Identify Standards to be followed.
This Tutorial is described in detail in a separate document, entitled Data_Integration_Tutorial.doc,
which is available on demand