Most don't consider the use of a staging area when looking at data integration, but they are indeed a fit for many problem domain when considering more complex data integration needs.
A staging approaching to data integration supports more complex and valuable data integration operations, including support for many large data sets and data operations that are more complex and of higher value. Using a staging area, or a temporary location where the data from the source system or systems is replicated, provides a logical location to perform complex operations on the data that would be difficult if not impossible to do when using direct integration approaches.
An example would be extracting data from multiple enterprise data stores, manipulating the data as to structure and content, performing complex operations such as data replication, data aggregation, and data cleansing, and then posting the data to a target database or applications.
Leveraging a staging area for integration is characterized by:
• The ability to perform more complex operations on data, including complete transformation of semantics and the data content using any number of dimensions since, in essence, you operate on an intermediary database that you control completely.
• The ability to leverage more coarse grained and complex data sets that may not always repeat.
• Informational focused, supporting valuable information externalization approaches, including business intelligence.
• More flexibility around business cycles, data processing cycles, widely disbursed systems, and hardware and network limitations, where it may not be feasible to extract all operational databases at the same time.
• The ability to better support complex database functions, including replication, cleansing, and aggregation.
Perhaps it's a fit for you? Certainly, a consideration with data integration.
Leave a comment