In this section, we look at a high-level architecture of a data quality monitor and examine more closely the parts we need to put this monitor together.
High-Level Architecture
In most data processing environments, information is collected, prepared, processed and passed along in stages (see Figure 1). Our data quality monitor can be used to validate information at any point (which we can call an "intervention point") where information is passed from one processing stage to any other.

Click on image to enlarge
Figure 1 (adapted from "Enterprise Knowledge Management: The Data Quality Approach,"
by David Loshin)
For each rule, the data quality monitor acts as a switch (see Figure 2). A data item that conforms to the rule is allowed to pass through to its destination, while invalid data items are routed out to a reconciliation process.

Click on image to enlarge
Figure 2 (adapted from "Enterprise Knowledge Management: The Data Quality Approach,"
by David Loshin)
We aggregate a collection of defined rules for any intervention point into one instance of a data quality monitor agent, which is connected to three processing stages: the data source, the data destination and a reconciliation process (see Figure 3).

Click on image to enlarge
Figure 3 (adapted from "Enterprise Knowledge Management: The Data Quality Approach,"
by David Loshin)
Rules Definition and Connectivity
-5-