Data Quality

Data quality is critical to achieving trusted data and is very closely related to the goals of data value. Quality has many dimensions including accuracy, completeness, consistency, precision, granularity, timeliness, integrity, and usability. Data governance is responsible to assess, monitor, communicate, and continuously improve data quality.

Data quality practices from BI and data warehousing are geared toward cleansing data to improve content correctness and structural integrity in data that is used by query and reporting processes. In the big data world, quality is more elusive. Correctness is difficult to determine when using data from external sources, and structural integrity can be difficult to test with unstructured and differently structured (non-relational) data.

Data quality control: These policies are intended to formally describe data quality expectations and specify the processes by which data values, instances, and sets are validated against those expectations, the methods for measuring conformance to data quality rules, the ways that data validity is monitored and reported, and the processes for assessing criticality, prioritization of discovered issues, root cause analysis, and remediation.