Data quality

What data quality means

Data quality describes how complete, correct and consistent data is. It is not a technical question but a commercial one. An analysis resting on incomplete data delivers a figure that helps nobody while influencing decisions. An AI system working with contradictory details returns contradictory answers.

The term is often confused with data volume. More data improves the result only when it is right. Anyone collecting ten thousand customer records half of which are outdated worsens the analysis compared with a smaller clean set.

The criteria quality can be measured by

• Completeness: are all fields filled that the analysis needs. A missing customer group prevents any segmentation.
• Correctness: do the values match reality. A wrong weight is more expensive than a missing one.
• Consistency: are dimensions held in the same unit, dates in the same format. Without a unit neither filtering nor sorting works.
• Uniqueness: does each record exist exactly once. Duplicate customer records distort every figure.
• Currency: do the data match today's state. An address from three years ago endangers every delivery.

How errors arise

The first reason is missing control at the source. An input field without validation accepts anything, including nonsense. The second reason is several collection points: when a value is maintained in three places, three versions exist within a short time.

The third reason is the missing reference. When a customer is deleted, their orders remain as orphaned records and distort the analysis. The fourth reason is time. Data ages without anyone doing anything wrong, and without regular reconciliation the state goes unnoticed.

What poor data costs

Four costs arise and are rarely added up. First wrong decisions, because the analysis is off. Second rework in operations, for instance a delivery to a wrong address. Third extra work in service, because customers ask. Fourth failed projects: a CRM or AI system starting with unusable data loses trust before it has achieved anything.

An indirect effect comes on top. Anyone who has once seen a figure that was wrong checks every further one. The analysis thereby loses its purpose, because its job is to answer a question quickly.

How quality can be checked

Four checks are machine feasible and find a large part of the errors. Missing values in decisive fields, plausibility of values against an expected range, uniformity of dimensions and weights, and duplicates in master data. A check against your own source comes on top, for instance against the supplier's catalogue. The approach is always the same: form a figure, report violations, name owners, repeat regularly. Without repetition the state returns within a few months.

Frequent questions

• Where do I start? With the fields appearing in the most frequent analyses.
• Is a tool enough? No, without rules and ownership it only finds what has already gone wrong.
• When is it good enough? When the analysis answers the questions being asked.