Data lineage is the history of data, including where the data has traveled through-out the its existence within an organization. Data lineage is a required part of corporate and government data policy compliance. Tracking the history of data is achieved through data lineage documentation and software. Without a way to identify where data errors are introduced into the environment, it is difficult for data stewards to identify and fix data quality issues.
With effective tools, data governance can be eased through the documentation of data’s entire journey through the organization. The documentation of data lineage helps simplify two of the main data governance concerns in for the effects of changes in data: root cause analysis and business impact analysis (BIA). Clear understanding of root causes and impacts of issues with data is aided by knowing everything that happened to the data since it came to be.
In software development, the tracking of data lineage can help with reconciling the difficulties between Agile development best practices, data governance regulations and company data policy. Data lineage tools and procedures help track where data flaws were introduced, which can ease diagnoses and correction. Implementing the tracking of data lineage can be difficult and often seen as a low priority, however, earlier correction means less error propagation, which means the implementation of data lineage tools early in the process often proves worth the effort.