data shadow

A data shadow is the collective body of data that is automatically generated and recorded as we go about our lives rather than intentionally created. Sources include sensors and IP surveillance, metadata from communciations and security and authentication mechanisms, among other possibilities. Shadow, in this context, means to follow: Our data shadow follows us.

Surveillance-related data is a particular driver for storage capacity requirements. This data is often recorded and stored long-term, providing a persistent record of activities online and in the physical world. The data presents a privacy concern for individuals and a burden on IT shoulders as it pushes storage demands and potentially exposes sensitive data.

Individuals are recorded on higher resolution video for significant portions of their days in many places. The number of these data files and their size are expected to increase as the number of devices, their resolution and other data detail increases. This massive volume of data calls for consideration into privacy policies, data ownership, retention and disposal. Administrators that are not adequately aware of the problem or keeping up on policy stand to be found responsible for leaks, should data be retained that should have been disposed of or, alternately, responsible for compliance failure should data be found missing that should have been saved.