Resource Data

The dangers of dark data: how to manage them and mitigate the risks


Dark data is a major challenge in business, and it’s not going away anytime soon. Fortunately, there are ways to reduce dark data and the risks that come with it.

Image: Shutterstock / sdecoret

One way to look at obscure (or invisible) data is to think of all the on-site and off-site paper files, files on cupboards and shelves, and any other type of information that has not been digitized, and that employees cannot easily access. Despite recent organizational digitization efforts, there is still an abundance of this hidden data circulating.

There’s also a broader, simpler definition of dark data – all data that you don’t have immediate visibility into. This can include electronic information that you may not be aware of, such as instant messages and video call transcripts, or other content that has not been centralized in a searchable data repository.

“Dark data is data that is not being managed or on an enterprise resource where it can be discovered,” said Brian Remmington, CTO at Hyland, which provides enterprise content management solutions. “Once discovered, dark data is no longer dark data. ”

SEE: Navigate Data Privacy (Free PDF) (TechRepublic)

During the COVID pandemic, many organizations worked in highly distributed and remote environments. This has spurred the growth of dark data.

“Employees also use company-appointed laptops for everything from video calls to document sharing and email access,” said Remmington. “This creates an excessive amount of dark data lurking in corporate storage. Additionally, employees can copy centralized content to their local laptops and not download it after editing it.”

Remmington believes the problem with the growing amount of obscure data will always be managing it and ensuring that it is revisited and used appropriately for business intelligence and governance purposes. Additionally, when employees leave the company, companies risk a loss of intellectual property or corporate memory by missing obscure data.

How serious is the risk of not handling dark data?

First, not handling your obscure data can create legal, security, and compliance risks. What if your business is the subject of legal action and needs to access this data as part of the legal discovery process?

Second, not having access to obscure data can lead key decision makers to make bad decisions for the organization.

Third, significant waste can result. This waste comes in the form of lost employee productivity as it will take them longer to find and get the information they need. Storage costs also increase, because storing data without using it is wasteful.

Fourth, the dark data problem contributes to enterprise data silos. “The number one goal must be to uncover obscure data so that it can be classified and handled appropriately, and potentially analyzed for business intelligence,” Remmington said. “One of the key things to look at is how each dark data silo came to be, and to identify the processes and tools that can help prevent this from happening again in the future.”

One proactive approach that companies can use to reduce obscure data is to create data retention policies for employees and to train and retrain employees on those policies.

SEE: Electronic data retention policy (TechRepublic Premium)

Audits can also be performed to ensure that data (and data changes) is synchronized between individual workstations and central data repositories.

These two steps can help eliminate the “black holes” in business information that dark data creates.

“It’s going to take a while for the dark data to go away,” Remmington admitted. “But in the meantime, organizations need to do their best to manage their data being created in real time as they deal with data backlogs. In the long run, this will help businesses meet their goals.”

See also