Dark Data 101: The good, the bad, the exciting

Exit big data, enter “dark data,” at least as far as buzzwords are concerned. You might hear about dark data a little more this year. Why? Because it’s a growing challenge—and opportunity.

What is dark data?

It’s not quite as sinister as it sounds. According to Gartner’s IT Glossary, dark data is “information assets organizations collect, process and store during business activities and generally fail to use for other purposes.” In other words, companies and organizations collect a nearly incomprehensible amount of unstructured data on everything from customer information to contact information, previous employee profiles and raw survey data. Much of it is never put to use for any valuable practice. Vertias reports that 52 percent of all stored data is dark. As we move forward with various big data and cloud computing services, we amass more and more data that we don’t use. Consider it data clutter, except a little more problematic than your National Geographic collection from 1971 on.

Why should I care about it?

Dark data doesn’t just take up space or gather dust. It poses security risks. Headlines like “A dark data startup you haven’t hear of is spilling secrets to Starbucks, McDonald’s,” can be a little unsettling and raise questions surrounding privacy and safety.  Dark data is largely unprotected, and the organizations that do strive to comply with government regulations or internal policies end up paying a pretty penny to store things safely. To repeat: government regulations. The European Union’s General Data Protection Regulation (GDPR) becomes in enforceable in May, and its protections for personal data are likely to impact global business.

What do I do about it?

Clear the clutter! R1Soft recommends that you audit your database, prune it, back it up and store it an encrypted form. Some experts suggest that dark date provides the perfect opportunity for artificial intelligence (can you imagine personally auditing endless hordes of data?). AI can not only clean out, organize and protect dark data. It can help you put it to good use.

What’s in it for me?

A lot, actually. Dark data can unlock many mysteries about your consumers—such as when they leave a web page or how consumers interact with loyalty programs. Dark social specifically gives insights on important trends and behavior patterns. Coupled with AI, dark data could transform the customer experience as we know it. The raw data, when properly mined, can inform investment decisions as well. Many large companies have quickly caught on. Last year, for example, Apple acquired Lattice Data, whose AI-enabled engine turns dark data into structured, usable data. Consider dark data an untapped vein of a nearly-endless information to improve your business or organization. You might strike gold.