The IT industry has been promoting big data for years. But some companies have found the data they needed was closer to home…dormant. And exploiting those dark data could be worth millions.
For at least 10 years, the IT industry promoted the concept of big data: the idea that huge volumes of external data will be indispensable to business. But some companies found the data they needed was closer to home. Exploiting dark data—internal files and logs which have served their immediate purpose and are now in storage—could be worth millions.
Seeing in the Dark
Quadis is a Spanish automotive business whose challenge was to improve the productivity of its call center while selling more products and enhancing control of inventory.
Its agents used a combination of the company’s industry-specific enterprise resource planning software and digital catalogues supplied by automotive manufacturers to take orders from repair shops across Spain. But since the two systems were not integrated, the agents needed to identify parts in the catalogues to find stock, and then place orders in the ERP system. After an order had gone through the system, the data was left unused and effectively dormant.
To exploit this untapped dark data, Quadis deployed Kosmos software from dark data specialist Datumize, on its network to read, capture and store data describing agent activity in both digital catalogues and the ERP system. It pulls data off internal networks and stores it locally or in the cloud.
The first outcome was to link caller ID with ERP records, avoiding the need for customers to introduce themselves and their recent business history. Quadis estimates this could save between 30 and 60 seconds per call, allowing the agents to take more calls each day, and potentially lifting revenue by €3.4 million.
The system also sifted through the history of requested parts. Using a combination of artificial intelligence (AI) and analytics, the software was able to forecast client parts needs based on those already ordered. The information gives agents the opportunity to up-sell or cross-sell parts when prompted by an on-screen dashboard. It also helps the business to forecast future demand for parts, making stock control more accurate and reducing inventory waste.
Quadis has been using the system for six months and estimates it has boosted service quality levels by 21%, reduced investment in assets by 25 to 50% and lowered operating costs by 10%.
Organizing in the Dark
Brown McFarlane, a UK-based steel firm, supplies safety-critical products to industries from oil and gas to mining, storing masses of information from suppliers to certify materials properties. IT manager Tim Wheatcroft explains.
These certificates were stored in PDF file formats and only read manually. The information was not easy to search or manage.
Brown McFarlane uses character recognition software from document specialist Kefron to read certificates and tag them with metadata to help organize information and allow engineers to search for materials properties more easily.
Wheatcroft estimates the system saves around 45 man-hours per week in research. It also provides opportunities to up-sell and improve service to clients.
We can suggest the substitution of products to clients more easily. We were always limited to specific properties the product was purchased for, but often it can do other things. For example, in some industries it is important whether the product is resistant to highly reactive chemical processes. Now we can search for that without looking through lots of test certificates.
Companies are also benefitting from dark data by looking deeper into their records to spur innovation. Knowledge-intensive businesses, such as pharmaceutical firms, tend to store research or patents for technologies which never made it to the market. These documents may contain data important to new areas of research or business opportunities. But the information will only come to light if someone in the business remembers the research and knows where to look.
Christopher Ganje is chief executive officer at Amplyfi, a specialist in using AI technologies to analyze dark data. He says that Amplyfi’s technology sifts through vast volumes of these historic data to pull up information relevant to new problems.
Companies can be nervous that they are missing a trick, that they have thrown away something that could later be valuable to a new technology or market opportunity.
The Risks of Staying in the Dark
While companies can benefit from insight gained by examining dark data, those that fail to adopt a strategy to manage it may be exposed to risks, says Ken Allan, cybersecurity leader at PA Consulting Group.
Businesses who are unaware of the dark data they hold are unlikely to protect it. Organized criminals or rogue nation states hacking into networks can exploit these data using their own analytics systems.
In doing so, they might expose personal information by cross-referencing the data or generate market insight or product information. To help defend against this kind of hacking, businesses can monitor networks for anomalous movement of data.
In any case, businesses need a strategy to manage and monitor dormant information to avoid being left in the dark.
