How to exploit the unfair advantages in unstructured data!

Unfair advantages? What is this madness, you may ask?

Most organisations are happy to tell me that that their crown jewels, that is to say, their customer and/or asset data is stored carefully, watched closely and guarded ferociously within structured data sources. These are often large-scale databases, fixed and static in nature; assets that are mined, analysed and scrutinised for Business Intelligence purposes. Specific roles are designated for contributions to these databases and only a select, chosen few can export and extract data from them.

The value, and costs associated to analysts, infrastructure and security of these assets are rarely questioned, and vast sums of enterprise money are sunk in running, understanding and maintaining them. And rightly so; after all, this has been an established pattern of winning, doing and retaining business for a long time.

But unstructured data sources are a much less glamorous topic of conversation. When pressed, many businesses will admit that much of their fileshare, email and sharepoint data is ‘dark’. Lost in the annals of time, these sprawling, unmanaged estates often feature orphaned ownership and huge quantities of duplicated and redundant information. Yet, there could be value locked away and forgotten within them.

The Scale of the Task

Let’s just look at unstructured data in isolation for a moment to better recognise its importance. Every organisation has unstructured data, and pays decent sums to various enterprise storage vendors (and increasingly cloud providers too) to flex the necessary capacity to contain it all. For many organisations, these environments have been around for 15+ years.

The bit that baffles me personally, is the lack of time and focus these vital stores of data have received. Blockers in technology, people and process historically, have meant this data has gone dark, and remains dark.

According to ITC research, the volume of unstructured data is set to grow from 33 zettabytes to 125 zettabytes globally, before 2025. Further research shows somewhere between 80-90% of the worlds data is unstructured and barely 0.5% of it is used in business intelligence right now (Dialani, 2020). So, don’t take my word for it and certainly don’t think this sprawl is limited or unique to your enterprise. This data is growing exponentially, unchecked and largely unrealised.

Risk? Of course 

I’ll blog more about risk in unstructured data in the near future. Suffice to say that the risk profile of unstructured data is close to catastrophic in many organisations. Without full visibility and quantifiable analysis of these data sets, organisations have little, to zero grasp, of just how big their problems are, and how much adjustment to the risk register is needed.

BUT! There is gold in those hills! 

Let’s grasp the obvious, low-hanging fruit here, for a short discourse. Yes … we’re starting with storage costs. You’ll no doubt be thinking that cloud storage is abundant and cheap, while on-premise costs for SSD and spinning disks has been falling consistently for years. And you’d be right, of course.

But as senior leaders, you’ll also recognise this is only a fraction of the cost incurred. Backups, redundancy and availability all require 2x or 3x cost for every byte of data kept, not to mention the human capital element of maintaining these estates. And cloud isn’t immune either, with AWS offering 16x tiers of storage now. When organisations are deploying more IOT, OT and RPA than ever before, we shouldn’t forget that the logs and output from all of these sources needs to go somewhere! If we consider that over 70% of data is not regularly accessed (Meehan, 2019), cold storage could be provisioned at a fraction of the cost. If only we knew which 70% that was.

The point is, if you can better understand what’s in your unstructured data, you can leverage significant cost savings in people and infrastructure.

Cost Reduction? Check!

There are other less-transparent and not immediately obvious ways in which many sectors can grasp a competitive advantage too. Scientific research, media, entertainment, legal and manufacturing industries are often found burning hundreds of hours of resource searching fruitlessly for aged research, papers and documentation. In all these fields of expertise, data often becomes valuable again many years after its first used. The most pertinent and obvious example would be SARS virus research when a global pandemic emerges!

Equally, sales teams, customer success organisations, programmers, bid teams and many more also have a need to find data accurately. Sometimes, especially in sales, the re-use of key client sales collateral and product slideware can be the difference between you and a competitor in a big deal.

You’d be unpleasantly surprised at just how much time your business is burning in such activities.
Accurate identification, tagging and speed at which this data can be found are critical to agile responses.

Cost Efficiencies & Productivity Realised? Check!

I’ve mentioned sales above, but the front-line teams are of course but one part of a well-oiled machine that generates income for organisations. Winning business involves significant time from analysts and BI teams, not only working with external data but internal data as well. Having access to previous bids, product information and similar, relating to a specific and highly targeted project is crucial.

In financial markets where debt and ledgers are traded, bought and sold for instance, being able to rapidly search billions of unstructured files and export for detailed analysis has simply been tedious, long-winded, time consuming and often too difficult technically for many organisations to achieve. But such analysis is the difference between a successful bid, or not.

The faster you get there, the faster your sales teams go to market, outmanoeuvre your competition and win big.

And let’s not forget the impact of privacy. Being able to demonstrate your organisation puts privacy at the forefront of its strategy means you’re seen to take CCPA and GDPR seriously instead of working around consumer privacy rights. It’s a fact that is hard to argue when you consider the erosion in consumer trust relating to big organisations, and the correlated 50% who would willingly share more of their data in return for higher value engagements (McKinsey, 2021); and Apple continue to grow new customers at an exponential rate with, at least in part, a privacy based strategy (Macworld, 2021).

New Revenue Streams? Check!

You're not alone!

According to Kyle Wiggers of Venture Beat (2021), just 13% of organisations have unlocked the potential of their data. Technology has, until now, been a blocker. But with a Data Discovery tool, and a Customer Success team to aid with alignment between people, process and the technology, the future is nearer than you ever thought possible!

To find out more about Exonar’s Data Discovery technology, just click here.