Even in this era of predictive analytics and AI-driven machine learning, a global business intelligence survey has found that 58% of respondents still base their decisions on gut feel. These respondents have the results of their expensive automated intelligence apparatus at hand but they couldn’t bring themselves to trust the conclusions reached by a black box whose inner workings they did not fully understand.
This is the “trust gap” and it is holding back the full deployment of a data-driven business culture.
Stephanie Reynolds explores the “trust gap” in this article on twdi.org:
To change the way we make decisions, we have to build trust. Trust in the data itself, trust in the specialist who transformed that data into analysis, and trust in the software that is storing, processing, and delivering insight.
Most current thought leadership about data governance describes the modern technical challenges of data security, quality, and lineage in a highly distributed data analytics architecture. However, solving those technical challenges won’t directly increase the number of managers making data-driven decisions. Instead, we need to shift our focus as data governance becomes more important to developing insights. It is a slight but critical shift to the people and processes that ultimately deliver trusted insights.
[…]
In today’s new self-service analytics world, in order to support trust in data and trust in analytics, you need a system for easily collecting, organizing, and accessing the documentation that describes analytics discovery. These catalogs need to be more than mere inventories of available data. If our goal is transparency and trust — the trust needed by managers to change behaviors and make data-driven decisions — data catalogs must embrace the notion that a data inventory is only a starting point. Inventories are not rich enough to deliver trust.
Data catalogs must capture the people, processes, and data involved in analytics discovery. The individuals asking curious questions and the methods applied during discovery are just as important as the quality, security, and lineage of the underlying data. The social information of who was involved in the analysis needs to be captured alongside the technical details of the analysis. This is the only way we can psychologically transfer trust in tribal knowledge to trust in the transparent scientific method performed during analysis. As managers — and as humans — we need this context to be able to trust the insight we receive.
