If the pandemic has taught us anything, companies in all industries need to leverage data better to stay competitive. Studies show that over the past 12 months, the digitization of customer and supply-chain interactions have accelerated by an average of three to four years. And while everyone is jumping on the data-driven decision making ‘train,’ a small word of caution: historically, digital transformation success rates are very low.
There are several reasons for this: lack of executive sponsorship, disconnection from business priorities, fear of the unknown, and overly ambitious targets. But, increasingly, there’s a more significant reason that gets in the way of transformation – a lack of trusted data. A data initiative that doesn’t correctly connect and analyze disparate data sources, so they are trusted across the enterprise, will fail.
Consider this. Your data has the potential to provide insights into go-to-market strategy, helps identify business leads, can determine who’s ready for promotion, and can help add features your customers will love (and your competitors will envy). It can uncover hidden needs or trends your company can leverage.
But if your sales team only trusts Salesforce.com data, your marketers won’t look beyond Marketo, and your HR team refuses to go outside of Gusto, then you have a problem. To build a trusted, enterprise-wide data mindset, you’ll need to create a solid data analytics foundation you can leverage. Setting this up requires three critical areas of consideration:
- Data collection: building processes where data is moved from its sources to a database
- Data modeling: organizing that raw data to create the business logic that determines how data can be created, stored, and changed
- Constructing a self-service system: how do you enable users to answer questions on their own?
The ultimate goal of building a data foundation is to establish an enterprise-wide single source of truth. This structured information model and associated data scheme ensure every data element is mastered in only one place, giving the business logic implicit in SQL queries somewhere to live. Everyone uses the same vocabulary to represent critical KPIs and data, which improves data quality, collaboration, productivity while reducing inconsistencies, which all lead to trust.
With the data model established, business users can answer questions in a self-service way. Getting the modeling layer right is key to letting end users explore data independently, so analysts are free to focus on ensuring the model’s integrity and evolving it based on business needs.
The Legacy Way: New Query, New Model
Business intelligence platforms have become pervasive throughout companies, either embedded in other applications or through a standalone platform’s self-service application. With more and more people wanting to access data sources and gain insights, the pressure for better-defined and organized central processes grows.
Data analysts play a crucial role here. With legacy systems, each time a user wants a report, the analysts build a data model. The model is built on what they know about the end user’s needs, the business rules based on how a specific organization operates, and the tribal knowledge through working with the organization.
The problem here is that the business logic implicit in analysts’ SQL queries lives in their heads or is scattered among files on their hard drive – but has nowhere to live externally. Each time a user needs to answer a question, the analyst must take that specialized knowledge and build a model the user can access to create reports.
This traditional model also means that users often make business decisions based on disparate data — whatever the analyst calls when building the data model. This legacy approach means that users can’t answer questions independently, and analysts aren’t free to work on higher-value activities. Instead, users should operate from a single source of truth, so they’re making business decisions based on consistent data.
Data Modelling Languages Formalize Analysts’ Knowledge
Forward-looking BI platforms have come on the market offering a new approach to solving this problem: a data modeling language. These languages typically have a lightweight structure that allows analysts to write a model, then automatically generate SQL queries against a particular database. The data model itself becomes a single source of truth — a universal dictionary of sorts. Users can then use this dictionary to explore data and create ad-hoc reports on their own.
These languages give the business logic implicit in SQL queries somewhere to ‘live.’ Variables such as which table has which data, what the column names mean, how to join table A with table B, and so on, are captured and consistent. The tribal knowledge inside the analyst’s brain gets sealed in the language.
Modeling language should resolve repetitive querying processes and streamline SQL queries by creating reusable powerful data models with custom columns and dynamic sub-querying.
They should create trust by improving data quality and reducing inconsistencies. This then empowers business users to explore trusted analyst-defined data models, saving time by reducing back-and-forth.
Businesses are now looking at data in different ways, with CIOs forecast to spend more on data and business analytics than any other technology. This creates an urgent need to build trust in the data to realize the return on this investment. Data modeling languages ensure everyone uses the same vocabulary to represent critical KPIs and data. They improve data quality, collaboration, productivity while reducing inconsistencies, which all lead to trust.
About the Author
Angshuman Guha is the CEO and Co-Founder of bipp, a new cloud BI platform recently released in beta. With a 20+ year career in applied machine learning at senior roles at Microsoft, Google, Yandex Labs, and Sears, Angshuman is a master of deep learning and holds 24+ patents for inventions as diverse as ‘risk premiums for conversion-based online advertising bidding’ to ‘determining a text string based on visual features of a shred’. He founded bipp to meet market and engineering needs that remain unsatisfied in the BI community.
Join us on Twitter: @InsideBigData1 – https://twitter.com/InsideBigData1
Sign up for the free insideBIGDATA newsletter.
Speak Your Mind