Post: Data vs. Intuition- Why is everybody in BI Talking about it?

Last month, we discussed the benefits of self-service BI in our blog. One risk of self-service BI is that data is open to each individual’s interpretation, and therefore may be subject to opinion and bias. As our society becomes more reliant on data, where does that leave intuition? Does one exist without the other?

According to Peter Sondergaard, senior VP of Gartner Research, “Information is the oil of the 21st century, and analytics is the combustion engine.” But as data integrates itself more and more into every aspect of society as we know it, where does that leave intuition?

After all, it was Einstein himself who said “I believe in intuitions and inspirations. I sometimes feel that I am right. I do not know that I am.”

But to truly understand the debate of Data vs Intuition, we need to understand what each of these represents.

Data is left-brain. It’s logical and calculated. It’s mathematical and scientific. You can collect it. You can test it. It’s based on knowns, facts, and reason.

Then there’s intuition. It’s an age-old embedded sense of “knowing.” It’s instinctive. It’s the right brain. It’s a physiological awareness of our environment and a sub-conscious gathering of data.

So which do you rely on more, data or intuition?

Let’s look at data first.

Data drives innovation. By gathering it we are able to not only shape the future but to predict it as well. It gives you or your business a competitive edge when it comes to making informed decisions. It helps you achieve desired results and helps you strategize when searching for solutions to problems.

By using data and following the law of supply and demand, Travis Kalanick, co-founder of Uber, implemented Dynamic Pricing, charging more when demand is higher and less when it’s not, which paid off big in the long run. Geoffrey Moore, author and high profile consultant for companies such as Salesforce, Microsoft, and Google said: “Without big data, you are blind and deaf and in the middle of a freeway.”

Intuition

On the other hand, the vision you have as a person or a company can’t be explained by data. That’s intuition. It helps you make decisions without solid reasoning which, given our increasing dependence on data, seems like a terrible idea, but many times it pays off BIG.

Bill Allen, CEO of Boeing in the 1950s had a vision that the rest of the company did not see. At that time, Boeing was only making airplanes for the defense industry but his intuition told him that he needed to build a commercial airliner. He fought hard to convince his board that his vision of a commercial plane would be the future of Boeing. That plane was the transcontinental airliner, the 707, and it revolutionized air travel as we know it.

A study at the New Jersey Institute of Technology revealed that 81% of CEOs with high intuition scores doubled their business every 5 years. Even the US military invests in helping soldiers refine their 6th sense because intuition is sometimes more valuable than intellect on the battlefield.

Intuition + data =

But why do data and intuition have to oppose each other? After all, utilizing them both is synergistic, balanced, whole-brain thinking. Data makes your findings relevant and intuition affirms that with a gut feeling. You can use intuition to create a theory and then test it with data. Essentially, intuition is merely another form of data.

Innovation expert Bernadette Jiwa says that we can’t make a decision without emotion, nor is data always an accurate representation of behavior. She cites the 2016 presidential election as the perfect example of this. “The data said one thing, and actually the “real” story was right under our noses, and we ignored it.” In fact, intuition and data science go hand in hand. Intuition helps give you an understanding of concepts and how to apply them, as well as contextualizing data. There is invisible and implied info in data that has to be interpreted with intuition.

I’ll leave you with a quote from author Malcolm Gladwell. “There can be as much value in the blink of an eye as in months of rational analysis.”

All that being said, which do you rely on more? Are you more data-driven, intuitive, or both?

 

Scroll to Top
As the BI space evolves, organizations must take into account the bottom line of amassing analytics assets.
The more assets you have, the greater the cost to your business. There are the hard costs of keeping redundant assets, i.e., cloud or server capacity. Accumulating multiple versions of the same visualization not only takes up space, but BI vendors are moving to capacity pricing. Companies now pay more if you have more dashboards, apps, and reports. Earlier, we spoke about dependencies. Keeping redundant assets increases the number of dependencies and therefore the complexity. This comes with a price tag.
The implications of asset failures differ, and the business’s repercussions can be minimal or drastic.
Different industries have distinct regulatory requirements to meet. The impact may be minimal if a report for an end-of-year close has a mislabeled column that the sales or marketing department uses, On the other hand, if a healthcare or financial report does not meet the needs of a HIPPA or SOX compliance report, the company and its C-level suite may face severe penalties and reputational damage. Another example is a report that is shared externally. During an update of the report specs, the low-level security was incorrectly applied, which caused people to have access to personal information.
The complexity of assets influences their likelihood of encountering issues.
The last thing a business wants is for a report or app to fail at a crucial moment. If you know the report is complex and has a lot of dependencies, then the probability of failure caused by IT changes is high. That means a change request should be taken into account. Dependency graphs become important. If it is a straightforward sales report that tells notes by salesperson by account, any changes made do not have the same impact on the report, even if it fails. BI operations should treat these reports differently during change.
Not all reports and dashboards fail the same; some reports may lag, definitions might change, or data accuracy and relevance could wane. Understanding these variations aids in better risk anticipation.

Marketing uses several reports for its campaigns – standard analytic assets often delivered through marketing tools. Finance has very complex reports converted from Excel to BI tools while incorporating different consolidation rules. The marketing reports have a different failure mode than the financial reports. They, therefore, need to be managed differently.

It’s time for the company’s monthly business review. The marketing department proceeds to report on leads acquired per salesperson. Unfortunately, half the team has left the organization, and the data fails to load accurately. While this is an inconvenience for the marketing group, it isn’t detrimental to the business. However, a failure in financial reporting for a human resource consulting firm with 1000s contractors that contains critical and complex calculations about sickness, fees, hours, etc, has major implications and needs to be managed differently.

Acknowledging that assets transition through distinct phases allows for effective management decisions at each stage. As new visualizations are released, the information leads to broad use and adoption.
Think back to the start of the pandemic. COVID dashboards were quickly put together and released to the business, showing pertinent information: how the virus spreads, demographics affected the business and risks, etc. At the time, it was relevant and served its purpose. As we moved past the pandemic, COVID-specific information became obsolete, and reporting is integrated into regular HR reporting.
Reports and dashboards are crafted to deliver valuable insights for stakeholders. Over time, though, the worth of assets changes.
When a company opens its first store in a certain area, there are many elements it needs to understand – other stores in the area, traffic patterns, pricing of products, what products to sell, etc. Once the store is operational for some time, specifics are not as important, and it can adopt the standard reporting. The tailor-made analytic assets become irrelevant and no longer add value to the store manager.