Big Data #9: Don’t Push It Too Far, Stick to Your Common Sense

Big data is about trying to unearth the right information from your structured and unstructured data sets. Done properly, it is no doubt that it will assist the enterprise to understand its environment, its customers better and make more informed decisions. However, in the current market dynamics, fear of missing out, wrong investments in the platforms, lack of experience are the most cited risks. There is more to that.
We have seen the bandwagon effect behaviors in big data as expected, companies rushed to invest in big data without proper knowledge, the fear of missing out rushed projects with too many items undone, which were in fact the same as the cold war era of arms race. During the big data timeframe, from the investment to the arms race, the companies failed to understand there are further risks that come with big data.
I can say that the foremost risk that companies fail to understand, or lose down the road is common sense, which this post is about. First of all, big data analytics are about giving insights, nothing more. It is being used by the scientific and government institutions for many years and by today we are sure that it is not hundred percent accurate. Think about the meteorologic research, which has far more data than many other sciences: are we absolutely definitely sure about the time when the next hurricane will strike? Looking at the big data, nobody can say which week, let alone day and time. We are sure about that and we are also sure that our big data will give us the information we need. It might, but we have to apply some judgement, we need to stop and think, before accepting whatever it gives us.

RELATED:   Big Data #4: Developing In-house Talent

Similarly, there are the inherent problems that come with any data analysis: choosing significance, relevance, eliminating noise, validity, extent of analysis, just to count a few. Staff training will eliminate some of these problems for sure, but still anyone working in the big data analytics should be aware that it is so easy and just a matter of minutes to lose sight. There are correlations that show the murder rates in U.S. fall with the loss of Internet Explorer market share, Facebook lowering the cholesterol-lowering effect of Justin Bieber and similar unrelated issues. Although these are just some funny statistics, they show that there are traps in the analytics that people may fall, such as correlation not implying causality.

RELATED:   Big Data #7: Preparing for the Big Data Workflows

Second, it does not replace the tried and tested methods in the business. Visiting customers, attending trade shows, conferences, building networking cannot be replaced entirely by monitoring mentions, hashtags, clicks, web site analytics and click monitoring, displayed in dashboards. The product or service that is offered by the enterprise, whether or not operating in a fully-automated Business-to-Business market has a human factor in it. Big data analytics does not replace the human element. Following my list above, suppose that big data analytics indicated a building trend in your market. In this case you have to make a little bit of research and then discuss with your colleagues, with your customers about their opinions. You cannot make the whole enterprise act on the big data results.
Third, big  data present users with the opportunity to dive deeper in the analysis, which is also a trap in itself. It is so easy to lose the sight of the forest for the trees. Big data analytics give you the tools to make queries for you to gain some insights. When you spot a trend, list a couple of questions to analyze the insight from different dimensions and when you are done, avoid the temptation to go further. Suppose that you are trying to analyze the trends with your product launch, you can try to see the impact of the marketing campaign, competitor’s reactions – such as increase in commercials etc.. Once done, and perhaps with some more questions from your colleagues, stop your analysis. Digging deeper will shift your focus from the product launch to less trivial things.

RELATED:   Big Data #6: Before Giving Your Big Data Project to a Contractor …

My advice is: try to see what works and what not, what works best where, how you should be using big data analytics and what technologies enable you to walk to your goals. Pushing big data too far and losing common sense may easily work against your company.
**Image credit: **business2community.com