Skip to content

Do we really need Big Data?


Informations presented on this page may be incomplete or outdated. You can find the most up-to-date version reading my book Engineering of Big Data Processing

In this part we cover the following topics


Where is a place for Big Data?

In this part we would like to give an answer for the following question: Why bussines should think about Big Data adoption? For me it's a little bit cumbersome cause I have no priori business experience so I will present my point of view on this case, which don't have to be the best one.

From very distance perspective, business is a cycle, from decision to action to measurement and assessment of results.

This creates opportunities for businesses to optimize their operations continuously. The essential issue is whether, and if yes so, at what level (stage) of business structures we get the added value resulting from the use of Big Data.

Where is a place for Big Data?


Bussines today


Globalization of trade

Traditionally, businesses were driven almost exclusively by internal data held in their information systems. Other words, any feedback data come from company's own sources. In some sense, these data would be the same even if it would be the only one company in the world. That was possible until the end of the 90s, because information flow was so slow that decisions and actions in one corner of the word didn't influence others. Business was run in isolation and the most important factor was effective cost-cutting. This days has gone. Now we act globaly. I live in Poland but buy microcontrollers directly from China, books from GB, go on holiday to Austria, etc. I select an option which is most optimal for me. Wether I buy from China or shop located on the corner of my street it takes me the same time (in terms on making order not getting it) because I do this on the way to work directly on my phone. I do this, because I can. Because borders are on maps and inside politics heads but not in business. Today we can't think locally about our bussines because we never know if just at this moment someone located thousands of kilometers from us but interested in our offer does not browse the content on the company's website.

Companies are learning that in order to execute their business in a worldwide marketplace (which is not their choice but the effect of social changes and technological changes), organizations need to consume data from the outside to sense directly the factors that influence their profitability. The use of such external data most often results in Big Data datasets.


The need for innovation

Globalization has another consequences that has left its mark on how we perceive business. Today we don't want to buy just a product A. Instead we want to buy an innovative product A. This makes a difference. I don't want to buy a phone, I want to buy a phone which has something that no one else has. I, as a customer, don't know what it should be (and saying the truth, in most cases I don't care), but I know that I want to have it. It is a very different market to the one that focuses on cost-cutting, for it is not about prices but instead about innovations.

So, to find new customers and keep existing customers, companies have to focus outward to be able to offer new innovative products and services, and delivering increased value propositions to customers. Along with the sale of the new Samsung S9 phone (2018), customers were offered a set of additional premium services

  • Replacement of the door-to-door device or quick repair up to 2 hours.
  • Additional year of protection.
  • Dedicated helpline and remote support of Samsung experts.

This is what an innovation is: Phone is expensive but there are many expensive phones. Convince me that I should buy exactly this one. Give me something extra, something innovative. If not in terms of hardware (which is more and more difficult) else in terms of innovative services.


A good tool is needed

For this reason, companies need to expand their Business Intelligence activities beyond retrospective reflection on internal information extracted from their corporate information systems. They need to open themselves to external data sources as a means of sensing the marketplace and their position within it.

With a correctly choosen set of tools supporting advanced and state of the art simulation and forecasting capabilities, a company can get advisory results that provide foresight. In this particular case, tools assist in bridging the gap between knowledge and wisdom.

This is the real power of Big Data -- enriching corporate perspective beyond introspection, from which a business can only infer information about marketplace as it is just now, to sensing the marketplace itself and so predict how it will be in a near and distant future.

As it has been shown above, we can think about business as a three stage cycle. This cycle corresponds to military based hierarchy

  • At the top there are executives and advisory groups. They make strategic decisions.
  • In the middle there are managers. They works on tactical level and to steer the organization in alignment with the strategy.
  • At the bottom strategy and tactic are bring to life. Here core processes are executed and final products are delivered to a customers.

Feedback data is provided by a set of metrics, for each layer defined differently. Regardless of the layer we are talking about, Big Data may enhances value as it may provides additional context through the integration of external point of view to help convert raw data into information and provide meaning to generate knowledge from information and finally wisdom.

At the operations level rich set of Performance Indicators (PIs), for both services and processes are generated. At this level metrics are generated that simply report on what is happening in the business. We are converting raw data through business concepts and context to generate information. It is about past, sometimes present.

At the tactical level we have Key Performance Indicators (KPIs) which aggregates metrices from the operational level. At this level, information (provided from operational level) can be examined through the corporate perspective to answer questions regarding how the business is performing. This way meaning to the information is given. Having information and its meaning we can answer a questions regarding why the business is performing at the level it is. It is about present, and explains why present is as it is.

At the strategic level we have Critical Success Factors (CSFs) to measure progress being made toward the achievement of strategic goals and objectives. Big Data may help us transform information and diagnostic into knowledge. At this level knowledge help answer questions of which strategy needs to change or be adopted in order to correct or enhance the performance. Knowledge workers provide the insight and experience to frame the available knowledge so that it can be integrated to form wisdom. It is about future, and explains what we should do.

Big Data makes possible transition from hindsight through insight to foresight:

  • Hindsight: (Google Translator, pl: spóźniony refleks) understanding of a situation or event only after it has happened or developed.
  • Insight: (Google Translator, pl: wnikliwość, intuicja) the capacity to gain an accurate and deep intuitive understanding of a person or thing.
  • Foresight: (Google Translator, pl: dalekowzroczność) the ability to predict or the action of predicting what will happen or be needed in the future.