Archive for November 2013

The Value of Big Data - Defining It Once and For All

Big Data is a huge buzz word.  Like so many "next big things", there is often misunderstanding about what it is and it's value.  This leaves many companies and industries with an open and lingering question about what they should be doing with Big Data.

However, unlike other "next big things", Big Data is not overhyped in its possibilities.  It truly opens up a tremendous frontier of business intelligence.  Unfortunately, given the misunderstandings, it is not always clear how to take advantage of it.

What is Big Data

Before answering the value of Big Data, it's worth a quick summary of what it is.  Big Data is generally thought of as the "3 Vs"; i.e., data that has

  • Volume (terabytes & beyond)
  • Velocity (streaming real time)
  • Variation (structured & unstructured).

Twitter is a great example.  It's data is very large, generated real time and unstructured (admittedly hashtags and handles provide some structure but no where near a traditional relational database).

It's also important to understand that not all "3 Vs" need to be present.  The concept of Big Data is relevant if Big Data processing technologies are needed to unlock the value of a company's data or take advantage of external data.

At a high level, Big Data processing technologies offer two key capabilities.  First, it is a novel way to store data that is especially well suited for any of the "3 Vs" with the focus of providing fast access in terms of queries and updates.  Second, it provides Map / Reduce functionality that identifies relationships between unstructured data elements and helps in building ‘keys’ or ‘indices’ that are needed for fast access and cross-data relationships.

Big Data's "4th V" - Value

It is critical to emphasize that processing Big Data is not an objective unto itself and value is not created by just implementing Big Data processing and storage.  Big Data must be directly used to help improve core objectives such as

  • Optimize offers (marketing offers, cross/up sell, product configuration)
  • Improve customer service (omnichannel, surprise & delight, recovery)
  • Predict and prevent customer churn
  • Improve inventory management / product forecasting.

Big Data can be leveraged to accomplish this and create value if used an input into any of the following business intelligence processes within your organization:

  1. Collection and maintenance of enterprise data assets
  2. Batch analytics and static decisions
  3. Real time analytics and decisions

Each process is described in more detail below as how Big Data can create value.  We've shared a Marketing related example for each but obviously there are many other operational processes that leverage business intelligence and can benefit from Big Data.

Enterprise Data Assets

By processing Big Data, intelligence can be extracted to build data assets over time which plug into existing enterprise data management solutions like customer relationship management (CRM).  These new data elements help improve performance for any process that leverages CRM data (e.g., marketing campaign management).  It is analogous to the objective of building marketing databases that contain customer profile and email address.  However, with Big Data, the profile information that is built contains detailed digital activity and social activity, sentiment and influence.  By knowing these attributes, marketing campaign target lists, channel selection and offer/content can all be improved.

Batch Analytics and Static Decisions

Batch analytics and static decisions are traditionally how companies make decisions.  Historical data is compiled and analysis is done to inform decisions like marketing mix / planning decisions.  However, with Big Data, a company can now bring in granular data about digital interactions that could be terabytes in size.  This granular, cross channel data enables much more sophisticated and accurate marketing optimization models leading to more effective marketing campaigns and resource allocation.

Real Time Analytics and Decisions

Real time decisions is the ability to intelligently engage customers and improve outcomes based on real time events. It requires processing events as they occur, combining the event with other valuable data, gaining intelligence from the data and deciding on an action to improve the customer interaction.  All of this done in real time.  Big Data opens up new data sources like granular data on digital interactions and external data like weather to feed algorithms that optimize which marketing offer to present on a customer by customer basis.

In the end, Big Data is similar to a lot of innovations.  It is a new and innovative way to improve solutions for existing core objectives like marketing offer optimization.  Perhaps, Big Data's value is not mysterious after all.

What do you think?  We'd love to hear your thoughts.