The 3 Vs of Big Data

Managing big data can be a nebulous task, especially with the vast amounts and varieties of data we have today. To make sense of the concept of big data, distinguished analyst Doug Laney of Gartner’s Data and Analytics research and advisory group introduced the 3Vs concept in a 2001 MetaGroup publication. The concept focuses on the 3 defining properties of big data – Volume, Velocity and Variety – and provides a means of efficient data analysis.

Volume

Volume refers to the amount of data generated. In today’s age, the volume of data that can be generated has the capacity to reach unprecedented heights. In fact, it is estimated that over 2.5 Quintilian bytes of data is generated each day and growing at compounding rates. With the data generated only set increase, it is not uncommon for many large companies to have Terabytes and Petabytes of accumulated and stored data. 

Velocity

Velocity refers to the rate at which new data is accumulated. Social media for example, deals with the task of accumulating vast amounts of data at an extremely high rate. Every day, 900 million photos are uploaded to Facebook, 500 million tweets are recorded by Twitter, and nearly half a million hours of video content is uploaded to YouTube. Big data helps large companies to accumulate and manage these massive numbers.

Variety

When big data is accumulated it is categorised into different classifications based on the source of the data. The data may be in the form of photos, text, video etc. The different classifications of data are important to understand, and consist of structured, unstructured and semi-structured data.

·        Structured data is data which is coded in such a format that it is easily understood. Structured data is organized and can be mapped into fixed pre-defined fields. Examples of structured data can include financial data. Such as demographic, location and transactional information.

·        Unstructured data is data which cannot be contained in an organized database. This type of data is difficult to manage and analyse and consequently, many companies have discarded unstructured data. Examples of unstructured data can include social media content, PDFs and call centre transcripts.

·        Semi-Structured data - although it has no conformity to a data model - has some manageable structure. Semi-structured data, with some process, can be stored in a relational database. Examples of semi-structured data can include email and XML. 

References

1: 3D data management: Controlling data volume, variety and velocity, 2001 Report: https://blogs.gartner.com/doug-laney/files/2012/01/ad949-3D-Data-Management-Controlling-Data-Volume-Velocity-and-Variety.pdf

2: Big Data LDN, 2019: https://bigdataldn.com/intelligence/big-data-the-3-vs-explained/

3: Volume, velocity, and variety: Understanding the three V's of big data, 2018: https://www.zdnet.com/article/volume-velocity-and-variety-understanding-the-three-vs-of-big-data/

4: Forbes: The Difference Between Structured, Semi-Structured And Unstructured Data, Bernard Marr: https://www.forbes.com/sites/bernardmarr/2019/10/18/whats-the-difference-between-structured-semi-structured-and-unstructured-data/#340a8d772b4d