The Big Deal on Big Data (Part 1)

by

In today’s world, making effective decisions depends on having good information at your fingertips. But as our ability to collect and analyze vasts amount of this information has grown over the past decate, our capability to effectively use this information hasn’t sufficiently matured. It’s very likely that the big investments in the collection and storage of this data isn’t paying off in better decision making.

Yet, while we struggle with that gap today, the pace of data continues to accelerate. There are now more than 2 billion users of the Internet[i] accessing and generating vast amounts of data. According to Cisco’s most recent Visual Networking Index[ii], Internet traffic increased eightfold over the last five years and will increase another fourfold over the next five. They estimate that by 2015, annual Internet traffic will approach one zetabyte. That’s a staggering amount of data. The gap will continue to grow.

To put that into perspective, let’s start smaller with a petabyte. A petabyte is 1015 bytes or 1 million gigabytes – capable of storing about 350 million MP3 songs[iii]. Using Gracenotes[iv] as an estimate on the total number of songs available (around 97 million) and the release of about 50 albums (or 500 songs) per week, it would take almost a thousand more years to have a petabyte of professionally recorded music. And a zetabyte is one million petabytes!

Figure: How Big is Each Byte

NameNumber of BytesNumber of SongsAll of Wikipedia[v]
Megabyte

1,000,000

< 1< 1
Gigabyte

1,000,000,000

350< 1
Terabyte

1,000,000,000,000

350 thousandOne tenth
Petabyte

1,000,000,000,000,000

350 million100 copies
Exabyte

1,000,000,000,000,000,000

350 billion100,000 copies
Zetabyte

1,000,000,000,000,000,000,000

350 trillion100,000,000 copies

 

The reason why this is so problematic to on-line marketers is that it continues to underscore the one-to-one marketing “data treadmill” – no matter how much data you collect about a single customer or potential customer, there’s always more to collect as their “digital exhaust”[vi] continues to expand in size and scope. While some of today’s largest marketing databases range into the terabytes of data, future database will need to expand significantly. But the emphasis will continue to be applying “big judgment” to “big data”; collecting more data and throwing it at existing processes is just throwing gasoline at the fire.

 


[i] http://www.internetworldstats.com/stats.htm

[ii] http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827/VNI_Hyperconnectivity_WP.html

[iii] Assuming about 2.8 megabytes per recorded song.

[iv] http://www.gracenote.com/

[v] http://en.wikipedia.org/wiki/Wikipedia:Database_download. Using 10 terabytes to make the math a big more straightforward. The size does not include images, just the text.

[vi] http://en.wikipedia.org/wiki/Digital_exhaust#cite_note-digital_exhaust_1-0

READ MORE

Why Does Change Management Keep Getting Put in Time Out?

Why Does Change Management Keep Getting Put in Time Out?

Buzzword Soup, Anyone? If I use words like “Transformation” or “Change Management” in a meeting, everyone nods like they get it. But in reality, there are usually as many understandings of those terms as there are people around the table. In today’s soup of ambiguous...

read more
Moving Forward From the Home Office

Moving Forward From the Home Office

It’s been almost 20 years since I’ve gone into an office five days a week. In that time, I’ve progressed in my consulting career from a Senior Manager, to a Principal, to a VP, to a Managing Partner, to a CEO. I barely remember what it’s like to “go to work” and to...

read more
The Leadership Dog Years

The Leadership Dog Years

As a business leader, I feel like I’ve been living in dog years — so much has happened that this year feels more like seven. In the spring, our company — like many others — had to throw out our annual plans and quickly pivot to new ones. In the summer, we increased...

read more
Shifting Perspectives: 3 Learnings From a 3-Day Training

Shifting Perspectives: 3 Learnings From a 3-Day Training

About a week ago, I completed the second live (virtual) training in the process of becoming a Certified Professional Coach through iPEC. Once again, my mind was blown! It reinforced for me that virtual workshops can, and do, work, and, in a lot of ways, I prefer them...

read more
Finding My Work-Life Balance

Finding My Work-Life Balance

In my previous post, I told the story of how I got back into consulting after becoming a mom. All of the diverse experiences I had during that journey have helped me to find my work-life balance by… Defining Boundaries “Go home,” my first boss said 12 years back —...

read more
How I Got Back to Work After Being a Full-Time Mom

How I Got Back to Work After Being a Full-Time Mom

I Landed My Dream Job Throwback to 2014, I had completed my MBA, landed my dream job as a consultant, and was hoping that my new consulting career would exponentially ramp up my career growth for the next 5 years. This would position me to take on critical decision...

read more