The Big Deal on Big Data (Part 1)

by

In today’s world, making effective decisions depends on having good information at your fingertips. But as our ability to collect and analyze vasts amount of this information has grown over the past decate, our capability to effectively use this information hasn’t sufficiently matured. It’s very likely that the big investments in the collection and storage of this data isn’t paying off in better decision making.

Yet, while we struggle with that gap today, the pace of data continues to accelerate. There are now more than 2 billion users of the Internet[i] accessing and generating vast amounts of data. According to Cisco’s most recent Visual Networking Index[ii], Internet traffic increased eightfold over the last five years and will increase another fourfold over the next five. They estimate that by 2015, annual Internet traffic will approach one zetabyte. That’s a staggering amount of data. The gap will continue to grow.

To put that into perspective, let’s start smaller with a petabyte. A petabyte is 1015 bytes or 1 million gigabytes – capable of storing about 350 million MP3 songs[iii]. Using Gracenotes[iv] as an estimate on the total number of songs available (around 97 million) and the release of about 50 albums (or 500 songs) per week, it would take almost a thousand more years to have a petabyte of professionally recorded music. And a zetabyte is one million petabytes!

Figure: How Big is Each Byte

Name Number of Bytes Number of Songs All of Wikipedia[v]
Megabyte

1,000,000

< 1 < 1
Gigabyte

1,000,000,000

350 < 1
Terabyte

1,000,000,000,000

350 thousand One tenth
Petabyte

1,000,000,000,000,000

350 million 100 copies
Exabyte

1,000,000,000,000,000,000

350 billion 100,000 copies
Zetabyte

1,000,000,000,000,000,000,000

350 trillion 100,000,000 copies

 

The reason why this is so problematic to on-line marketers is that it continues to underscore the one-to-one marketing “data treadmill” – no matter how much data you collect about a single customer or potential customer, there’s always more to collect as their “digital exhaust”[vi] continues to expand in size and scope. While some of today’s largest marketing databases range into the terabytes of data, future database will need to expand significantly. But the emphasis will continue to be applying “big judgment” to “big data”; collecting more data and throwing it at existing processes is just throwing gasoline at the fire.

 


[i] http://www.internetworldstats.com/stats.htm

[ii] http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827/VNI_Hyperconnectivity_WP.html

[iii] Assuming about 2.8 megabytes per recorded song.

[iv] http://www.gracenote.com/

[v] http://en.wikipedia.org/wiki/Wikipedia:Database_download. Using 10 terabytes to make the math a big more straightforward. The size does not include images, just the text.

[vi] http://en.wikipedia.org/wiki/Digital_exhaust#cite_note-digital_exhaust_1-0

READ MORE

Shifting Perspectives: 3 Learnings From a 3-Day Training

Shifting Perspectives: 3 Learnings From a 3-Day Training

About a week ago, I completed the second live (virtual) training in the process of becoming a Certified Professional Coach through iPEC. Once again, my mind was blown! It reinforced for me that virtual workshops can, and do, work, and, in a lot of ways, I prefer them...

read more
Finding My Work-Life Balance

Finding My Work-Life Balance

In my previous post, I told the story of how I got back into consulting after becoming a mom. All of the diverse experiences I had during that journey have helped me to find my work-life balance by… Defining Boundaries “Go home,” my first boss said 12 years back —...

read more
How I Got Back to Work After Being a Full-Time Mom

How I Got Back to Work After Being a Full-Time Mom

I Landed My Dream Job Throwback to 2014, I had completed my MBA, landed my dream job as a consultant, and was hoping that my new consulting career would exponentially ramp up my career growth for the next 5 years. This would position me to take on critical decision...

read more
Self-Awareness is Key to Belonging

Self-Awareness is Key to Belonging

In August of this year, as part of our annual company meeting, our team at Thought Ensemble participated in the foundational session of Diversity, Equity, and Inclusion (DEI) training led by Dr. Nika White, IOM, CDE (she/her/hers). One of the most meaningful moments...

read more
Finding Your Organization’s Magic Pixie Dust

Finding Your Organization’s Magic Pixie Dust

It is often said that organizational culture is like a fog — it is all around us; it impacts our ability to see, to move quickly, and to deliver; but we cannot quite put our finger on it. Indeed, some organizations see their culture as a byproduct of operations,...

read more
We’ve Refreshed Our Brand!

We’ve Refreshed Our Brand!

Why have we refreshed our brand, you ask? Well, as we have grown and matured as an organization, we felt that our previous brand elements no longer represented us as well as they could. You see, we founded Thought Ensemble back in 2008 to help companies better compete...

read more
Thought Ensemble’s Purpose — Inspired in 2020

Thought Ensemble’s Purpose — Inspired in 2020

I recently wrote about how company purpose is being tested and inspired by all the events of 2020. This topic is very real for us at Thought Ensemble. We’ve been thinking a lot about what really matters as we’ve navigated the...

read more
How 2020 Is Testing and Inspiring Corporate Purpose

How 2020 Is Testing and Inspiring Corporate Purpose

In August 2019, the Business Roundtable rewrote their statement of corporate purpose. I followed this with significant interest being that I have never forgotten the debates about corporate purpose in business school almost two decades ago. We were taught that the...

read more
Why Purpose-Driven Organizations May Struggle With Change

Why Purpose-Driven Organizations May Struggle With Change

I love working with companies who really want to make a difference, beyond just making money for their shareholders. I mean, making money is fun and all, but it is even more rewarding to join in on a just cause. Plus, as this HBR article explains, companies who have...

read more