Big data changes everything – especially ads

'Throughout human history, we have been dependent on machines to survive.' Morpheus, The Matrix

Predictive modelling

You may remember the advertising news headlines about an angry dad in Minneapolis who stormed into a Target store demanding an explanation as to why his teenage daughter had been sent ads for maternity clothes and nursery furniture. She wasn’t pregnant – or was she? And how did Target know before her own family?

You can thank big data and predictive advertising for helping Target figure out how to tell if women are pregnant long before they start buying diapers. Predictive modelling helps a retailer know that if a customer (female, in this case) is searching for lotion, pre-natal vitamins, baby furniture, and swaddling blankets – there’s a good chance there’s a baby in the picture somewhere.

And remember, the search info and product preferences is knowledge freely shared by most people, so why wouldn’t Target use that data to predict needs and be seen as a helpful brand reaching out a friendly hand to the a expectant mom (or sister, or aunt, or Grandma)?

This is the promise of big data and predictive advertising. The data produces actionable information for the brand and agency around the needs and desires of the customer. The brand then serves up products and services to meet those anticipated needs and wants. A good thing? Or kind of creepy?

At a crossroad

Advertisers, brands and customers are finding themselves at this crossroads now and in the foreseeable future. A perfect storm is brewing that is a combination of personal data freely given by the consumer, coupled with big data analytics, predictive modelling, and inference advertising, all in the smartphone in the palm of our hand. It's exciting, terrifying and fascinating all at the same time.

It is indeed a brave new world for advertisers. 'Data-driven” advertising has been bandied about over the past decade inside agencies as almost a necessary evil. No more. Even the comments around big data from the advertising giants Publicis and Omnicom in their merger announcement indicated that 'they hope to be better equipped to participate in an industry that's quickly become dominated by data analysis and automated ad buying.'

'Big data in the cloud'

Meanwhile, tech companies like Google and Apple, and social companies like Facebook and Twitter, are on the scene with the all-data-all-the-time approach to advertising, convincing brands that data is the only path to true ROI. Their promise is that somehow, contained within that mysterious 'big data in the cloud', is the magic formula brands have been waiting for to unlock the purchasing power and brand loyalty of the consumer.

This party is clearly getting crowded. The lure of big data and huge profits is just too tantalizing to ignore. But all these new stakeholders are quickly realizing that big data can mean really HUGE data and HUGE can be hard (and expensive) to manage. Let’s pause a moment to get a good feel for what big data actually means. Back in 2010, the Economist noted that: 'Wal-Mart, a retail giant, handles more than 1m customer transactions every hour, feeding databases estimated at more than 2.5 petabytes—the equivalent of 167 times the books in America's Library of Congress.' Imagine how sky-high that number is today.

And what’s also worth mentioning here is that the brand actually owns the data – in this case Wal-Mart. Facebook, Google, and the ad agency might have access to some customer data, but the truly valuable and big data is under lock and key, firmly in the hands of the brand. The actual capturing, storing, parsing, and producing actionable data might move around a bit, but the data itself belongs to the brand.

Data mining

Some brands are proving to be pretty good at mining their own big data reservoirs for marketing – and R&D. Way back in 2009 Netflix made waves by offering a $1 million prize for the best algorithm based on massive amounts of Netflix subscriber data to recommend movies based on past preferences. Entrants could be tracked in real-time on the Netflix leaderboard with participants racing to out-do each other. The competition was fierce right up to the moment that BellKor’s Pragmatic Chaos was awarded the ultimate prize.

No one can afford to sit this one out. For advertisers, publishers and anyone in the business of digital media, it’s time to appoint a really good Chief Creative Technologist and get busy.

Beverly Macy is the CEO of Gravity Summit, co-author of The Power of Real-Time Social Media Marketing, podcast host,and frequently writes about Big Data. Follow her on Twitter @BeverlyMacy.

Read more from Say Media.