Big Data, Big Deal (part 1)

Alex Batchelor, COO of BrainJuicer and former Chairman of The Marketing Society, writes the first chapter of his story of data, drawing on experiences at Capital One and Royal Mail, and confessing to being a huge fan of data, big or otherwise.

I keep being invited to conferences about Big Data. Even more strangely, I actually get asked to speak at some. It is an indication of my age that I started work when not everybody had a computer at their desk. I actually worked for IBM when it introduced the first “portable computer”, which seemed to involve having a strong handle on the side of a normal computer along with a flip-down keyboard, and required people to have very strong arms.

When I first started at Unilever, we used to throw away the prize draw entry forms, all painstakingly completed with names and addresses. This was because it cost a lot to transcribe and keep the information and it was deemed to be more expensive than it was worth. Indeed, the first introduction I had to the potential value of this data was when I was responsible for the call centre we established. We only put a phone number on the packs in 1990. Before then you had to write to us.

I had been asked to give a financial justification for keeping the phone numbers and information about the people who contacted us. We started recruiting people from this database to test products. It was considerably cheaper than plucking potential trialists from the general population.

Predicting the unpredictable

My first experience with genuinely large data sets was when working as a consultant with Capital One.  I was amazed to discover that they ran almost 60,000 different trials on their database each year, and had exhaustive data on response rates to different mailings, even down to variances caused by font size and colour of print.   For the first time I saw the pure commercial benefits of repeated controlled tests - and also the risks.

The responses to some of the trials were similar every time. For other trials, the effects seemed to decay over time, and in extreme cases change completely, even though we felt we had controlled all the relevant variables.  Sometimes you just can’t predict human behaviour – or we were just predicting it wrongly.

Postcode plus

That experience helped me when I got to the Royal Mail.  You may not know this, but of the 85 million items handled every night, about 65 million envelopes go in a machine. (I am sure the numbers are smaller now than seven years ago.) A picture is taken of the front and back to read the address.  It uses missile recognition technology provided by Lockheed Martin to read the addresses and print a delivery point suffix – like your postcode but with a bit more data unique to your house – on each item.

What was more amazing was that all of this data was just thrown away – that is, until I wrote a business case suggesting that we kept it. I admitted that I wasn’t even sure what we were going to use it for, just that it would be useful in three ways:

1.    It was a way of understanding load across our business. You can tell when you first and last saw an item in the automated sorting system and the volumes moving between different mail centres and their variability.

2.    It was a way of understanding individual addresses, in terms of the frequency, volume and type of mail that they received.

3.    It was potentially a source of information to sell to other data users.

Google quite happily allows companies to target their advertising at the sites of their competitors, and indeed charges you more for the privilege, but, as far as I am aware, Royal Mail has not yet started letting companies know who else is sending you mail.

People understanding people

I am actually a huge fan of data, big, or otherwise. And I am a huge fan of businesses making informed decisions.  If I have learned one thing – it is that no matter how much actual data you have, you still have to have a view about human behaviour.  Why are people doing that, what would make them do it more, or less, often?  You also need a way of testing whether that view is right. That needs to be in situations as close to reality as possible.  

Human decisions are made using a variety of lenses, affected by the environment in which the decision is made, by the actions of others and by our own individual biases.  They are affected in ways that we don’t understand and can’t articulate – and we are certainly not rational, individual agents as implied by a lot of economic theory.  

Using our understanding of human behaviour to help companies is what good marketing is all about.  While that might be enhanced by access to large data sets, it is not predicated on them – and still ultimately requires people understanding people.

Read more from Alex Batchelor.