To get a god’s eye view of human behavior you have to ask

by User Not Found | June 06, 2011

I know that many of my colleagues despair of me when I question the value of marketing based on passively collected, digital data. They either think I don’t get it, or I am being hopelessly old fashioned. So maybe I need to make my case more clearly.

It is not that I am anti-listening, anti-targeting or anti-social – media that is – it’s just that I believe there is far too much hype and hope surrounding the use of digital data. However, there is incredible value to be gained, but only if we approach it in the right way.

Let me try to explain the root cause of my concern. Every time we use the Internet on our mobile phones, we leave behind a trail of digital clues as to our needs, wants and interests. This data exhaust is vast and free, but it is far from exhaustive. What is missing is any insight into our underlying motivations or any understanding of the influence of context and cues, the triggers that cause us to behave in certain ways or make certain decisions.

Data mining will tell us what people did, but we can only infer why they did it. And all too often that inference will be wrong. The beliefs, expectations and objectives of the people charged with making sense of the data will shape their interpretation, just as much as the quality of the raw data.

But there is a solution to this problem. We can ask questions to clarify why people behave the way they do. Don’t take my word for it. Listen to this interview on “Here and Now” with Professor Alex “Sandy” Pentland of the Massachusetts Institute of Technology (MIT) Human Dynamics Laboratory.

In the course of the interview, we learn that mobile phone data can be used to predict flue epidemics, the onset of obesity and changes in voting behavior. How? Sandy answers:

We look at the cell phone data and we also pester people with questions.

It turns out that the predictive power of digital data really lies in its integration with active questioning. Sandy and colleagues essentially conduct surveys to gain insight into why people behave the way they do. They ask questions by phone. They ask people to complete diaries. They ask people to weigh themselves. This additional data gives them tremendous insight into what the patterns in the behavioral data really mean. It gives them what Sandy refers to as, “a god’s eye view of human behavior.”

If we want to make effective and efficient use of passively collected digital data for ad targeting or making sense of online word of mouth, then we need to be prepared to emulate Sandy and colleagues.

We need to dig deep to find out why people behave the way they do. Then, once we have established reliable and predictable patterns from the behavioral data, we can downplay the role of direct questioning to the odd confirmatory text or tweet.

So what do you think? Can we make sense of digital data without integrating it with active questioning? If so, how?


Leave a comment
  1. Nigel, June 16, 2011
    Hi Graham,
    Thanks for providing some real evidence to support the proposition. I think this ties back to another, related, topic. Many of the people who "like" a  brand are looking for deals and freebies. Are they really the people you want to listen to?
    Cheers, Nigel
  2. Graham, June 15, 2011
    A current case in point:
    Client's brand has been measured in terms of sentiment in social media vs. its key competitor.  The data says ... both have a similar proportion of positive vs. neutral vs. negative comment.  We also surveyed these two brands.  Attitudes were wildly different, with one having more negative and the other way more positive perceptions - on a rational and an emotional level.

    Most of the online comment was superficial and related to deals.  The real problems with the 'negative' brand surfaced through personal conversations and prompted metrics.  Both sets of data have their place - but it seems to me that you need to do a lot of sifting, filtering, sorting and mining of the 'free comment' to get anywhere near to the truth about how consumers really feel about brands.
  3. Nigel, June 10, 2011
    Hi Jack, thanks for the comment but I have to disagree with you.
    Your rationale makes fine sense if I do not want to change people's behavior. But marketers have two basic objectives: keep a current buyer buying and get non-buyers to buy. Past behavior is a useless measure when it comes to the latter objective. More later...gotta run now.
  4. Nigel, June 07, 2011

    Hi Matt, good point about the siloed approach. It really is debilitating when the two disciplines should be complementary. Structure should follow strategy, as Bob Meyers likes to say, and that applies everything about a business including location. Maybe the reason people are content to let each group pursue a separate agenda is because they don't really have a strategy?

    Hi Phil, yes, you might be employed a while longer but maybe your objective should be to understand patterns in behavioral data so we do not need to ask so many questions?

  5. Jack Vrooman, June 07, 2011
    A consumer's past behavior is a better predictor of future behavior than the reasons or motivations that person ascribes to the behavior. Digital data offers facts. Not enormously useful ones yet, but those interested in selling a product, I think, are more interested in sales growth than a 'God's eye view of human behavior'. While those fighting obesity and trying to figuring out voter behavior should obviously do everything in their power to enhance their data.

    There are a number of problems with the accuracy of respondent generated data. How honest, how insightful and how well intentioned the subjects are, as well as how well conceived the survey is, all leave a great margin of error.

    Ad to that the advances in neuroscience, that I know you are aware of, that suggests much of our decision making  is "preconscious". So if you ask an obese person why they ate two desserts, socially, psychologically and neurologically you really have no idea how accurate her answer will be.

    But you ask the cafe owner, he could care less. And he would have a vested interest in tracking her digitally as well as saying to her, "Have you lost some weight?".
  6. Philip Herr, June 06, 2011
    Hi Nigel, this is a pet issue of mine and I was excited to see you take on the topic. I have been concerned over the past few years that our role as researchers -- uncovering human motivations -- may eventually be replaced by passively collected data. After all, good analysts are able to infer motivation based on multiple behavioral data points. But on further thought I realized that if the entire body of literature and philosophy and most of the social sciences have been grappling with understanding the human condition for so long, how then can behavioral analysts replace the role of direct enquiry? No, I get to be employed for a while longer!
  7. Matt, June 06, 2011
    Whether it's research, marketing, or whatever, when someone says "I only need digital to solve this" I get concerned. Just because people spend a lot of time online, doesn't mean we should invest 100% of our efforts using online tools. 

    Like Professor Pentland did, we should let traditional tools work with digital tools. The issue is that quite often, the digital team sits in a different office or city from the not-as-digital team. But maybe hat can be changed with video conferencing. 

    Leave a comment