ESOMAR & the Gamesters of Irrationality
We're counting down the Top 10 posts of 2011 on "Innovation Evolved". Originally published September 21, here's #10 on the countdown.
The themes of this year’s ESOMAR Annual Congress, which concluded today, are best represented by two of the winning papers: “The Game Experiments”, by Jon Puleston of GMI and Deborah Sleep of Engage Research; and “Research in a World of Irrational Expectations”, by Stephen Philips and Abigail Hill of Spring Research. References to consumer irrationality and gamification ran through many of the presentations, in fact, starting with the keynote by Richard Wiseman.
Richard Wiseman, a professor at the University of Hertsfordshire and author of Quirkology and many other books, kicked off the congress discussing our failure to recognize our own perceptual limits, and how significant those limits can be. “We are tripping ourselves up perceptually. We think we have a good perception system but we are making assumptions.” Optical illusions provide great ways to showcase those assumptions, and Richard shared many illusions, including this great video: Colour-Changing Card Trick. In effect, people don’t always see what they should have.
For those new to behavioral economics, Orlando Wood of BrainJuicer presented a great review of the many studies that have revealed our “bounded rationality”. Highlighting the disconnect between the rational brain and the irrational brain, the hit Heineken commercial “Walk-In Fridge” was almost rejected at the storyboard stage because consumers didn’t rate it high enough. For that ad, perhaps it was because social desirability bias makes respondents reluctant to acknowledge gender differences. Realizing that surveys might not be the best way to evaluate story boards of potential commercials, Heineken and TNS decided to test a “bioquali” approach, and found that biometrics (neuroscience and eye tracking) proved complementary to surveys, offering new insights when assessing ads.
In another presentation, Kyle Findlay and Kirsty Alberts from TNS in South Africa presented “Gamification”. They found that those who participated in games were more likely to comment and were preconditioned to be more positive. They weren’t the only ones to find that gamification changed the answers of surveys, and not always in predictable ways. Theo Downes-Le Guin of Market Strategies International presented “All fun and games? Myth and realities of respondent engagement in online surveys”. He tested a truly gamified survey – respondents first created a fantasy roleplaying game avatar and selected a fantasy world to explore and then were rewarded with accoutrements for answering questions; completing the survey presented them with an image of their avatar with all its accessories. Theo had hypothesized that this UI would be alienating to many respondents, yet most respondents who abandoned simply did so because they didn’t wait for the Flash file to load in their browser; there was in fact no material difference in the types or attitudes of people who answered the gamified survey vs. more traditional web surveys.
For a survey researcher, what was great about this year’s ESOMAR was the integration of survey methodologies into many of the papers, recognizing the strengths and limits of surveys and countering them with other methodologies or new approaches. This was all brought to a culmination with Jon Puleston’s presentation on his winning paper, which is a must-read for every survey researcher. In a way, GMI is using irrationality on respondents themselves, giving them small nudges that produce much richer feedback. Gamification can take many forms, and Jon and Deborah have tested many of these forms. For instance, games often have arbitrary rules. In a survey, a gamified question with an arbitrary rule may say: “Please describe yourself in exactly 7 words”. In Jon and Deborah's research, this kind of rule produced much more thoughtful answers than more traditional wording. They also tested question-based forms, including betting games, exercises with timers, a Space Invaders-style shooter and a ski game. Not every experiment worked (the ski game was too hard for respondents), and those that did work sometimes produced materially different answers (so don’t fold them into tracking studies). But most worked. Their overall conclusion from their research? “More fun = more feedback”.
Words any market researcher can rally around!
