Debating Research Gamification
GreenBook and NewMR hosted today's debate on research gamification. Before the debate officially kicked off, each of the four participants made opening remarks:
- Reg Baker, the "Pensive Pessimist," started off by saying that it's "especially important, given the upheaval in the industry, that we are realistic about which ideas are worth pursuing" and that we have good ways of validating these ideas before taking them to clients.
- Bernie Malinoff, the "Observer," appreciates the challenge gamification poses to larger research organizations, which rely on scalability, cost efficiency and normative databases to do their business. How can we scale creativity and maintain quality?
- Jon Puleston, the "Experimenter," has plenty of his own research to show that gamification is an extremely powerful means of getting someone's attention and getting a consumer to provide more feedback. That said, the technique is in need of more validation, and scalability can be a challenge as well.
- For Betty Adamou, the "Adventurer," the matter comes down to fact that "gamification has proven to increase engagement." This is especially important in the context of falling response rates.
Now on to the main questions, which were crowd-sourced through IdeaScale:
Q: Can games be used to bridge the context gap? One of the big issues that survey research faces is that the contexts in which questions are asked often bear no resemblance to the context someone reaches a decision in. -- Tom Ewing, BrainJuicer
A: According to Jon, there are techniques that need to be evolved, but there are absolutely possibilities. For example, you can simulate impulse purchases by giving consumers time constraints or use projective techniques that ask them to think of a specific scenario to get consumers in the right frame of mind. Reg's view is more conservative; the circumstances around how people make any decision are very complex and difficult to simulate in an online survey. Case in point: virtual shopping. There is some great graphical technology at play, but it is very hard to simulate the real experience of going into a store and picking up a product.
Q: Are surveys just really bad games? -- Peter Harrison, BrainJuicer
A: Betty states that most surveys lack two core pieces of any game: feedback and goals. But adding game mechanics to a bad survey doesn’t make it a good survey! Bernie introduces his position on the 'purposed use of innovation,' in which — rather than ask which technology to use — researchers instead must ask what issue they are trying to address with technology. For example, if your goal is to eradicate straightlining, the research he and Jon have done shows that visual and linguistic changes to the survey can have a significant impact.
Q: Won't the inherent and diverse game incentives affect the recruitment, retention and attrition of players? -- Cam Davis, Vancouver Focus
A: Betty's short answer is "yes." But the longer answer is that it depends on what your respondents actually find useful as an incentive. For example, her studies offer points where consumers can edit their avatars or share graphics with each other, but the incentive itself should really be a bonus. Recognizing consumers for their contribution and giving them feedback on the outcome or goal of the research is more valuable than any incentive.
Q: What are the economics of gamification in a "better, quicker, cheaper" world? -- Olivier de Gaudemar, Ipsos Open Thinking Exchange
A: Jon points out an example in which he's transformed a segmentation into a personality test, and consumers find out their personality type at the end. The set up is not much work, but it improves engagement, as measured by the amount of time consumers spend answering each question. For Reg, the economics come down to the definition of gamification. Jon’s segmentation approach could be standardized and scaled across many surveys. But if your goal is to produce a rich and unique graphical experience, that is a problem in terms of cost and development time. Betty closed out this portion of the debate by boiling gamification down to three components: a goal, rules, and feedback on survey progress. You don't need fancy graphics for that!
After a bonus round, Bernie was declared the winner of the debate (based on listener votes). Here's one of the quotes that probably helped win people over: "The overwhelming majority of interviews online today are the ones we should be focusing on. Ad tests, ad trackers, and concept tests — anything the big firms are doing. We're looking at small studies to understand gaming; we need to shift focus and figure out how to impact the largest volume of surveys today and what low hanging fruit will help us get there."
