Under cognitively demanding listening situations, listeners may expend more listening effort (LE) but only if they are motivated to achieve listening success. We aimed to evaluate aspects of gamification (by using performance-based financial rewards, financial penalties and feedback) as agents to motivate young (18-35 years) listeners with normal hearing to exert LE during a speech recognition task.

In addition to a standard remuneration, participants were given £5 at the start of the speech recognition task and informed that correct responses would be rewarded via financial gain whereas incorrect responses would incur a financial penalty. Feedback on performance (i.e. correct/incorrect responses) and the amount of financial reward retained was provided at the end of each trial. The listening demands of the speech recognition task were manipulated using prior knowledge (perceptual ‘pop out’) of tone-vocoded speech sentences. We measured multimodal LE outcomes on a trial-by-trial basis, including performance accuracy on the speech task (correct response rates), response times (RTs), self-rated work and pupil diameter (peak pupil dilation, PPD).

Results revealed that high listening demands decreased correct response rates as well as PPD, and increased RTs and self-rated work. High motivation (high rewards/penalties) increased self-reported listening effort, PPD and RTs but had no effect on correct response rates. Motivation had no effect on self-rated fatigue measured at the end of each block of trials. Interactions between listening demands and motivation were not found.

The use of game design elements in our novel speech recognition task captured the effects of motivation and listening demands on multimodal measures of listening effort. The operationalisation of financial reward through performance-based rewards/penalties and feedback on a trial-by-trial basis may be a more effective and robust motivational agent than simply offering financial rewards based on achieving a target performance level over a block of trials. Application of gamification elements to “conventional” speech recognition tasks may improve listener experience and engagement, highlighting the importance of listener motivation in LE research.