Objective: The ability to segregate competing auditory streams is a necessary prerequisite to focus attention on the target speech during speech-on-speech masking. Stream build-up typically takes place across the duration of a few seconds. The Visual World Paradigm (VWP) is considered as a powerful tool for auditory research enabling gaze fixation acquisition at millisecond level during the time course of speech processing. The aim of this study was to introduce a new VWP to capture the time course of speech-on-speech segregation when competing sentences are presented and to collect pupil responses as a measure of listening effort.

Procedure: Young adult participants with normal hearing (n=12) and normal or corrected-to-normal vision were recruited. A matrix sentence test was used as speech material. These sentences have the structure “name-verb-number-adjective-object.” In each trial a target and a masker sentence were diotically presented via headphones. In parallel the VWP visually presented the number and the object word of both the target and the masker sentences. Participants were instructed to focus their gaze to the number and the object belonging to the target sentence. Stimuli were presented at different target-to-masker ratios (TMR) of 0, 2.5, 4.5, and 6.5 dB. Additionally, speech recognition performance was determined in an offline experiment to compare the results with the gaze fixations and pupil dilations.

Results: The gaze fixations consistently reflected the different TMRs on both the number and the object word. Slopes of the fixation curves were steeper and the proportion of target fixations was higher for larger TMRs, reflecting that segregation of the competing sentences was more efficient in the favourable conditions. Furthermore, temporal analysis of pupillometry using series of Bayesian paired sample t-tests showed a reduction in listening effort with increasing TMR. This was also true for the conditions where target speech recognition was nearly perfect.

Conclusions: Our results reveal that the proposed VWP is generally suited for an objective assessment of sentence-based speech-on-speech segregation and corresponding listening effort. Since it captures speech processing on a fine-grained temporal level it gives information that is not provided by offline performance measures.