14th Speech in Noise Workshop, 12-13 January 2023, Split, Croatia 14th Speech in Noise Workshop, 12-13 January 2023, Split, Croatia

P12Session 2 (Friday 13 January 2023, 09:00-11:00)
Neural mechanisms of audio-tactile speech integration

Pierre Guilleminot
Imperial College London, UK

Tobias Reichenbach
Imperial College London, UK
FAU Nuremberg-Erlangen, Germany

Speech is organized hierarchically, from phonemes to syllables, words and sentences. To extract semantic content, its processing requires segmentation based on those tokens. In particular, it relies on neural oscillations in the delta and theta frequency ranges (1-4Hz and 4-8Hz) in the auditory cortex, which track the rhythm of syllables. Using tactile stimulation, the neural activity in these frequency bands can be modulated and therefore potentially affect speech processing, similarly to the way speech comprehension has been found to be affected by transcranial current stimulation.

Here, we used vibrotactile pulses to modulate speech comprehension (Guilleminot and Reichenbach, 2022, doi:10.1073/pnas.2117000119). The pulses were aligned to the centres of syllables, with different delays between auditory and tactile streams. We assessed speech comprehension using semantically unpredictable sentences. The neural encoding of speech and tactile pulses were measured using electroencephalographic recordings (EEG) during continuous speech.

The tactile stimulation modulated speech comprehension in a manner that showed an approximate sinusoidal dependence on the time lag. When the auditory and tactile streams were in synchrony, we found significant speech comprehension enhancement compared to an audio-only and sham conditions. Similarly to these behavioural findings, we observed that the audio-tactile stimulation modulated the neural responses to both tactile and speech streams, also in a sinusoidal manner depending on the delay of the tactile stimuli. Finally, we demonstrated that the comfort at which subjects reported that they could understand the sentences was correlated with electrophysiological markers of audiotactile integration in the auditory cortex.

Our results therefore provide evidence of the link between neural oscillations in theta range and the syllable rate as well as the possibility of using vibrotactile pulses to modulate and enhance speech-in-noise comprehension.

Last modified 2023-01-06 23:41:06