Speech production and Perception in Challenging Listening Conditions: Investigating the Effects of Noise, Ear Occlusion, and Hearing Impairement

When we speak, hearing ourselves in the acoustic environment we are in results in a “calibration” of our own speech production. In my research, I investigate this process in various contexts. For example, in noise, we raise our voices “automatically,” a phenomenon known as the Lombard effect. In noisy environments, we often need to wear hearing protection, which attenuates the noise but also creates ear occlusion. Speaking with ear occlusion alters our perception of our own voice and further modifies how we produce speech. In my first project, we study the characteristics of occluded and occluded Lombard speech by extracting acoustic features of vowels and consonants from an existing speech database (SpEAR), where the ear occlusion and noise conditions varied, and analyzing how the features differed. Hearing loss is another important factor to consider in relation to this speech calibration process. In my second project, I am creating a comprehensive database to study the effects of hearing thresholds on speech production in varying noise and ear occlusion levels. Through acoustic and statistical analysis, I will investigate the effect of hearing thresholds and, if possible, determine if we can detect hearing loss from one’s speech production. In addition to audition, vision also plays a role in speech production. In my final project, we investigate the role of vision along with room acoustics and how people integrate these two streams of information. Participants were prompted to speak while being immersed in a visual room with a VR headset and an auditory room with room impressions. There were a total of 9 trials, combining three visually and acoustically distinctive rooms, meaning that the visuals and acoustics may or may not be congruent. We extracted the vowels and measured their sound pressure levels, analyzing the differences in levels in relation to the time course and the audio-visual conditions.