Multisensory Integration in Music Perception: How Visual Stimuli Influence the Perception of Ambiguous Auditory Rhythms

Project Title

Multisensory Integration in Music Perception: How Visual Stimuli Influence the Perception of Ambiguous Auditory Rhythms

Faculty Mentor(s)

Project Description

Music perception relies on the integration of information from different sensory modalities. Although rhythm perception has traditionally been studied as an auditory dominant process, visual information also contributes to how rhythmic patterns are perceived. This project investigates how visual stimuli influences the perception of ambiguous auditory rhythms. We hypothesize that visual cues have a greater influence on rhythm perception when auditory rhythms are ambiguous rather than when they are unambiguous.Participants are presented with audiovisual stimuli consisting of clear and ambiguous rhythmic patterns. Electroencephalography (EEG) is used to measure neural activity during the task. Behavioral responses and EEG recordings are analyzed to examine perceptual judgments, neural entrainment, and their possible neural correlates. By integrating behavioral and neural measures, this research aims to provide broader insight into how the brain integrates information across sensory modalities during rhythm perception.

 

Why is your research important?

Music perception matters to general public since it is a real life experience that we are experience but rarely asked why or focus on the mechanism behind it. By investigating the multisensory integration of music perception, we got a chance to better understand how humans are able to get information from the outside world and form our own perception. It’s about music but never just limited to this, it’s one of the small steps forward to the goals of better understanding how our brain works with the mind and world.
 

What does the process of doing your research look like?

I am responsible for building the experiment, testing participants, and doing the data analysis. 

What knowledge has your research contributed to your field?

This project focuses on a specific situation when the dominant sensory modality becomes ambiguous or not that reliable, and studies how the brain responds and forms our perception based on other sensory information to compensate. By pairing the behavior results with EEG data, we aimed to get a better understanding of how the brain integrates information across different sensory modalities under this situation.

How did you get involved in research? What drove you to seek out research experiences in college?

I first started with a really good experience through the STRONG program in Oberlin College, which introduced me to scientific research and made me feel comfortable pursuing it. The community and experience was amazing, and it gave me a view of what I wanted to have more of in my college life even with so many accessible resources on campus. 

What is your favorite aspect of the research?

Troubleshooting is a bittersweet journey, but research is essentially a continuous series of troubleshooting cycles. Whether we are designing studies, building experimental setups, analyzing data, or interacting with participants, we are constantly met with challenges. It might be hardware glitches, programming bugs, or the steep learning curves of new knowledge. While troubleshooting can be demanding, the sense of accomplishment when you finally overcome the challenge is truly unparalleled.

What advice would you give to a younger student wanting to get involved in research in your field?

 If there is one piece of advice I can give, it is this: never assume you aren’t qualified. 

Coming from a first-generation background, my entry into the research world was filled with anxiety about worries that I lacked the experience or capability to earn an opportunity. But the reality is, if you are patient, passionate, and willing to put effort into the field, you will realize that resources and people around you are willing to support you more than you thought.

That is exactly how I started my research journey at Oberlin, where I found my community. I got incredible help and support from my mentor Professor Leslie Kwakye, not only in research but also in my personal growth.

The very first step always seems like the steepest mountain to climb, but just treat it as your very first troubleshooting practice on this path. Once you overcome this, everything else follows.