CASE STUDY: The Integration of Auditory and Textual Input in Vocabulary Learning from Subtitled Viewing
When you turn on foreign-language subtitles while watching a movie, what is your brain actually doing? Are your eyes tracking every spoken word in real time, or are you just skim-reading text while the soundtrack plays in the background?
A study by Dr. Andi Wang looked to see how viewers integrate what they see with what they hear. By tracking eye movements down to the millisecond, the research reveals how different subtitle formats fundamentally alter our learning strategies.
Testing How People Watch Subtitled Videos in a Foreign Language
The study followed 76 intermediate-to-advanced Chinese learners of English. Participants watched 23 minutes of the English BBC documentary Animal Odd Couples featuring 17 target vocabulary words. Viewers were divided into three groups:
- English Captions: Spoken English paired with written English text.
- Chinese Subtitles (L1): Spoken English paired with Chinese translation.
- Bilingual Subtitles: Spoken English paired with both English captions and Chinese translations on screen.
To see exactly how participants coordinated reading with listening, the researcher used an EyeLink 1000 Plus eye tracker, recording monocular eye gaze at 500 Hz. By pairing high-speed gaze tracking with timestamped audio, the researcher was able to determine whether a viewer’s eyes hit an unfamiliar word in sync with the speech (aligned), before the speech (ahead), after the speech (behind), or skipped it entirely.
Subtitle Language Affects Audio Processing
The type of subtitle on screen completely shifted how viewers processed the audio:
- English Captions Keep You in Sync: Viewers reading English captions looked at unfamiliar words at the exact moment they were spoken 60% of the time—far more than in any other group.
- Native Subtitles Encourage Shortcuts: Viewers using Chinese subtitles frequently read ahead or skipped words entirely. The research suggests that they used native text as a fast lane for plot comprehension rather than connecting speech to meaning.
- Bilingual Subtitles Cause “Split Attention”: While bilingual subtitles sound ideal, viewers spent most of their effort matching Chinese translations to the English audio, skipping the written English line over 51% of the time.
Surprise Finding: Reading Ahead Predicts Better Learning
It might be assumed that reading text in perfect synchronization with spoken audio is the gold standard for learning. However, post-viewing vocabulary tests revealed a surprise: participants who read captions slightly ahead of the audio actually achieved higher vocabulary recognition scores.
Why? The author argues that proficient learners often scan upcoming text to build a quick mental framework. When the spoken audio arrives a split second later, it confirms what they just read, strengthening the memory link.
The Takeaway for Learners of English as a Second Language
If the goal is simply to enjoy a foreign English film, native subtitles work great. However, if you want to expand your vocabulary, English captions are best. Reading target text directly forces your brain to build strong connections between spoken sounds and written forms
有关眼动追踪如何帮助您的研究的信息，请查看我们的解决方案和产品页面或联系我们。我们很乐意为您提供帮助！