Adding Video and Subtitles to an Audio Lesson

If you are creating an educational video or online lecture, it may be tempting to load the screen with dynamic background footage, animations, and graphics. The logic seems simple: more visual stimulation equals better engagement. However, a study by Yueyuan Zheng, Xinchen Ye, and Janet H. Hsiao challenges this assumption. Their research proves that when visual media isn’t essential to the lesson, extra video graphics can actually hurt comprehension by scattering the viewer’s attention. Instead, the authors found that adding subtitles can boost memory for the video content.

Eye Tracking Methodology to Test the Impact of Video and Subtitles in Learning

The researchers ran experiments testing how native Mandarin university students absorbed 75-second factual lessons on topics like astronomy, geography, and ecology. Participants learned under four distinct conditions:

1. Audio-Only: Spoken lesson paired with a blank screen.
2. Audio + Subtitles: Spoken lesson paired with verbatim text.
3. Audio + Video: Spoken lesson paired with relevant full-screen background video.
4. Audio + Video + Subtitles: Spoken lesson paired with both background video and subtitles.

To understand why some students learned better than others, the team tracked participants’ eye movements using an EyeLink 1000 system, sampling gaze at 1000 Hz, while participants completed memory tests and cognitive assessments.

Subtitles Help, Video Distracts Learning

When analyzing student test scores, the researchers found two clear trends:

• Subtitles Boost Memory: Adding verbatim subtitles to an audio lesson significantly improved test performance. Seeing the written text reinforced spoken facts.
• Background Video Adds No Value: Adding background video clips provided no learning benefit over listening to the audio alone. The team argue that because the key facts were in the narration, extra video footage acted merely as visual noise.

Eye Movements Predict Comprehension

Using advanced machine-learning models to process the EyeLink gaze data, the researchers discovered two distinct viewing strategies among participants:

• Centralized Viewers: Kept their gaze anchored near the center of the screen or directly on the subtitle line.
• Distributed Viewers: Allowed their eyes to wander across peripheral regions and other background graphics.

The results were striking: Centralized Viewers scored significantly higher on comprehension tests. When viewers allowed their eyes to explore decorative video elements, they expended mental energy that should have been used to process the spoken lecture. Crucially, the eye-tracking data showed that adding subtitles helped stabilize visual behavior, anchoring viewers’ eyes to a consistent spot and preventing their attention from wandering.

The Takeaway for Educators and Creators

According to the study authors, when designing audio-driven lectures or informational presentations, the best advice is to keep visuals intentional:

Guide the viewer’s gaze: Encourage learners to maintain a focused, centralized visual strategy during online lectures.

Include verbatim subtitles: Text reinforces spoken facts and anchors visual attention.

Cut decorative background video: Unless video footage carries essential information that cannot be explained in speech, extra visual fluff steals mental bandwidth.

有关眼动追踪如何帮助您的研究的信息，请查看我们的解决方案和产品页面或联系我们。我们很乐意为您提供帮助！