全部EyeLink出版物
以下按年份列出了截至2024年（2025年初）的所有13000多篇同行评审的EyeLink研究出版物。您可以使用视觉搜索、平滑追求、帕金森氏症等关键字搜索出版物库。您还可以搜索单个作者的姓名。按研究领域分组的眼动追踪研究可以在解决方案页面上找到。如果我们错过了任何EyeLink眼动追踪论文，请给我们发电子邮件！
2025 |
Masataka Yano; Keiyu Niikuni; Ruri Shimura; Natsumi Funasaki; Masatoshi Koizumi Producing non-basic word orders in (in)felicitous contexts: Evidence from pupillometry and functional near-infrared spectroscopy (fNIRS) Journal Article In: Language, Cognition and Neuroscience, vol. 40, no. 1, pp. 1–22, 2025. @article{Yano2025,The present study examined why speakers of languages with flexible word orders are more likely to use syntactically complex non-basic word orders when they provide discourse-given information earlier in sentences. This may be because they are more efficient for speakers to produce (the Speaker Economy Hypothesis). Alternatively, speakers may produce them to help listeners understand sentences more efficiently (the Listener Economy Hypothesis), given that previous studies showed that the processing of non-basic word orders was facilitated when the felicitous context was provided (i.e. a displaced object refers to discourse-given information). We addressed this issue by conducting a picture-description experiment, in which participants uttered sentences with syntactically basic Subject-Object-Verb (SOV) or non-basic Object-Subject-Verb (OSV) in felicitous or infelicitous contexts while cognitive load was tracked using pupillometry and functional near-infrared spectroscopy. The results showed that the felicitous context facilitated the filler-gap dependency formation of OSVs in production, supporting the Speaker Economy Hypothesis. |
Lukas Suveg; Tanvi Thakkar; Emily Burg; Shelly P. Godar; Daniel Lee; Ruth Y. Litovsky The relationship between spatial release from masking and listening effort among cochlear implant users with single-sided deafness Journal Article In: Ear & Hearing, pp. 1–16, 2025. @article{Suveg2025,Objectives: To examine speech intelligibility and listening effort in a group of patients with single-sided deafness (SSD) who received a cochlear implant (CI). There is limited knowledge on how effectively SSD-CI users can integrate electric and acoustic inputs to obtain spatial hearing benefits that are important for navigating everyday noisy environments. The present study examined speech intelligibility in quiet and noise simultaneously with measuring listening effort using pupillometry in individuals with SSD before, and 1 year after, CI activation. The study was designed to examine whether spatial separation between target and interfering speech leads to improved speech understanding (spatial release from masking [SRM]), and is associated with a decreased effort (spatial release from listening effort [SRE]) measured with pupil dilation (PPD). Design: Eight listeners with adult-onset SSD participated in two visits: (1) pre-CI and (2) post-CI (1 year after activation). Target speech consisted of Electrical and Electronics Engineers sentences and masker speech consisted of AzBio sentences. Outcomes were measured in three target-masker configurations with the target fixed at 0° azimuth: (1) quiet, (2) co-located target/maskers, and (3) spatially separated (±90° azimuth) target/maskers. Listening effort was quantified as change in peak proportional PPD on the task relative to baseline dilation. Participants were tested in three listening modes: acoustic-only, CI-only, and SSD-CI (both ears). At visit 1, the acoustic-only mode was tested in all three target-masker configurations. At visit 2, the acoustic-only and CI-only modes were tested in quiet, and the SSD-CI listening mode was tested in all three target-masker configurations. Results: Speech intelligibility scores in quiet were at the ceiling for the acoustic-only mode at both visits, and in the SSD-CI listening mode at visit 2. In quiet, at visit 2, speech intelligibility scores were significantly worse in the CI-only listening modes than in all other listening modes. Comparing SSD-CI listening at visit 2 with pre-CI acoustic-only listening at visit 1, speech intelligibility scores for co-located and spatially separated configurations showed a trend toward improvement (higher scores) that was not significant. However, speech intelligibility was significantly higher in the separated compared with the co-located configuration in acoustic-only and SSD-CI listening modes, indicating SRM. PPD evoked by speech presented in quiet was significantly higher with CI-only listening at visit 2 compared with acoustic-only listening at visit 1. However, there were no significant differences between co-located and spatially separated configurations on PPD, likely due to the variability among this small group of participants. There was a negative correlation between SRM and SRE, indicating that improved speech intelligibility with spatial separation of target and masker is associated with a greater decrease in listening effort on those conditions. Conclusions: The small group of patients with SSD-CI in the present study demonstrated improved speech intelligibility from spatial separation of target and masking speech, but PPD measures did not reveal the effects of spatial separation on listening effort. However, there was an association between the improvement in speech intelligibility (SRM) and the reduction in listening effort (SRE) from spatial separation of target and masking speech. |
Jens Schmidtke; Dana Bsharat-Maalouf; Tamar Degani; Hanin Karawani How lexical frequency, language dominance and noise affect listening effort–insights from pupillometry insights from pupillometry Journal Article In: Language, Cognition and Neuroscience, vol. 40, no. 2, pp. 195–208, 2025. @article{Schmidtke2025a,Acoustic, listener, and stimulus-related factors modulate speech-in-noise processes. This study examined how noise, listening experience, manipulated at two levels, native [L1] vs. second language [L2], and lexical frequency impact listening effort. Forty-seven participants, tested in their L1 Hebrew and L2 English, completed a word recognition test in quiet and noisy conditions while pupil size was recorded to assess listening effort. Results showed that listening in L2 was overall more effortful than in L1, with frequency effects modulated by language and noise. In L1, pupil responses to high and low frequency words were similar in both conditions. In L2, low frequency words elicited a larger pupil response, indicating greater effort, but this effect vanished in noise. A time-course analysis of the pupil response suggests that L1–L2 processing differences occur during lexical selection, indicating that L2 listeners may struggle to match acoustic-phonetic signals to long-term memory representations. |
Sharif I. Kronemer; Victoria E. Gobo; Catherine R. Walsh; Joshua B. Teves; Diana C. Burk; Somayeh Shahsavarani; Javier Gonzalez; Castillo Peter Cross‑species real‑time detection of trends in pupil size fluctuation Journal Article In: Behavior Research Methods, vol. 57, no. 1, pp. 1–14, 2025. @article{Kronemer2025,Pupillometry is a popular method because pupil size is easily measured and sensitive to central neural activity linked to behavior, cognition, emotion, and perception. Currently, there is no method for online monitoring phases of pupil size fluctuation. We introduce rtPupilPhase—an open-source software that automatically detects trends in pupil size in real time. This tool enables novel applications of real-time pupillometry for achieving numerous research and translational goals. We validated the performance of rtPupilPhase on human, rodent, and monkey pupil data, and we propose future implementations of real-time pupillometry. |
Magdalena Gruner; Andreas Widmann; Stefan Wöhner; Erich Schröger; Jörg D. Jescheniak Semantic context effects in picture and sound naming: Evidence from event-related potentials and pupillometric data Journal Article In: Journal of cognitive neuroscience, vol. 37, no. 2, pp. 443–463, 2025. @article{Gruner2025,When a picture is repeatedly named in the context of semantically related pictures (homogeneous context), responses are slower than when the picture is repeatedly named in the context of unrelated pictures (heterogeneous context). This semantic interference effect in blocked-cyclic naming plays an important role in devising theories of word production. Wöhner, Mädebach, and Jescheniak [Wöhner, S., Mädebach, A., & Jescheniak, J. D. Naming pictures and sounds: Stimulus type affects semantic context effects. Journal of Experimental Psychology: Human Perception and Performance, 47, 716-730, 2021] have shown that the effect is substantially larger when participants name environmental sounds than when they name pictures. We investigated possible reasons for this difference, using EEG and pupillometry. The behavioral data replicated Wöhner and colleagues. ERPs were more positive in the homogeneous compared with the heterogeneous context over central electrode locations between 140-180 msec and 250-350 msec for picture naming and between 250 and 350 msec for sound naming, presumably reflecting semantic interference during semantic and lexical processing. The later component was of similar size for pictures and sounds. ERPs were more negative in the homogeneous compared with the heterogeneous context over frontal electrode locations between 400 and 600 msec only for sounds. The pupillometric data showed a stronger pupil dilation in the homogeneous compared with the heterogeneous context only for sounds. The amplitudes of the late ERP negativity and pupil dilation predicted naming latencies for sounds in the homogeneous context. The latency of the effects indicates that the difference in semantic interference between picture and sound naming arises at later, presumably postlexical processing stages closer to articulation. We suggest that the processing of the auditory stimuli interferes with phonological response preparation and self-monitoring, leading to enhanced semantic interference. |
Joshua O. Eayrs; Haya Serena Tobing; S. Tabitha Steendam; Nicoleta Prutean; Wim Notebaert; Jan R. Wiersema; Ruth M. Krebs; C. Nico Boehler Reward and efficacy modulate the rate of anticipatory pupil dilation Journal Article In: Psychophysiology, vol. 62, no. 1, pp. 1–12, 2025. @article{Eayrs2025,Pupil size is a well-established marker of cognitive effort, with greater efforts leading to larger pupils. This is particularly true for pupil size during task performance, whereas findings on anticipatory effort triggered by a cue stimulus are less consistent. For example, a recent report by Frömer et al. found that in a cued-Stroop task, behavioral performance and electrophysiological markers of preparatory effort allocation were modulated by cued reward and ‘efficacy' (the degree to which rewards depended on good performance), but pupil size did not show a comparable pattern. Here, we conceptually replicated this study, employing an alternative approach to the pupillometry analyses. In line with previous findings, we found no modulation of absolute pupil size in the cue-to-target interval. Instead, we observed a significant difference in the rate of pupil dilation in anticipation of the target: pupils dilated more rapidly for high-reward trials in which rewards depended on good performance. This was followed by a significant difference in absolute pupil size within the first hundreds of milliseconds following Stroop stimulus onset, likely reflecting a lagging effect of anticipatory effort allocation. Finally, the slope of pupil dilation was significantly correlated with behavioral response times, and this association was strongest for the high-reward, high-efficacy trials, further supporting that the rate of anticipatory pupil dilation reflects anticipatory effort. We conclude that pupil size is modulated by anticipatory effort, but in a highly temporally-specific manner, which is best reflected by the rate of dilation in the moments just prior to stimulus onset. |
Yi Hsuan Chang; Rachel Yep; Chin An Wang In: Psychophysiology, vol. 62, pp. 1–22, 2025. @article{Chang2025,Pupil size is a non-invasive index for autonomic arousal mediated by the locus coeruleus–norepinephrine (LC-NE) system. While pupil size and its derivative (velocity) are increasingly used as indicators of arousal, limited research has investigated the relationships between pupil size and other well-known autonomic responses. Here, we simultaneously recorded pupillometry, heart rate, skin conductance, pulse wave amplitude, and respiration signals during an emotional face–word Stroop task, in which task-evoked (phasic) pupil dilation correlates with LC-NE responsivity. We hypothesized that emotional conflict and valence would affect pupil and other autonomic responses, and trial-by-trial correlations between pupil and other autonomic responses would be observed during both tonic and phasic epochs. Larger pupil dilations, higher pupil size derivative, and lower heart rates were observed in the incongruent condition compared to the congruent condition. Additionally, following incongruent trials, the congruency effect was reduced, and arousal levels indexed by previous-trial pupil dilation were correlated with subsequent reaction times. Furthermore, linear mixed models revealed that larger pupil dilations correlated with higher heart rates, higher skin conductance responses, higher respiration amplitudes, and lower pulse wave amplitudes on a trial-by-trial basis. Similar effects were seen between positive and negative valence conditions. Moreover, tonic pupil size before stimulus presentation significantly correlated with all other tonic autonomic responses, whereas tonic pupil size derivative correlated with heart rates and skin conductance responses. These results demonstrate a trial-by-trial relationship between pupil dynamics and other autonomic responses, highlighting pupil size as an effective real-time index for autonomic arousal during emotional conflict and valence processing. |
Khaled H. A. Abdel-Latif; Thomas Koelewijn; Deniz Başkent; Hartmut Meister Assessment of speech processing and listening effort associated with speech-on-speech masking using the visual world paradigm and pupillometry Journal Article In: Trends in hearing, vol. 29, pp. 1–13, 2025. @article{AbdelLatif2025,Speech-on-speech masking is a common and challenging situation in everyday verbal communication. The ability to segregate competing auditory streams is a necessary requirement for focusing attention on the target speech. The Visual World Paradigm (VWP) provides insight into speech processing by capturing gaze fixations on visually presented icons that reflect the speech signal. This study aimed to propose a new VWP to examine the time course of speech segregation when competing sentences are presented and to collect pupil size data as a measure of listening effort. Twelve young normal-hearing participants were presented with competing matrix sentences (structure "name-verb-numeral-adjective-object") diotically via headphones at four target-to-masker ratios (TMRs), corresponding to intermediate to near perfect speech recognition. The VWP visually presented the number and object words from both the target and masker sentences. Participants were instructed to gaze at the corresponding words of the target sentence without providing verbal responses. The gaze fixations consistently reflected the different TMRs for both number and object words. The slopes of the fixation curves were steeper, and the proportion of target fixations increased with higher TMRs, suggesting more efficient segregation under more favorable conditions. Temporal analysis of pupil data using Bayesian paired sample t-tests showed a corresponding reduction in pupil dilation with increasing TMR, indicating reduced listening effort. The results support the conclusion that the proposed VWP and the captured eye movements and pupil dilation are suitable for objective assessment of sentence-based speech-on-speech segregation and the corresponding listening effort. |
2024 |
Fangshu Yao; Xiaoyue Chang; Bin Zhou; Wen Zhou Olfaction modulates cortical arousal independent of perceived odor intensity and pleasantness Journal Article In: NeuroImage, vol. 299, pp. 1–11, 2024. @article{Yao2024a,Throughout history, various odors have been harnessed to invigorate or relax the mind. The mechanisms underlying odors' diverse arousal effects remain poorly understood. We conducted five experiments (184 participants) to investigate this issue, using pupillometry, electroencephalography, and the attentional blink paradigm, which exemplifies the limit in attentional capacity. Results demonstrated that exposure to citral, compared to vanillin, enlarged pupil size, reduced resting-state alpha oscillations and alpha network efficiency, augmented beta-gamma oscillations, and enhanced the coordination between parietal alpha and frontal beta-gamma activities. In parallel, it attenuated the attentional blink effect. These effects were observed despite citral and vanillin being comparable in perceived odor intensity, pleasantness, and nasal pungency, and were unlikely driven by semantic biases. Our findings reveal that odors differentially alter the small-worldness of brain network architecture, and thereby brain state and arousal. Furthermore, they establish arousal as a unique dimension in olfactory space, distinct from intensity and pleasantness. |
Mario A. Svirsky; Jonathan D. Neukam; Nicole Hope Capach; Nicole M. Amichetti; Annette Lavender; Arthur Wingfield Communication under sharply degraded auditory input and the “2-sentence” problem Journal Article In: Ear & Hearing, vol. 45, no. 4, pp. 1045–1058, 2024. @article{Svirsky2024,Objectives: Despite performing well in standard clinical assessments of speech perception, many cochlear implant (CI) users report experiencing significant difficulties when listening in real-world environments. We hypothesize that this disconnect may be related, in part, to the limited ecological validity of tests that are currently used clinically and in research laboratories. The challenges that arise from degraded auditory information provided by a CI, combined with the listener's finite cognitive resources, may lead to difficulties when processing speech material that is more demanding than the single words or single sentences that are used in clinical tests. Design: Here, we investigate whether speech identification performance and processing effort (indexed by pupil dilation measures) are affected when CI users or normal-hearing control subjects are asked to repeat two sentences presented sequentially instead of just one sentence. Results: Response accuracy was minimally affected in normal-hearing listeners, but CI users showed a wide range of outcomes, from no change to decrements of up to 45 percentage points. The amount of decrement was not predictable from the CI users' performance in standard clinical tests. Pupillometry measures tracked closely with task difficulty in both the CI group and the normal-hearing group, even though the latter had speech perception scores near ceiling levels for all conditions. Conclusions: Speech identification performance is significantly degraded in many (but not all) CI users in response to input that is only slightly more challenging than standard clinical tests; specifically, when two sentences are presented sequentially before requesting a response, instead of presenting just a single sentence at a time. This potential “2-sentence problem” represents one of the simplest possible scenarios that go beyond presentation of the single words or sentences used in most clinical tests of speech perception, and it raises the possibility that even good performers in single-sentence tests may be seriously impaired by other ecologically relevant manipulations. The present findings also raise the possibility that a clinical version of a 2-sentence test may provide actionable information for counseling and rehabilitating CI users, and for people who interact with them closely. |
Yanliang Sun; Lixue Wang; Wenhao Yu; Xue Yang; Jiaru Song; Shouxin Li Mechanisms of visual working memory processing task-irrelevant information retrieved from visual long-term memory Journal Article In: Cognition, vol. 250, pp. 1–10, 2024. @article{Sun2024d,Visual working memory (VWM) can selectively filter task-irrelevant information from incoming visual stimuli. However, whether a similar filtering process applies to task-irrelevant information retrieved from visual long-term memory (VLTM) remains elusive. We assume a “resource-limited retrieval mechanism” in VWM in charge of the retrieval of irrelevant VLTM information. To make a comprehensive understanding of this mechanism, we conducted three experiments using both a VLTM learning task and a VWM task combined with pupillometry. The presence of a significant pupil light response (PLR) served as empirical evidence that VLTM information can indeed make its way into VWM. Notably, task-relevant VLTM information induced a sustained PLR, contrasting with the transient PLR observed for task-irrelevant VLTM information. Importantly, the transience of the PLR occurred under conditions of low VWM load, but this effect was absent under conditions of high load. Collectively, these results show that task-irrelevant VLTM information can enter VWM and then fade away only under conditions of low VWM load. This dynamic underscores the resource-limited retrieval mechanism within VWM, exerting control over the entry of VLTM information. |
S. Tabitha Steendam; Nicoleta Prutean; Fleur Clybouw; Joshua O. Eayrs; Nanne Kukkonen; Wim Notebaert; Ruth M. Krebs; Jan R. Wiersema; C. Nico Boehler Compensating for the mobile menace with extra effort: A pupillometry investigation of the mere presence effect of smartphones Journal Article In: Biological Psychology, vol. 193, pp. 1–11, 2024. @article{Steendam2024,Previous research suggests that the mere presence of a smartphone can detrimentally affect performance. However, other studies failed to observe such detrimental effects. A limitation of existing studies is that no indexes of (potentially compensating) effort were included. Further, time-on-task effects have been unexplored. Here, we address these limitations by investigating the mere-presence effect of a smartphone on performance in two continuous-performance experiments (Experiment 1 using an n-back and a number judgement task at two difficulty levels, and Experiment 2 using a pure, challenging n-back task), measuring pupil size to assess invested effort, and taking into account time-on-task effects. Finally, contrary to previous studies that predominantly used between-subject designs, we utilized within-subject designs in both experiments. Contrary to expectations, Experiment 1 largely yielded no significant effects of smartphone presence on performance. Nonetheless, the presence of a smartphone triggered larger tonic pupil size in the more difficult task, and a more rapid decrease over time. Experiment 2 similarly failed to demonstrate smartphone effects on performance, but replicated the finding of larger tonic pupil size in the presence of a smartphone. In addition, tonic pupil size showed a slower decrease over time when a smartphone was present. In Experiment 2, we could furthermore look at phasic pupil size, which decreased over time in the absence of a phone but not in its presence. These findings suggest a complex relationship between smartphone presence, effort, and time-on-task, which does not necessarily express itself behaviorally, highlighting in particular the need to also explore potential contributions of (compensatory) effort. |
Connor Spiech; Anne Danielsen; Bruno Laeng; Tor Endestad Oscillatory attention in groove Journal Article In: Cortex, vol. 174, pp. 137–148, 2024. @article{Spiech2024,Attention is not constant but rather fluctuates over time and these attentional fluctuations may prioritize the processing of certain events over others. In music listening, the pleasurable urge to move to music (termed ‘groove' by music psychologists) offers a particularly convenient case study of oscillatory attention because it engenders synchronous and oscillatory movements which also vary predictably with stimulus complexity. In this study, we simultaneously recorded pupillometry and scalp electroencephalography (EEG) from participants while they listened to drumbeats of varying complexity that they rated in terms of groove afterwards. Using the intertrial phase coherence of the beat frequency, we found that while subjects were listening, their pupil activity became entrained to the beat of the drumbeats and this entrained attention persisted in the EEG even as subjects imagined the drumbeats continuing through subsequent silent periods. This entrainment in both the pupillometry and EEG worsened with increasing rhythmic complexity, indicating poorer sensory precision as the beat became more obscured. Additionally, sustained pupil dilations revealed the expected, inverted U-shaped relationship between rhythmic complexity and groove ratings. Taken together, this work bridges oscillatory attention to rhythmic complexity in relation to musical groove. |
Blanca T. M. Spee; Jozsef Arato; Jan Mikuni; Ulrich S. Tran; Matthew Pelowski; Helmut Leder In: Frontiers in Psychology, vol. 15, pp. 1–15, 2024. @article{Spee2024,Introduction: Gestalt perception refers to the cognitive ability to perceive various elements as a unified whole. In our study, we delve deeper into the phenomenon of Gestalt recognition in visual cubist art, a transformative process culminating in what is often described as an Aha moment. This Aha moment signifies a sudden understanding of what is seen, merging seemingly disparate elements into a coherent meaningful picture. The onset of this Aha moment can vary, either appearing almost instantaneously, which is in line with theories of hedonic fluency, or manifesting after a period of time, supporting the concept of delayed but more in-depth meaningful insight. Methods: We employed pupillometry to measure cognitive and affective shifts during art interaction, analyzing both maximum pupil dilation and average dilation across the trial. The study consisted of two parts: in the first, 84 participants identified faces in cubist paintings under various conditions, with Aha moments and pupil dilation measured. In part 2, the same 84 participants assessed the artworks through ratings in a no-task free-viewing condition. Results: Results of part 1 indicate a distinctive pattern of pupil dilation, with maximum dilation occurring at both trial onset and end. Longer response times were observed for high-fluent, face-present stimuli, aligning with a delayed but accurate Aha-moment through recognition. Additionally, the time of maximum pupil dilation, rather than average dilation, exhibited significant associations, being later for high-fluent, face-present stimuli and correct detections. In part 2, average, not the time of maximum pupil dilation emerged as the significant factor. Face-stimuli and highly accessible art evoked stronger dilations, also reflecting high clearness and negative valence ratings. Discussion: The study underscores a complex relationship between the timing of recognition and the Aha moment, suggesting nuanced differences in emotional and cognitive responses during art viewing. Pupil dilation measures offer insight into these processes especially for moments of recognition, though their application in evaluating emotional responses through artwork ratings warrants further exploration. |
Jack W. Silcox; Karen Bennett; Allyson Copeland; Sarah Hargus Ferguson; Brennan R. Payne The costs (and benefits?) of effortful listening for older adults: Insights from simultaneous electrophysiology, pupillometry, and memory Journal Article In: Journal of Cognitive Neuroscience, vol. 36, no. 6, pp. 997–1020, 2024. @article{Silcox2024,Although the impact of acoustic challenge on speech processing and memory increases as a person ages, older adults may engage in strategies that help them compensate for these demands. In the current preregistered study, older adults (n = 48) listened to sentences—presented in quiet or in noise—that were high constraint with either expected or unexpected endings or were low constraint with unexpected endings. Pupillometry and EEG were simultaneously recorded, and subsequent sentence recognition and word recall were measured. Like young adults in prior work, we found that noise led to increases in pupil size, delayed and reduced ERP responses, and decreased recall for unexpected words. However, in contrast to prior work in young adults where a larger pupillary response predicted a recovery of the N400 at the cost of poorer memory performance in noise, older adults did not show an associated recovery of the N400 despite decreased memory performance. Instead, we found that in quiet, increases in pupil size were associated with delays in N400 onset latencies and increased recognition memory performance. In conclusion, we found that transient variation in pupil-linked arousal predicted trade-offs between real-time lexical processing and memory that emerged at lower levels of task demand in aging. Moreover, with increased acoustic challenge, older adults still exhibited costs associated with transient increases in arousal without the corresponding benefits. |
Jing Shen; Elizabeth Heller Murray Breathy vocal quality, background noise, and hearing loss: How do these adverse conditions affect speech perception by older adults? Journal Article In: Ear & Hearing, vol. 46, no. 2, pp. 474–482, 2024. @article{Shen2024,Objectives: Although breathy vocal quality and hearing loss are both prevalent age-related changes, their combined impact on speech communication is poorly understood. This study investigated whether breathy vocal quality affected speech perception and listening effort by older listeners. Furthermore, the study examined how this effect was modulated by the adverse listening environment of background noise and the listener's level of hearing loss. Design: Nineteen older adults participated in the study. Their hearing ranged from near-normal to mild-moderate sensorineural hearing loss. Participants heard speech material of low-context sentences, with stimuli resynthesized to simulate original, mild-moderately breathy, and severely breathy conditions. Speech intelligibility was measured using a speech recognition in noise paradigm, with pupillometry data collected simultaneously to measure listening effort. Results: Simulated severely breathy vocal quality was found to reduce intelligibility and increase listening effort. Breathiness and background noise level independently modulated listening effort. The impact of hearing loss was not observed in this dataset, which can be due to the use of individualized signal to noise ratios and a small sample size. Conclusion: Results from this study demonstrate the challenges of listening to speech with a breathy vocal quality. Theoretically, the findings highlight the importance of periodicity cues in speech perception in noise by older listeners. Breathy voice could be challenging to separate from the noise when the noise also lacks periodicity. Clinically, it suggests the need to address both listener-and talker-related factors in speech communication by older adults. |
Eser Sendesen; Meral Didem Türkyılmaz In: Auris Nasus Larynx, vol. 51, no. 4, pp. 659–665, 2024. @article{Sendesen2024b,Objective: In previous studies, the results regarding the presence of listening effort or fatigue in tinnitus patients were inconsistent. The reason for this inconsistency could be that extended high frequencies, which can cause listening handicap, were not within normal limits. Therefore, this study aimed to evaluate the listening skills in tinnitus patients by matching the normal hearing thresholds at all frequencies, including the extended high frequency. Methods: Eighteen chronic tinnitus patients and thirty matched healthy controls having normal pure-tone average with symmetrical hearing thresholds was included. Subjects were evaluated with 0.125–20 kHz pure-tone audiometry, Montreal cognitive assessment test (MoCA), Tinnitus Handicap Inventory (THI), Matrix Test, Pupillometry. Results: Pupil dilatation in the 'coding' phase of the sentence presented in tinnitus patients was less than in the control group (p < 0.05). There was no difference between the groups for Matrix test scores (p > 0.05) Also, there was no statistically significant correlation between THI and Pupillometry components nor between MoCA (p > 0.05). Conclusion: Even though tinnitus patients had normal hearing in the range of 0.125–20 kHz, their autonomic nervous system responses during listening differed from healthy subjects. This difference was interpreted for potential listening fatigue in tinnitus patients. |
Eser Sendesen; Didem Turkyilmaz Listening handicap in tinnitus patients by controlling extended high frequencies - Effort or fatigue? Journal Article In: Auris Nasus Larynx, vol. 51, no. 1, pp. 198–205, 2024. @article{Sendesen2024,Objective: In previous studies, the results regarding the presence of listening effort or fatigue in tinnitus patients were inconsistent. The reason for this inconsistency could be that extended high frequencies, which can cause listening handicap, were not considered. Therefore, this study aimed to evaluate the listening skills in tinnitus patients by matching the hearing thresholds at all frequencies, including the extended high frequency. Methods: Eighteen chronic tinnitus patients and thirty matched healthy controls having normal pure-tone average with symmetrical hearing thresholds was included. Subjects were evaluated with 0.125-20 kHz pure-tone audiometry, Montreal cognitive assessment test (MoCA), Tinnitus Handicap Inventory (THI), Matrix Test, Pupillometry. Results: Pupil dilatation in the 'coding' phase of the sentence presented in tinnitus patients was less than in the control group (p<0.05). There was no difference between the groups for Matrix test scores (p> 0.05) Also, there was no statistically significant correlation between THI and Pupillometry components nor between MoCA (p>0.05). Conclusion: The results were interpreted for potential listening fatigue in tinnitus patients. Considering the possible listening handicap in tinnitus patients, reducing the listening difficulties especially in noisy environments, can be added to the goals of tinnitus therapy protocols. |
Eser Sendesen; Didem Turkyilmaz Investigation of the behavior of tinnitus patients under varying listening conditions with simultaneous electroencephalography and pupillometry Journal Article In: Brain and Behavior, vol. 14, no. 6, pp. 1–11, 2024. @article{Sendesen2024a,Objective: This study aims to control all hearing thresholds, including extended high frequencies (EHFs), presents stimuli of varying difficulty levels, and measures electroencephalography (EEG) and pupillometry responses to determine whether listening difficulty in tinnitus patients is effort or fatigue-related. Methods: Twenty-one chronic tinnitus patients and 26 matched healthy controls having normal pure-tone averages with symmetrical hearing thresholds were included. Subjects were evaluated with 0.125−20 kHz pure-tone audiometry, Montreal Cognitive Assessment Test (MoCA), Tinnitus Handicap Inventory (THI), EEG, and pupillometry. Results: Pupil dilatation and EEG alpha power during the “encoding” phase of the presented sentence in tinnitus patients were less in all listening conditions (p <.05). Also, there was no statistically significant relationship between EEG and pupillometry components for all listening conditions and THI or MoCA (p >.05). Conclusion: EEG and pupillometry results under various listening conditions indicate potential listening effort in tinnitus patients even if all frequencies, including EHFs, are controlled. Also, we suggest that pupillometry should be interpreted with caution in autonomic nervous system-related conditions such as tinnitus. |
Anna Render; Hedwig Eisenbarth; Matt Oxner; Petra Jansen Arousal, interindividual differences and temporal binding a psychophysiological study Journal Article In: Psychological Research, vol. 88, no. 5, pp. 1653–1677, 2024. @article{Render2024,The sense of agency varies as a function of arousal in negative emotional contexts. As yet, it is unknown whether the same is true for positive affect, and how inter-individual characteristics might predict these effects. Temporal binding, an implicit measure of the sense of agency, was measured in 59 participants before and after watching either an emotionally neutral film clip or a positive film clip with high or low arousal. Analyses included participants' individual differences in subjective affective ratings, physiological arousal (pupillometry, skin conductance, heart rate), striatal dopamine levels via eye blink rates, and psychopathy. Linear mixed models showed that sexual arousal decreased temporal binding whereas calm pleasure had no facilitation effect on binding. Striatal dopamine levels were positively linked whereas subjective and physiological arousal may be negatively associated with binding towards actions. Psychopathic traits reduced the effect of high arousal on binding towards actions. These results provide evidence that individual differences influence the extent to which the temporal binding is affected by high arousing states with positive valence. |
Jourdan J. Pouliot; Richard T. Ward; Caitlin M. Traiser; Payton Chiasson; Faith E. Gilbert; Andreas Keil Neurophysiological and autonomic dynamics of threat processing during sustained social fear generalization Journal Article In: Journal of Cognitive Neuroscience, vol. 37, no. 2, pp. 482–497, 2024. @article{Pouliot2024,Survival in rapidly changing environments requires that organisms learn to predict noxious outcomes based on situational cues. One key facet of successful threat prediction is generalization from a specific predictive cue to similar cues, ensuring that a cue-outcome contingency is applied beyond the original learning environment. Generalization has also been observed in laboratory studies of human aversive conditioning: Most behavioral and physiological processes generalize responses from a stimulus paired with threat, (the CS+), to unpaired stimuli, with response magnitudes varying as a function of stimulus similarity. In contrast, work focusing on sensory responses in visual cortex has found a sharpening pattern, in which responses to stimuli closely resembling the CS+ are maximally suppressed, potentially reflecting lateral inhibitory interactions with the CS+ representation. Originally demonstrated with simple visual cues, changes in visuocortical tuning have also been observed in threat generalization learning across facial identity cues. It is however unclear to what extent these visuocortical changes represent transient or sustained effects and if generalization learning requires prior conditioning to the CS+. The present study addressed these questions using EEG and pupillometry in a paradigm involving several hundreds of trials of aversive generalization learning along a gradient of facial identities. Visuocortical ssVEP sharpening occurred after dozens of trials of generalization learning without prior differential conditioning, but diminished as learning progressed further. By contrast, generalization of EEG alpha power suppression, pupil dilation, and self-reported valence and arousal ratings was seen throughout the experimental session. Findings are consistent with models of threat processing emphasizing the role of changing visucocortical and attention dynamics in the formation, curation, and shaping of fear memories as observers continue learning about stimulus-outcome contingencies. |
Jasmine Pan; Xuelin Sun; Edison Park; Marine Kaufmann; Michaela Klimova; Joseph T. McGuire; Sam Ling The effects of emotional arousal on pupil size depend on luminance Journal Article In: Scientific reports, vol. 14, no. 1, pp. 1–18, 2024. @article{Pan2024,Pupillometry is widely used to measure arousal states. The primary functional role of the pupil, however, is to respond to the luminance of visual inputs. We previously demonstrated that cognitive effort-related arousal interacted multiplicatively with luminance, with the strongest pupillary effects of arousal occurring at low-to-mid luminances (< 37 cd/m2), implying a narrow range of conditions ideal for assessing cognitive arousal-driven pupillary differences. Does this generalize to other forms of arousal? To answer this, we assessed luminance-driven pupillary response functions while manipulating emotional arousal, using well-established visual and auditory stimulus sets. At the group level, emotional arousal interacted with the pupillary light response differently from cognitive arousal: the effects occurred primarily at much lower luminances (< 20 cd/m2). Analyses at the individual-participant level revealed qualitatively distinct patterns of modulation, with a sizable number of individuals displaying no arousal response to the visual or auditory stimuli, regardless of luminance. Together, our results suggest that effects of arousal on pupil size are not monolithic: different forms of arousal exert different patterns of effects. More practically, our findings suggest that lower luminances create better conditions for measuring pupil-linked arousal, and when selecting ambient luminance levels, consideration of the arousal manipulation and individual differences is critical. |
Sean R. O'Bryan; Mindi M. Price; Jessica L. Alquist; Tyler Davis; Miranda Scolari Changes in pupil size track self-control failure Journal Article In: Experimental Brain Research, vol. 242, no. 4, pp. 829–841, 2024. @article{OBryan2024a,People are more likely to perform poorly on a self-control task following a previous task requiring self-control (ego-depletion), but the mechanism for this effect remains unclear. We used pupillometry to test the role of attentional effort in ego-depletion. We hypothesized that an elevated pupil diameter (PD)—a common physiological measure of effort—during an initial task requiring self-control should be negatively associated with performance on a subsequent control task. To test this hypothesis, participants were first assigned to either a high- or low-demand attention task (manipulation; a standard ego-depletion paradigm), after which all participants completed the same Stroop task. We then separately extracted both sustained (low-frequency) and phasic (high-frequency) changes in PD from both tasks to evaluate possible associations with lapses of cognitive control on the Stroop task. We first show that in the initial task, sustained PD was larger among participants who were assigned to the demanding attention condition. Furthermore, ego-depletion effects were serially mediated by PD: an elevated PD response emerged rapidly among the experimental group during the manipulation, persisted as an elevated baseline response during the Stroop task, and predicted worse accuracy on incongruent trials, revealing a potential indirect pathway to ego-depletion via sustained attention. Secondary analyses revealed another, independent and direct pathway via high levels of transient attentional control: participants who exhibited large phasic responses during the manipulation tended to perform worse on the subsequent Stroop task. We conclude by exploring the neuroscientific implications of these results within the context of current theories of self-control. |
Leonie Nowack; Hermann J. Müller; Markus Conci Changes in attentional breadth scale with the demands of Kanizsa-figure object completion–evidence from pupillometry Journal Article In: Attention, Perception, & Psychophysics, vol. 86, no. 2, pp. 439–456, 2024. @article{Nowack2024,The present study investigated whether the integration of separate parts into a whole-object representation varies with the amount of available attentional resources. To this end, two experiments were performed, which required observers to maintain central fixation while searching in peripheral vision for a target among various distractor configurations. The target could either be a “grouped” whole-object Kanizsa figure, or an “ungrouped” configuration of identical figural parts, but which do not support object completion processes to the same extent. In the experiments, accuracies and changes in pupil size were assessed, with the latter reflecting a marker of the covert allocation of attention in the periphery. Experiment 1 revealed a performance benefit for grouped (relative to ungrouped) targets, which increased with decreasing distance from fixation. By contrast, search for ungrouped targets was comparably poor in accuracy without revealing any eccentricity-dependent variation. Moreover, measures of pupillary dilation mirrored this eccentricity-dependent advantage in localizing grouped targets. Next, in Experiment 2, an additional attention-demanding foveal task was introduced in order to further reduce the availability of attentional resources for the peripheral detection task. This additional task hampered performance overall, alongside with corresponding pupil size changes. However, there was still a substantial benefit for grouped over ungrouped targets in both the behavioral and the pupillometric data. This shows that perceptual grouping scales with the allocation of attention even when only residual attentional resources are available to trigger the representation of a complete (target) object, thus illustrating that object completion operates in the “near absence” of attention. |
Hannah Mechtenberg; Cristal Giorio; Emily B. Myers Pupil dilation reflects perceptual priorities during a receptive speech task Journal Article In: Ear & Hearing, vol. 45, no. 2, pp. 425–440, 2024. @article{Mechtenberg2024,Objectives: The listening demand incurred by speech perception fluctuates in normal conversation. At the acoustic-phonetic level, natural variation in pronunciation acts as speedbumps to accurate lexical selection. Any given utterance may be more or less phonetically ambiguous - a problem that must be resolved by the listener to choose the correct word. This becomes especially apparent when considering two common speech registers - clear and casual - that have characteristically different levels of phonetic ambiguity. Clear speech prioritizes intelligibility through hyperarticulation which results in less ambiguity at the phonetic level, while casual speech tends to have a more collapsed acoustic space. We hypothesized that listeners would invest greater cognitive resources while listening to casual speech to resolve the increased amount of phonetic ambiguity, as compared with clear speech. To this end, we used pupillometry as an online measure of listening effort during perception of clear and casual continuous speech in two background conditions: quiet and noise. Design: Forty-eight participants performed a probe detection task while listening to spoken, nonsensical sentences (masked and unmasked) while recording pupil size. Pupil size was modeled using growth curve analysis to capture the dynamics of the pupil response as the sentence unfolded. Results: Pupil size during listening was sensitive to the presence of noise and speech register (clear/casual). Unsurprisingly, listeners had overall larger pupil dilations during speech perception in noise, replicating earlier work. The pupil dilation pattern for clear and casual sentences was considerably more complex. Pupil dilation during clear speech trials was slightly larger than for casual speech, across quiet and noisy backgrounds. Conclusions: We suggest that listener motivation could explain the larger pupil dilations to clearly spoken speech. We propose that, bounded by the context of this task, listeners devoted more resources to perceiving the speech signal with the greatest acoustic/phonetic fidelity. Further, we unexpectedly found systematic differences in pupil dilation preceding the onset of the spoken sentences. Together, these data demonstrate that the pupillary system is not merely reactive but also adaptive - sensitive to both task structure and listener motivation to maximize accurate perception in a limited resource system. |
Drew J. McLaughlin; Jackson S. Colvett; Julie M. Bugg; Kristin J. Van Engen Sequence effects and speech processing: cognitive load for speaker-switching within and across accents Journal Article In: Psychonomic Bulletin & Review, vol. 31, no. 1, pp. 1–11, 2024. @article{McLaughlin2024,Prior work in speech processing indicates that listening tasks with multiple speakers (as opposed to a single speaker) result in slower and less accurate processing. Notably, the trial-to-trial cognitive demands of switching between speakers or switching between accents have yet to be examined. We used pupillometry, a physiological index of cognitive load, to examine the demands of processing first (L1) and second (L2) language-accented speech when listening to sentences produced by the same speaker consecutively (no switch), a novel speaker of the same accent (within-accent switch), and a novel speaker with a different accent (across-accent switch). Inspired by research on sequential adjustments in cognitive control, we aimed to identify the cognitive demands of accommodating a novel speaker and accent by examining the trial-to-trial changes in pupil dilation during speech processing. Our results indicate that switching between speakers was more cognitively demanding than listening to the same speaker consecutively. Additionally, switching to a novel speaker with a different accent was more cognitively demanding than switching between speakers of the same accent. However, there was an asymmetry for across-accent switches, such that switching from an L1 to an L2 accent was more demanding than vice versa. Findings from the present study align with work examining multi-talker processing costs, and provide novel evidence that listeners dynamically adjust cognitive processing to accommodate speaker and accent variability. We discuss these novel findings in the context of an active control model and auditory streaming framework of speech processing. |
Priscila López-Beltrán; Paola E. Dussias Heritage speakers' processing of the Spanish subjunctive: A pupillometric study Priscila Journal Article In: Linguistic Approaches to Bilingualism, vol. 14, no. 6, pp. 809–855, 2024. @article{LopezBeltran2024,We investigated linguistic knowledge of subjunctive mood in heritage speakers of Spanish who live in a long-standing English-Spanish bilingual community in Albuquerque, New Mexico. Three experiments examine the constraints on subjunctive selection. Experiment 1 and Experiment 2 employed pupillometry to investigate heritage speakers' online sensitivity to the presence of the subjunctive with non-variable governors (Lexical conditioning) and with negated governors (Structural conditioning). Experiment 3 employed an elicited production task to examine production of subjunctive in the same contexts. The findings of the heritage group were compared to those of a group ofSpanish-dominant Mexican bilinguals. Results showed that in comprehension and production, heritage speakers were as sensitive as the Spanish-dominant bilinguals to the lexical and structural factors that condition mood selection. In comprehension, the two groups experienced an increased pupillary dilation in conditions where the indicative was used but the subjunctive was expected. In addition, high- frequency governors and irregular subordinate verbs boosted participants' sensitivity to the presence of the subjunctive. In production, there were no significant differences between heritage speakers and Spanish-dominant bilinguals when producing the subjunctive with non-variable and negated governors. |
Ye Liu; Bridget W. Mahony; Xiaochun Wang; Pierre M. Daye; Wei Wang; Patrick Cavanagh; Pierre Pouget; Ian Max Andolina Assessing perceptual chromatic equiluminance using a reflexive pupillary response Journal Article In: Scientific Reports, vol. 14, no. 1, pp. 1–12, 2024. @article{Liu2024i,Equiluminant stimuli help assess the integrity of colour perception and the relationship of colour to other visual features. As a result of individual variation, it is necessary to calibrate experimental visual stimuli to suit each individual's unique equiluminant ratio. Most traditional methods rely on training observers to report their subjective equiluminance point. Such paradigms cannot easily be implemented on pre-verbal or non-verbal observers. Here, we present a novel Pupil Frequency-Tagging Method (PFTM) for detecting a participant's unique equiluminance point without verbal instruction and with minimal training. PFTM analyses reflexive pupil oscillations induced by slow (< 2 Hz) temporal alternations between coloured stimuli. Two equiluminant stimuli will induce a similar pupil dilation response regardless of colour; therefore, an observer's equiluminant point can be identified as the luminance ratio between two colours for which the oscillatory amplitude of the pupil at the tagged frequency is minimal. We compared pupillometry-based equiluminance ratios to those obtained with two established techniques in humans: minimum flicker and minimum motion. In addition, we estimated the equiluminance point in non-human primates, demonstrating that this new technique can be successfully employed in non-verbal subjects. |
Frauke Kraus; Bernhard Ross; Björn Herrmann; Jonas Obleser Neurophysiology of effortful listening: Decoupling motivational modulation from task demands Journal Article In: The Journal of Neuroscience, vol. 44, no. 44, pp. 1–15, 2024. @article{Kraus2024,In demanding listening situations, a listener's motivational state may affect their cognitive investment. Here, we aim to delineate how domain-specific sensory processing, domain-general neural alpha power, and pupil size as a proxy for cognitive investment encode influences of motivational state under demanding listening. Participants performed an auditory gap-detection task while pupil size and the magnetoencephalogram (MEG) were simultaneously recorded. Task demand and a listener's motivational state were orthogonally manipulated through changes in gap duration and monetary-reward prospect, respectively. Whereas task difficulty impaired performance, reward prospect enhanced it. Pupil size reliably indicated the modulatory impact of an individual's motivational state. At the neural level, the motivational state did not affect auditory sensory processing directly but impacted attentional post-processing of an auditory event as reflected in the late evoked-response field and alpha power change. Both pre-gap pupil dilation and higher parietal alpha power predicted better performance at the single-trial level. The current data support a framework wherein the motivational state acts as an attentional top-down neural means of post-processing the auditory input in challenging listening situations. Significance Statement How does an individual's motivational state affect cognitive investment during effortful listening? In this simultaneous pupillometry and MEG study, participants performed an auditory gap-detection task while their motivational state was manipulated through varying prospect of a monetary reward. The pupil size directly mirrored this motivational modulation of the listening demand. The individual's motivational state also enhanced top-down attentional post-processing of the auditory event but did neither change auditory sensory processing nor pre-gap parietal alpha power. These data suggest that a listener's motivational state acts as a late attentional top-down effect on auditory neural processes in challenging listening situations. ### Competing Interest Statement The authors have declared no competing interest. |
Aleksandra Koprowska; Dorothea Wendt; Maja Serman; Torsten Dau; Jeremy Marozeau The effect of auditory training on listening effort in hearing-aid users: Insights from a pupillometry study Journal Article In: International Journal of Audiology, vol. 64, no. 1, pp. 59–69, 2024. @article{Koprowska2024,Objective: The study investigated how auditory training affects effort exerted by hearing-impaired listeners in speech-in-noise task. Design: Pupillometry was used to characterise listening effort during a hearing in noise test (HINT) before and after phoneme-in-noise identification training. Half of the study participants completed the training, while the other half formed an active control group. Study sample: Twenty 63-to-79 years old experienced hearing-aid users. Results: Higher peak pupil dilations (PPDs) were obtained at the end of the study compared to the beginning in both groups of the participants. The analysis of pupil dilation in an extended time window revealed, however, that the magnitude of pupillary response increased more in the training than in the control group. The effect of training on effort was observed in pupil responses even when no improvement in HINT was found. Conclusion: The results demonstrate that using a listening effort metric adds additional insights into the effectiveness of auditory training compared to the situation when only speech-in-noise performance is considered. Trends observed in pupil responses suggested increased effort—both after the training and the placebo intervention—most likely reflecting the effect of the individual's motivation. |
Damian Koevoet; Marnix Naber; Stefan Stigchel The intensity of internal and external attention assessed with pupillometry Journal Article In: Journal of Cognition, vol. 7, pp. 1–10, 2024. @article{Koevoet2024,Not only is visual attention shifted to objects in the external world, attention can also be directed to objects in memory. We have recently shown that pupil size indexes how strongly items are attended externally, which was reflected in more precise encoding into visual working memory. Using a retro-cuing paradigm, we here replicated this finding by showing that stronger pupil constrictions during encoding were reflective of the depth of encoding. Importantly, we extend this previous work by showing that pupil size also revealed the intensity of internal attention toward content stored in visual working memory. Specifically, pupil dilation during the prioritization of one among multiple internally stored representations predicted the precision of the prioritized item. Furthermore, the dynamics of the pupillary responses revealed that the intensity of internal and external attention independently determined the precision of internalized visual representations. Our results show that both internal and external attention are not all-or-none processes, but should rather be thought of as continuous resources that can be deployed at varying intensities. The employed pupillometric approach allows to unravel the intricate interplay between internal and external attention and their effects on visual working memory. |
Samet Kılıç; Eser Sendesen; Filiz Aslan; Nurhan Erbil; Özgür Aydın; Didem Türkyılmaz Investigating sensitivity to auditory cognition in listening effort assessments: A simultaneous EEG and pupillometry study Journal Article In: Brain and Behavior, vol. 14, no. 11, pp. 1–12, 2024. @article{Kilic2024,Background: It is still not fully explained what kind of cognitive sources the methods used in the assessment of listening effort are more sensitive to and how these measurement results are related to each other. The aim of the study is to ascertain which neural resources crucial for listening effort are most sensitive to objective measurement methods using differently degraded speech stimuli. Methods: A total of 49 individuals between the ages of 19 and 34 with normal hearing participated in the study. In the first stage, simultaneous pupillometry, electroencephalogram (EEG), and single-task paradigm reaction time (RT) measurements were made during the challenging listening and repetition task with noise-vocoded speech. Two speech reception thresholds (SRT) (50% and 80%) for two vocoding conditions (16 and 6 channels) were collected, resulting in 4 conditions. In the second stage, the Rey Auditory Verbal Learning Test (RAVLT) and the test of attention in listening (TAIL) were applied. Stepwise linear regression analyses were conducted to examine the predictors of listening effort measurements. Results: A significant difference was found between 6 and 16 channel stimuli in both pupil dilation change and EEG alpha band power change. In the hardest listening condition, whereas RAVLT scores are significant predictors of pupil dilation change, TAIL scores are significant predictors of EEG alpha power. As the stimulus difficulty decreased, the factors that predicted both EEG and pupillometry results decreased. In the single-task paradigm, a significant regression model could not be obtained at all four difficulty levels. Conclusion: As a result of the study, it was found that the pupil dilation change was more sensitive to auditory memory skills and the EEG alpha power change was more sensitive to auditory attention skills. To our knowledge, this study is the first to investigate the sensitivity of different listening effort measurement methods to auditory cognitive skills. |
M. A. Johns; R. C. Calloway; I. M. D. Karunathilake; L. P. Decruy; S. Anderson; J. Z. Simon; S. E. Kuchinsky Attention mobilization as a modulator of listening effort: Evidence from pupillometry Journal Article In: Trends in Hearing, vol. 28, pp. 1–20, 2024. @article{Johns2024,Listening to speech in noise can require substantial mental effort, even among younger normal-hearing adults. The task-evoked pupil response (TEPR) has been shown to track the increased effort exerted to recognize words or sentences in increasing noise. However, few studies have examined the trajectory of listening effort across longer, more natural, stretches of speech, or the extent to which expectations about upcoming listening difficulty modulate the TEPR. Seventeen younger normal-hearing adults listened to 60-s-long audiobook passages, repeated three times in a row, at two different signal-to-noise ratios (SNRs) while pupil size was recorded. There was a significant interaction between SNR, repetition, and baseline pupil size on sustained listening effort. At lower baseline pupil sizes, potentially reflecting lower attention mobilization, TEPRs were more sustained in the harder SNR condition, particularly when attention mobilization remained low by the third presentation. At intermediate baseline pupil sizes, differences between conditions were largely absent, suggesting these listeners had optimally mobilized their attention for both SNRs. Lastly, at higher baseline pupil sizes, potentially reflecting overmobilization of attention, the effect of SNR was initially reversed for the second and third presentations: participants initially appeared to disengage in the harder SNR condition, resulting in reduced TEPRs that recovered in the second half of the story. Together, these findings suggest that the unfolding of listening effort over time depends critically on the extent to which individuals have successfully mobilized their attention in anticipation of difficult listening conditions. |
Ronen Hershman; David L. Share; Elisabeth M. Weiss; Avishai Henik; Adi Shechter Insights from eye blinks into the cognitive processes involved in visual word recognition Journal Article In: Journal of Cognition, vol. 7, no. 1, pp. 1–9, 2024. @article{Hershman2024,Behavioral differences in speed and accuracy between reading familiar and unfamiliar words are well-established in the empirical literature. However, these standard measures of skill proficiency are limited in their ability to capture the moment-to-moment processing involved in visual word recognition. In the present study, the effect of word familiarity was initially investigated using an eye blink rate among adults and children. The probability of eye blinking was higher for familiar (real) words than for unfamiliar (pseudo)words. This counterintuitive pattern of results suggests that the processing of unfamiliar (pseudo)words is more demanding and perhaps less rewarding than the processing of familiar (real) words, as previously observed in both behavioral and pupillometry data. Our findings suggest that the measurement of eye blinks might shed new light on the cognitive processes involved in visual word recognition and other domains of human cognition. |
Björn Herrmann; Jennifer D. Ryan Pupil size and eye movements differently index effort in both younger and older adults Journal Article In: Journal of Cognitive Neuroscience, vol. 36, no. 7, pp. 1325–1340, 2024. @article{Herrmann2024,The assessment of mental effort is increasingly relevant in neurocognitive and life span domains. Pupillometry, the measure of the pupil size, is often used to assess effort but has disadvantages. Analysis of eye movements may provide an alternative, but research has been limited to easy and difficult task demands in younger adults. An effort measure must be sensitive to the whole effort profile, including “giving up” effort investment, and capture effort in different age groups. The current study comprised three experiments in which younger (n = 66) and older (n = 44) adults listened to speech masked by background babble at different signal-to-noise ratios associated with easy, difficult, and impossible speech comprehension. We expected individuals to invest little effort for easy and impossible speech (giving up) but to exert effort for difficult speech. Indeed, pupil size was largest for difficult but lower for easy and impossible speech. In contrast, gaze dispersion decreased with increasing speech masking in both age groups. Critically, gaze dispersion during difficult speech returned to levels similar to easy speech after sentence offset, when acoustic stimulation was similar across conditions, whereas gaze dispersion during impossible speech continued to be reduced. These findings show that a reduction in eye movements is not a byproduct of acoustic factors, but instead suggest that neurocognitive processes, different from arousal-related systems regulating the pupil size, drive reduced eye movements during high task demands. The current data thus show that effort in one sensory domain (audition) differentially impacts distinct functional properties in another sensory domain (vision). |
Nabil Hasshim; Molly Carruthers; Ludovic Ferrand; Maria Augustinova; Benjamin A. Parris No pupillometric evidence for effortful proactive control in the proportion-congruent Stroop paradigm Journal Article In: Quarterly Journal of Experimental Psychology, pp. 1–13, 2024. @article{Hasshim2024,Cognitive control is the ability to allocate attention away from stimuli that are irrelevant to achieving a goal, towards stimuli that are. When conflict is anticipated, attention is biased in a global, top-down manner called proactive control and this effortful type of cognitive control is engaged before stimulus onset. The list-wise congruency proportion (LWPC) effect, where the Stroop congruency effect is reduced when there are more incongruent than congruent trials compared to vice versa, has been viewed as one of the prime signatures of this type of cognitive control. However, there has been recent debate about the extent to which this effect should be attributed to proactive control instead of alternative explanations such as simpler associative learning or reactive control. Thus, by using pupillometry (i.e., an indicator of cognitive effort), the present study investigated the extent to which LWPC effects result from effortful proactive control. Experiment 1 employed a classic proportion congruency manipulation, while Experiment 2 replaced congruent trials with neutral trials to control for potential effects of associative learning. While in line with past findings, proportion congruency effects were obtained in response times of both experiments and pupillometry showed both proportion congruency and Stroop effects after stimulus onset, no differences in pupil sizes were found during the preparatory phase. Therefore, these results do not support the idea that the observed LWPC effects are due to participants engaging in effortful proactive control. |
Laura S. Geurts; Sam Ling; Janneke F. M. Jehee Pupil-linked arousal modulates precision of stimulus representation in cortex Journal Article In: The Journal of Neuroscience, vol. 44, no. 42, pp. 1–11, 2024. @article{Geurts2024,Neural responses are naturally variable from one moment to the next, even when the stimulus is held constant. What factors might underlie this variability in neural population activity? We hypothesized that spontaneous fluctuations in cortical stimulus representations are created by changes in arousal state. We tested the hypothesis using a combination of fMRI, probabilistic decoding methods, and pupillometry. Human participants (20 female, 12 male) were presented with gratings of random orientation. Shortly after viewing the grating, participants reported its orientation and gave their level of confidence in this judgment. Using a probabilistic fMRI decoding technique, we quantified the precision of the stimulus representation in the visual cortex on a trial-by-trial basis. Pupil size was recorded and analyzed to index the observer's arousal state. We found that the precision of the cortical stimulus representation, reported confidence, and variability in the behavioral orientation judgments varied from trial to trial. Interestingly, these trial-by-trial changes in cortical and behavioral precision and confidence were linked to pupil size and its temporal rate of change. Specifically, when the cortical stimulus representation was more precise, the pupil dilated more strongly prior to stimulus onset and remained larger during stimulus presentation. Similarly, stronger pupil dilation during stimulus presentation was associated with higher levels of subjective confidence, a secondary measure of sensory precision, as well as improved behavioral performance. Taken together, our findings support the hypothesis that spontaneous fluctuations in arousal state modulate the fidelity of the stimulus representation in the human visual cortex, with clear consequences for behavior. |
Natsumi Funasaki; Masataka Yano Role of prosody and word order in identifying focus: Evidence from pupillometry Journal Article In: Language, Cognition and Neuroscience, vol. 40, no. 2, pp. 23–40, 2024. @article{Funasaki2024,This study investigated the role of prosody and word order in identifying the focus of sentences in Japanese. Native Japanese speakers listened to sentences with different types of word order (subject–object–verb (SOV) vs. object–subject–verb (OSV)), prosody (whether the first noun phrase is stressed or not) and preceding contexts (object- vs. subject-wh questions), while processing costs were measured using pupillometry. Although syntactically non-basic OSV was more difficult to process than basic SOV, this processing difficulty was considerably reduced when the supportive context (the subject-wh question) required S to be focused. The time–course analysis of pupillometry revealed that the Japanese speakers immediately used prosodic cues to determine the focus of sentences, but the effect of word order cues for focus was delayed until the sentence-final verb was encountered. This study advances our understanding of the temporal dynamics of focus processing and the interplay between syntactic and information structures in sentence comprehension. |
Julia Fietz; Dorothee Pöhlchen; BeCOME Working Group; Tanja M. Brückl; Anna-Katharine Brem; Frank Padberg; Michael Czisch; Philipp G. Sämann; Victor I. Spoormaker In: Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, vol. 9, no. 6, pp. 580–587, 2024. @article{Fietz2024,Background: Neurocognitive functioning is a relevant transdiagnostic dimension in psychiatry. As pupil size dynamics track cognitive load during a working memory task, we aimed to explore if this parameter allows identification of psychophysiological subtypes in healthy participants and patients with affective and anxiety disorders. Methods: Our sample consisted of 226 participants who completed the n-back task during simultaneous functional magnetic resonance imaging and pupillometry measurements. We used latent class growth modeling to identify clusters based on pupil size in response to cognitive load. In a second step, these clusters were compared on affective and anxiety symptom levels, performance in neurocognitive tests, and functional magnetic resonance imaging activity. Results: The clustering analysis resulted in two distinct pupil response profiles: one with a stepwise increasing pupil size with increasing cognitive load (reactive group) and one with a constant pupil size across conditions (nonreactive group). A larger increase in pupil size was significantly associated with better performance in neurocognitive tests in executive functioning and sustained attention. Statistical maps of parametric modulation of pupil size during the n-back task showed the frontoparietal network in the positive contrast and the default mode network in the negative contrast. The pupil response profile of the reactive group was associated with more thalamic activity, likely reflecting better arousal upregulation and less deactivation of the limbic system. Conclusions: Pupil measurements have the potential to serve as a highly sensitive psychophysiological readout for detection of neurocognitive deficits in the core domain of executive functioning, adding to the development of valid transdiagnostic constructs in psychiatry. |
Hadeel Ershaid; Mikel Lizarazu; D. J. Drew McLaughlin; Martin Cooke; Olympia Simantiraki; Maria Koutsogiannaki; Marie Lallier Contributions of listening effort and intelligibility to cortical tracking of speech in adverse listening conditions Journal Article In: Cortex, vol. 172, pp. 54–71, 2024. @article{Ershaid2024,Cortical tracking of speech is vital for speech segmentation and is linked to speech intelligibility. However, there is no clear consensus as to whether reduced intelligibility leads to a decrease or an increase in cortical speech tracking, warranting further investigation of the factors influencing this relationship. One such factor is listening effort, defined as the cognitive resources necessary for speech comprehension, and reported to have a strong negative correlation with speech intelligibility. Yet, no studies have examined the relationship between speech intelligibility, listening effort, and cortical tracking of speech. The aim of the present study was thus to examine these factors in quiet and distinct adverse listening conditions. Forty-nine normal hearing adults listened to sentences produced casually, presented in quiet and two adverse listening conditions: cafeteria noise and re- verberant speech. Electrophysiological responses were registered with electroencephalogram, and listening effort was estimated subjectively using self-reported scores and objectively using pupillometry. Results indicated varying impacts of adverse conditions on intelligibility, listening effort, and cortical tracking of speech, depending on the preservation of the speech temporal envelope. The more distorted envelope in the reverberant condition led to higher listening effort, as reflected in higher subjective scores, increased pupil diameter, and stronger cortical tracking of speech in the delta band. These findings suggest that using measures of listening effort in addition to those of intelligibility is useful for interpreting cortical tracking of speech results. Moreover, reading and phonological skills of participants were positively correlated with listening effort in the cafeteria condition, suggesting a special role of expert language skills in processing speech in this noisy condition. Implications for future research and theories linking atypical cortical tracking of speech and reading disorders are further discussed. |
Laura Doll; Andrew R. Dykstra; Alexander Gutschalk Perceptual awareness of near-threshold tones scales gradually with auditory cortex activity and pupil dilation Journal Article In: iScience, vol. 27, no. 8, pp. 1–20, 2024. @article{Doll2024,Negative-going responses in sensory cortex co-vary with perceptual awareness of sensory stimuli. Given that this awareness negativity has also been observed for undetected stimuli, some have challenged its role for perception. To address this question, we combined magnetoencephalography, electroencephalography, and pupillometry to study how sustained attention and response criterion affect the auditory awareness negativity. Participants first detected distractor sounds and denied hearing task-irrelevant near-threshold tones, which evoked neither awareness negativity nor pupil dilation. These same tones evoked both responses when task-relevant, stronger for hit but also present for miss trials. Participants then rated their perception on a six-point scale to test whether response criterion explains the presence of these responses for miss trials. Decreasing perception ratings were associated with gradually reduced evoked responses, consistent with signal detection theory. These results support the concept of an awareness negativity that is modulated by attention but does not require a non-linear threshold mechanism. |
Roos A. Doekemeijer; Quinn Cabooter; Intan K. Wardhani; Frederick Verbruggen; C. Nico Boehler From pupil to performance: Exploring the role of tonic norepinephrine levels in response inhibition using pretrial pupil measures Journal Article In: Psychophysiology, pp. 1–13, 2024. @article{Doekemeijer2024,Response inhibition is key to flexible behavior. Importantly, performance in any task, including response inhibition tasks, fluctuates on a moment-to-moment basis. Using pupillometry, we investigated the relationship between these behavioral fluctuations in response inhibition and naturally occurring fluctuations of norepinephrine (NE) levels in the brain before a given trial has even started. This was motivated by earlier pharmacological work suggesting a pivotal role of NE in response inhibition, in particular. We specifically used two pupillometry proxies for pretrial (tonic) NE levels, the absolute pretrial pupil size and its derivative, and investigated whether and to which degree they were related to response-inhibition performance in a stop-signal task. Specifically, we investigated the relationship to stopping success, and the speed of the go response (GoRT) and that of the stop response (SSRT). In two experiments, we showed that larger pretrial pupil measures predicted (1) lower stopping success, (2) faster GoRTs (particularly so when the go response needed to be executed in a stop context), and some evidence for (3) faster SSRTs. Taken together, our findings show a clear pattern that pretrial pupil measures predict behavioral fluctuations in response inhibition, which suggests that tonic levels of NE are involved in the regulation of these behavioral fluctuations. Yet, our work furthermore indicates that this involvement is not stopping-specific, given its effect on both the go and the stop response. |
Hsin-Hua Chin; Ying-Hsuan Tai; Rachel Yep; Yi-Hsuan Chang; Chun-Hsien Hsu; Chin-An Wang Investigating causal effects of pupil size on visual discrimination and visually evoked potentials in an optotype discrimination task Journal Article In: Frontiers in Neuroscience, vol. 18, pp. 1–10, 2024. @article{Chin2024,Pupil size primarily changes to regulate the amount of light entering the retina, optimizing the balance between visual acuity and sensitivity for effective visual processing. However, research directly examining the relationship between pupil size and visual processing has been limited. While a few studies have recorded pupil size and EEG signals to investigate the role of pupil size in visual processing, these studies have predominantly focused on the domain of visual sensitivity. Causal effects of pupil size on visual acuity, therefore, remain poorly understood. By manipulating peripheral background luminance levels and target stimulus contrast while simultaneously recording pupillometry and EEG signals, we examined how absolute pupil size affects visual discrimination and visually evoked potentials (VEP) in a task using optotype mimicking the Snellen eye chart, the most common assessment of visual acuity. Our findings indicate that both higher background luminance levels and higher target contrast were associated with improved target discrimination and faster correct reaction times. Moreover, while higher contrast visual stimuli evoked larger VEPs, the effects of pupil size on VEPs were not significant. Additionally, we did not observe inter-individual correlations between absolute pupil size and discrimination performance or VEP amplitude. Together, our results demonstrate that absolute pupil size, regulated by global luminance level, played a functional role in enhancing visual discrimination performance in an optotype discrimination task. The differential VEP effects of pupil size compared to those of stimulus contrast further suggested distinct neural mechanisms involved in facilitating visual acuity under small pupils. |
Yuqing Cai; Christoph Strauch; Stefan Van der Stigchel; Marnix Naber Open-DPSM: An open-source toolkit for modeling pupil size changes to dynamic visual inputs Journal Article In: Behavior Research Methods, vol. 56, no. 6, pp. 5605–5621, 2024. @article{Cai2024,Pupil size change is a widely adopted, sensitive indicator for sensory and cognitive processes. However, the interpretation of these changes is complicated by the influence of multiple low-level effects, such as brightness or contrast changes, posing challenges to applying pupillometry outside of extremely controlled settings. Building on and extending previous models, we here introduce Open Dynamic Pupil Size Modeling (Open-DPSM), an open-source toolkit to model pupil size changes to dynamically changing visual inputs using a convolution approach. Open-DPSM incorporates three key steps: (1) Modeling pupillary responses to both luminance and contrast changes; (2) Weighing of the distinct contributions of visual events across the visual field on pupil size change; and (3) Incorporating gaze-contingent visual event extraction and modeling. These steps improve the prediction of pupil size changes beyond the here-evaluated benchmarks. Open-DPSM provides Python functions, as well as a graphical user interface (GUI), enabling the extension of its applications to versatile scenarios and adaptations to individualized needs. By obtaining a predicted pupil trace using video and eye-tracking data, users can mitigate the effects of low-level features by subtracting the predicted trace or assess the efficacy of the low-level feature manipulations a priori by comparing estimated traces across conditions. |
Dana Bsharat-Maalouf; Jens Schmidtke; Tamar Degani; Hanin Karawani Through the pupils' lens: Multilingual effort in first and second language listening Journal Article In: Ear & Hearing, vol. 46, no. 2, pp. 494–511, 2024. @article{BsharatMaalouf2024,Objectives: The present study aimed to examine the involvement of listening effort among multilinguals in their first (L1) and second (L2) languages in quiet and noisy listening conditions and investigate how the presence of a constraining context within sentences influences listening effort. Design: A group of 46 young adult Arabic (L1)–Hebrew (L2) multilinguals participated in a listening task. This task aimed to assess participants' perceptual performance and the effort they exert (as measured through pupillometry) while listening to single words and sentences presented in their L1 and L2, in quiet and noisy environments (signal to noise ratio = 0 dB). Results: Listening in quiet was easier than in noise, supported by both perceptual and pupillometry results. Perceptually, multilinguals performed similarly and reached ceiling levels in both languages in quiet. However, under noisy conditions, perceptual accuracy was significantly lower in L2, especially when processing sentences. Critically, pupil dilation was larger and more prolonged when listening to L2 than L1 stimuli. This difference was observed even in the quiet condition. Contextual support resulted in better perceptual performance of high-predictability sentences compared with low-predictability sentences, but only in L1 under noisy conditions. In L2, pupillometry showed increased effort when listening to high-predictability sentences compared with low-predictability sentences, but this increased effort did not lead to better understanding. In fact, in noise, speech perception was lower in high-predictability L2 sentences compared with low-predictability ones. Conclusions: The findings underscore the importance of examining listening effort in multilingual speech processing and suggest that increased effort may be present in multilingual's L2 within clinical and educational settings. |
Andy Brendler; Max Schneider; Immanuel G. Elbau; Rui Sun; Taechawidd Nantawisarakul; Dorothee Pöhlchen; Tanja Brückl; A. K. Brem; E. B. Binder; A. Erhardt; J. Fietz; N. C. Grandi; Y. Kim; S. Ilić-Ćoćić; L. Leuchs; S. Lucae; T. Namendorf; J. Pape; L. Schilbach; I. Mücke-Heim; J. Ziebula; Michael Czisch; Philipp G. Sämann; Michael D. Lee; Victor I. Spoormaker In: Scientific Reports, vol. 14, no. 344, pp. 1–11, 2024. @article{Brendler2024,Major depressive disorder (MDD) is a devastating and heterogenous disorder for which there are no approved biomarkers in clinical practice. We recently identified anticipatory hypo-arousal indexed by pupil responses as a candidate mechanism subserving depression symptomatology. Here, we conducted a replication and extension study of these findings. We analyzed a replication sample of 40 unmedicated patients with a diagnosis of depression and 30 healthy control participants, who performed a reward anticipation task while pupil responses were measured. Using a Bayesian modelling approach taking measurement uncertainty into account, we could show that the negative correlation between pupil dilation and symptom load during reward anticipation is replicable within MDD patients, albeit with a lower effect size. Furthermore, with the combined sample of 136 participants (81 unmedicated depressed and 55 healthy control participants), we further showed that reduced pupil dilation in anticipation of reward is inversely associated with anhedonia items of the Beck Depression Inventory in particular. Moreover, using simultaneous fMRI, particularly the right anterior insula as part of the salience network was negatively correlated with depressive symptom load in general and anhedonia items specifically. The present study supports the utility of pupillometry in assessing noradrenergically mediated hypo-arousal during reward anticipation in MDD, a physiological process that appears to subserve anhedonia. |
Elvio Blini; Roberto Arrighi; Giovanni Anobile Pupillary manifolds: Uncovering the latent geometrical structures behind phasic changes in pupil size Journal Article In: Scientific reports, vol. 14, no. 1, pp. 1–10, 2024. @article{Blini2024a,The size of the pupils reflects directly the balance of different branches of the autonomic nervous system. This measure is inexpensive, non-invasive, and has provided invaluable insights on a wide range of mental processes, from attention to emotion and executive functions. Two outstanding limitations of current pupillometry research are the lack of consensus in the analytical approaches, which vary wildly across research groups and disciplines, and the fact that, unlike other neuroimaging techniques, pupillometry lacks the dimensionality to shed light on the different sources of the observed effects. In other words, pupillometry provides an integrated readout of several distinct networks, but it is unclear whether each has a specific fingerprint, stemming from its function or physiological substrate. Here we show that phasic changes in pupil size are inherently low-dimensional, with modes that are highly consistent across behavioral tasks of very different nature, suggesting that these changes occur along pupillary manifolds that are highly constrained by the underlying physiological structures rather than functions. These results provide not only a unified approach to analyze pupillary data, but also the opportunity for physiology and psychology to refer to the same processes by tracing the sources of the reported changes in pupil size in the underlying biology. |
Isabel Bleimeister; Inbar Avni; Michael Granovetter; Gal Meiri; Michal Ilan; Analya Michaelovski; Idan Menashe; Marlene Behrmann; Ilan Dinstein Idiosyncratic pupil regulation in autistic children Journal Article In: Autism Research, vol. 17, pp. 2503–2513, 2024. @article{Bleimeister2024,Recent neuroimaging and eye tracking studies have suggested that children with autism spectrum disorder (ASD) may exhibit more variable and idiosyncratic brain responses and eye movements than typically developing (TD) children. Here we extended this research for the first time to pupillometry recordings. We successfully completed pupillometry recordings with 103 children (66 with ASD), 4.5-years-old on average, who viewed three 90 second movies, twice. We extracted their pupillary time-course for each movie, capturing their stimulus evoked pupillary responses. We then computed the correlation between the time-course of each child and those of all others in their group. This yielded an average inter-subject correlation value per child, representing how similar their pupillary responses were to all others in their group. ASD participants exhibited significantly weaker inter-subject correlations than TD participants, reliably across all three movies. Differences across groups were largest in responses to a naturalistic movie containing footage of a social interaction between two TD children. This measure enabled classification of ASD and TD children with a sensitivity of 0.82 and specificity of 0.73 when trained and tested on independent datasets. Using the largest ASD pupillometry dataset to date, we demonstrate the utility of a new technique for measuring the idiosyncrasy of pupil regulation, which can be completed even by young children with co-occurring intellectual disability. These findings reveal that a considerable subgroup of ASD children have significantly more unstable, idiosyncratic pupil regulation than TD children, indicative of more variable, weakly regulated, underlying neural activity. |
Omer Ben Barak-Dror; Barak Hadad; Hani Barhum; David Haggiag; Michal Tepper; Israel Gannot; Yuval Nir Touchless short-wave infrared imaging for dynamic rapid pupillometry and gaze estimation in closed eyes Journal Article In: Communications Medicine, vol. 4, no. 1, pp. 1–12, 2024. @article{BenBarakDror2024,Background: Assessments of gaze direction (eye movements), pupil size, and the pupillary light reflex (PLR) are critical for neurological examination and neuroscience research and constitute a powerful tool in diverse clinical settings ranging from critical care through endocrinology and drug addiction to cardiology and psychiatry. However, current bedside pupillometry is typically intermittent, qualitative, manual, and limited to open-eye cases, restricting its use in sleep medicine, anesthesia, and intensive care. Methods: We combined short-wave infrared (SWIR, ~0.9-1.7μm) imaging with image processing algorithms to perform rapid (~30 ms) pupillometry and eye tracking behind closed eyelids. Forty-three healthy volunteers participated in two experiments with PLR evoked by visible light stimuli or directing eye movements towards screen targets. Imaging was performed simultaneously on one eye closed, and the other open eye serving as ground truth. Data analysis was performed with a custom approach quantifying changes in brightness around the pupil area or with a deep learning U-NET-based procedure. Results: Here we show that analysis of SWIR imaging data can successfully measure stimulus-evoked PLR in closed-eye conditions, revealing PLR events in single trials and significant PLRs in nearly all individual subjects, as well as estimating gaze direction. The neural net-based analysis could successfully use closed-eye SWIR data to recreate estimates of open-eye images and assess pupil size. Conclusions: Continuous touchless monitoring of rapid dynamics in pupil size and gaze direction through closed eyes paves the way for developing devices with wide-ranging applications, fulfilling long-standing goals in clinical and research fields. |
Janika Becker; Marvin Viertler; Christoph W. Korn; Helen Blank The pupil dilation response as an indicator of visual cue uncertainty and auditory outcome surprise Journal Article In: European Journal of Neuroscience, vol. 59, no. 10, pp. 2686–2701, 2024. @article{Becker2024a,In everyday perception, we combine incoming sensory information with prior expectations. Expectations can be induced by cues that indicate the probability of following sensory events. The information provided by cues may differ and hence lead to different levels of uncertainty about which event will follow. In this experiment, we employed pupillometry to investigate whether the pupil dilation response to visual cues varies depending on the level of cue-associated uncertainty about a following auditory outcome. Also, we tested whether the pupil dilation response reflects the amount of surprise about the subsequently presented auditory stimulus. In each trial, participants were presented with a visual cue (face image) which was followed by an auditory outcome (spoken vowel). After the face cue, participants had to indicate by keypress which of three auditory vowels they expected to hear next. We manipulated the cue-associated uncertainty by varying the probabilistic cue-outcome contingencies: One face was most likely followed by one specific vowel (low cue uncertainty), another face was equally likely followed by either of two vowels (intermediate cue uncertainty) and the third face was followed by all three vowels (high cue uncertainty). Our results suggest that pupil dilation in response to task-relevant cues depends on the associated uncertainty, but only for large differences in the cue-associated uncertainty. Additionally, in response to the auditory outcomes, the pupil dilation scaled negatively with the cue-dependent probabilities, likely signalling the amount of surprise. |
Janika Becker; Christoph W. Korn; Helen Blank Pupil diameter as an indicator of sound pair familiarity after statistically structured auditory sequence Journal Article In: Scientific Reports, vol. 14, no. 1, pp. 1–14, 2024. @article{Becker2024,Inspired by recent findings in the visual domain, we investigated whether the stimulus-evoked pupil dilation reflects temporal statistical regularities in sequences of auditory stimuli. We conducted two preregistered pupillometry experiments (experiment 1 |
Shira Baror; Thomas J. Baumgarten; Biyu J. He Neural mechanisms determining the duration of task-free, self-paced visual perception Journal Article In: Journal of Cognitive Neuroscience, vol. 36, no. 5, pp. 756–775, 2024. @article{Baror2024,Humans spend hours each day spontaneously engaging with visual content, free from specific tasks and at their own pace. Currently, the brain mechanisms determining the duration of self-paced perceptual behavior remain largely unknown. Here, participants viewed naturalistic images under task-free settings and self-paced each image's viewing duration while undergoing EEG and pupillometry recordings. Across two independent data sets, we observed large inter-and intra-individual variability in viewing duration. However, beyond an image's presentation order and category, specific image content had no consistent effects on spontaneous viewing duration across participants. Overall, longer viewing durations were associated with sustained enhanced posterior positivity and anterior negativity in the ERPs. Individual-specific variations in the spontaneous viewing duration were consistently correlated with evoked EEG activity amplitudes and pupil size changes. By contrast, presentation order was selectively correlated with baseline alpha power and baseline pupil size. Critically, spontaneous viewing duration was strongly predicted by the temporal stability in neural activity patterns starting as early as 350 msec after image onset, suggesting that early neural stability is a key predictor for sustained perceptual engagement. Interestingly, neither bottom–up nor top–down predictions about image category influenced spontaneous viewing duration. Overall, these results suggest that individual-specific factors can influence perceptual processing at a surprisingly early time point and influence the multifaceted ebb and flow of spontaneous human perceptual behavior in naturalistic settings. |
Evgeniia I. Alshanskaia; Galina V. Portnova; Krystsina Liaukovich; Olga V. Martynova Pupillometry and autonomic nervous system responses to cognitive load and false feedback: An unsupervised machine learning approach Journal Article In: Frontiers in Neuroscience, vol. 18, pp. 1–18, 2024. @article{Alshanskaia2024,Objectives: Pupil dilation is controlled both by sympathetic and parasympathetic nervous system branches. We hypothesized that the dynamic of pupil size changes under cognitive load with additional false feedback can predict individual behavior along with heart rate variability (HRV) patterns and eye movements reflecting specific adaptability to cognitive stress. To test this, we employed an unsupervised machine learning approach to recognize groups of individuals distinguished by pupil dilation dynamics and then compared their autonomic nervous system (ANS) responses along with time, performance, and self-esteem indicators in cognitive tasks. Methods: Cohort of 70 participants were exposed to tasks with increasing cognitive load and deception, with measurements of pupillary dynamics, HRV, eye movements, and cognitive performance and behavioral data. Utilizing machine learning k-means clustering algorithm, pupillometry data were segmented to distinct responses to increasing cognitive load and deceit. Further analysis compared clusters, focusing on how physiological (HRV, eye movements) and cognitive metrics (time, mistakes, self-esteem) varied across two clusters of different pupillary response patterns, investigating the relationship between pupil dynamics and autonomic reactions. Results: Cluster analysis of pupillometry data identified two distinct groups with statistically significant varying physiological and behavioral responses. Cluster 0 showed elevated HRV, alongside larger initial pupil sizes. Cluster 1 participants presented lower HRV but demonstrated increased and pronounced oculomotor activity. Behavioral differences included reporting more errors and lower self-esteem in Cluster 0, and faster response times with more precise reactions to deception demonstrated by Cluster 1. Lifestyle variations such as smoking habits and differences in Epworth Sleepiness Scale scores were significant between the clusters. Conclusion: The differentiation in pupillary dynamics and related metrics between the clusters underlines the complex interplay between autonomic regulation, cognitive load, and behavioral responses to cognitive load and deceptive feedback. These findings underscore the potential of pupillometry combined with machine learning in identifying individual differences in stress resilience and cognitive performance. Our research on pupillary dynamics and ANS patterns can lead to the development of remote diagnostic tools for real-time cognitive stress monitoring and performance optimization, applicable in clinical, educational, and occupational settings. |
Miriam Acquafredda; Paola Binda Pupillometry indexes ocular dominance plasticity Journal Article In: Vision Research, vol. 222, pp. 1–8, 2024. @article{Acquafredda2024,Short-term monocular deprivation in normally sighted adult humans produces a transient shift of ocular dominance, boosting the deprived eye. This effect has been documented with both perceptual tests and through physiological recordings, but no previous study simultaneously measured physiological responses and the perceptual effects of deprivation. Here we propose an integrated experimental paradigm that combines binocular rivalry with pupillometry, to introduce an objective physiological index of ocular dominance plasticity, acquired concurrently with perceptual testing. Ten participants reported the perceptual dynamics of binocular rivalry, while we measured pupil diameter. Stimuli were a white and a black disk, each presented monocularly. Rivalry dynamics and pupil-size traces were compared before and after 2 h of monocular deprivation, achieved by applying a translucent patch over the dominant eye. Consistent with prior research, we observed that monocular deprivation boosts the deprived-eye signal and consequently increases ocular dominance. In line with previous studies, we also observed subtle but systematic modulations of pupil size that tracked alternations between exclusive dominance phases of the black or white disk. Following monocular deprivation, the amplitude of these pupil-size modulations increased, which is consistent with the post-deprivation boost of the deprived eye and the increase of ocular dominance. This provides evidence that deprivation impacts the effective strength of monocular visual stimuli, coherently affecting perceptual reports and the automatic and unconscious regulation of pupil diameter. Our results show that a combined paradigm of binocular rivalry and pupillometry gives new insights into the physiological mechanisms underlying deprivation effects. |
Jordan C. Abramowitz; Matthew J. Goupell; Kristina DeRoy Milvae Cochlear–implant simulated signal degradation exacerbates listening effort in older listeners Journal Article In: Ear & Hearing, vol. 45, no. 2, pp. 441–450, 2024. @article{Abramowitz2024,Objectives: Individuals with cochlear implants (CIs) often report that listening requires high levels of effort. Listening effort can increase with decreasing spectral resolution, which occurs when listening with a CI, and can also increase with age. What is not clear is whether these factors interact; older CI listeners potentially experience even higher listening effort with greater signal degradation than younger CI listeners. This study used pupillometry as a physiological index of listening effort to examine whether age, spectral resolution, and their interaction affect listening effort in a simulation of CI listening. Design: Fifteen younger normal-hearing listeners (ages 18 to 31 years) and 15 older normal-hearing listeners (ages 65 to 75 years) participated in this experiment; they had normal hearing thresholds from 0.25 to 4 kHz. Participants repeated sentences presented in quiet that were either unprocessed or vocoded, simulating CI listening. Stimuli frequency spectra were limited to below 4 kHz (to control for effects of age-related high-frequency hearing loss), and spectral resolution was decreased by decreasing the number of vocoder channels, with 32-, 16-, and 8-channel conditions. Behavioral speech recognition scores and pupil dilation were recorded during this task. In addition, cognitive measures of working memory and processing speed were obtained to examine if individual differences in these measures predicted changes in pupil dilation. Results: For trials where the sentence was recalled correctly, there was a significant interaction between age and spectral resolution, with significantly greater pupil dilation in the older normal-hearing listeners for the 8- and 32-channel vocoded conditions. Cognitive measures did not predict pupil dilation. Conclusions: There was a significant interaction between age and spectral resolution, such that older listeners appear to exert relatively higher listening effort than younger listeners when the signal is highly degraded, with the largest effects observed in the eight-channel condition. The clinical implication is that older listeners may be at higher risk for increased listening effort with a CI. |
2023 |
Laoura Ziaka; Athanassios Protopapas Cognitive control beyond single-item tasks: insights from pupillometry, gaze, and behavioral measures Journal Article In: Journal of Experimental Psychology: Human Perception and Performance, vol. 49, no. 7, pp. 968–988, 2023. @article{Ziaka2023,Cognitive control has been typically examined using single-item tasks. This has implications for the generalizability of theories of control implementation. Previous studies have revealed that different control demands are posed by tasks depending on whether they present stimuli individually (i.e., single-item) or simultaneously in array format (i.e., multi-item). In the present study, we tracked within-task performance in single-item and multi-item Stroop tasks using simultaneous pupillometry, gaze, and behavioral response measures, aiming to explore the implications of format differences for cognitive control. The results indicated within-task performance decline in the multi-item version of the Stroop task, accompanied by pupil constriction and dwell time increase, in both the incongruent and the neutral condition. In contrast, no performance decline or dwell time increase was observed in the course of the single-item version of the task.We interpret these findings in terms of capacity constraints on cognitive control, with implications for cognitive control research, and highlight the need for better understanding of the cognitive demands of multi-item tasks. |
Junyi Zhou; Wenjie Zhuang Physically active undergraduates perform better on executive-related oculomotor control: Evidence from the antisaccade task and pupillometry Journal Article In: PsyCh Journal, vol. 12, no. 1, pp. 17–24, 2023. @article{Zhou2023e,Previous studies have shown that exercise can improve executive function in young and older adults. However, it remains controversial whether a sufficient amount of physical activity leads to higher-level executive function. To examine the effect of physical activity on executive function, we used eye-tracking technology and the antisaccade task in 41 young undergraduates with various levels of physical activity. Moreover, we also investigated their differences in cognitive ability by examining their pupil size during the antisaccade task. Eye-tracking results showed that physically active individuals showed shorter saccade latency and higher accuracy in the antisaccade task than their physically inactive counterparts. Furthermore, the former showed larger pupil size during the preparatory period of antisaccade. These findings suggest that individuals with higher-level physical activity have higher-level executive function. The larger pupil sizes of physically active individuals may imply that their locus coeruleus-norepinephrine system and executive-related prefrontal cortex are more active, which contributes to their higher-level cognitive ability. |
Yao Yao; Katrina Connell; Stephen Politzer-Ahles Hearing emotion in two languages : A pupillometry study of Cantonese – Mandarin bilinguals ' perception of affective cognates in L1 and L2 Journal Article In: Bilingualism: Language and Cognition, pp. 1–14, 2023. @article{Yao2023,Differential affective processing has been widely documented for bilinguals: L1 affective words elicit higher levels of arousal and stronger emotionality ratings than L2 affective words (Pavlenko, 2012). In this study, we focus on two closely related Chinese languages, Mandarin and Cantonese, whose affective lexicons are highly overlapping, with shared lexical items that only differ in pronunciation across languages. We recorded L1 Cantonese – L2 Mandarin bilinguals' pupil responses to auditory tokens of Cantonese and Mandarin affective words. Our results showed that Cantonese–Mandarin bilinguals had stronger pupil responses when the affective words were pronounced in Cantonese (L1) than when the same words were pronounced in Mandarin (L2). The effect was most evident in taboo words and among bilinguals with lower L2 proficiency. We discuss the theoretical implications of the findings in the frameworks of exemplar theory and models of the bilingual lexicon. © |
Yao Yao; Katrina Connell; Stephen Politzer-Ahles Hearing emotion in two languages: A pupillometry study of Cantonese–Mandarin bilinguals' perception of affective cognates in L1 and L2 Journal Article In: Bilingualism: Language and Cognition, vol. 26, no. 4, pp. 795–808, 2023. @article{Yao2023b,Differential affective processing has been widely documented for bilinguals: L1 affective words elicit higher levels of arousal and stronger emotionality ratings than L2 affective words (Pavlenko, 2012). In this study, we focus on two closely related Chinese languages, Mandarin and Cantonese, whose affective lexicons are highly overlapping, with shared lexical items that only differ in pronunciation across languages. We recorded L1 Cantonese – L2 Mandarin bilinguals' pupil responses to auditory tokens of Cantonese and Mandarin affective words. Our results showed that Cantonese–Mandarin bilinguals had stronger pupil responses when the affective words were pronounced in Cantonese (L1) than when the same words were pronounced in Mandarin (L2). The effect was most evident in taboo words and among bilinguals with lower L2 proficiency. We discuss the theoretical implications of the findings in the frameworks of exemplar theory and models of the bilingual lexicon. |
Weizhen Xie; Weiwei Zhang Pupillary evidence reveals the influence of conceptual association on brightness perception Journal Article In: Psychonomic Bulletin & Review, vol. 30, no. 4, pp. 1388–1395, 2023. @article{Xie2023a,Our visual experience often varies based on momentary thoughts and feelings. For example, when positive concepts are invoked, visual objects may appear brighter (e.g., a “brighter” smile). However, it remains unclear whether this phenomenological experience is driven by a genuine top-down modulation of brightness perception or by a mere response bias. To investigate this issue, we use pupillometry as a more objective measure of perceived brightness. We asked participants to judge the brightness level of an isoluminant gray color patch after evaluating the valence of a positive or negative word. We found that the gray color patch elicited greater pupillary light reflex and more frequent “brighter” responses after observers had evaluated the valence of a positive word. As pupillary light reflex is unlikely driven by voluntary control, these results suggest that the conceptual association between affect and luminance can modulate brightness perception. |
Christian Wienke; Marcus Grueschow; Aiden Haghikia; Tino Zaehle In: Journal of Neuroscience, vol. 43, no. 36, pp. 6306–6319, 2023. @article{Wienke2023,Transcutaneous auricular vagus nerve stimulation (taVNS) has been proposed to activate the locus ceruleus-noradrenaline (LC-NA) system. However, previous studies failed to find consistent modulatory effects of taVNS on LC-NA biomarkers. Previous studies suggest that phasic taVNS may be capable of modulating LC-NA biomarkers such as pupil dilation and alpha oscillations. However, it is unclear whether these effects extend beyond pure sensory vagal nerve responses. Critically, the potential of the pupillary light reflex as an additional taVNS biomarker has not been explored so far. Here, we applied phasic active and sham taVNS in 29 subjects (16 female, 13 male) while they performed an emotional Stroop task (EST) and a passive pupil light reflex task (PLRT). We recorded pupil size and brain activity dynamics using a combined Magnetoencephalography (MEG) and pupillometry design. Our results show that phasic taVNS significantly increased pupil dilation and performance during the EST. During the PLRT, active taVNS reduced and delayed pupil constriction. In the MEG, taVNS increased frontal-midline theta and alpha power during the EST, whereas occipital alpha power was reduced during both the EST and PLRT. Our findings provide evidence that phasic taVNS systematically modulates behavioral, pupillary, and electrophysiological parameters of LC-NA activity during cognitive processing. Moreover, we demonstrate for the first time that the pupillary light reflex can be used as a simple and effective proxy of taVNS efficacy. These findings have important implications for the development of noninvasive neuromodulation interventions for various cognitive and clinical applications. |
Sarah Villard; Tyler K. Perrachione; Sung-Joo Lim; Ayesha Alam; Gerald Kidd In: The Journal of the Acoustical Society of America, vol. 154, no. 2, pp. 1152–1167, 2023. @article{Villard2023,The task of processing speech masked by concurrent speech/noise can pose a substantial challenge to listeners. However, performance on such tasks may not directly reflect the amount of listening effort they elicit. Changes in pupil size and neural oscillatory power in the alpha range (8–12 Hz) are prominent neurophysiological signals known to reflect listening effort; however, measurements obtained through these two approaches are rarely correlated, suggesting that they may respond differently depending on the specific cognitive demands (and, by extension, the specific type of effort) elicited by specific tasks. This study aimed to compare changes in pupil size and alpha power elicited by different types of auditory maskers (highly confusable intelligible speech maskers, speech-envelope-modulated speech-shaped noise, and unmodulated speech-shaped noise maskers) in young, normal-hearing listeners. Within each condition, the target-to-masker ratio was set at the participant's individually estimated 75% correct point on the psychometric function. The speech masking condition elicited a significantly greater increase in pupil size than either of the noise masking conditions, whereas the unmodulated noise masking condition elicited a significantly greater increase in alpha oscillatory power than the speech masking condition, suggesting that the effort needed to solve these respective tasks may have different neural origins. |
Anna Cornelia Stausberg Who are you to judge? Investigating narcissism through pupil dilation at witness testimonials Journal Article In: Psychology, vol. 14, no. 02, pp. 144–157, 2023. @article{Stausberg2023,Various literature has explored narcissistic behaviour and its distinct emotional and aggressive nature. However, there is a significant gap in research when exploring the physiological differences of individual classified as having high narcissistic traits and tendencies. Pupillometry is a measure often utilised to measure distinct differences in emotional arousal. This measure allows for excellent insight into physiological responses to various stimuli presented. This exploratory study was set up to investigate pupillometry responses to auditory stimulation in narcissistic versus control participants. Findings were consistent with previous research; however, various limitations hindered significant findings. This pilot study is a guideline for future research as it is a first attempt at exploring a physiological relationship between emotionality and narcissism in the context of a criminal hearing. |
Olympia Simantiraki; Anita E. Wagner; Martin Cooke The impact of speech type on listening effort and intelligibility for native and non-native listeners Journal Article In: Frontiers in Neuroscience, vol. 17, pp. 1–16, 2023. @article{Simantiraki2023,Listeners are routinely exposed to many different types of speech, including artificially-enhanced and synthetic speech, styles which deviate to a greater or lesser extent from naturally-spoken exemplars. While the impact of differing speech types on intelligibility is well-studied, it is less clear how such types affect cognitive processing demands, and in particular whether those speech forms with the greatest intelligibility in noise have a commensurately lower listening effort. The current study measured intelligibility, self-reported listening effort, and a pupillometry-based measure of cognitive load for four distinct types of speech: (i) plain i.e. natural unmodified speech; (ii) Lombard speech, a naturally-enhanced form which occurs when speaking in the presence of noise; (iii) artificially-enhanced speech which involves spectral shaping and dynamic range compression; and (iv) speech synthesized from text. In the first experiment a cohort of 26 native listeners responded to the four speech types in three levels of speech-shaped noise. In a second experiment, 31 non-native listeners underwent the same procedure at more favorable signal-to-noise ratios, chosen since second language listening in noise has a more detrimental effect on intelligibility than listening in a first language. For both native and non-native listeners, artificially-enhanced speech was the most intelligible and led to the lowest subjective effort ratings, while the reverse was true for synthetic speech. However, pupil data suggested that Lombard speech elicited the lowest processing demands overall. These outcomes indicate that the relationship between intelligibility and cognitive processing demands is not a simple inverse, but is mediated by speech type. The findings of the current study motivate the search for speech modification algorithms that are optimized for both intelligibility and listening effort. |
Eser Sendesen; Samet Kılıç; Nurhan Erbil; Özgür Aydın; Didem Turkyilmaz An exploratory study of the effect of tinnitus on listening effort using EEG and pupillometry Journal Article In: Otolaryngology-Head and Neck Surgery, vol. 169, no. 5, pp. 1259–1267, 2023. @article{Sendesen2023,Objective: Previous behavioral studies on listening effort in tinnitus patients did not consider extended high-frequency hearing thresholds and had conflicting results. This inconsistency may be related that listening effort is not evaluated by the central nervous system (CNS) and autonomic nervous system (ANS), which are directly related to tinnitus pathophysiology. This study matches hearing thresholds at all frequencies, including the extended high-frequency and reduces hearing loss to objectively evaluate listening effort over the CNS and ANS simultaneously in tinnitus patients. Study Design: Case-control study. Setting: University hospital. Methods: Sixteen chronic tinnitus patients and 23 matched healthy controls having normal pure-tone averages with symmetrical hearing thresholds were included. Subjects were evaluated with 0.125 to 20 kHz pure-tone audiometry, Montreal Cognitive Assessment Test (MoCA), Tinnitus Handicap Inventory (THI), Visual Analog Scale (VAS), electroencephalography (EEG), and pupillometry. Results: Pupil dilation and EEG alpha band in the “coding” phase of the sentence presented in tinnitus patients was less than in the control group (p <.05). VAS score was higher in the tinnitus group (p <.01). Also, there was no statistically significant relationship between EEG and pupillometry components and THI or MoCA (p >.05). Conclusion: This study suggests that tinnitus patients may need to make an extra effort to listen. Also, pupillometry may not be sufficiently reliable to assess listening effort in ANS-related pathologies. Considering the possible listening difficulties in tinnitus patients, reducing the listening difficulties, especially in noisy environments, can be added to the goals of tinnitus therapy protocols. |
Mihaela Beatrice Neagu; Abigail A. Kressner; Helia Relaño-Iborra; Per Bækgaard; Torsten Dau; Dorothea Wendt Investigating the reliability of pupillometry as a measure of individualized listening effort Journal Article In: Trends in Hearing, vol. 27, pp. 1–20, 2023. @article{Neagu2023,Recordings of the pupillary response have been used in numerous studies to assess listening effort during a speech-in-noise task. Most studies focused on averaged responses across listeners, whereas less is known about pupil dilation as an indicator of the individuals' listening effort. The present study investigated the reliability of several pupil features as potential indicators of individual listening effort and the impact of different normalization procedures on the reliability. The pupil diameters of 31 normal-hearing listeners were recorded during multiple visits while performing a speech-in-noise task. The signal-to-noise ratios (SNRs) of the stimuli ranged from (Formula presented.) 12 dB to (Formula presented.) 4 dB. All listeners were measured twice at separate visits, and 11 were re-tested at a third visit. To examine the reliability of the pupil responses across visits, the intraclass correlation coefficient was applied to the peak and mean pupil dilation and to the temporal features of the pupil response, extracted using growth curve analysis. The reliability of the pupillary response was assessed in relation to SNR and different normalization procedures over multiple visits. The most reliable pupil features were the traditional mean and peak pupil dilation. The highest reliability results were obtained when the data were baseline-corrected and normalized to the individual pupil response range across all visits. Moreover, the present study results showed only a minor impact of the SNR and the number of visits on the reliability of the pupil response. Overall, the results may provide an important basis for developing a standardized test for pupillometry in the clinic. |
Yoichi Mukai; Juhani Järvikivi; Benjamin V. Tucker The role of phonology-to-orthography consistency in predicting the degree of pupil dilation induced in processing reduced and unreduced speech Journal Article In: Applied Psycholinguistics, vol. 44, no. 5, pp. 784–815, 2023. @article{Mukai2023,The relationship between the ways in which words are pronounced and spelled has been shown to affect spoken word processing, and a consistent relationship between pronunciation and spelling has been reported as a possible cause of unreduced pronunciations being easier to process than reduced counterparts although reduced pronunciations occur more frequently. In the present study, we investigate the effect of pronunciation-to-spelling consistency for reduced and unreduced pronunciations in L1 and L2 listeners of a logographic language. More precisely, we compare L1 and L2 Japanese listeners to probe whether they use orthographic information differently when processing reduced and unreduced speech. Using pupillometry, the current study provides evidence that extends the hypothesis about the role of orthography in the processing of reduced speech. Orthographic realization matters in processing for L1 and L2 advanced listeners. More specifically, how consistent the orthographic realization is with its phonological form (phonology-to-orthography consistency) modulates the extent to which reduced pronunciation induces additional processing costs. The results are further discussed in terms of their implications for how listeners process reduced speech and the role of the orthographic form in speech processing. |
Sreenivasan Meyyappan; Abhijit Rajan; George R. Mangun; Mingzhou Ding Top-down control of the left visual field bias in cued visual spatial attention Journal Article In: Cerebral Cortex, vol. 33, no. 9, pp. 5097–5107, 2023. @article{Meyyappan2023,A left visual field (LVF) bias in perceptual judgments, response speed, and discrimination accuracy has been reported in humans. Cognitive factors, such as visual spatial attention, are known to modulate or even eliminate this bias. We investigated this problem by recording pupillometry together with functional magnetic resonance imaging (fMRI) in a cued visual spatial attention task. We observed that (i) the pupil was significantly more dilated following attend-right than attend-left cues, (ii) the task performance (e.g. reaction time [RT]) did not differ between attend-left and attend-right trials, and (iii) the difference in cue-related pupil dilation between attend-left and attend-right trials was inversely related to the corresponding difference in RT. Neuroscientically, correlating the difference in cue-related pupil dilation with the corresponding cue-related fMRI difference yielded activations primarily in the right hemisphere, including the right intraparietal sulcus and the right ventrolateral prefrontal cortex. These results suggest that (i) there is an asymmetry in visual spatial attention control, with the rightward attention control being more effortful than the leftward attention control, (ii) this asymmetry underlies the reduction or the elimination of the LVF bias, and (iii) the components of the attentional control networks in the right hemisphere are likely part of the neural substrate of the observed asymmetry in attentional control. |
Drew J. McLaughlin; Maggie E. Zink; Lauren Gaunt; Jamie Reilly; Mitchell S. Sommers; Kristin J. Van Engen; Jonathan E. Peelle Give me a break! Unavoidable fatigue effects in cognitive pupillometry Journal Article In: Psychophysiology, vol. 60, no. 7, pp. 1–20, 2023. @article{McLaughlin2023,Pupillometry has a rich history in the study of perception and cognition. One perennial challenge is that the magnitude of the task-evoked pupil response diminishes over the course of an experiment, a phenomenon we refer to as a fatigue effect. Reducing fatigue effects may improve sensitivity to task effects and reduce the likelihood of confounds due to systematic physiological changes over time. In this paper, we investigated the degree to which fatigue effects could be ameliorated by experimenter intervention. In Experiment 1, we assigned participants to one of three groups—no breaks, kinetic breaks (playing with toys, but no social interaction), or chatting with a research assistant—and compared the pupil response across conditions. In Experiment 2, we additionally tested the effect of researcher observation. Only breaks including social interaction significantly reduced the fatigue of the pupil response across trials. However, in all conditions we found robust evidence for fatigue effects: that is, regardless of protocol, the task-evoked pupil response was substantially diminished (at least 60%) over the duration of the experiment. We account for the variance of fatigue effects in our pupillometry data using multiple common statistical modeling approaches (e.g., linear mixed-effects models of peak, mean, and baseline pupil diameters, as well as growth curve models of time-course data). We conclude that pupil attenuation is a predictable phenomenon that should be accommodated in our experimental designs and statistical models. |
Jacie R. McHaney; William L. Schuerman; Matthew K. Leonard; Bharath Chandrasekaran Transcutaneous auricular vagus nerve stimulation modulates performance but not pupil size during nonnative speech category learning Journal Article In: Journal of Speech, Language, and Hearing Research, vol. 66, no. 10, pp. 3825–3843, 2023. @article{McHaney2023,Purpose: Subthreshold transcutaneous auricular vagus nerve stimulation (taVNS) synchronized with behavioral training can selectively enhance nonnative speech category learning in adults. Prior work has demonstrated that behavioral performance increases when taVNS is paired with easier-to-learn Mandarin tone categories in native English listeners, relative to when taVNS is paired with harder-to-learn Mandarin tone categories or without taVNS. Mechanistically, this temporally precise plasticity has been attributed to noradrenergic modulation. However, prior work did not specifically utilize methodologies that indexed nor-adrenergic modulation and, therefore, was unable to explicitly test this hypothesis. Our goal for this study was to use pupillometry to gain mechanistic insights into taVNS behavioral effects. Method: Thirty-eight participants learned to categorize Mandarin tones while pupil-lometry was recorded. In a double-blinded design, participants were divided into two taVNS groups that, as in the prior study, differed according to whether taVNS was paired with easier-to-learn tones or harder-to-learn tones. Learning performance and pupillary responses were measured using linear mixed-effects models. Results: We found that taVNS did not have any tone-specific or group behav-ioral or pupillary effects. However, in an exploratory analysis, we observed that taVNS did lead to faster rates of learning on trials paired with stimulation, particularly for those who were stimulated at lower amplitudes. Conclusions: Our results suggest that pupillary responses may not be a reliable marker of locus coeruleus–norepinephrine system activity in humans. However, future research should systematically examine the effects of stimulation amplitude on both behavior and pupillary responses. |
Audrey Mazancieux; Franck Mauconduit; Alexis Amadon; Jan Willem de Gee; Tobias H. Donner; Florent Meyniel Brainstem fMRI signaling of surprise across different types of deviant stimuli Journal Article In: Cell Reports, vol. 42, no. 11, pp. 1–15, 2023. @article{Mazancieux2023,Detection of deviant stimuli is crucial to orient and adapt our behavior. Previous work shows that deviant stimuli elicit phasic activation of the locus coeruleus (LC), which releases noradrenaline and controls central arousal. However, it is unclear whether the detection of behaviorally relevant deviant stimuli selectively triggers LC responses or other neuromodulatory systems (dopamine, serotonin, and acetylcholine). We combine human functional MRI (fMRI) recordings optimized for brainstem imaging with pupillometry to perform a mapping of deviant-related responses in subcortical structures. Participants have to detect deviant items in a “local-global” paradigm that distinguishes between deviance based on the stimulus probability and the sequence structure. fMRI responses to deviant stimuli are distributed in many cortical areas. Both types of deviance elicit responses in the pupil, LC, and other neuromodulatory systems. Our results reveal that the detection of task-relevant deviant items recruits the same multiple subcortical systems across computationally different types of deviance. |
Sebastiaan Mathôt; Ana Vilotijević Methods in cognitive pupillometry: Design, preprocessing, and statistical analysis Journal Article In: Behavior Research Methods, vol. 55, no. 6, pp. 3055–3077, 2023. @article{Mathot2023a,Cognitive pupillometry is the measurement of pupil size to investigate cognitive processes such as attention, mental effort, working memory, and many others. Currently, there is no commonly agreed-upon methodology for conducting cognitive-pupillometry experiments, and approaches vary widely between research groups and even between different experiments from the same group. This lack of consensus makes it difficult to know which factors to consider when conducting a cognitive-pupillometry experiment. Here we provide a comprehensive, hands-on guide to methods in cognitive pupillometry, with a focus on trial-based experiments in which the measure of interest is the task-evoked pupil response to a stimulus. We cover all methodological aspects of cognitive pupillometry: experimental design, preprocessing of pupil-size data, and statistical techniques to deal with multiple comparisons when testing pupil-size data. In addition, we provide code and toolboxes (in Python) for preprocessing and statistical analysis, and we illustrate all aspects of the proposed workflow through an example experiment and example scripts. |
Elke B. Lange; Lauren K. Fink Eye blinking, musical processing, and subjective states—A methods account Journal Article In: Psychophysiology, vol. 60, no. 10, pp. 1–20, 2023. @article{Lange2023,Affective sciences often make use of self-reports to assess subjective states. Seeking a more implicit measure for states and emotions, our study explored spontaneous eye blinking during music listening. However, blinking is understudied in the context of research on subjective states. Therefore, a second goal was to explore different ways of analyzing blink activity recorded from infra-red eye trackers, using two additional data sets from earlier studies differing in blinking and viewing instructions. We first replicate the effect of increased blink rates during music listening in comparison with silence and show that the effect is not related to changes in self-reported valence, arousal, or to specific musical features. Interestingly, but in contrast, felt absorption reduced participants' blinking. The instruction to inhibit blinking did not change results. From a methodological perspective, we make suggestions about how to define blinks from data loss periods recorded by eye trackers and report a data-driven outlier rejection procedure and its efficiency for subject-mean analyses, as well as trial-based analyses. We ran a variety of mixed effects models that differed in how trials without blinking were treated. The main results largely converged across accounts. The broad consistency of results across different experiments, outlier treatments, and statistical models demonstrates the reliability of the reported effects. As recordings of data loss periods come for free when interested in eye movements or pupillometry, we encourage researchers to pay attention to blink activity and contribute to the further understanding of the relation between blinking, subjective states, and cognitive processing. |
Charlene L. M. Lam; Tom J. Barry; Jenny Yiend; Tatia M. C. Lee The role of consciousness in threat extinction learning Journal Article In: Consciousness and Cognition, vol. 116, pp. 1–12, 2023. @article{Lam2023,Extinction learning is regarded as a core mechanism underlying exposure therapy. The extent to which learned threats can be extinguished without conscious awareness is a controversial and on-going debate. We investigated whether implicit vs. explicit exposure to a threatened stimulus can modulate defence responses measured using pupillometry. Healthy participants underwent a threat conditioning paradigm in which one of the conditioned stimuli (CS) was perceptually suppressed using continuous flash suppression (CFS). Participants' pupillary responses, CS pleasantness ratings, and trial-by-trial awareness of the CS were recorded. During Extinction, participants' pupils dilated more in the trials in which they were unaware of the CS than in those in which they were aware of it (Cohen's d = 0.57). After reinstatement, the percentage of fear recovery was greater for the CFS-suppressed CS than the CS with full awareness. The current study suggests that the modulation of fear responses by extinction with reduced visual awareness is weaker compared to extinction with full perceptual awareness. |
Damian Koevoet; Christoph Strauch; Marnix Naber; Stefan Van Stigchel The costs of paying overt and covert attention assessed with pupillometry Journal Article In: Psychological Science, vol. 34, no. 8, pp. 887–898, 2023. @article{Koevoet2023a,Attention can be shifted with or without an accompanying saccade (i.e., overtly or covertly, respectively). Thus far, it is unknown how cognitively costly these shifts are, yet such quantification is necessary to understand how and when attention is deployed overtly or covertly. In our first experiment (N = 24 adults), we used pupillometry to show that shifting attention overtly is more costly than shifting attention covertly, likely because planning saccades is more complex. We pose that these differential costs will, in part, determine whether attention is shifted overtly or covertly in a given context. A subsequent experiment (N = 24 adults) showed that relatively complex oblique saccades are more costly than relatively simple saccades in horizontal or vertical directions. This provides a possible explanation for the cardinal-direction bias of saccades. The utility of a cost perspective as presented here is vital to furthering our understanding of the multitude of decisions involved in processing and interacting with the external world efficiently. |
Damian Koevoet; Marnix Naber; Christoph Strauch; Rosyl S. Somai; Stefan Van der Stigchel Differential aspects of attention predict the depth of visual working memory encoding: Evidence from pupillometry Journal Article In: Journal of Vision, vol. 23, no. 6, pp. 1–11, 2023. @article{Koevoet2023,What determines how much one encodes into visual working memory? Traditionally, encoding depth is considered to be indexed by spatiotemporal properties of gaze, such as gaze position and dwell time. Although these properties inform about where and how long one looks, they do not necessarily inform about the current arousal state or how strongly attention is deployed to facilitate encoding. Here, we found that two types of pupillary dynamics predict how much information is encoded during a copy task. The task involved encoding a spatial pattern of multiple items for later reproduction. Results showed that smaller baseline pupil sizes preceding and stronger pupil orienting responses during encoding predicted that more information was encoded into visual working memory. Additionally, we show that pupil size reflects not only how much but also how precisely material is encoded. We argue that a smaller pupil size preceding encoding is related to increased exploitation, whereas larger pupil constrictions signal stronger attentional (re)orienting to the to-be-encoded pattern. Our findings support the notion that the depth of visual working memory encoding is the integrative outcome of differential aspects of attention: how alert one is, how much attention one deploys, and how long it is deployed. Together, these factors determine how much information is encoded into visual working memory. |
Jiae Kim; Jiyeon Lee; Sang Beom Jun; Jee Eun Sung Pupillometry as a window to detect cognitive aging in the brain Journal Article In: Biomedical Engineering Letters, pp. 91–101, 2023. @article{Kim2023b,This study investigated whether there are aging-related differences in pupil dilation (pupillometry) while the cognitive load is manipulated using digit- and word-span tasks. A group of 17 younger and 15 cognitively healthy older adults performed digit- and word-span tasks. Each task comprised three levels of cognitive loads with 10 trials for each level. For each task, the recall accuracy and the slope of pupil dilation were calculated and analyzed. The raw signal of measured pupil size was low-pass filtered and interpolated to eliminate blinking artifacts and spike noises. Two-way ANOVA was used for statistical analyses. For the recall accuracy, the significant group differences emerged as the span increases in digit-span (5- vs. 7-digit) and word-span (4- vs. 5-word) tasks, while the group differences were not significant on 3-digit- and 3-word-span tasks with lower cognitive load. In digit-span tasks, there was no aging-related difference in the slope of pupil dilation. However, in word-span tasks, the slope of pupil dilation differed significantly between two groups as cognitive load increased, indicating that older adults presented a higher pupil dilation slope than younger adults especially under the conditions with higher cognitive load. The current study found significant aging effects in the pupil dilations under the more cognitive demanding span tasks when the types of span tasks varied (e.g., digit vs. word). The manipulations successfully elicited differential aging effects, given that the aging effects became most salient under word-span tasks with greater cognitive load especially under the maximum length. |
Jeff Huang; Donald Brien; Brian C. Coe; Giulia Longoni; Donald J. Mabbott; Douglas P. Munoz; E. Ann Yeh Delayed oculomotor response associates with optic neuritis in youth with demyelinating disorders Journal Article In: Multiple Sclerosis and Related Disorders, vol. 79, pp. 1–9, 2023. @article{Huang2023a,Introduction: Impairment in visual and cognitive functions occur in youth with demyelinating disorders such as multiple sclerosis, neuromyelitis optica spectrum disorder, and myelin oligodendrocyte glycoprotein antibody-associated disease. Quantitative behavioral assessment using eye-tracking and pupillometry can provide functional metrics for important prognostic and clinically relevant information at the bedside. Methods: Children and adolescents diagnosed with demyelinating disorders and healthy, age-matched controls completed an interleaved pro- and anti-saccade task using video-based eye-tracking and underwent spectral-domain optical coherence tomography examination for evaluation of retinal nerve fiber layer and ganglion cell inner plexiform layer thickness. Low-contrast visual acuity and Symbol Digit Modalities Test were performed for visual and cognitive functional assessments. We assessed saccade and pupil parameters including saccade reaction time, direction error rate, pupil response latency, peak constriction time, and peak constriction and dilation velocities. Generalized Estimating Equations were used to examine the association of eye-tracking parameters with optic neuritis history, structural metrics, and visual and cognitive scores. Results: The study included 36 demyelinating disorders patients, aged 8–18 yrs. (75% F; median = 15.22 yrs. |
Hengda He; Linbi Hong; Paul Sajda Pupillary response is associated with the reset and switching of functional brain networks during salience processing Journal Article In: PLoS Computational Biology, vol. 19, no. 5, pp. 1–29, 2023. @article{He2023a,The interface between processing internal goals and salient events in the environment involves various top-down processes. Previous studies have identified multiple brain areas for salience processing, including the salience network (SN), dorsal attention network, and the locus coeruleus-norepinephrine (LC-NE) system. However, interactions among these systems in salience processing remain unclear. Here, we simultaneously recorded pupillometry, EEG, and fMRI during an auditory oddball paradigm. The analyses of EEG and fMRI data uncovered spatiotemporally organized target-associated neural correlates. By modeling the target-modulated effective connectivity, we found that the target-evoked pupillary response is associated with the network directional couplings from late to early subsystems in the trial, as well as the network switching initiated by the SN. These findings indicate that the SN might cooperate with the pupil-indexed LC-NE system in the reset and switching of cortical networks, and shed light on their implications in various cognitive processes and neurological diseases. |
Althea Frisanco; Marco Biella; Marco Brambilla; Mariska E. Kret All that meets the eye: The contribution of reward processing and pupil mimicry on pupillary reactions to facial trustworthiness Journal Article In: Current Psychology, vol. 42, no. 14, pp. 11685–11692, 2023. @article{Frisanco2023,The present work investigates pupillary reactions induced by exposure to faces with different levels of trustworthiness. Participants' (N = 69) pupillary changes were recorded while they viewed white male faces with a neutral expression varying on facial trustworthiness. Results suggest that reward processing and pupil mimicry are relevant mechanisms driving participants' pupil reactions. However, when including both factors in one statistical model, pupil mimicry seems to be a stronger predictor than reward processing of participants' pupil dilation. Results are discussed in light of pupillometry evidence. |
Jamal Esmaily; Sajjad Zabbah; Reza Ebrahimpour; Bahador Bahrami Interpersonal alignment of neural evidence accumulation to social exchange of confidence Journal Article In: eLife, vol. 12, pp. 1–27, 2023. @article{Esmaily2023,Private, subjective beliefs about uncertainty have been found to have idiosyncratic computational and neural substrates yet, humans share such beliefs seamlessly and cooperate successfully. Bringing together decision making under uncertainty and interpersonal alignment in communication, in a discovery plus pre-registered replication design, we examined the neuro-computational basis of the relationship between privately held and socially shared uncertainty. Examining confidence-speed-accuracy trade-off in uncertainty-ridden perceptual decisions under social vs isolated context, we found that shared (i.e. reported confidence) and subjective (inferred from pupillometry) uncertainty dynamically followed social information. An attractor neural network model incorporating social information as top-down additive input captured the observed behavior and demonstrated the emergence of social alignment in virtual dyadic simulations. Electroencephalography showed that social exchange of confidence modulated the neural signature of perceptual evidence accumulation in the central parietal cortex. Our findings offer a neural population model for interpersonal alignment of shared beliefs. |
Ciara Egan; Joshua S. Payne; Manon W. Jones In: Neuropsychologia, vol. 184, pp. 1–8, 2023. @article{Egan2023,Readers with developmental dyslexia are known to be impaired in representing and accessing phonology, but their ability to process meaning is generally considered to be intact. However, neurocognitive studies show evidence of a subtle semantic processing deficit in dyslexic readers, relative to their typically-developing peers. Here, we compared dyslexic and typical adult readers on their ability to judge semantic congruency (congruent vs. inconcongruent) in short, two-word phrases, which were further manipulated for phonological relatedness (alliterating vs. non-alliterating); “dazzling-diamond”; “sparkling-diamond”; “dangerous-diamond”; and “creepy-diamond”. At the level of behavioural judgement, all readers were less accurate when evaluating incongruent alliterating items compared with incongruent non-aliterating, suggesting that phonological patterning creates the illusion of semantic congruency (as per Egan et al., 2020). Dyslexic readers showed a similar propensity for this form-meaning relationship despite a phonological processing impairment as evidenced in the cognitive and literacy indicative assessments. Dyslexic readers also showed an overall reduction in the ability to accurately judge semantic congruency, suggestive of a subtle semantic impairment. Whilst no group differences emerged in the electrophysiological measures, our pupil dilation measurements revealed a global tendency for dyslexic readers to manifest a reduced attentional response to these word stimuli, compared with typical readers. Our results show a broad manifestation of neurocognitive differences in adult dyslexic and typical readers' processing of print, at the level of autonomic arousal as well as in higher level semantic judgements. |
Tjerk T. Dercksen; Andreas Widmann; Nicole Wetzel Salient omissions—pupil dilation in response to unexpected omissions of sound and touch Journal Article In: Frontiers in Psychiatry, vol. 14, pp. 1–11, 2023. @article{Dercksen2023,Introduction: Recent theories describe perception as an inferential process based on internal predictive models adjusted by means of prediction violations (prediction error). To study and demonstrate predictive processing in the brain the use of unexpected stimulus omissions has been suggested as a promising approach as the evoked brain responses are uncontaminated by responses to stimuli. Here, we aimed to investigate the pupil's response to unexpected stimulus omissions in order to better understand surprise and orienting of attention resulting from prediction violation. So far only few studies have used omission in pupillometry research and results have been inconsistent. Methods: This study adapted an EEG paradigm that has been shown to elicit omission responses in auditory and somatosensory modalities. Healthy adults pressed a button at their own pace, which resulted in the presentation of sounds or tactile stimuli in either 88%, 50% or 0% (motor-control) of cases. Pupil size was recorded continuously and averaged to analyze the pupil dilation response associated with each condition. Results: Results revealed that omission responses were observed in both modalities in the 88%-condition compared to motor-control. Similar pupil omission responses were observed between modalities, suggesting modality-unspecific activation of the underlying brain circuits. Discussion: In combination with previous omission studies using EEG, the findings demonstrate predictive models in brain processing and point to the involvement of subcortical structures in the omission response. Our pupillometry approach is especially suitable to study sensory prediction in vulnerable populations within the psychiatric field. |
Wolf Culemann; Leana Neuber; Angela Heine Pupil vs. eyelid: Evaluating the accuracy of blink detection in pupil-based eye tracking devices Journal Article In: Procedia Computer Science, vol. 225, pp. 2008–2017, 2023. @article{Culemann2023,Spontaneous eye blinks as a means to tap into human cognitive and affective processes have been demonstrated to offer valuable Spontaneous eye blinks as a means to tap into human cognitive and affective processes have been demonstrated to offer valuable insight in various fields of application, such as human-computer-interaction and driver drowsiness detection. Furthermore, studies insight in various fields of application, such as human-computer-interaction and driver drowsiness detection. Furthermore, studies focusing on gaze tracking and pupillometry have to deal with eye blinks as artifacts, leading to distortions in gaze and pupil data. insight in various fields of application, such as human-computer-interaction and driver drowsiness detection. Furthermore, studies focusing on gaze tracking and pupillometry have to deal with eye blinks as artifacts, leading to distortions in gaze and pupil data. focusing on gaze tracking and pupillometry have to deal with eye blinks as artifacts, leading to distortions in gaze and pupil data. Accurate detection of blinks and calculation of their duration is, consequently, essential for both areas of research and development. In the present study, the accuracy of standard blink detection approaches based on pupil-based eye trackers was investigated with Accurate detection of blinks and calculation of their duration is, consequently, essential for both areas of research and development. Accurate detection of blinks and calculation of their duration is, consequently, essential for both areas of research and development. In the present study, the accuracy of standard blink detection approaches based on pupil-based eye trackers was investigated with In the present study, the accuracy of standard blink detection approaches based on pupil-based eye trackers was investigated with respect to blink durations as well as blink occurrence detection for different eye closure thresholds. We compared the results respect to blink durations as well as blink occurrence detection for different eye closure thresholds. We compared the results respect to blink durations as well as blink occurrence detection for different eye closure thresholds. We compared the results from two different detection algorithms and two eye tracking devices against ground truth data, i.e. blink events that were labeled manually on a frame-by-frame basis from video recordings. The Eye Aspect Ratio was used for the estimation of blink amplitudes from two different detection algorithms and two eye tracking devices against ground truth data, i.e. blink events that were labeled manually on a frame-by-frame basis from video recordings. The Eye Aspect Ratio was used for the estimation of blink amplitudes and eye closure. The results reveal significant differences in blink detection rate and blink duration depending on the method from two different detection algorithms and two eye tracking devices against ground truth data, i.e. blink events that were labeled manually on a frame-by-frame basis from video recordings. The Eye Aspect Ratio was used for the estimation of blink amplitudes and eye closure. The results reveal significant differences in blink detection rate and blink duration depending on the method used. Blink detection by pupil-based eye tracking devices is demonstrated to be systematically influenced by the choice of device, and eye closure. The results reveal significant differences in blink detection rate and blink duration depending on the method used. Blink detection by pupil-based eye tracking devices is demonstrated to be systematically influenced by the choice of device, detection algorithm, and blink definition. Our results highlight the importance of discussing different approaches to blink detection used. Blink detection by pupil-based eye tracking devices is demonstrated to be systematically influenced by the choice of device, when comparing blink metrics across different devices and applications. In addition, we encourage dialogue within the community to transparently communicate blink detection methods and discuss their impact on study outcomes. detection algorithm, and blink definition. Our results highlight the importance of discussing different approaches to blink detection detection algorithm, and blink definition. Our results highlight the importance of discussing different approaches to blink detection when comparing blink metrics across different devices and applications. In addition, we encourage dialogue within the community when comparing blink metrics across different devices and applications. In addition, we encourage dialogue within the community to transparently communicate blink detection methods and discuss their impact on study outcomes. |
M. Eric Cui; Björn Herrmann Eye movements decrease during effortful speech listening Journal Article In: Journal of Neuroscience, vol. 43, no. 32, pp. 5856–5869, 2023. @article{Cui2023,Hearing impairment affects many older adults but is often diagnosed decades after speech comprehension in noisy situations has become effortful. Accurate assessment of listening effort may thus help diagnose hearing impairment earlier. However, pupillometry—the most used approach to assess listening effort—has limitations that hinder its use in practice. The current study explores a novel way to assess listening effort through eye movements. Building on cognitive and neurophysiological work, we examine the hypothesis that eye movements decrease when speech listening becomes challenging. In three experiments with human participants from both sexes, we demonstrate, consistent with this hypothesis, that fixation duration increases and spatial gaze dispersion decreases with increasing speech masking. Eye movements decreased during effortful speech listening for different visual scenes (free viewing, object tracking) and speech materials (simple sentences, naturalistic stories). In contrast, pupillometry was less sensitive to speech masking during story listening, suggesting pupillometric measures may not be as effective for the assessments of listening effort in naturalistic speech-listening paradigms. Our results reveal a critical link between eye movements and cognitive load, suggesting that neural activity in the brain regions that support the regulation of eye movements, such as frontal eye field and superior colliculus, are modulated when listening is effortful. |
Sophie L. Cronin; Ottmar V. Lipp; Welber Marinovic Pupil dilation during encoding, but not type of auditory stimulation, predicts recognition success in face memory Journal Article In: Biological Psychology, vol. 178, pp. 1–10, 2023. @article{Cronin2023,We encounter and process information from multiple sensory modalities in our daily lives, and research suggests that learning can be more efficient when contexts are multisensory. In this study, we were interested in whether face identity recognition memory might be improved in multisensory learning conditions, and to explore associated changes in pupil dilation during encoding and recognition. In two studies participants completed old/new face recognition tasks wherein visual face stimuli were presented in the context of sounds. Faces were learnt alongside no sound, low arousal sounds (Experiment 1), high arousal non-face relevant, or high arousal face relevant (Experiment 2) sounds. We predicted that the presence of sounds during encoding would improve later recognition accuracy, however, the results did not support this with no effect of sound condition on memory. Pupil dilation, however, was found to predict later successful recognition both at encoding and during recognition. While these results do not provide support to the notion that face learning is improved under multisensory conditions relative to unisensory conditions, they do suggest that pupillometry may be a useful tool to further explore face identity learning and recognition. |
Gonçalo Cosme; Patrícia Arriaga; Pedro J. Rosa; Mitul A. Mehta; Diana Prata Temporal profile of intranasal oxytocin in the human autonomic nervous system at rest: An electrocardiography and pupillometry study Journal Article In: Journal of Psychopharmacology, vol. 37, no. 6, pp. 566–576, 2023. @article{Cosme2023,Background: Human social behavior is modulated by oxytocin (OT). Intranasal administration of OT (IN-OT) is a noninvasive route shown to elicit changes in the autonomic nervous system (ANS) activity; however, IN-OT's effect on the temporal profile of ANS activity at rest is yet to be described. Aims: We aimed to describe the temporal profile of IN-OT at six 10-min time windows from 15- to 100-min post-administration in 20 male participants at rest while continuously recording their pupillary in an eyes-open condition and cardiac activity in eyes-open and eyes-closed conditions. Methods: We used a double-blind, placebo-controlled, within-subjects design study where we extracted two proxies of parasympathetic nervous system (PNS) activity: high-frequency heart rate variability (HF-HRV) and pupillary unrest index (PUI); and a proxy of sympathetic nervous system activity: sample entropy of the pupillary unrest. Results: In the eyes-open condition, we found an effect of IN-OT on the proxies of PNS activity: decreased PUI in the three-time windows post-administration spanning 65–100 min, and as an exploratory finding, an increased HF-HRV in the 80–85 min time window. Conclusions: We suggest there is a role of OT in PNS regulation that may be consistent with OT's currently theorized role in the facilitation of alertness and approach behavior. |
Joseph Colantonio; Igor Bascandziev; Maria Theobald; Garvin Brod; Elizabeth Bonawitz In: Entropy, vol. 25, no. 2, pp. 1–24, 2023. @article{Colantonio2023,Bayesian models allow us to investigate children's belief revision alongside physiological states, such as “surprise”. Recent work finds that pupil dilation (or the “pupillary surprise response”) following expectancy violations is predictive of belief revision. How can probabilistic models inform the interpretations of “surprise”? Shannon Information considers the likelihood of an observed event, given prior beliefs, and suggests stronger surprise occurs following unlikely events. In contrast, Kullback–Leibler divergence considers the dissimilarity between prior beliefs and updated beliefs following observations—with greater surprise indicating more change between belief states to accommodate information. To assess these accounts under different learning contexts, we use Bayesian models that compare these computational measures of “surprise” to contexts where children are asked to either predict or evaluate the same evidence during a water displacement task. We find correlations between the computed Kullback–Leibler divergence and the children's pupillometric responses only when the children actively make predictions, and no correlation between Shannon Information and pupillometry. This suggests that when children attend to their beliefs and make predictions, pupillary responses may signal the degree of divergence between a child's current beliefs and the updated, more accommodating beliefs. |
Ivory Y. Chen; Aytaç Karabay; Sebastiaan Mathȏt; Howard Bowman; Elkan G. Akyürek Concealed identity information detection with pupillometry in rapid serial visual presentation Journal Article In: Psychophysiology, vol. 60, no. 1, pp. 1–14, 2023. @article{Chen2023n,The concealed information test (CIT) relies on bodily reactions to stimuli that are hidden in mind. However, people can use countermeasures, such as purposely focusing on irrelevant things, to confound the CIT. A new method designed to prevent countermeasures uses rapid serial visual presentation (RSVP) to present stimuli on the fringe of awareness. Previous studies that used RSVP in combination with electroencephalography (EEG) showed that participants exhibit a clear reaction to their real first name, even when they try to prevent such a reaction (i.e., when their name is concealed information). Because EEG is not easily applicable outside the laboratory, we investigated here whether pupil size, which is easier to measure, can also be used to detect concealed identity information. In our first study, participants adopted a fake name, and searched for this name in an RSVP task, while their pupil sizes were recorded. Apart from this fake name, their real name and a control name also appeared in the task. We found pupil dilation in response to the task-irrelevant real name, as compared to control names. However, while most participants showed this effect qualitatively, it was not statistically significant for most participants individually. In a second study, we preregistered the proof-of-concept methodology and replicated the original findings. Taken together, our results show that the current RSVP task with pupillometry can detect concealed identity information at a group level. Further development of the method is needed to create a valid and reliable concealed identity information detector at the individual level. |
Tom Bullock; Mary H. Maclean; Tyler Santander; Alexander P. Boone; Viktoriya Babenko; Neil M. Dundon; Alexander Stuber; Liann Jimmons; Jamie Raymer; Gold N. Okafor; Michael B. Miller; Barry Giesbrecht; Scott T. Grafton; Janice Marshall Habituation of the stress response multiplex to repeated cold pressor exposure Journal Article In: Frontiers in Psychology, vol. 13, pp. 1–23, 2023. @article{Bullock2023,Humans show remarkable habituation to aversive events as reflected by changes of both subjective report and objective measures of stress. Although much experimental human research focuses on the effects of stress, relatively little is known about the cascade of physiological and neural responses that contribute to stress habituation. The cold pressor test (CPT) is a common method for inducing acute stress in human participants in the laboratory; however, there are gaps in our understanding of the global state changes resulting from this stress-induction technique and how these responses change over multiple exposures. Here, we measure the stress response to repeated CPT exposures using an extensive suite of physiologic measures and state-of-the-art analysis techniques. In two separate sessions on different days, participants underwent five 90 s CPT exposures of both feet and five warm water control exposures, while electrocardiography (ECG), impedance cardiography, continuous blood pressure, pupillometry, scalp electroencephalography (EEG), salivary cortisol and self-reported pain assessments were recorded. A diverse array of adaptive responses are reported that vary in their temporal dynamics within each exposure as well as habituation across repeated exposures. During cold-water exposure there was a cascade of changes across several cardiovascular measures (elevated heart rate (HR), cardiac output (CO) and Mean Arterial Pressure (MAP) and reduced left ventricular ejection time (LVET), stroke volume (SV) and high- frequency heart rate variability (HF)). Increased pupil dilation was observed, as was increased power in low-frequency bands (delta and theta) across frontal EEG electrode sites. Several cardiovascular measures also habituated over repeated cold-water exposures (HR, MAP, CO, SV, LVET) as did pupil dilation and alpha frequency activity across the scalp. Anticipation of cold water induced stress effects in the time-period immediately prior to exposure, indexed by increased pupil size and cortical disinhibition in the alpha and beta frequency bands across central scalp sites. These results provide comprehensive insight into the evolution of a diverse array of stress responses to an acute noxious stressor, and how these responses adaptively contribute to stress habituation. |
Andreas Bär; Hannah E. Bär; Max Schneider; Fritz Renner The pupil as a window to the mind's eye: Greater emotionality of episodic imagery than verbal visualisation of rewarding activities Journal Article In: Journal of Experimental Psychopathology, vol. 14, no. 2, pp. 1–11, 2023. @article{Baer2023,Episodic imagery has been shown to amplify emotion more than abstract verbal representations. This may prove useful for clinical interventions aiming to motivate adaptive behaviours. However, most findings rely on self-report measures and verbal control conditions not designed to actively prevent automatic engagement in episodic imagery. We thus investigated the difference in emotionality between Episodic Imagery (EI) and Verbal Visualisation (VV) using pupil dilation as a physiological measure of emotional arousal. A sample of 75 participants listened to audio recordings describing activities in a positive manner. Subjects were randomly assigned to the EI or VV condition. Participants in the EI condition imagined performing the described activity, while participants in the VV condition visualised the words constituting the descriptions. As predicted, EI led to greater pupil dilation than VV, independent of mental effort. Self-reported anticipatory reward assessed throughout the task was also greater for EI than VV, yet no difference was found for arousal, anticipated reward or motivation. Our findings extend previous work demonstrating the property of episodic imagery to amplify emotion to a physiological level using pupillometry. However, we did not find a transfer to motivation, which is in line with previous studies using verbal control conditions for episodic imagery. |
2022 |
Cherie Zhou; Monicque M. Lorist; Sebastiaan Mathôt Is categorization in visual working memory a way to reduce mental effort? A pupillometry study Journal Article In: Cognitive Science, vol. 46, no. 9, pp. 1–15, 2022. @article{Zhou2022a,Recent studies on visual working memory (VWM) have shown that visual information can be stored in VWM as continuous (e.g., a specific shade of red) as well as categorical representations (e.g., the general category red). It has been widely assumed, yet never directly tested, that continuous representations require more VWM mental effort than categorical representations; given limited VWM capacity, this would mean that fewer continuous, as compared to categorical, representations can be maintained simultaneously. We tested this assumption by measuring pupil size, as a proxy for mental effort, in a delayed estimation task. Participants memorized one to four ambiguous (boundaries between adjacent color categories) or prototypical colors to encourage continuous or categorical representations, respectively; after a delay, a probe indicated the location of the to-be-reported color. We found that, for memory load 1, pupil size was larger while maintaining ambiguous as compared to prototypical colors, but without any difference in memory precision; this suggests that participants relied on an effortful continuous representation to maintain a single ambiguous color, thus resulting in pupil dilation while preserving precision. Strikingly, this effect gradually inverted, such that for memory load 4, pupil size was smaller while maintaining ambiguous and prototypical colors, but memory precision was now substantially reduced for ambiguous colors; this suggests that with increased memory load participants increasingly relied on categorical representations for ambiguous colors (which are by definition a poor fit to any category). Taken together, our results suggest that continuous representations are more effortful than categorical representations and that very few continuous representations (perhaps only one) can be maintained simultaneously. |
Atsushi Yokoi; Jeffrey Weiler Pupil diameter tracked during motor adaptation in humans Journal Article In: Journal of Neurophysiology, vol. 128, no. 5, pp. 1224–1243, 2022. @article{Yokoi2022,Pupil diameter is known as a noninvasive window into individuals' internal states. Despite the growing use of pupillometry in cognitive learning studies, it receives little attention in motor learning studies. Here, we characterized the pupil responses in a short-term reach adaptation paradigm by measuring pupil diameter of human participants while they adapted to abrupt, gradual, or switching force field conditions. Our results demonstrate how surprise and uncertainty reflected in pupil diameter develop during motor adaptation.Pupil diameter, under constant illumination, is known to reflect individuals' internal states, such as surprise about observation and environmental uncertainty. Despite the growing use of pupillometry in cognitive learning studies as an additional measure for examining internal states, few studies have used pupillometry in human motor learning studies. Here, we provide the first detailed characterization of pupil diameter changes in a short-term reach adaptation paradigm. We measured pupil changes in 121 human participants while they adapted to abrupt, gradual, or switching force field conditions. Sudden increases in movement error caused by the introduction/reversal of the force field resulted in strong phasic pupil dilation during movement accompanied by a transient increase in tonic premovement baseline pupil diameter in subsequent trials. In contrast, pupil responses were reduced when the force field was gradually introduced, indicating that large, unexpected errors drove the changes in pupil responses. Interestingly, however, error-induced pupil responses gradually became insensitive after experiencing multiple force field reversals. We also found an association between baseline pupil diameter and incidental knowledge of the gradually introduced perturbation. Finally, in all experiments, we found a strong co-occurrence of larger baseline pupil diameter with slower reaction and movement times after each rest break. Collectively, these results suggest that tonic baseline pupil diameter reflects one's belief about environmental uncertainty, whereas phasic pupil dilation during movement reflects surprise about a sensory outcome (i.e., movement error), and both effects are modulated by novelty. Our results provide a new approach for nonverbally assessing participants' internal states during motor learning. |
Weizhen Xie; JC Lynne Lu Sing; Ana Martinez-Flores; Weiwei Zhang Induced negative arousal modulates the speed of visual working memory consolidation Journal Article In: Emotion, vol. 22, no. 1, pp. 179–197, 2022. @article{Xie2022a,This study examines how induced negative arousal influences the consolidation of fragile sensory inputs into durable working memory (WM) representations. Participants performed a visual WM change detection task with different amounts of encoding time manipulated by random pattern masks inserted at different levels of memory-and-mask Stimulus Onset Asynchrony (SOA). Prior to the WM task, negative or neutral emotion was induced using audio clips from the International Affective Digital Sounds (IADS). Pupillometry was simultaneously recorded to provide an objective measure of induced arousal. Self-report measures of early-life stress (i.e., adverse childhood experiences) and current mood states (i.e., depressed mood and anxious feeling) were also collected as covariates. We find that participants initially remember a comparable number of WM items under a short memory-and-mask SOA of 100 ms across emotion conditions, but then encode more items into WM at a longer memory-and-mask SOA of 333 ms under induced negative arousal. These findings suggest that induced negative arousal speeds up WM consolidation. Yet, induced negative arousal does not seem to significantly affect participants' WM storage capacity estimated from a separate no mask condition. Furthermore, this emotional effect on WM consolidation speed is moderated by key affect-related individual differences. Participants who have greater pupil responses to negative IADS sounds or have more early-life stress show faster WM consolidation under induced negative arousal. Collectively, our findings reveal a critical role of phasic adrenergic responses in the rapid consolidation of visual WM content and identify potential moderators of this association. |
Thomas Wilschut; Sebastiaan Mathôt Interactions between visual working memory, attention, and color categories: A pupillometry study Journal Article In: Journal of Cognition, vol. 5, no. 1, pp. 1–12, 2022. @article{Wilschut2022,Recent studies have found that visual working memory (VWM) for color shows a categorical bias: Observers typically remember colors as more prototypical to the category they belong to than they actually are. Here, we further examine color-category effects on VWM using pupillometry. Participants remembered a color for later reproduction on a color wheel. During the retention interval, a colored probe was presented, and we measured the pupil constriction in response to this probe, assuming that the strength of constriction reflects the visual saliency of the probe. We found that the pupil initially constricted most strongly for non-matching colors that were maximally different from the memorized color; this likely reflects a lack of visual adaptation for these colors, which renders them more salient than memory-matching colors (which were shown before). Strikingly, this effect reversed later in time, such that pupil constriction was more prolonged for memory-matching colors as compared to non-matching colors; this likely reflects that memory-matching colors capture attention more strongly, and perhaps for a longer time, than non-matching colors do. We found no effects of color categories on pupil constriction: After controlling for color distance, (non-matching) colors from the same category as the memory color did not result in a different pupil response as compared to colors from a different category; however, we did find that behavioral responses were biased by color categories. In summary, we found that pupil constriction to colored probes reflects both visual adaptation and VWM content, but, unlike behavioral measures, is not notably affected by color categories. |
I. K. Wardhani; B. H. Janssen; C. N. Boehler Investigating the relationship between background luminance and self-reported valence of auditory stimuli Journal Article In: Acta Psychologica, vol. 224, pp. 1–10, 2022. @article{Wardhani2022a,The present study investigated the effect of background luminance on the self-reported valence ratings of auditory stimuli, as suggested by some earlier work. A secondary aim was to better characterise the effect of auditory valence on pupillary responses, on which the literature is inconsistent. Participants were randomly presented with sounds of different valence categories (negative, neutral, and positive) obtained from the IADS-E database. At the same time, the background luminance of the computer screen (in blue hue) was manipulated across three levels (i.e., low, medium, and high), with pupillometry confirming the expected strong effect of luminance on pupil size. Participants were asked to rate the valence of the presented sound under these different luminance levels. On a behavioural level, we found evidence for an effect of background luminance on the self-reported valence rating, with generally more positive ratings as background luminance increased. Turning to valence effects on pupil size, irrespective of background luminance, interestingly, we observed that pupils were smallest in the positive valence and the largest in negative valence condition, with neutral valence in between. In sum, the present findings provide evidence concerning a relationship between luminance perception (and hence pupil size) and self-reported valence of auditory stimuli, indicating a possible cross-modal interaction of auditory valence processing with completely task-irrelevant visual background luminance. We furthermore discuss the potential for future applications of the current findings in the clinical field. |
Shamini Warda; Jaana Simola; Devin B. Terhune Pupillometry tracks errors in interval timing Journal Article In: Behavioral Neuroscience, vol. 13, no. 2, pp. 495–502, 2022. @article{Warda2022,Recent primate studies suggest a potential link between pupil size and subjectively elapsed duration. Here, we sought to investigate the relationship between pupil size and perceived duration in human participants performing two temporal bisection tasks in the subsecond and suprasecond interval ranges. In the subsecond task, pupil diameter was greater during stimulus processing when shorter intervals were overestimated but also during and after stimulus offset when longer intervals were underestimated. By contrast, in the suprasecond task, larger pupil diameter was observed only in the late stimulus offset phase prior to response prompts when longer intervals were underestimated. This pattern of results suggests that pupil diameter relates to an error monitoring mechanism in interval timing. These results are at odds with a direct relationship between pupil size and the perception of duration but suggest that pupillometric variation might play a key role in signifying errors related to temporal judgments. |
Stella D. Voulgaropoulou; Fasya Fauzani; Janine Pfirrmann; Claudia Vingerhoets; Thérèse Amelsvoort; Dennis Hernaus Asymmetric effects of acute stress on cost and benefit learning Journal Article In: Psychoneuroendocrinology, vol. 138, pp. 1–10, 2022. @article{Voulgaropoulou2022,Background: Humans are continuously exposed to stressful challenges in everyday life. Such stressful events trigger a complex physiological reaction – the fight-or-flight response – that can hamper flexible decision-making and learning. Inspired by key neural and peripheral characteristics of the fight-or-flight response, here, we ask whether acute stress changes how humans learn about costs and benefits. Methods: Healthy adults were randomly exposed to an acute stress (age mean=23.48, 21/40 female) or no-stress control (age mean=23.80, 22/40 female) condition, after which they completed a reinforcement learning task in which they minimize cost (physical effort) and maximize benefits (monetary rewards). During the task pupillometry data were collected. A computational model of cost-benefit reinforcement learning was employed to investigate the effect of acute stress on cost and benefit learning and decision-making. Results: Acute stress improved learning to maximize rewards relative to minimizing physical effort (Condition-by-Trial Type interaction: F(1,78)= 6.53 |
Jeshua Tromp; Sander Nieuwenhuis; Peter Murphy The effects of neural gain on reactive cognitive control Journal Article In: Computational Brain and Behavior, vol. 5, no. 3, pp. 422–433, 2022. @article{Tromp2022,Fluctuations in global neural gain, arising from brainstem arousal systems, have been found to shape attention, learning, and decision-making as well as cortical state. Comparatively, little is known about how fluctuations in neural gain affect cognitive control. In the present study, we examined this question using a combination of behavioral methods, pupillometry, and computational modeling. Simulations of a comprehensive model of the Stroop task incorporating task conflict and both proactive and reactive forms of control indicated that increasing global gain led to an overall speeding of reaction times, increased Stroop interference, and decreased Stroop facilitation. Pupil analyses revealed that the pre-trial pupil derivative (i.e., rate of change), a putative non-invasive index of global gain, showed the same diagnostic relationships with the Stroop-task performance of human participants. An analysis of the internal model dynamics suggested that a gain-related increase in task conflict and corresponding (within-trial) increase in reactive control are vital for understanding this pattern of behavioral results. Indeed, a similar connectionist model without this task-conflict-control loop could not account for the results. Our study suggests that spontaneous fluctuations in neural gain can have a significant impact on reactive cognitive control. |
Chiara Tortelli; Antonella Pomè; Marco Turi; Roberta Igliozzi; David C. Burr; Paola Binda Contextual information modulates pupil size in autistic children Journal Article In: Frontiers in Neuroscience, vol. 16, pp. 1–10, 2022. @article{Tortelli2022,Recent Bayesian models suggest that perception is more “data-driven” and less dependent on contextual information in autistic individuals than others. However, experimental tests of this hypothesis have given mixed results, possibly due to the lack of objectivity of the self-report methods typically employed. Here we introduce an objective no-report paradigm based on pupillometry to assess the processing of contextual information in autistic children, together with a comparison clinical group. After validating in neurotypical adults a child-friendly pupillometric paradigm, in which we embedded test images within an animation movie that participants watched passively, we compared pupillary response to images of the sun and meaningless control images in children with autism vs. age- and IQ-matched children presenting developmental disorders unrelated to the autistic spectrum. Both clinical groups showed stronger pupillary constriction for the sun images compared with control images, like the neurotypical adults. However, there was no detectable difference between autistic children and the comparison group, despite a significant difference in pupillary light responses, which were enhanced in the autistic group. Our report introduces an objective technique for studying perception in clinical samples and children. The lack of statistically significant group differences in our tests suggests that autistic children and the comparison group do not show large differences in perception of these stimuli. This opens the way to further studies testing contextual processing at other levels of perception. |