Chuanli Zang; Zhichao Zhang; Zhang Manman; Federica Degno; Simon P. Liversedge
Examining semantic parafoveal-on-foveal effects using a Stroop boundary paradigm Journal Article
In: Journal of Memory and Language, vol. 128, pp. 1–14, 2023.
The issue of whether lexical processing occurs serially or in parallel has been a central and contentious issue in respect of models of eye movement control in reading for well over a decade. A critical question in this regard concerns whether lexical parafoveal-on-foveal effects exist in reading. Because Chinese is an unspaced and densely packed language, readers may process parafoveal words to a greater extent than they do in spaced alphabetic languages. In two experiments using a novel Stroop boundary paradigm (Rayner, 1975), participants read sentences containing a single-character color-word whose preview was manipulated (identity or pseu- docharacter, printed in black [no-color], or in a color congruent or incongruent with the character meaning). Two boundaries were used, one positioned two characters before the target and one immediately to the left of the target. The previews changed from black to color and then back to black as the eyes crossed the first and then the second boundary respectively. In Experiment 1 four color-words (red, green, yellow and blue) were used and in Experiment 2 only red and green color-words were used as targets. Both experiments showed very similar patterns such that reading times were increased for colored compared to no-color previews indicating a parafoveal visual interference effect. Most importantly, however, there were no robust interactive effects. Preview effects were comparable for congruent and incongruent color previews at the pretarget region when the data were combined from both experiments. These results favour serial processing accounts and indicate that even under very favourable experimental conditions, lexical semantic parafoveal-on-foveal effects are minimal.
Yuyang Zhang; Jing Yang; Zhisheng Edward Wen
Learners with low working memory capacity benefit more from the presence of an instructor 's face in video lectures Journal Article
In: Journal of Intelligence, vol. 11, no. 5, pp. 1–14, 2023.
This current study explores the influence of learners' working memory capacity (WMC) on the facilitation effect of an instructor's presence during video lectures. Sixty-four undergraduates were classified into high and low WMC groups based on their performance in an operation span task. They watched three types of video lectures on unfamiliar topics in a random order: video lectures with an instructor's voiceover but without presence (VN), video lectures with the instructor's face picture (VP), and video lectures with the same instructor talking (VV). We collected their eye movement data during the video lectures and their learning performance in the comprehension tests following each video. Two-way ANOVA and post-hoc analyses showed that the instructor's presence significantly improved comprehension performance in only the low WMC group. They allocated more attention to the instructor's face picture and talking head than the high WMC group. Our results highlight the value of the instructor's presence as a social cue in video lectures, which is particularly beneficial for learners with a low WMC.
Tania S. Zamuner; Theresa Rabideau; Margarethe McDonald; H. Henny Yeung
Developmental change in children's speech processing of auditory and visual cues: An eyetracking study Journal Article
In: Journal of Child Language, vol. 50, pp. 27–51, 2023.
This study investigates how children aged two to eight years ( N = 129) and adults ( N = 29) use auditory and visual speech for word recognition. The goal was to bridge the gap between apparent successes of visual speech processing in young children in visual-looking tasks, with apparent difficulties of speech processing in older children from explicit behavioural measures. Participants were presented with familiar words in audio-visual (AV), audio-only (A-only) or visual-only (V-only) speech modalities, then presented with target and distractor images, and looking to targets was measured. Adults showed high accuracy, with slightly less target-image looking in the V-only modality. Developmentally, looking was above chance for both AV and A-only modalities, but not in the V-only modality until 6 years of age (earlier on /k/-initial words). Flexible use of visual cues for lexical access develops throughout childhood.
Yao Yao; Katrina Connell; Stephen Politzer-ahles
Hearing emotion in two languages : A pupillometry study of Cantonese – Mandarin bilinguals ' perception of affective cognates in L1 and L2 Journal Article
In: Bilingualism: Language and Cognition, pp. 1–14, 2023.
Differential affective processing has been widely documented for bilinguals: L1 affective words elicit higher levels of arousal and stronger emotionality ratings than L2 affective words (Pavlenko, 2012). In this study, we focus on two closely related Chinese languages, Mandarin and Cantonese, whose affective lexicons are highly overlapping, with shared lexical items that only differ in pronunciation across languages. We recorded L1 Cantonese – L2 Mandarin bilinguals' pupil responses to auditory tokens of Cantonese and Mandarin affective words. Our results showed that Cantonese–Mandarin bilinguals had stronger pupil responses when the affective words were pronounced in Cantonese (L1) than when the same words were pronounced in Mandarin (L2). The effect was most evident in taboo words and among bilinguals with lower L2 proficiency. We discuss the theoretical implications of the findings in the frameworks of exemplar theory and models of the bilingual lexicon. textcopyright
Robert L. Whitwell; Mehul A. Garach; Melvyn A. Goodale; Irene Sperandio
Looking at the Ebbinghaus illusion: differences in neurocomputational requirements, not gaze-mediated attention, explain a classic perception-action dissociation Journal Article
In: Philosophical Transactions of the Royal Society B: Biological Sciences, vol. 378, pp. 1–11, 2023.
Perceiving and grasping an object present an animal with different sets of computational problems. The solution in primates entails the specialization of separate neural networks for visual processing with different object representations. This explains why the Ebbinghaus illusion minimally affects the grasping hand's in-flight aperture, which normally scales with target size, even though the size of the target disc remains misperceived. An attractive alternative account, however, posits that grasps are refractory to the illusion because participants fixate on the target and fail to attend to the surrounding context. To test this account, we tracked both limb and gaze while participants made forced-choice judgments of relative disc size in the Ebbinghaus illusion or did so in combination with grasping or manually estimating the size of one of the discs. We replicated the classic dissociation: grasp aperture was refractory to the measured illusory effect on perceived size, while judgments and manual estimates of disc size were not. Importantly, the number of display-wide saccades per second and the percentage of total fixation time or fixations directed at the selected disc failed to explain the dissociation. Our findings support the contention that object perception and goal-directed action rely on distinct visual representations. This article is part of a discussion meeting issue 'New approaches to 3D vision'.
Danhui Wang; Man Zeng; Han Zhao; Lei Gao; Shan Li; Zibei Niu; Xuejun Bai; Xiaolei Gao
Effects of syllable boundaries in Tibetan reading Journal Article
In: Scientific Reports, vol. 13, no. 314, pp. 1–10, 2023.
Interword spaces exist in the texts of many languages that use alphabetic writing systems. In most cases, interword spaces, as a kind of word boundary information, play an important role in the reading process of readers. Tibetan also uses alphabetic writing, its text has no spaces between words as word boundary markers. Instead, there are intersyllable tshegs (“."), which are superscript dots. Interword spaces play an important role in reading as word boundary information. Therefore, it is interesting to investigate the role of tshegs and what effect replacing tshegs with spaces will have on Tibetan reading. To answer these questions, Experiment 1 was conducted in which 72 Tibetan undergraduates read three-syllable-boundary conditions (normal, spaced, and untsheged). However, in Experiment 1, because we performed the experimental operations of deleting tshegs and replacing tshegs, the spatial information distribution of Tibetan sentences under different operating conditions was different, which may have a certain potential impact on the experimental results. To rule out the underlying confounding factor, in Experiment 2, 58 undergraduates read sentences for both untsheged and alternating-color conditions. Overall, the global and local analyses revealed that tshegs, spaces, and alternating-color markers as syllable boundaries can help readers segment syllables in Tibetan reading. In Tibetan reading, both spaces and tshegs are effective visual syllable segmentation cues, and spaces are more effective visual syllable segmentation cues than tshegs.
Yingjia Wan; Yipu Wei; Baorui Xu; Liqi Zhu; Michael K. Tanenhaus
Musical coordination affects children's perspective-taking, but musical synchrony does not Journal Article
In: Developmental Science, pp. 1–13, 2023.
Perspective-taking, which is important for communication and social activities, can be cultivated through joint actions, including musical activities in children. We examined how rhythmic activities requiring coordination affect perspective-taking in a referential communication task with 100 Chinese 4- to 6-year-old children. In Study 1, 5- to 6-year-old children played an instrument with a virtual partner in one of three coordination conditions: synchrony, asynchrony, and antiphase synchrony. Eye movements were then monitored with the partner giving instructions to identify a shape referent which included a pre-nominal scalar adjective (e.g., big cubic block). When the target contrast (a small cubic block) was in the shared ground and a competitor contrast was occluded for the partner, participants who used perspective differences could, in principle, identify the intended referent before the shape was named. We hypothesized that asynchronous and antiphase synchronous musical activities, which require self- other distinction, might have stronger effects on perspective-taking than synchronous activity. Children in the asynchrony and antiphase synchrony conditions, but not the synchrony condition, showed anticipatory looks at the target, demonstrating real-time use of the partner's perspective. Study 2 was conducted to determine if asynchrony and antiphase asynchrony resulted in perspective-taking that otherwise would not have been observed, or if synchronous coordination inhibited perspective-taking that wouldotherwise have occurred. We found no evidence for online perspective-taking in 4-to 6-year-old children without music manipulation. Therefore, playing instruments asynchronously or in alternation, but not synchronously, increases perspective-taking in children of this age, likely by training self-other distinction and control.
Josefine Waldthaler; Alexander Sperlich; Aylin König; Charlotte Stüssel; Frank Bremmer; Lars Timmermann; David Pedrosa
High (130 Hz)- and mid (60 Hz)-frequency deep brain stimulation in the subthalamic nucleus differentially modulate response inhibition: A preliminary combined EEG and eye tracking study Journal Article
In: NeuroImage: Clinical, vol. 37, pp. 1–11, 2023.
While deep brain stimulation (DBS) in the subthalamic nucleus (STN) improves motor functions in Parkinson's disease (PD), it may also increase impulsivity by interfering with the inhibition of reflexive responses. The aim of this study was to investigate if varying the pulse frequency of STN-DBS has a modulating effect on response inhibition and its neural correlates. For this purpose, 14 persons with PD repeated an antisaccade task in three stimulation settings (DBS off, high-frequency DBS (130 Hz), mid-frequency DBS (60 Hz)) in a randomized order, while eye movements and brain activity via high-density EEG were recorded. On a behavioral level, 130 Hz DBS stimulation had no effect on response inhibition measured as antisaccade error rate, while 60 Hz DBS induced a slight but significant reduction of directional errors compared with the DBS-off state and 130 Hz DBS. Further, stimulation with both frequencies decreased the onset latency of correct antisaccades, while increasing the latency of directional errors. Time-frequency domain analysis of the EEG data revealed that 60 Hz DBS was associated with an increase in preparatory theta power over a midfrontal region of interest compared with the off-DBS state which is generally regarded as a marker of increased cognitive control. While no significant differences in brain activity over mid- and lateral prefrontal regions of interest emerged between the 60 Hz and 130 Hz conditions, both stimulation frequencies were associated with a stronger midfrontal beta desynchronization during the mental preparation for correct antisaccades compared with DBS off-state which is discussed in the context of potentially enhanced proactive recruitment of the oculomotor network. Our preliminary findings suggest that mid-frequency STN-DBS may provide beneficial effects on response inhibition, while both 130 Hz- and 60 Hz STN-DBS may promote voluntary actions at the expense of slower reflexive responses.
Ria Vormbrock; Maximilian Bruchmann; Lucas Menne; Thomas Straube; Sebastian Schindler
Testing stimulus exposure time as the critical factor of increased EPN and LPP amplitudes for fearful faces during perceptual distraction tasks Journal Article
In: Cortex, vol. 160, pp. 9–23, 2023.
Fearful facial expressions are prioritized across different information processing stages, as evident in early, intermediate, and late components of event-related brain potentials (ERPs). Recent studies showed that, in contrast to early N170 modulations, mid-latency (Early Posterior Negativity, EPN) and late (Late Positive Potential, LPP) emotional modula- tions depend on the attended perceptual feature. Nevertheless, several studies reported significant differences between emotional and neutral faces for the EPN or LPP components during distraction tasks. One cause for these conflicting findings might be that when faces are presented sufficiently long, participants attend to task-irrelevant features of the faces. In this registered report, we tested whether the presentation duration of faces is the critical factor for differences between reported emotional modulations during perceptual distraction tasks. To this end, 48 participants were required to discriminate the orientation of lines overlaid onto fearful or neutral faces, while face presentation varied (100 msec, 300 msec, 1,000 msec, 2,000 msec). While participants did not need to pay attention to the faces, we observed main effects of emotion for the EPN and LPP, but no interaction between emotion and presentation duration. Of note, unregistered exploratory tests per presenta- tion duration showed no significant EPN and LPP emotion differences during short dura- tions (100 and 300 msec) but significant differences with longer durations. While the presentation duration seems not to be a critical factor for EPN and LPP emotion effects, future studies are needed to investigate the role of threshold effects and the applied an- alytic designs to explain conflicting findings in the literature.
Lara Koch; Norbert Kathmann; Benedikt Reuter
Lack of speeded disengagement from facial expressions of disgust in remitted major depressive disorder: Evidence from an eye-movement study Journal Article
In: Behaviour Research and Therapy, vol. 160, pp. 1–11, 2023.
Acute major depression is characterized by specific abnormalities in the way emotional material is attended to. In late stages of stimulus processing, clinically depressed and dysphoric individuals show difficulties to disengage attention from emotionally negative material. It is unclear, however, whether aberrant disengagement is a transitory attentional phenomenon tied to depressive symptoms, or whether it constitutes a more stable disposition that outlast the symptomatic episode. To address this issue, the current study examined 39 currently euthymic individuals previously affected by major depression (RMD) and 40 healthy control participants reporting no lifetime psychopathology (ND). We used a gaze-contingent eye tracking paradigm designed to separately assess the attentional components of engagement and disengagement when viewing facial expressions of sadness, disgust and happiness. Never-depressed healthy participants, but not remitted euthymic individuals, showed speeded disengagement from facial expressions of disgust. We propose that the lack of this distinct acceleration in previously depressed but fully remitted individuals might reflect an attentional disposition that carries over to euthymic phases of the disease. On the other hand, a tendency to disengage quickly from areas in the visual field that convey social disdain could potentially act as a protective, possibly mood-stabilizing bias in resilient individuals.
Florence Van Meenen; Nicolas Masson; Leen Catrysse; Liesje Coertjens
Taking a closer look at how higher education students process and use (discrepant) peer feedback Journal Article
In: Learning and Instruction, vol. 84, pp. 1–11, 2023.
Little is known on how students process peer feedback (PF) and use it to improve their work. We asked 59 participants to read the feedback of two peers on a fictional essay and to revise it, while we recorded their gaze behaviour. Regarding the PF processing subphase, discrepant PF led to more transitions, but only for participants who reported the discrepancy afterwards. Counterintuitively, participants who did not report the discrepancy, showed longer first-pass reading times. Concerning the PF use subphase, dwell time on essay correlated positively with the quality of the revised essays assessed by professors. Participants with a high-quality revision spent more time addressing higher order comments, corrected one or two lower order aspects at a time and proofread in the end, in which they went beyond the suggestions provided in the PF. These insights can be used when designing training to foster students' uptake of (discrepant) PF.
Joeri F. L. Helden; Marnix Naber
Effects of natural scene inversion on visual-evoked brain potentials and pupillary responses: A matter of effortful processing of unfamiliar configurations Journal Article
In: Neuroscience, vol. 509, pp. 201–209, 2023.
The inversion of a picture of a face hampers the accuracy and speed at which observers can perceptually process it. Event-related potentials and pupillary responses, successfully used as biomarkers of face inversion in the past, suggest that the perception of visual features, that are organized in an unfamiliar manner, recruits demanding additional processes. However, it remains unclear whether such inversion effects generalize beyond face stimuli and whether indeed more mental effort is needed to process inverted images. Here we aimed to study the effects of natural scene inversion on visual evoked potentials and pupil dilations. We simultaneously measured responses of 47 human participants to presentations of images showing upright or inverted natural scenes. For inverted scenes, we observed relatively stronger occipito-temporo-parietal N1 peak amplitudes and larger pupil dilations (on top of an initial orienting response) than for upright scenes. This study revealed neural and physiological markers of natural scene inversion that are in line with inversion effects of other stimulus types and demonstrates the robustness and generalizability of the phenomenon that unfamiliar configurations of visual content require increased processing effort.
Caterina Trentin; Heleen A. Slagter; Christian N. L. Olivers
Visual working memory representations bias attention more when they are the target of an action plan Journal Article
In: Cognition, vol. 230, pp. 1–10, 2023.
Attention has frequently been regarded as an emergent property of linking sensory representations to action plans. It has recently been proposed that similar mechanisms may operate within visual working memory (VWM), such that linking an object in VWM to an action plan strengthens its sensory memory representation, which then expresses as an attentional bias. Here we directly tested this hypothesis by comparing attentional biases induced by VWM representations which were the target of a future action, to those induced by VWM representations that were equally task-relevant, but not the direct target of action. We predicted that the first condition would result in a more prioritized memory state and hence stronger attentional biases. Specifically, participants memorized a geometric shape for a subsequent memory test. At test, in case of a match, participants either had to perform a grip movement on the matching object (action condition), or perform the same movement, but on an unrelated object (control condition). To assess any attentional biases, during the delay period between memorandum and test, participants performed a visual selection task in which either the target was surrounded by the memorized shape (congruent trials) or a distractor (incongruent trials). Eye movements were measured as a proxy for attentional priority. We found a significant interaction for saccade latencies between action condition and shape congruency, reflecting more pronounced VWM-based attentional biases in the action condition. Our results are consistent with the idea that action plans prioritize sensory representations in VWM.
Vladislava Staroverova; Anastasiya Lopukhina; Nina Zdorova; Nina Ladinskaya; Olga Vedenina; Sofya Goldina; Anastasiia Kaprielova; Ksenia Bartseva; Olga Dragoy
Phonological and orthographic parafoveal processing during silent reading in Russian children and adults Journal Article
In: Journal of Experimental Child Psychology, vol. 226, pp. 1–11, 2023.
Studies on German and English have shown that children and adults can rely on phonological and orthographic information from the parafovea during reading, but this reliance differs between ages and languages. In the current study, we investigated the development of phonological and orthographic parafoveal processing during silent reading in Russian-speaking 8-year-old children, 10-year-old children, and adults using the gaze-contingent boundary paradigm. The participants read sentences with embedded nouns that were presented in original, pseudohomophone, control for pseudohomophone, transposed-letter, and control for transposed-letter conditions in the parafoveal area to assess phonological and orthographic preview benefit effects. The results revealed that all groups of participants relied only on orthographic but not phonological parafoveal information. These findings indicate that 8-year-old children already preprocess parafoveal information similarly to adults.
Katrine Falcon Søby; Evelyn Arko Milburn; Line Burholt Kristensen; Valentin Vulchanov; Mila Vulchanova
In the native speaker's eye: Online processing of anomalous learner syntax Journal Article
In: Applied Psycholinguistics, pp. 1–28, 2023.
How do native speakers process texts with anomalous learner syntax? Second-language learners of Norwegian, and other verb-second (V2) languages, frequently place the verb in third position (e.g., *Adverbial-Subject-Verb), although it is mandatory for the verb in these languages to appear in second position (Adverbial-Verb-Subject). In an eye-tracking study, native Norwegian speakers read sentences with either grammatical V2 or ungrammatical verb-third (V3) word order. Unlike previous eye-tracking studies of ungrammaticality, which have primarily addressed morphosyntactic anomalies, we exclusively manipulate word order with no morphological or semantic changes. We found that native speakers reacted immediately to ungrammatical V3 word order, indicated by increased fixation durations and more regressions out on the subject, and subsequently on the verb. Participants also recovered quickly, already on the following word. The effects of grammaticality were unaffected by the length of the initial adverbial. The study contributes to future models of sentence processing which should be able to accommodate various types of “noisy” input, that is, non-standard variation. Together with new studies of processing ofother L2 anomalies in Norwegian, the current findings can help language instructors and students prioritize which aspects of grammar to focus on.
Johannes J. D. Singer; Radoslaw M. Cichy; Martin N. Hebart
The spatiotemporal neural dynamics of object recognition for natural images and line drawings Journal Article
In: Journal of Neuroscience, vol. 43, no. 3, pp. 484–500, 2023.
Drawings offer a simple and efficient way to communicate meaning. While line drawings capture only coarsely how objects look in reality, we still perceive them as resembling real-world objects. Previous work has shown that this perceived similarity is mirrored by shared neural representations for drawings and natural images, which suggests that similar mechanisms underlie the recognition of both. However, other work has proposed that representations of drawings and natural images become similar only after substantial processing has taken place, suggesting distinct mechanisms. To arbitrate between those alternatives, we measured brain responses resolved in space and time using fMRI and MEG, respectively, while human participants (female and male) viewed images of objects depicted as photographs, line drawings, or sketch-like drawings. Using multivariate decoding, we demonstrate that object category information emerged similarly fast and across overlapping regions in occipital, ventral-temporal, and posterior parietal cortex for all types of depiction, yet with smaller effects at higher levels of visual abstraction. In addition, cross-decoding between depiction types revealed strong generalization of object category information from early processing stages on. Finally, by combining fMRI and MEG data using representational similarity analysis, we found that visual information traversed similar processing stages for all types of depiction, yet with an overall stronger representation for photographs. Together, our results demonstrate broad commonalities in the neural dynamics of object recognition across types of depiction, thus providing clear evidence for shared neural mechanisms underlying recognition of natural object images and abstract drawings.
Summer Sheremata; George L. Malcolm; Sarah Shomstein
Behavioral asymmetries in visual short-term memory occur in retinotopic coordinates Journal Article
In: Attention, Perception, and Psychophysics, vol. 85, pp. 113–119, 2023.
Visual short-term memory (VSTM) is an essential store that creates continuous representations from disjointed visual input. However, severe capacity limits exist, reflecting constraints in supporting brain networks. VSTM performance shows spatial biases predicted by asymmetries in the brain based upon the location of the remembered object. Visual representations are retinotopic, or relative to location of the representation on the retina. It therefore stands to reason that memory performance may also show retinotopic biases. Here, eye position was manipulated to tease apart retinotopic coordinates from spatiotopic coordinates, or location relative to the external world. Memory performance was measured while participants performed a color change-detection task for items presented across the visual field while subjects fixated central or peripheral position. VSTM biases reflected the location of the stimulus on the retina, regardless of where the stimulus appeared on the screen. Therefore, spatial biases occur in retinotopic coordinates in VSTM and suggest a fundamental link between behavioral VSTM measures and visual representations.
Camilo R. Ronderos; Ernesto Guerra; Pia Knoeferle
When sequence matters: the processing of contextually biased German verb–object metaphors Journal Article
In: Language and Cognition, vol. 15, no. 1, pp. 1–28, 2023.
Several studies have investigated the comprehension of decontextualized English nominal metaphors. However, not much is known about how contextualized, non-nominal, non-English metaphors are processed, and how this might inform existing theories of metaphor comprehension. In the current work, we investigate the effects of context and of sequential order for an under-studied type of construction: German verb–object metaphors. In two visual-world, eye-tracking experiments, we manipulated whether a discourse context biased a spoken target utterance toward a metaphoric or a literal interpretation. We also manipulated the order of verb and object in the target utterances (e.g., Stefan interviewt eine Hyäne , ‘Stefan interviews a hyena', verb→object; and Stefan wird eine Hyäne interviewen , ‘Stefan will a hyena interview', object→verb). Experiment 1 shows that contextual cues interacted with sequential order, mediating the processing of verb–object metaphors: When the context biased toward a metaphoric interpretation, participants readily understood the object metaphorically for the verb→object sequence, whereas they likely first understood it literally for the object→verb sequence. Crucially, no such effect of sequential order was found when context biased toward a literal interpretation. Experiment 2 suggests that differences in processing found in Experiment 1 were brought on by the interaction of discourse context and sequential order and not by sequential order alone. We propose ways in which existing theoretical views could be extended to account for these findings. Overall, our study shows the importance of context during figurative language comprehension and highlights the need to test the predictions of metaphor theories on non-English and non-nominal metaphors.
Helen Rodger; Nayla Sokhn; Junpeng Lao; Yingdi Liu; Roberto Caldara
Developmental eye movement strategies for decoding facial expressions of emotion Journal Article
In: Journal of Experimental Child Psychology, vol. 229, pp. 1–23, 2023.
In our daily lives, we routinely look at the faces of others to try to understand how they are feeling. Few studies have examined the perceptual strategies that are used to recognize facial expressions of emotion, and none have attempted to isolate visual information use with eye movements throughout development. Therefore, we recorded the eye movements of children from 5 years of age up to adulthood during recognition of the six “basic emotions” to investigate when perceptual strategies for emotion recognition become mature (i.e., most adult-like). Using iMap4, we identified the eye movement fixation patterns for recognition of the six emotions across age groups in natural viewing and gaze-contingent (i.e., expanding spotlight) conditions. While univariate analyses failed to reveal significant differences in fixation patterns, more sensitive multivariate distance analyses revealed a U-shaped developmental trajectory with the eye movement strategies of the 17- to 18-year-old group most similar to adults for all expressions. A developmental dip in strategy similarity was found for each emotional expression revealing which age group had the most distinct eye movement strategy from the adult group: the 13- to 14-year-olds for sadness recognition; the 11- to 12-year-olds for fear, anger, surprise, and disgust; and the 7- to 8-year-olds for happiness. Recognition performance for happy, angry, and sad expressions did not differ significantly across age groups, but the eye movement strategies for these expressions diverged for each group. Therefore, a unique strategy was not a prerequisite for optimal recognition performance for these expressions. Our data provide novel insights into the developmental trajectories underlying facial expression recognition, a critical ability for adaptive social relations.
Miriam Rivero-Contreras; Paul E. Engelhardt; David Saldaña
Do easy-to-read adaptations really facilitate sentence processing for adults with a lower level of education? An experimental eye-tracking study Journal Article
In: Learning and Instruction, vol. 84, pp. 1–13, 2023.
The Easy-to-Read guidelines recommend visual support and lexical simplification to facilitate text processing, but few studies have empirically verified the efficacy of these guidelines. This study examined the influence of these recommendations on sentence processing by examining eye movements at the text- and word-level in adult readers. We tested 30 non-university adults (low education level) and 30 university adults (high education level). The experimental task consisted of 60 sentences. Half were accompanied by an image and half were not, and half contained a low-frequency word and half a high-frequency word. Results showed that visual support and lexical simplification facilitated processing in both groups of adults, and non-university adults were significantly slower than university adults at sentence processing. However, lexical simplification resulted in faster processing in the non-university adults' group. Conclusions focus on the mechanisms in which both adaptations benefit readers, and practical implications for reading comprehension.
Brendan L. Portengen; Giorgio L. Porro; Saskia M. Imhof; Marnix Naber
The trade-off between luminance and color contrast assessed with pupil responses Journal Article
In: Translational Vision Science & Technology, vol. 12, no. 1, pp. 1–12, 2023.
Purpose: A scene consisting of a white stimulus on a black background incorporates strong luminance contrast. When both stimulus and background receive different colors, luminance contrast decreases but color contrast increases. Here, we sought to characterize the pattern of stimulus salience across varying trade-offs of color and luminance contrasts by using the pupil light response. Methods: Three experiments were conducted with 17, 16, and 17 healthy adults. For all experiments, a flickering stimulus (2 Hz; alternating color to black) was presented superimposed on a background with a complementary color to the stimulus (i.e., opponency colors in human color perception: blue and yellow for Experiment 1, red and green for Experiment 2, and equiluminant red and green for Experiment 3). Background luminance varied between 0% and 45% to trade off luminance and color contrast with the stimulus. By comparing the locus of the optimal trade-off between color and luminance across different color axes, we explored the generality of the trade-off. Results: The strongest pupil responses were found when a substantial amount of color contrast was present (at the expense of luminance contrast). Pupil response amplitudes increased by 15% to 30% after the addition of color contrast. An optimal pupillary responsiveness was reached at a background luminance setting of 20% to 35% color contrast across several color axes. Conclusions: These findings suggest that a substantial component of pupil light responses incorporates color processing. More sensitive pupil responses and more salient stimulus designs can be achieved by adding subtle levels of color contrast between stimulus and background. Translational Relevance: More robust pupil responses will enhance tests of the visual field with pupil perimetry.
Stefan Pollmann; Lei Zheng
Right-dominant contextual cueing for global configuration cues, but not local position cues Journal Article
In: Neuropsychologia, vol. 178, pp. 1–7, 2023.
Contextual cueing can depend on global configuration or local item position. We investigated the role of these two kinds of cues in the lateralization of contextual cueing effects. Cueing by item position was tested by recombining two previously learned displays, keeping the individual item locations intact, but destroying the global configuration. In contrast, cueing by configuration was investigated by rotating learned displays, thereby keeping the configuration intact but changing all item positions. We observed faster search for targets in the left display half, both for repeated and new displays, along with more first fixation locations on the left. Both position and configuration cues led to faster search, but the search time reduction compared to new displays due to position cues was comparable in the left and right display half. In contrast, configural cues led to increased search time reduction for right half targets. We conclude that only configural cues enabled memory-guided search for targets across the whole search display, whereas position cueing guided search only to targets in the vicinity of the fixation. The right-biased configural cueing effect is a consequence of the initial leftward search bias and does not indicate hemispheric dominance for configural cueing.
Alexis Pérez-Bellido; Eelke Spaak; Floris P. Lange
Magnetoencephalography recordings reveal the neural mechanisms of auditory contributions to improved visual detection Journal Article
In: Communications biology, vol. 6, no. 12, pp. 1–16, 2023.
Sounds enhance the detection of visual stimuli while concurrently biasing an observer's decisions. To investigate the neural mechanisms that underlie such multisensory interactions, we decoded time-resolved Signal Detection Theory sensitivity and criterion parameters from magneto-encephalographic recordings of participants that performed a visual detection task. We found that sounds improved visual detection sensitivity by enhancing the accumulation and maintenance of perceptual evidence over time. Meanwhile, criterion decoding analyses revealed that sounds induced brain activity patterns that resembled the patterns evoked by an actual visual stimulus. These two complementary mechanisms of audiovisual interplay differed in terms of their automaticity: Whereas the sound-induced enhancement in visual sensitivity depended on participants being actively engaged in a detection task, we found that sounds activated the visual cortex irrespective of task demands, potentially inducing visual illusory percepts. These results challenge the classical assumption that sound-induced increases in false alarms exclusively correspond to decision-level biases.
Adam J. Parker; Milla Räsänen; Timothy J. Slattery
What is the optimal position of low-frequency words across line boundaries? An eye movement investigation Journal Article
In: Applied Cognitive Psychology, pp. 1–13, 2023.
When displaying text on a page or a screen, only a finite number of characters can be presented on a single line. If the text exceeds that finite value, then text wrapping occurs. Often this process results in longer, more difficult to process words being positioned at the start of a line. We conducted an eye movement study to examine how this artefact of text wrapping affects passage reading. This allowed us to answer the question: should word difficulty be used when determining line breaks? Thirty-nine participants read 20 passages where low-frequency target words were either line-initial or line-final. There was no statistically reliable effect of our manipulation on passage reading time or comprehension despite several effects at a local level. Regarding our primary research question, the evidence suggests that word difficulty may not need to be accounted for when determining line breaks and assigning words to new lines.
Arthur Pabst; Zoé Bollen; Nicolas Masson; Pauline Billaux; Philippe Timary; Pierre Maurage
An eye-tracking study of biased attentional processing of emotional faces in severe alcohol use disorder Journal Article
In: Journal of Affective Disorders, vol. 323, pp. 778–787, 2023.
Background: Social cognition impairments in severe alcohol use disorder (SAUD) are increasingly established. However, fundamental aspects of social cognition, and notably the attentional processing of socio-affective information, remain unexplored, limiting our understanding of underlying mechanisms. Here, we determined whether patients with SAUD show attentional biases to specific socio-affective cues, namely emotional faces. Method: In a modified dot-probe paradigm, 30 patients with SAUD and 30 demographically matched healthy controls (HC) were presented with pairs of neutral-emotional (angry, disgusted, happy, sad) faces while having their eye movements recorded. Indices of early/automatic (first fixations, latency to first fixations) and later/controlled (number of fixations, dwell-time) processes were computed. Results: Patients with SAUD did not differ from HC in their attention to angry/disgusted/sad vs. neutral faces. However, patients with SAUD fixated/dwelled less on happy vs. neutral faces in the first block of stimuli than HC, who presented an attentional bias to happy faces. Limitations: Sample-size was determined to detect medium-to-large effects and subtler ones may have been missed. Further, our cross-sectional design provides no explanation as to whether the evidenced biases precede or are a consequence of SAUD. Conclusions: These results extend the social cognition literature in SAUD to the attentional domain, by evidencing the absence of a controlled attentional bias toward positive social cues in SAUD. This may reflect reduced sensitivity to social reward and could contribute to higher order social cognition difficulties and social dysfunction.
Julie Ouerfelli-Ethier; Isabella Comtois Bona; Romain Fournet; Laure Pisella; Aarlenne Z. Khan
Pre-saccadic attention relies more on suppression than does covert attention Journal Article
In: Journal of Vision, vol. 23, no. 1, pp. 1–19, 2023.
During covert and pre-saccadic attentional shifts, it is unclear how facilitation and suppression processes interact for target selection. A recent countermanding task pointed to greater suppression at unattended locations during trials with saccades compared to trials without saccades (i.e., fixation and successful stop trials), whereas target facilitation did not differ. It is unknown whether this finding is restricted to countermanding paradigms that involve inhibitory processes. To test this, we adapted Gaspelin and colleagues (2015)'s attention capture task where, within the same block, one location was primed with frequent line discrimination trials, and all locations were occasionally probed using letters report trials. Participants also performed a baseline condition without priming. We tested 15 participants and examined how performance at non-primed locations was affected by covert versus pre-saccadic attention in blocks of four or six items, as well as by position from the primed location and timing from saccade onset. For both attention conditions, letter report at non-primed locations was worse compared to baseline, demonstrating suppression, and letter report at primed location was better, demonstrating facilitation. In saccades trials, letter report was better at primed locations and worse at non-primed locations compared to fixation trials. The timing of this additional pre-saccadic suppression differed from saccadic suppression. In both attention conditions, suppression was greater when primed and non-primed locations were within the same hemifield or in diagonal opposite quadrants. These results confirmed that attention preceding saccade execution suppressed non-primed locations to a larger extent than covert attention, with the same spatial quadrant effect.
Corinna Osterbrink; Arvid Herwig
What determines location specificity or generalization of transsaccadic learning? Journal Article
In: Journal of Vision, vol. 23, no. 1, pp. 1–18, 2023.
Humans incorporate knowledge of transsaccadic associations into peripheral object perception. Several studies have shown that learning of new manipulated transsaccadic associations leads to a presaccadic perceptual bias. However, there was still disagreement whether this learning effect was location specific (Herwig, Weiß, & Schneider, 2018) or generalizes to new locations (Valsecchi & Gegenfurtner, 2016). The current study investigated under what conditions location generalization of transsaccadic learning occurs. In all experiments, there were acquisition phases in which the spatial frequency (Experiment 1) or the size (Experiment 2 and 3) of objects was changed transsaccadically. In the test phases, participants judged the respective feature of peripheral objects. These could appear either at the location where learning had taken place or at new locations. All experiments replicated the perceptual bias effect at the old learning locations. In two experiments, transsaccadic learning remained location specific even when learning occurred at multiple locations (Experiment 1) or with the feature of size (Experiment 2) for which a transfer had previously been shown. Only in Experiment 3 was a transfer of the learning effect to new locations observable. Here, learning only took place for one object and not for several objects that had to be discriminated. Therefore, one can conclude that, when specific associations are learned for multiple objects, transsaccadic learning stays location specific and when a transsaccadic association is learned for only one object it allows a generalization to other locations.
Sunny Nigam; Russell Milton; Sorin Pojoga; Valentin Dragoi
Adaptive coding across visual features during free-viewing and fixation conditions Journal Article
In: Nature Communications, vol. 14, no. 87, pp. 1–12, 2023.
Theoretical studies have long proposed that adaptation allows the brain to effectively use the limited response range of sensory neurons to encode widely varying natural inputs. However, despite this influential view, experimental studies have exclusively focused on how the neural code adapts to a range of stimuli lying along a single feature axis, such as orientation or contrast. Here, we performed electrical recordings in macaque visual cortex (area V4) to reveal significant adaptive changes in the neural code of single cells and populations across multiple feature axes. Both during free viewing and passive fixation, populations of cells improved their ability to encode image features after rapid exposure to stimuli lying on orthogonal feature axes even in the absence of initial tuning to these stimuli. These results reveal a remarkable adaptive capacity of visual cortical populations to improve network computations relevant for natural viewing despite the modularity of the functional cortical architecture.
Simar Moussaoui; Christina F. Pereira; Matthias Niemeier
Working memory in action: Transsaccadic working memory deficits in the left visual field and after transcallosal remapping Journal Article
In: Cortex, vol. 159, pp. 26–38, 2023.
Every waking second, we make three saccadic eye movements that move our retinal images. Thus, to attain a coherent image of the world we need to remember visuo-spatial information across saccades. But transsaccadic working memory (tWM) remains poorly understood. Crucially, there has been a debate whether there are any differences in tWM for the left vs. right visual field and depending on saccade direction. However, previous studies have probed tWM with minimal loads whereas spatial differences might arise with higher loads. Here we employed a task that probed higher memory load for spatial information in the left and right visual field and with horizontal as well as vertical saccades. We captured several measures of precision and accuracy of performance that, when submitted to principal component analysis, produced two components. Component 1, mainly associated with precision, yielded greater error for the left than the right visual field. Component 2 was associated with performance accuracy and unexpectedly produced a disadvantage after rightward saccades. Both components showed that performance was worse when rightward or leftward saccades afforded a shift of memory representations between visual fields compared to remapping within the same field. Our study offers several novel findings. It is the first to show that tWM involves at least two components likely reflecting working memory capacity and strategic aspects of working memory, respectively. Reduced capacity for the left, rather than the right visual field is consistent with how the left and right visual fields are known to be represented in the two hemispheres. Remapping difficulties between visual fields is consistent with the limited information transfer across the corpus callosum. Finally, the impact of rightward saccades on working memory might be due to greater interference of the accompanying shifts of attention. Our results highlight the dynamic nature of transsaccadic working memory.
Joshua B. Moskowitz; Sarah A. Berger; Monica S. Castelhano; Jason P. Gallivan; J. Randall Flanagan
The influence of movement-related costs when searching to act and acting to search Journal Article
In: Journal of Neurophysiology, vol. 129, pp. 115–130, 2023.
Real world search behaviour often involves limb movements, either during search or following search. Here we investigated whether movement-related costs influence search behaviour in two kinds of search tasks. In our visual search tasks, participants made saccades to find a target object among distractors and then moved a cursor, controlled by the handle of a robotic manipulandum, to the target. In our manual search tasks, participants moved the cursor to perform the search, placing it onto objects to reveal their identity as either a target or a distractor. Across experiments, we manipulated either the effort or time costs associated with movement such that these costs varied across the search space. We varied effort by applying different resistive forces to the handle and we varied time costs by altering the speed of the cursor. Our analysis of cursor and eye movements during manual and visual search, respectively, showed that effort influenced manual search but did not influence visual search. In contrast, time costs influenced both visual and manual search. Our results demonstrate that, in addition to perceptual and cognitive factors, movement-related costs can also influence search behaviour.
Hiu Mei Chow; Miriam Spering
Eye movements during optic flow perception Journal Article
In: Vision Research, vol. 204, pp. 1–11, 2023.
Optic flow is an important visual cue for human perception and locomotion and naturally triggers eye movements. Here we investigate whether the perception of optic flow direction is limited or enhanced by eye movements. In Exp. 1, 23 human observers localized the focus of expansion (FOE) of an optic flow pattern; in Exp. 2, 18 observers had to detect brief visual changes at the FOE. Both tasks were completed during free viewing and fixation conditions while eye movements were recorded. Task difficulty was varied by manipulating the coherence of radial motion from the FOE (4 %-90 %). During free viewing, observers tracked the optic flow pattern with a combination of saccades and smooth eye movements. During fixation, observers nevertheless made small-scale eye movements. Despite differences in spatial scale, eye movements during free viewing and fixation were similarly directed toward the FOE (saccades) and away from the FOE (smooth tracking). Whereas FOE localization sensitivity was not affected by eye movement instructions (Exp. 1), observers' sensitivity to detect brief changes at the FOE was 27 % higher (p <.001) during free-viewing compared to fixation (Exp. 2). This performance benefit was linked to reduced saccade endpoint errors, indicating the direct beneficial impact of foveating eye movements on performance in a fine-grain perceptual task, but not during coarse perceptual localization.
Siobhan M. McAteer; Anthony McGregor; Daniel T. Smith
Oculomotor rehearsal in visuospatial working memory Journal Article
In: Attention, Perception, and Psychophysics, vol. 85, pp. 261–275, 2023.
The neural and cognitive mechanisms of spatial working memory are tightly coupled with the systems that control eye movements, but the precise nature of this coupling is not well understood. It has been argued that the oculomotor system is selectively involved in rehearsal of spatial but not visual material in visuospatial working memory. However, few studies have directly compared the effect of saccadic interference on visual and spatial memory, and there is little consensus on how the underlying working memory representation is affected by saccadic interference. In this study we aimed to examine how working memory for visual and spatial features is affected by overt and covert attentional interference across two experiments. Participants were shown a memory array, then asked to either maintain fixation or to overtly or covertly shift attention in a detection task during the delay period. Using the continuous report task we directly examined the precision of visual and spatial working memory representations and fit psychophysical functions to investigate the sources of recall error associated with different types of interference. These data were interpreted in terms of embodied theories of attention and memory and provide new insights into the nature of the interactions between cognitive and motor systems.
Marcello Maniglia; Kristina M. Visscher; Aaron R. Seitz
Consistency of preferred retinal locus across tasks and participants trained with a simulated scotoma Journal Article
In: Vision Research, vol. 203, pp. 1–9, 2023.
After loss of central vision following retinal pathologies such as macular degeneration (MD), patients often adopt compensatory strategies including developing a “preferred retinal locus” (PRL) to replace the fovea in tasks involving fixation. A key question is whether patients develop multi-purpose PRLs or whether their oculomotor strategies adapt to the demands of the task. While most MD patients develop a PRL, clinical evidence suggests that patients may develop multiple PRLs and switch between them according to the task at hand. To understand this, we examined a model of central vision loss in normally seeing individuals and tested whether they used the same or different PRLs across tasks after training. Nineteen participants trained for 10 sessions on contrast detection while in conditions of gaze-contingent, simulated central vision loss. Before and after training, peripheral looking strategies were evaluated during tasks measuring visual acuity, reading abilities and visual search. To quantify strategies in these disparate, naturalistic tasks, we measured and compared the amount of task-relevant information at each of 8 equally spaced, peripheral locations, while participants performed the tasks. Results showed that some participants used consistent viewing strategies across tasks whereas other participants' strategies differed depending on task. This novel method allows quantification of peripheral vision use even in relatively ecological tasks. These results represent one of the first examinations of peripheral viewing strategies across tasks in simulated vision loss. Results suggest that individual differences in peripheral looking strategies following simulated central vision loss may model those developed in pathological vision loss.
Changlin Luo; Siyuan Qiao; Xiangling Zhuang; Guojie Ma
Dynamic attentional bias for pictorial and textual food cues in the visual search paradigm Journal Article
In: Appetite, vol. 180, pp. 1–11, 2023.
Previous studies have found that individuals have an attentional bias for food cues, which may be related to the energy level or the type of stimulus (e.g., pictorial or textual food cues) of the food cues. However, the available evidence is inconsistent, and there is no consensus about how the type of stimulus and food energy modulate food-related attentional bias. Searching for food is one of the most important daily behaviors. In this study, a modified visual search paradigm was used to explore the attentional bias for food cues, and eye movements were recorded. Food cues consisted of both food words and food pictures with different energy levels (i.e., high- and low-calorie foods). The results showed that there was an attentional avoidance in the early stage but a later-stage attentional approach for all food cues in the pictorial condition. This was especially true for high-calorie food pictures. Participants showed a later-stage conflicting attentional bias for foods with different energy levels in the textual condition. They showed an attentional approach to high-calorie food words but an attentional avoidance of low-calorie food words. These data show that food-related attentional bias varied along with different time courses, which was also modulated by the type of stimulus and food energy. These findings regarding dynamic attentional bias could be explained using the Goal Conflict Model of eating behavior.
Guangsheng Liang; John E. Poquiz; Miranda Scolari
Space- and feature-based attention operate both independently and interactively within latent components of perceptual decision making Journal Article
In: Journal of Vision, vol. 23, no. 1, pp. 1–17, 2023.
Top-down visual attention filters undesired stimuli while selected information is afforded the lion's share of limited cognitive resources. Multiple selection mechanisms can be deployed simultaneously, but how unique influences of each combine to facilitate behavior remains unclear. Previously, we failed to observe an additive perceptual benefit when both space-based attention (SBA) and feature-based attention (FBA) were cued in a sparse display (Liang & Scolari, 2020): FBA was restricted to higher order decision-making processes when combined with a valid spatial cue, whereas SBA additionally facilitated target enhancement. Here, we introduced a series of design modifications across three experiments to elicit both attention mechanisms within signal enhancement while also investigating the impacts on decision making. First, we found that when highly reliable spatial and feature cues made unique contributions to search (experiment 1), or when each cue component was moderately reliable (experiments 2a and 2b), both mechanisms were deployed independently to resolve the target. However, the same manipulations produced interactive attention effects within other latent decision-making components that depended on the probability of the integrated cueing object. Time spent before evidence accumulation was reduced and responses were more conservative for the most likely pre-cue combination—even when it included an invalid component. These data indicate that selection mechanisms operate on sensory signals invariably in an independent manner, whereas a higher-order dependency occurs outside of signal enhancement.
Mathieu Lesourd; Alia Afyouni; Franziska Geringswald; Fabien Cignetti; Lisa Raoul; Julien Sein; Bruno Nazarian; Jean-luc Anton; Marie-Hélène Grosbras
Action observation network activity related to object-directed and socially-directed actions in adolescents Journal Article
In: Journal of Neuroscience, vol. 43, no. 1, pp. 125–141, 2023.
The human action observation network (AON) encompasses brain areas consistently engaged when we observe other's actions. Although the core nodes of the AON are present from childhood, it is not known to what extent they are sensitive to different action features during development. Because social cognitive abilities continue to mature during adolescence, the AON response to socially-oriented actions, but not to object-related actions, may differ in adolescents and adults. To test this hypothesis, we scanned with functional magnetic resonance imaging (fMRI) male and female typically-developing teenagers (n = 28; 13 females) and adults (n = 25; 14 females) while they passively watched videos of manual actions varying along two dimensions: sociality (i.e., directed toward another person or not) and transitivity (i.e., involving an object or not). We found that action observation recruited the same fronto-parietal and occipito-temporal regions in adults and adolescents. The modulation of voxel-wise activity according to the social or transitive nature of the action was similar in both groups of participants. Multivariate pattern analysis, however, revealed that decoding accuracies in intraparietal sulcus (IPS)/superior parietal lobe (SPL) for both sociality and transitivity were lower for adolescents compared with adults. In addition, in the lateral occipital temporal cortex (LOTC), generalization of decoding across the orthogonal dimension was lower for sociality only in adolescents. These findings indicate that the representation of the content of others' actions, and in particular their social dimension, in the adolescent AON is still not as robust as in adults.
Rony Lemel; Lilach Shalev; Gal Nitsan; Boaz M. Ben-David
Listen up! ADHD slows spoken-word processing in adverse listening conditions: Evidence from eye movements Journal Article
In: Research in Developmental Disabilities, vol. 133, pp. 1–15, 2023.
Background: Cognitive skills such as sustained attention, inhibition and working memory are essential for speech processing, yet are often impaired in people with ADHD. Offline measures have indicated difficulties in speech recognition on multi-talker babble (MTB) background for young adults with ADHD (yaADHD). However, to-date no study has directly tested online speech processing in adverse conditions for yaADHD. Aims: Gauging the effects of ADHD on segregating the spoken target-word from its sound-sharing competitor, in MTB and working-memory (WM) load. Methods and procedures: Twenty-four yaADHD and 22 matched controls that differ in sustained attention (SA) but not in WM were asked to follow spoken instructions presented on MTB to touch a named object, while retaining one (low-load) or four (high-load) digit/s for later recall. Their eye fixations were tracked. Outcomes and results: In the high-load condition, speech processing was less accurate and slowed by 140ms for yaADHD. In the low-load condition, the processing advantage shifted from early perceptual to later cognitive stages. Fixation transitions (hesitations) were inflated for yaADHD. Conclusions and implications: ADHD slows speech processing in adverse listening conditions and increases hesitation, as speech unfolds in time. These effects, detected only by online eyetracking, relate to attentional difficulties. We suggest online speech processing as a novel purview on ADHD. What this paper adds?: We suggest speech processing in adverse listening conditions as a novel vantage point on ADHD. Successful speech recognition in noise is essential for performance across daily settings: academic, employment and social interactions. It involves several executive functions, such as inhibition and sustained attention. Impaired performance in these functions is characteristic of ADHD. However, to date there is only scant research on speech processing in ADHD. The current study is the first to investigate online speech processing as the word unfolds in time using eyetracking for young adults with ADHD (yaADHD). This method uncovered slower speech processing in multi-talker babble noise for yaADHD compared to matched controls. The performance of yaADHD indicated increased hesitation between the spoken word and sound-sharing alternatives (e.g., CANdle-CANdy). These delays and hesitations, on the single word level, could accumulate in continuous speech to significantly impair communication in ADHD, with severe implications on their quality of life and academic success. Interestingly, whereas yaADHD and controls were matched on WM standardized tests, WM load appears to affect speech processing for yaADHD more than for controls. This suggests that ADHD may lead to inefficient deployment of WM resources that may not be detected when WM is tested alone. Note that these intricate differences could not be detected using traditional offline accuracy measures, further supporting the use of eyetracking in speech tasks. Finally, communication is vital for active living and wellbeing. We suggest paying attention to speech processing in ADHD in treatment and when considering accessibility and inclusion.
Rosa Lafer-sousa; Karen Wang; Reza Azadi; Emily Lopez; Simon Bohn; Arash Afraz
Behavioral detectability of optogenetic stimulation of inferior temporal cortex varies with the size of concurrently viewed objects Journal Article
In: Current Research in Neurobiology, vol. 4, pp. 1–7, 2023.
We have previously demonstrated that macaque monkeys can behaviorally detect a subtle optogenetic impulse delivered to their inferior temporal (IT) cortex. We have also shown that the ability to detect the cortical stimulation impulse varies depending on some characteristics of the visual images viewed at the time of brain stimulation, revealing the visual nature of the perceptual events induced by stimulation of the IT cortex. Here we systematically studied the effect of the size of viewed objects on behavioral detectability of optogenetic stimulation of the central IT cortex. Surprisingly, we found that behavioral detection of the same optogenetic impulse highly varies with the size of the viewed object images. Reduction of the object size in four steps from 8 to 1 degree of visual angle significantly decreased detection performance. These results show that identical stimulation impulses delivered to the same neural population induce variable perceptual events depending on the mere size of the objects viewed at the time of brain stimulation. 1.
Justin B. Kueser; Ryan Peters; Arielle Borovsky
The role of semantic similarity in verb learning events: Vocabulary-related changes across early development Journal Article
In: Journal of Experimental Child Psychology, vol. 226, pp. 1–19, 2023.
Verb meaning is challenging for children to learn across varied events. This study examined how the taxonomic semantic similarity of the nouns in novel verb learning events in a progressive alignment learning condition differed from the taxonomic dissimilarity of nouns in a dissimilar learning condition in supporting near (similar) and far (dissimilar) verb generalization to novel objects in an eye-tracking task. A total of 48 children in two age groups (23 girls; younger: 21–24 months
Anastasia Kerr-German; A. Caglar Tas; Aaron T. Buss
A multi-method approach to addressing the toddler data desert in attention research Journal Article
In: Cognitive Development, vol. 65, pp. 1–14, 2023.
Visual attention skills undergo robust development change during infancy and continue to co-develop with other cognitive processes in early childhood. Despite this, this is a general disconnect between measures of the earliest foundations of attention during infancy and later development of attention in relation to executive functioning during the toddler years. To examine associations between these different measures of attention, the current study administered an oculomotor task (infant orienting with attention, IOWA) and a manual response (Flanker) task with a group of toddlers. We collected simultaneous neural recordings (using functional near-infrared spectroscopy), eye-tracking, and behavioral responses in 2.5- and 3.5-year-olds to examine the neural and behavioral associations between these skills. Results revealed that oculomotor facilitation in the IOWA task was negatively associated with accuracy on neutral trials in the Flanker task. Second, conflict scores between the two tasks were positively associated. At the neural level, however, the tasks showed distinct patterns of activation. Left frontal cortex was engaged during the Flanker task whereas right frontal and parietal cortex was engaged during the IOWA task. Activation during the IOWA task differed based on how well children could control oculomotor behavior during the task. Children with high levels of stimulus reactivity activated parietal cortex more strongly, but children with more controlled oculomotor behavior activated frontal cortex more strongly.
Yiru Huang; Zitian Liu; Zidong Chen; Zongyi Zhan; Le Gao; Jingyi Hu; Yanyan Wu; Fang-Fang Yan; Daming Deng; Chang-Bing Huang; Minbin Yu
Visual crowding reveals field- and axis-specific cortical miswiring after long-term axial misalignment in strabismic patients without amblyopia Journal Article
In: Investigative Ophthalmology & Visual Science, vol. 64, no. 1, pp. 1–11, 2023.
PURPOSE. Inspired by physiological and neuroimaging findings that revealed squint- induced modification of cortical volume and visual receptive field in early visual areas, we hypothesized that strabismic eyes without amblyopia manifest an increase in critical spacing of visual crowding, an essential bottleneck on object recognition and reliable psychophysical index of cortical organization. METHODS. We used real-time eye tracking to ensure gaze-contingent display and examined visual crowding in patients with horizontal concomitant strabismus (both esotropia and exotropia) but without amblyopia and age-matched normal controls. RESULTS. Nineteen patients with exotropia (12 men, mean ± SD = 22.89 ± 7.82 years), matched normal controls (7 men, mean ± SD = 23.07 ± 1.07 years) participated in this 21 patients with esotropia (10 men, mean ± SD = 23.48 ± 6.95 years), and 14 age- study. We found that patients with strabismus without amblyopia showed significantly larger critical spacing with nasotemporal asymmetry in only the radial axis that related to the strabismus pattern, with exotropia exhibiting stronger temporal hemifield crowding and esotropia exhibiting stronger nasal hemifield crowding, in both the deviated and fixating eyes. Moreover, the magnitude of crowding change was related to the duration and degree of strabismic deviation. CONCLUSIONS. Using visual crowding as a psychophysical index of cortical organization, our study demonstrated significantly greater peripheral visual crowding with nasotemporal asymmetry in only the radial axis in patients with strabismus without amblyopia, indicating the existence of hemifield- and axis-specific miswiring of cortical processing in object recognition induced by long-term adaptation to ocular misalignment.
Scott S. Hsieh; David A. Cook; Akitoshi Inoue; Hao Gong; Parvathy Sudhir Pillai; Matthew P. Johnson; Shuai Leng; Lifeng Yu; Jeff L. Fidler; David R. Holmes Iii; Rickey E. Carter; Cynthia H. Mccollough; Joel G. Fletcher
Understanding reader variability: A 25-radiologist study on liver metastasis detection at CT Journal Article
In: Radiology, vol. 306, no. 2, pp. 1–10, 2023.
Background: Substantial interreader variability exists for common tasks in CT imaging, such as detection of hepatic metastases. This variability can undermine patient care by leading to misdiagnosis. Purpose: To determine the impact of interreader variability associated with (a) reader experience, (b) image navigation patterns (eg, eye movements, workstation interactions), and (c) eye gaze time at missed liver metastases on contrast-enhanced abdominal CT images. Materials and Methods: In a single-center prospective observational trial at an academic institution between December 2020 and February 2021, readers were recruited to examine 40 contrast-enhanced abdominal CT studies (eight normal, 32 containing 91 liver metastases). Readers circumscribed hepatic metastases and reported confidence. The workstation tracked image navigation and eye movements. Performance was quantified by using the area under the jackknife alternative free-response receiver operator charac- teristic (JAFROC-1) curve and per-metastasis sensitivity and was associated with reader experience and image navigation variables. Differences in area under JAFROC curve were assessed with the Kruskal-Wallis test followed by the Dunn test, and effects of image navigation were assessed by using the Wilcoxon signed-rank test. Results: Twenty-five readers (median age, 38 years; IQR, 31–45 years; 19 men) were recruited and included nine subspecialized abdominal radiologists, five nonabdominal staff radiologists, and 11 senior residents or fellows. Reader experience explained differences in area under the JAFROC curve, with abdominal radiologists demonstrating greater area under the JAFROC curve (mean, 0.77; 95% CI: 0.75, 0.79) than trainees (mean, 0.71; 95% CI: 0.69, 0.73) (P = .02) or nonabdominal subspecialists (mean, 0.69; 95% CI: 0.60, 0.78) (P = .03). Sensitivity was similar within the reader experience groups (P = .96). Image navigation variables that were associated with higher sensitivity included longer interpretation time (P = .003) and greater use of coronal images (P < .001). The eye gaze time was at least 0.5 and 2.0 seconds for 71% (266 of 377) and 40% (149 of 377) of missed metastases, respectively. Conclusion: Abdominal radiologists demonstrated better discrimination for the detection of liver metastases on abdominal contrast-enhanced CT images. Missed metastases frequently received at least a brief eye gaze. Higher sensitivity was associated with longer interpretation time and greater use of liver display windows and coronal images.
Monja Hoven; Alejandro Hirmas; Jan Engelmann; Ruth Holst
The role of attention in decision-making under risk in gambling disorder: An eye-tracking study Journal Article
In: Addictive Behaviors, vol. 138, pp. 1–10, 2023.
Gambling disorder (GD) is a behavioural addiction characterized by impairments in decision-making, favouring risk- and reward-prone choices. One explanatory factor for this behaviour is a deviation in attentional processes, as increasing evidence indicates that GD patients show an attentional bias toward gambling stimuli. However, previous attentional studies have not directly investigated attention during risky decision-making. 26 patients with GD and 29 healthy matched controls (HC) completed a mixed gambles task combined with eye-tracking to investigate attentional biases for potential gains versus losses during decision-making under risk. Results indicate that compared to HC, GD patients gambled more and were less loss averse. GD patients did not show a direct attentional bias towards gains (or relative to losses). Using a recent (neuro)economics model that considers average attention and trial-wise deviations in average attention, we conducted fine-grained exploratory analyses of the attentional data. Results indicate that the average attention for gains in GD patients moderated the effect of gain value on gambling choices, whereas this was not the case for HC. GD patients with high average attention for gains started gambling at less high gain values. A similar trend-level effect was found for losses, where GD patients with high average attention for losses stopped gambling at lower loss values. This study gives more insight into how attentional processes in GD play a role in gambling behaviour, which could have implications for the development of future treatments focusing on attentional training or for the development of interventions that increase the salience of losses.
Liu Hong-Zhi; Yang Xing-Lan; Li Qiu-Yue; Wei Zi-Han
Preference of dimension-based difference in intertemporal choice: Eye-tracking evidence Journal Article
In: Acta Psychologica Sinica, vol. 55, no. 4, pp. 612–625, 2023.
In the field of intertemporal choice, considerable empirical evidence from behavioral and process data supports the use of dimension-based models. The existing dimension-based models provide qualitative explanations for an individual's intertemporal choice and focus on “which dimension is the greater difference dimension”, but ignore the preference of dimension-based difference (i.e., “how much different of the difference between the two dimensions”). In the present study, we used eye-tracking technology to examine the relationship between the preference of dimension-based difference and the information searching process. The results in the two experiments consistently revealed that response time, gaze transition entropy (a measure of visual scanning efficiency), and stationary gaze entropy (a measure of the level of even distribution across different areas of interest) could negatively predict the preference of dimension-based difference. Our findings highlighted the correlation between the preference of dimension-based difference and the information searching process, providing further process evidence for dimension-based intertemporal models.
Elle Van Heusden; Christian N. L. Olivers; Mieke Donk
The eyes prefer targets nearby fixation: Quantifying eccentricity-dependent attentional biases in oculomotor selection Journal Article
In: Vision Research, vol. 205, pp. 1–11, 2023.
An important function of peripheral vision is to provide the target of the next eye movement. Here we investigate the extent to which the eyes are biased to select a target closer to fixation over one further away. Participants were presented with displays containing two identical singleton targets and were asked to move their eyes to either one of them. The targets could be presented at three different eccentricities relative to central fixation. In one condition both singletons were presented at the same eccentricity, providing an estimate of the speed of selection at each of the eccentricities. The saccadic latency distributions from this same-eccentricity condition were then used to predict the selection bias when both targets were presented at different eccentricities. The results show that when targets are presented at different eccentricities, participants are biased to select the item closest to fixation. This eccentricity-based bias was considerably stronger than predicted on the basis of saccadic latency distributions in the same-eccentricity condition. This rules out speed of processing per se as a sole explanation for such a bias. Instead, the results are consistent with attentional competition being weighted in favour of items close to fixation.
Dongcheng He; Haluk Öğmen
A neural model for vector decomposition and relative-motion perception Journal Article
In: Vision Research, vol. 202, pp. 1–21, 2023.
The perception of motion not only depends on the detection of motion signals but also on choosing and applying reference-frames according to which motion is interpreted. Here we propose a neural model that implements the common-fate principle for reference-frame selection. The model starts with a retinotopic layer of directionally-tuned motion detectors. The Gestalt common-fate principle is applied to the activities of these detectors to implement in two neural populations the direction and the magnitude (speed) of the reference-frame. The output activities of retinotopic motion-detectors are decomposed using the direction of the reference-frame. The direction and magnitude of the reference-frame are then applied to these decomposed motion-vectors to generate activities that reflect relative-motion perception, i.e., the perception of motion with respect to the prevailing reference-frame. We simulated this model for classical relative motion stimuli, viz., the three-dot, rotating-wheel, and point-walker (biological motion) paradigms and found the model performance to be close to theoretical vector decomposition values. In the three-dot paradigm, the model made the prediction of perceived curved-trajectories for the target dot when its horizontal velocity was slower or faster than the flanking dots. We tested this prediction in two psychophysical experiments and found a good qualitative and quantitative agreement between the model and the data. Our results show that a simple neural network using solely motion information can account for the perception of group and relative motion.
Yuki Harada; Junji Ohyama; Makoto Wada
Effects of temporal properties of facial expressions on the perceived intensity of emotion Journal Article
In: Royal Society Open Science, vol. 10, pp. 1–14, 2023.
A series of multiple facial expressions can be temporally perceived as an averaged facial expression in a process known as ensemble perception. This study examined the effect of temporal parameters on the perceived intensity of facial expression in each emotion, and how the effect varies with autistic traits in typically developing people. In the experiment, we presented facial expressions that switched from emotional to neutral expressions, and vice versa, for 3s. Participants rated the overall perceived intensity of the facial emotions as a whole rather than rating individual items within the set. For the two tasks, a ratio of duration of emotional faces to duration of neutral faces (emotional ratio) and the timing for transitions were manipulated individually. The results showed that the intensity of facial emotion was perceived more strongly when the presentation ratio increased and when the emotional expression was presented last. The effects were different among the emotions (e.g. relatively weak in the anger expression). Moreover, the perceived intensity of angry expressions decreased with autistic traits. These results suggest that the properties and individual differences in the facial ensemble of each emotion affect emotional perceptions.
Zachary Hamblin-Frohman; Jia Xuan Low; Stefanie I. Becker
Attentional prioritisation and facilitation for similar stimuli in visual working memory Journal Article
In: Psychological Research, pp. 1–8, 2023.
Visual working memory (VWM) allows for the brief retention of approximately three to four items. Interestingly, when these items are similar to each other in a feature domain, memory recall performance is elevated compared to when they are dissimilar. This similarity benefit is currently not accounted for by models of VWM. Previous research has suggested that this similarity benefit may arise from selective attentional prioritisation in the maintenance phase. However, the similarity effect has not been contrasted under circumstances where dissimilar item types can adequately compete for memory resources. In Experiment 1, similarity benefits were seen for all-similar over all-dissimilar displays. This was also seen in mixed displays, change detection performance was higher when one of the two similar items changed, compared to when the dissimilar item changed. Surprisingly, the similarity effect was stronger in these mixed displays then when comparing the all-similar and all-dissimilar. Experiment 2 investigated this further by examining how attention was allocated in the memory encoding phase via eye movements. Results revealed that attention prioritised similar over dissimilar items in the mixed displays. Similar items were more likely to receive the first fixation and were fixated more often than dissimilar items. Furthermore, dwell times were elongated for dissimilar items, suggesting that encoding was less efficient. These results suggest that there is an attentional strategy towards prioritising similar items over dissimilar items, and that this strategy's influence can be observed in the memory encoding phase.
Michael Hahn; Frank Keller
Modeling task effects in human reading with neural network-based attention Journal Article
In: Cognition, vol. 230, pp. 1–25, 2023.
Research on human reading has long documented that reading behavior shows task-specific effects, but it has been challenging to build general models predicting what reading behavior humans will show in a given task. We introduce NEAT, a computational model of the allocation of attention in human reading, based on the hypothesis that human reading optimizes a tradeoff between economy of attention and success at a task. Our model is implemented using contemporary neural network modeling techniques, and makes explicit and testable predictions about how the allocation of attention varies across different tasks. We test this in an eyetracking study comparing two versions of a reading comprehension task, finding that our model successfully accounts for reading behavior across the tasks. Our work thus provides evidence that task effects can be modeled as optimal adaptation to task demands.
Xianliang Ge; Yunxian Pan; Sujie Wang; Linze Qian; Jingjia Yuan; Jie Xu; Nitish Thakor; Yu Sun
Improving intention detection in single-trial classification through fusion of EEG and eye-tracker data Journal Article
In: IEEE Transactions on Human-Machine Systems, vol. 53, no. 1, pp. 132–141, 2023.
Intention decoding is an indispensable procedure in hands-free human-computer interaction (HCI). Conventional eye-tracking system using single-model fixation duration possibly issues commands ignoring users' real expectation. In the current study, an eye-brain hybrid brain-computer interface (BCI) interaction system was introduced for intention detection through fusion of multi-modal eye-track and ERP (a measurement derived from EEG) features. Eye-track and EEG data were recorded from 64 healthy participants as they performed a 40-min customized free search task of a fixed target icon among 25 icons. The corresponding fixation duration of eye-trackingw and ERP were extracted. Five previously-validated LDA-based classifiers (including RLDA, SWLDA, BLDA, SKLDA, and STDA) and the widely-used CNN method were adopted to verify the efficacy of feature fusion from both offline and pseudo-online analysis, and optimal approach was evaluated through modulating the training set and system response duration. Our study demonstrated that the input of multi-modal eye-track and ERP features achieved superior performance of intention detection in the single trial classification of active search task. And compared with single-model ERP feature, this new strategy also induced congruent accuracy across different classifiers. Moreover, in comparison with other classification methods, we found that the SKLDA exhibited the superior performance when fusing feature in offline test (ACC=0.8783
Rui Fu; Tao Huang; Mingyue Li; Qinyu Sun; Yunxing Chen
A multimodal deep neural network for prediction of the driver's focus of attention based on anthropomorphic attention mechanism and prior knowledge Journal Article
In: Expert Systems with Applications, vol. 214, pp. 1–12, 2023.
The prediction of the driver's focus of attention (DFoA) is becoming essential research for the driver distraction detection and intelligent vehicle. Therefore, this work makes an attempt to predict DFoA. However, traffic driving environment is a complex and dynamic changing scene. The existing methods lack full utilization of driving scene information and ignore the importance of different objects or regions of the driving scene. To alleviate this, we propose a multimodal deep neural network based on anthropomorphic attention mechanism and prior knowledge (MDNN-AAM-PK). Specifically, a more comprehensive information of driving scene (RGB images, semantic images, optical flow images and depth images of successive frames) is as the input of MDNN-AAM-PK. An anthropomorphic attention mechanism is developed to calculate the importance of each pixel in the driving scene. A graph attention network is adopted to learn semantic context features. The convolutional long short-term memory network (ConvLSTM) is used to achieve the transition of fused features in successive frames. Furthermore, a training method based on prior knowledge is designed to improve the efficiency of training and the performance of DFoA prediction. These experiments, including experimental comparison with the state-of-the-art methods, the ablation study of the proposed method, the evaluation on different datasets and the visual assessment experiment in vehicle simulation platform, show that the proposed method can accurately predict DFoA and is better than the state-of-the-art methods.
Nick Fogt; Andrew J Toole; Xiangrui Li; Emmanuel Owusu; Steven T Manning; Marjean T Kulp
Functional magnetic resonance imaging activation for different vergence eye movement subtypes Journal Article
In: Ophthalmic and Physiological Optics, vol. 43, pp. 93–104, 2023.
Introduction: Maddox suggested that there were four convergence subtypes, each driven by a different stimulus. The purpose of this study was to assess the neural correlates for accommodative convergence, proximal convergence (convergence stimulus provided), disparity convergence and voluntary convergence (no specific convergence stimulus provided) using functional magnetic resonance imaging (fMRI). Methods: Ten subjects (mean age = 24.4 years) with normal binocular vision participated. The blood oxygenation level- dependent (BOLD) signals of the brain from fMRI scans were measured when subjects made vergence eye movements while: (1) alternately viewing letters monocularly where one eye viewed through a −2.00 D lens, (2) alternately viewing Difference of Gaussian targets monocularly at distance and near, (3) viewing random dot stereograms with increasing disparity and (4) voluntarily converging the eyes with binocular viewing. Results: The accommodative convergence paradigm resulted in activation on the right side in the right fusiform cortex and the right middle occipital cortex. The proximal convergence stimulus mainly activated areas in the right occipital lobe. The disparity stimulus activated areas in the left occipital cortex and the left frontal cortex. Finally, the voluntary convergence paradigm resulted in activation primarily in the occipital lobe and mostly bilaterally. Conclusion: The accommodative, proximal, disparity and voluntary convergence paradigms resulted in activation in unique areas in the brain with functional MRI. Activation was found in more areas in the proximal and voluntary conditions compared with the accommodative and disparity conditions.
Laura Fernández-Arroyo; Nuria Sagarra; Kaylee Fernández
Differential effects of language proficiency and use on L2 lexical prediction Journal Article
In: The Mental Lexicon, pp. 1–26, 2023.
Language experience is essential for SLA. Yet, studies comparing the role of L2 proficiency and L2 use on L2 processing are scant, and there are no studies examining how these variables modulate learners' ability to generalize grammatical associations to new instances. This study investigates whether L2 proficiency and L2 use affect L2 stress-tense suffix associations (a stressed syllable cuing a present suffix, and an unstressed syllable cuing a preterit suffix) using eye-tracking. Spanish monolinguals and English learners of Spanish varying in L2 proficiency and L2 use saw two verbs (e.g., firma-firmó ‘(s)he signs/signed'), heard a sentence containing one of the verbs, and chose the verb they had heard. Both groups looked at target verbs above chance before hearing the suffix, but the monolinguals did so more accurately and earlier than the learners. The learners recognized past verbs faster than present verbs, were faster with higher than lower L2 proficiency, and later with higher than lower L2 use. Finally, higher L2 proficiency yielded earlier morphological activation but higher L2 use produced later morphological activation, indicating that L2 proficiency and L2 use affect L2 word processing differently. We discuss the contribution of these findings to language acquisition and processing models, as well as models of general cognition.
Julia Farrell; Stefania Conte; Ryan Barry-Anwar; Lisa S. Scott
Face race and sex impact visual fixation strategies for upright and inverted faces in 3- to 6-year-old children Journal Article
In: Developmental Psychobiology, vol. 65, pp. 1–15, 2023.
Everyday face experience tends to be biased, such that infants and young children interact more often with own-race and female faces leading to differential processing of faces within these groups relative to others. In the present study, visual fixation strategies were recorded using eye tracking to determine the extent to which face race and sex/gender impact a key index of face processing in 3- to 6-year-old children (n = 47). Children viewed male and female upright and inverted White and Asian faces while visual fixations were recorded. Face orientation was found to have robust effects on children's visual fixations, such that children exhibited shorter first fixation and average fixation durations and a greater number of fixations for inverted compared to upright face trials. First fixations to the eye region were also greater for upright compared to inverted faces. Fewer fixations and longer duration fixations were found for trials with male compared to female faces and for upright compared to inverted unfamiliar-race faces, but not familiar-race faces. These findings demonstrate evidence of differential fixation strategies toward different types of faces in 3- to 6-year-old chil- dren, illustrating the importance of experience in the development of visual attention to faces.
Elizabeth H. Evans; Martin J. Tovée; Peter J. B. Hancock; Piers L. Cornelissen
How do looking patterns, anti-fat bias, and causal weight attributions relate to adults' judgements of child weight? Journal Article
In: Body Image, vol. 44, pp. 9–23, 2023.
Prevailing weight-normative approaches to health pressure adults to visually categorise children's weight, despite little understanding of how such judgements are made. There is no evidence this strategy improves child health, and it may harm children with higher weights. To understand decision-making processes and identify potential mechanisms of harm we examined perceptual and attitudinal factors involved in adults' child weight category judgements. Eye movements of 42 adults were tracked while categorizing the weight of 40 computer-generated images of children (aged 4–5 & 10–11 years) varying in size. Questionnaires assessed child-focused weight bias and causal attributions for child weight. Participants' eye movement patterns resembled those previously reported for adult bodies. Categorisation data showed a perceptual bias towards the ‘mid-range' category. For higher weight stimuli, participants whose category judgements most closely matched the stimulus's objective weight had higher child-focused anti-fat bias and weaker genetic attributions for child weight – i.e,. adults who ‘label' higher weight in children in line with BMI categories report more stigmatising beliefs about such children, suggesting a possible mechanism of harm. Overall, adults' judgements reflect both unalterable perceptual biases and potentially harmful attitudinal factors, calling into question the feasibility and appropriateness of public health efforts to promote visual child weight categorisation.
Bret Eschman; Shannon Ross-Sheehy
Visual short-term memory persists across multiple fixations: An n-back approach to quantifying capacity in infants and adults Journal Article
In: Psychological Science, pp. 1–14, 2023.
Visual short-term memory (STM) is a foundational component of general cognition that develops rapidly during the first year of life. Although previous research has revealed important relations between overt visual fixation and memory formation, it is unknown whether infants can maintain distinct memories for sequentially fixated items or remember nonfixated array items. Participants (5-month-olds, 11-month-olds, and adults; n = 24 at each age) from the United States were tested in a passive change-detection paradigm with an n-back manipulation to examine memory for the last fixated item (one-back), second-to-last fixated item (two-back), or nonfixated item (change-other). Eye tracking was used to measure overt fixation while participants passively viewed arrays of colored circles. Results for all ages revealed convergent evidence of memory for up to two sequentially fixated objects (i.e., one-back, two-back), with moderate evidence for nonfixated array items (change-other). A permutation analysis examining change preference over time suggested that differences could not be explained by perseverative looking or location biases.
Seth Elkin-Frankston; Carlene Horner; Reem Alzahabi; Matthew S. Cain
Characterizing motion prediction in small autonomous swarms Journal Article
In: Applied Ergonomics, vol. 106, pp. 1–7, 2023.
The use of robotic swarms has become increasingly common in research, industrial, and military domains for tasks such as collective exploration, coordinated movement, and collective localization. Despite the expanded use of robotic swarms, little is known about how swarms are perceived by human operators. To characterize human-swarm interactions, we evaluate how operators perceive swarm characteristics, including movement patterns, control schemes, and occlusion. In a series of experiments manipulating movement patterns and control schemes, participants tracked swarms on a computer screen until they were occluded from view, at which point participants were instructed to estimate the spatiotemporal dynamics of the occluded swarm by mouse click. In addition to capturing mouse click responses, eye tracking was used to capture participants eye movements while visually tracking swarms. We observed that manipulating control schemes had minimal impact on the perception of swarms, and that swarms are easier to track when they are visible compared to when they were occluded. Regarding swarm movements, a complex pattern of data emerged. For example, eye tracking indicates that participants more closely track a swarm in an arc pattern compared to sinusoid and linear movement patterns. When evaluating behavioral click-responses, data show that time is underestimated, and that spatial accuracy is reduced in complex patterns. Results suggest that measures of performance may capture different patterns of behavior, underscoring the need for multiple measures to accurately characterize performance. In addition, the lack of generalizable data across different movement patterns highlights the complexity involved in the perception of swarms of objects.
Jay A. Edelman; Tim A. Ahles; Neelam Prashad; Madalyn Fernbach; Yuelin Li; Robert D. Melara; James C. Root
The effect of visual target presence and age on antisaccade performance Journal Article
In: Journal of Neurophysiology, vol. 129, pp. 307–319, 2023.
Antisaccade and prosaccade (PS) performance were studied in a large cohort of females (age range 42–74 yr). Antisaccade performance was assessed in two variants of the task, a “traditional” antisaccade (TA) task, in which no visual stimuli were present at the saccade goal, and a visually guided antisaccade (VGA) task, in which small visual stimuli were present at the possible saccade goals prior to the imperative visual stimulus. Directional error frequency was similar in the two antisaccade tasks. However, reaction time (RT) was $sim$33 ms longer in the VGA task than in the TA task. Across participants, the average saccade amplitudes of prosaccades and TAs were both correlated with those of VGAs but not with each other. TAs had a hypermetria that increased with age. Saccade amplitude variability was much higher for TAs than for PSs and VGAs. Saccade polar angle variability was low for all three tasks. Age diminished performance with modest task dependence, except for an increase in TA hypermetria. These results suggest that the generation of antisaccade directional errors does not depend on visual target presence at the saccade goal, that antisaccade RT can be affected by target presence, that age can increase saccade hypermetria in the absence of visual guidance, and that visually guided antisaccades are governed by distinct voluntary and visually guided saccade mechanisms. Moreover, these results suggest that an understanding of human motor performance benefits from the use of a participant pool with a larger age range than that used in most studies.
Tiantian Dong; Juncai Sun; Wen He
Positive and spontaneous facial expressions convey kindness and competence personality traits: Visual reasoning in personality attribution to faces Journal Article
In: Personality and Individual Differences, vol. 200, no. 57, pp. 1–7, 2023.
People often make personality attribution from facial emotional cues. Previous studies have demonstrated the importance of the valence of facial expressions on personality attribution. However, authenticity is an essential dimension of facial expressions, and it is more important to know how does facial authenticity affect personality attribution. This study combined the personality judgments task and eye-tracking methods to explore the effect of valence and authenticity of expressions on personality attribution and the visual reasoning modes in different expressions. Results showed that expressions with more spontaneous and positive were more likely to be considered kind and competent. Facial expressions affected the extent to which gaze was directed toward the eyebrows for kindness and the mouth for competence. These results suggest that personality attribution is influenced by facial emotional cues, such as valence and authenticity, and this effect is mapped onto specific face regions related to the personality.
Carmen Julia Coloma; Ernesto Guerra; Zulema De Barbieri; Andrea Helo; Carmen Julia; Ernesto Guerra; Zulema De Barbieri; Andrea Helo
Article comprehension in monolingual Spanish-speaking children with developmental language disorder: A longitudinal eye tracking study Journal Article
In: International Journal of Speech-Language Pathology, pp. 1–13, 2023.
Purpose: Article-noun disagreement in spoken language is a marker of children with developmental language disorder (DLD). However, the evidence is less clear regarding article comprehension. This study investigates article comprehension in monolingual Spanish-speaking children with and without DLD. Method: Eye tracking methodology used in a longitudinal experimental design enabled the examination of real time article comprehension. The children at the time 1 were 40 monolingual Spanish-speaking preschoolers (20 with DLD and 20 with typical language development [TLD]). A year later (time 2), 27 of these children (15 with DLD and 12 with TLD) were evaluated. Children listened to simple phrases while inspecting a four object visual context. The article in the phrase agreed in number and gender with only one of the objects. Result: At the time 1, children with DLD did not use articles to identify the correct image, while children with TLD anticipated the correct picture. At the time 2, both groups used the articles' morphological markers, but children with DLD showed a slower and weaker preference for the correct referent compared to their age-matched peers. Conclusion: These findings suggest a later emergence, but a similar developmental trajectory, of article comprehension in children with DLD compared to their peers with TLD.
Sebastian C. Coleman; Zelekha A. Seedat; Anna C. Whittaker; Agatha Lenartowicz; Karen J. Mullinger
Beyond the beta rebound: Post-task responses in oscillatory activity follow cessation of working memory processes Journal Article
In: NeuroImage, vol. 265, pp. 1–11, 2023.
Post-task responses (PTRs) are transitionary responses occurring for several seconds between the end of a stimulus/task and a period of rest. The most well-studied of these are beta band (13 – 30 Hz) PTRs in motor networks following movement, often called post-movement beta rebounds, which have been shown to differ in patients with schizophrenia and autism. Previous studies have proposed that beta PTRs reflect inhibition of task-positive networks to enable a return to resting brain activity, scaling with cognitive demand and reflecting cortical self-regulation. It is unknown whether PTRs are a phenomenon of the motor system, or whether they are a more general self-modulatory property of cortex that occur following cessation of higher cognitive processes as well as movement. To test this, we recorded magnetoencephalography (MEG) responses in 20 healthy participants to a working-memory task, known to recruit cortical networks associated with higher cognition. Our results revealed PTRs in the theta, alpha and beta bands across many regions of the brain, including the dorsal attention network (DAN) and lateral visual regions. These PTRs increased significantly (p < 0.05) in magnitude with working-memory load, an effect which is independent of oscillatory modulations occurring over the task period as well as those following individual stimuli. Furthermore, we showed that PTRs are functionally related to reaction times in left lateral visual (p < 0.05) and left parietal (p < 0.1) regions, while the oscillatory responses measured during the task period are not. Importantly, motor PTRs following button presses did not modulate with task condition, suggesting that PTRs in different networks are driven by different aspects of cognition. Our findings show that PTRs are not limited to motor networks but are widespread in regions which are recruited during the task. We provide evidence that PTRs have unique properties, scaling with cognitive load and correlating significantly with behaviour. Based on the evidence, we suggest that PTRs inhibit task-positive network activity to enable a transition to rest, however, further investigation is required to uncover their role in neuroscience and pathology.
Laura Cacciamani; Rachel M. Skocypec; Colin S. Flowers; Diana C. Perez; Mary A. Peterson
BOLD activation on the groundside of figures: More suppression of grounds that competed more for figural status Journal Article
In: Cortex, vol. 158, pp. 96–109, 2023.
A fundamental aspect of object detection is assigning a border to one (figure) side but not the other (ground) side. Figures are shaped; grounds appear shapeless near the figure border. Accumulating evidence supports the view that the mechanism of figure assignment is inhibitory competition with the figure perceived on the winning side. Suppression has been observed on the groundside of figure borders. One prediction is that more suppression will be observed when the groundside competes more for figural status. We tested this prediction by assessing BOLD activation on the groundside of two types of stimuli with articulated borders: AEnov and AEfam stimuli. In both stimulus types, multiple image-based priors (symmetry, closure, small area, enclosure by a larger region) favored the inside as the figure. In AEfam but not AEnov stimuli, the figural prior of familiar configuration present on the outside competes for figural status. Observers perceived the insides of both types of stimuli as novel figures and the outsides as shapeless grounds. Previously, we observed lower BOLD activation in early visual areas representing the grounds of AEfam than AEnov stimuli, although unexpectedly, activation was above baseline. With articulated borders, it can be difficult to exclude figure activation from ground ROIs. Here, our ground ROIs better excluded figure activation; we also added straight-edge (SE) control stimuli and increased the sample size. In early visual areas representing the grounds, we observed lower BOLD activation on the groundside of AEfam than AEnov stimuli and below-baseline BOLD activation on the groundside of SE and AEfam stimuli. These results, indicating that greater suppression is applied to groundsides that competed more for figural status but lost the competition, support a Bayesian model of figure assignment in which proto-objects activated at both low and high levels where image features and familiar configurations are represented, respectively, compete for figural status.
Tom Bullock; Mary H. Maclean; Tyler Santander; Alexander P. Boone; Viktoriya Babenko; Neil M. Dundon; Alexander Stuber; Liann Jimmons; Jamie Raymer; Gold N. Okafor; Michael B. Miller; Barry Giesbrecht; Scott T. Grafton; Janice Marshall
Habituation of the stress response multiplex to repeated cold pressor exposure Journal Article
In: Frontiers in Psychology, vol. 13, pp. 1–23, 2023.
Humans show remarkable habituation to aversive events as reflected by changes of both subjective report and objective measures of stress. Although much experimental human research focuses on the effects of stress, relatively little is known about the cascade of physiological and neural responses that contribute to stress habituation. The cold pressor test (CPT) is a common method for inducing acute stress in human participants in the laboratory; however, there are gaps in our understanding of the global state changes resulting from this stress-induction technique and how these responses change over multiple exposures. Here, we measure the stress response to repeated CPT exposures using an extensive suite of physiologic measures and state-of-the-art analysis techniques. In two separate sessions on different days, participants underwent five 90 s CPT exposures of both feet and five warm water control exposures, while electrocardiography (ECG), impedance cardiography, continuous blood pressure, pupillometry, scalp electroencephalography (EEG), salivary cortisol and self-reported pain assessments were recorded. A diverse array of adaptive responses are reported that vary in their temporal dynamics within each exposure as well as habituation across repeated exposures. During cold-water exposure there was a cascade of changes across several cardiovascular measures (elevated heart rate (HR), cardiac output (CO) and Mean Arterial Pressure (MAP) and reduced left ventricular ejection time (LVET), stroke volume (SV) and high- frequency heart rate variability (HF)). Increased pupil dilation was observed, as was increased power in low-frequency bands (delta and theta) across frontal EEG electrode sites. Several cardiovascular measures also habituated over repeated cold-water exposures (HR, MAP, CO, SV, LVET) as did pupil dilation and alpha frequency activity across the scalp. Anticipation of cold water induced stress effects in the time-period immediately prior to exposure, indexed by increased pupil size and cortical disinhibition in the alpha and beta frequency bands across central scalp sites. These results provide comprehensive insight into the evolution of a diverse array of stress responses to an acute noxious stressor, and how these responses adaptively contribute to stress habituation.
Rolando Bonandrini; Eraldo Paulesu; Daniela Traficante; Elena Capelli; Marco Marelli; Claudio Luzzatti
Lateralized reading in the healthy brain: A behavioral and computational study on the nature of the visual field effect Journal Article
In: Neuropsychologia, vol. 180, pp. 1–16, 2023.
Despite its widespread use to measure functional lateralization of language in healthy subjects, the neuro- cognitive bases of the visual field effect in lateralized reading are still debated. Crucially, the lack of knowledge on the nature of the visual field effect is accompanied by a lack of knowledge on the relative impact of psycholinguistic factors on its measurement, thus potentially casting doubts on its validity as a functional laterality measure. In this study, an eye-tracking-controlled tachistoscopic lateralized lexical decision task (Experiment 1) was administered to 60 right-handed and 60 left-handed volunteers and word length, orthographic neighbor- hood, word frequency, and imageability were manipulated. The magnitude of visual field effect was bigger in right-handed than in left-handed participants. Across the whole sample, a visual field-by-frequency interaction was observed, whereby a comparatively smaller effect of word frequency was detected in the left visual field/ right hemisphere (LVF/RH) than in the right visual field/left hemisphere (RVF/LH). In a subsequent computational study (Experiment 2), efficient (LH) and inefficient (RH) activation of lexical orthographic nodes was modelled by means of the Naïve Discriminative Learning approach. Computational data simulated the effect of visual field and its interaction with frequency observed in the Experiment 1. Data suggest that the visual field effect can be biased by word frequency. Less distinctive connections between orthographic cues and lexical/ semantic output units in the RH than in the LH can account for the emergence of the visual field effect and its interaction with word frequency.
Christina Blomquist; Bob MCMurray
The development of lexical inhibition in spoken word recognition Journal Article
In: Developmental Psychology, vol. 59, no. 1, pp. 186–206, 2023.
As a spoken word unfolds over time, similar sounding words (cap and cat) compete until one word “wins”. Lexical competition becomes more efficient from infancy through adolescence. We examined one potential mechanism underlying this development: lexical inhibition, by which activated candidates suppress competitors. In Experiment 1, younger (7–8 years) and older (12–13 years) children heard words (cap) in which the onset was manipulated to briefly boost competition from a cohort competitor (cat). This was compared to a condition with a nonword (cack) onset that would not inhibit the target. Words were presented in a visual world task during which eye movements were recorded. Both groups showed less looking to the target when perceiving the competitor-splice relative to the nonword-splice, showing engagement of lexical inhibition. Exploratory analyses of linguistic adaptation across the experiment revealed that older children demonstrated consistent lexical inhibition across the experiment and younger children did not, initially showing no effect in the first half of trials and then a robust effect in the latter half. In Experiment 2, adults also displayed consistent lexical inhibition in the same task. These findings suggest that younger children do not consistently engage lexical inhibition in typical listening but can quickly bring it online in response to certain linguistic experiences. Computational modeling showed that age-related differences are best explained by increased engagement of inhibition rather than growth in activation. These findings suggest that continued development of lexical inhibition in later childhood may underlie increases in efficiency of spoken word recognition.
Christina M. Blomquist; Rochelle S. Newman; Jan Edwards
The development of spoken word recognition in informative and uninformative sentence contexts Journal Article
In: Journal of Experimental Child Psychology, vol. 227, pp. 1–10, 2023.
Although there is ample evidence documenting the development of spoken word recognition from infancy to adolescence, it is still unclear how development of word-level processing interacts with higher-level sentence processing, such as the use of lexical–semantic cues, to facilitate word recognition. We investigated how the ability to use an informative verb (e.g., draws) to predict an upcoming word (picture) and suppress competition from similar-sounding words (pickle) develops throughout the school-age years. Eye movements of children from two age groups (5–6 years and 9–10 years) were recorded while the children heard a sentence with an informative or neutral verb (The brother draws/gets the small picture) in which the final word matched one of a set of four pictures, one of which was a cohort competitor (pickle). Both groups demonstrated use of the informative verb to more quickly access the target word and suppress cohort competition. Although the age groups showed similar ability to use semantic context to facilitate processing, the older children demonstrated faster lexical access and more robust cohort suppression in both informative and uninformative contexts. This suggests that development of word-level processing facilitates access of top-down linguistic cues that support more efficient spoken language processing. Whereas developmental differences in the use of semantic context to facilitate lexical access were not explained by vocabulary knowledge, differences in the ability to suppress cohort competition were explained by vocabulary. This suggests a potential role for vocabulary knowledge in the resolution of lexical competition and perhaps the influence of lexical competition dynamics on vocabulary development.
Nathaniel J. Blanco; Brandon M. Turner; Vladimir M. Sloutsky
The benefits of immature cognitive control: How distributed attention guards against learning traps Journal Article
In: Journal of Experimental Child Psychology, vol. 226, pp. 1–16, 2023.
Cognitive control allows one to focus one's attention efficiently on relevant information while filtering out irrelevant information. This ability provides a means of rapid and effective learning, but using this control also brings risks. Importantly, useful information may be ignored and missed, and learners may fall into “learning traps” (e.g., learned inattention) wherein they fail to realize that what they ignore carries important information. Previous research has shown that adults may be more prone to such traps than young children, but the mechanisms underlying this difference are unclear. The current study used eye tracking to examine the role of attentional control during learning in succumbing to these learning traps. The participants, 4-year-old children and adults, completed a category learning task in which an unannounced switch occurred wherein the feature dimensions most relevant to correct categorization became irrelevant and formerly irrelevant dimensions became relevant. After the switch, adults were more likely than children to ignore the new highly relevant dimension and settle on a suboptimal categorization strategy. Furthermore, eye-tracking analyses reveal that greater attentional selectivity during learning (i.e., optimizing attention to focus only on the most relevant sources of information) predicted this tendency to miss important information later. Children's immature cognitive control, leading to broadly distributed attention, appears to protect children from this trap—although at the cost of less efficient and slower learning. These results demonstrate the double-edged sword of cognitive control and suggest that immature control may serve an adaptive function early in development.
B. Bellana; N. Ladyka-Wojcik; S. Lahan; M. Moscovitch; C. L. Grady
Recollection and prior knowledge recruit the left angular gyrus during recognition Journal Article
In: Brain Structure and Function, vol. 228, pp. 197–217, 2023.
The human angular gyrus (AG) is implicated in recollection, or the ability to retrieve detailed memory content from a specific episode. A separate line of research examining the neural bases of more general mnemonic representations, extracted over multiple episodes, also highlights the AG as a core region of interest. To reconcile these separate views of AG function, the present fMRI experiment used a Remember-Know paradigm with famous (prior knowledge) and non-famous (no prior knowledge) faces to test whether AG activity could be modulated by both task-specific recollection and general prior knowledge within the same individuals. Increased BOLD activity in the left AG was observed during both recollection in the absence of prior knowledge (recollected > non-recollected or correctly rejected non-famous faces) and when prior knowledge was accessed in the absence of experiment-specific recollection (famous > non-famous correct rejections). This pattern was most prominent for the left AG as compared to the broader inferior parietal lobe. Recollection-related responses in the left AG increased with encoding duration and prior knowledge, despite prior knowledge being incidental to the recognition decision. Overall, the left AG appears sensitive to both task-specific recollection and the incidental access of general prior knowledge, thus broadening our notions of the kinds of mnemonic representations that drive activity in this region.
Manuel Beiran; Nicolas Meirhaeghe; Hansem Sohn; Mehrdad Jazayeri; Manuel Beiran; Nicolas Meirhaeghe; Hansem Sohn; Mehrdad Jazayeri; Srdjan Ostojic
Parametric control of flexible timing through low- dimensional neural manifolds Journal Article
In: Neuron, vol. 111, pp. 1–15, 2023.
Biological brains possess an unparalleled ability to adapt behavioral responses to changing stimuli and environments. How neural processes enable this capacity is a fundamental open question. Previous works have identified two candidate mechanisms: a low-dimensional organization of neural activity and a modulation by contextual inputs. We hypothesized that combining the two might facilitate generalization and adaptation in complex tasks. We tested this hypothesis in flexible timing tasks where dynamics play a key role. Examining trained recurrent neural networks, we found that confining the dynamics to a low-dimensional subspace allowed tonic inputs to parametrically control the overall input-output transform, enabling generalization to novel inputs and adaptation to changing conditions. Reverse-engineering and theoretical analyses demonstrated that this parametric control relies on a mechanism where tonic inputs modulate the dynamics along non-linear manifolds while preserving their geometry. Comparisons with data from behaving monkeys confirmed the behavioral and neural signatures of this mechanism.
Anthony Beh; Paul V. McGraw; Denis Schluppeck
The effects of simulated hemianopia on eye movements during text reading Journal Article
In: Vision Research, vol. 204, pp. 1–14, 2023.
Vision loss is a common, devastating complication of cerebral strokes. In some cases the complete contra-lesional visual field is affected, leading to problems with routine tasks and, notably, the ability to read. Although visual information crucial for reading is imaged on the foveal region, readers often extract useful parafoveal information from the next word or two in the text. In hemianopic field loss, parafoveal processing is compromised, shrinking the visual span and resulting in slower reading speeds. Recent approaches to rehabilitation using perceptual training have been able to demonstrate some recovery of useful visual capacity. As gains in visual sensitivity were most pronounced at the border of the scotoma, it may be possible to use training to restore some of the lost visual span for reading. As restitutive approaches often involve prolonged training sessions, it would be beneficial to know how much recovery is required to restore reading ability. To address this issue, we employed a gaze-contingent paradigm using a low-pass filter to blur one side of the text, functionally simulating a visual field defect. The degree of blurring acts as a proxy for visual function recovery that could arise from restitutive strategies, and allows us to evaluate and quantify the degree of visual recovery required to support normal reading fluency in patients. Because reading ability changes with age, we recruited a group of younger participants, and another with older participants who are closer in age to risk groups for ischaemic strokes. Our results show that changes in patterns of eye movement observed in hemianopic loss can be captured using this simulated reading environment. This opens up the possibility of using participants with normal visual function to help identify the most promising strategies for ameliorating hemianopic loss, before translation to patient groups.
Stefanie I. Becker; James D. Retell; Jeremy M. Wolfe
Mirror blindness: Our failure to recognize the target in search for mirror-reversed shapes Journal Article
In: Attention, Perception, and Psychophysics, pp. 1–20, 2023.
It is well known that visual search for a mirror target (i.e., a horizontally flipped item) is more difficult than search for other-oriented items (e.g., vertically flipped items). Previous studies have typically attributed costs of mirror search to early, attention-guiding processes but could not rule out contributions from later processes. In the present study we used eye tracking to distinguish between early, attention-guiding processes and later target identification processes. The results of four experiments revealed a marked human weakness in identifying mirror targets: Observers appear to frequently fail to classify a mirror target as a target on first fixation and to continue with search even after having directly looked at the target. Awareness measures corroborated that the location of a mirror target could not be reported above chance level after it had been fixated once. This mirror blindness effect explained a large proportion (45–87%) of the overall costs of mirror search, suggesting that part of the difficulties with mirror search are rooted in later, object identification processes (not attentional guidance). Mirror blindness was significantly reduced but not completely eliminated when both the target and non-targets were held constant, which shows that perfect top-down knowledge can reduce mirror blindness, without completely eliminating it. The finding that non-target certainty reduced mirror blindness suggests that object identification is not solely achieved by comparing a selected item to a target template. These results demonstrate that templates that guide search toward targets are not identical to the templates used to conclusively identify those targets.
Monica Barbira; Mireille J. Babineaua; Anne-Caroline Fiévét; Anne Christophe
Rapid infant learning of syntactic–semantic links Journal Article
In: Proceedings of the National Academy of Sciences, vol. 120, no. 1, pp. 1–6, 2023.
In the second year of life, infants begin to rapidly acquire the lexicon of their native language. A key learning mechanism underlying this acceleration is syntactic bootstrapping: the use of hidden cues in grammar to facilitate vocabulary learning. How infants forge the syntactic–semantic links that underlie this mechanism, however, remains specula- tive. A hurdle for theories is identifying computationally light strategies that have high precision within the complexity of the linguistic signal. Here, we presented 20-mo-old infants with novel grammatical elements in a complex natural language environment and measured their resultant vocabulary expansion. We found that infants can learn and exploit a natural language syntactic–semantic link in less than 30 min. The rapid speed of acquisition of a new syntactic bootstrap indicates that even emergent syntactic–semantic links can accelerate language learning. The results suggest that infants employ a cognitive network of efficient learning strategies to self-supervise language development.
Zahra Azizi; Reza Ebrahimpour
Explaining integration of evidence separated by temporal gaps with frontoparietal circuit models Journal Article
In: Neuroscience, vol. 509, pp. 74–95, 2023.
Perceptual decisions rely on accumulating sensory evidence over time. However, the accumulation process is complicated in real life when evidence resulted from separated cues over time. Previous studies demonstrate that participants are able to integrate information from two separated cues to improve their performance invariant to an interval between the cues. However, there is no neural model that can account for accuracy and confidence in decisions when there is a time interval in evidence. We used behavioral and EEG datasets from a visual choice task —Random dot motion— with separated evidence to investigate three candid distributed neural networks. We showed that decisions based on evidence accumulation by separated cues over time are best explained by the interplay of recurrent cortical dynamics of centro-parietal and frontal brain areas while an uncertainty-monitoring module included in the model.
Reza Azadi; Simon Bohn; Emily Lopez; Rosa Lafer-Sousa; Karen Wang; Mark A. G. Eldridge; Arash Afraz
Image-dependence of the detectability of optogenetic stimulation in macaque inferotemporal cortex Journal Article
In: Current Biology, vol. 33, pp. 1–8, 2023.
Artificial activation of neurons in early visual areas induces perception of simple visual flashes. Accordingly, stimulation in high-level visual cortices is expected to induce perception of complex features. However, results from studies in human patients challenge this expectation. Stimulation rarely induces any detectable visual event, and never a complex one, in human subjects with closed eyes. Stimulation of the face-selective cortex in a human patient led to remarkable hallucinations only while the subject was looking at faces. In contrast, stimulations of color- and face-selective sites evoke notable hallucinations independent of the object being viewed. These anecdotal observations suggest that stimulation of high-level visual cortex can evoke perception of complex visual features, but these effects depend on the availability and content of visual input. In this study, we introduce a novel psychophysical task to systematically investigate characteristics of the perceptual events evoked by optogenetic stimulation of macaque inferior temporal (IT) cortex. We trained macaque monkeys to detect and report optogenetic impulses delivered to their IT cortices while holding fixation on object images. In a series of experiments, we show that detection of cortical stimulation is highly dependent on the choice of images presented to the eyes and it is most difficult when fixating on a blank screen. These findings suggest that optogenetic stimulation of high-level visual cortex results in easily detectable distortions of the concurrent contents of vision.
Amir Assouline; Avi Mendelsohn
Weaving a story: Narrative formation over prolonged time scales engages social cognition and frontoparietal networks Journal Article
In: European Journal of Neuroscience, pp. 1–15, 2023.
Forming narratives is of key importance to human experience, enabling one to render large amounts of information into relatively compacted stories for future retrieval, giving meaning to otherwise fragmented occurrences. The neural mechanisms that underlie coherent narrative construction of causally connected information over prolonged temporal periods are yet unclear. Participants in this fMRI study observed consecutive scenes from a full-length movie either in their original order, enabling causal inferences over time, or in reverse order, impeding a key component of coherent narratives—causal inference. In between scenes, we presented short periods of blank screens for examining post-encoding processing effects. Using multivariate pattern analysis (MVPA) followed by seed-base correlation analysis, we hypothesized that net- works involved in online monitoring of incoming information on the one hand, and offline processing of previous occurrences on the other would differ between the groups. We found that despite the exposure to the same scenes, the chronological-order condition exhibited enhanced functional connectivity in frontoparietal regions associated with information integration and working memory. The reverse-order condition yielded offline, post-scene coactivation of neural networks involved in social cognition and particularly theory of mind and action comprehension. These findings shed light on offline processes of narrative construction efforts, highlighting the role of social cognition networks in seeking for narrative coherence.
Susana Mouga; Isabel Catarina Duarte; Cátia Café; Daniela Sousa; Frederico Duque; Guiomar Oliveira; Miguel Castelo-Branco
Parahippocampal deactivation and hyperactivation of central executive, saliency and social cognition networks in autism spectrum disorder Journal Article
In: Journal of Neurodevelopmental Disorders, vol. 14, no. 9, pp. 1–12, 2022.
Background: The concomitant role of the Central Executive, the Saliency and the Social Cognition networks in autism spectrum disorder (ASD) in demanding ecological tasks remains unanswered. We addressed this question using a novel task-based fMRI virtual-reality task mimicking a challenging daily-life chore that may present some difficulties to individuals with ASD: the EcoSupermarketX. Methods: Participants included 29 adolescents: 15 with ASD and 15 with typical neurodevelopment (TD). They performed the EcoSupermarketX (a shopping simulation with three goal-oriented sub-tasks including “no cue”, “non-social” or “social” cues), during neuroimaging and eye-tracking. Results: ASD differed from TD only in total time and distance to complete the “social cue” sub-task with matched eye-tracking measures. Neuroimaging revealed simultaneous hyperactivation across social, executive, and saliency circuits in ASD. In contrast, ASD showed reduced activation in the parahippocampal gyrus, involved in scene recognition. Conclusions: When performing a virtual shopping task matching the performance of controls, ASD adolescents hyperactivate three core networks: executive, saliency and social cognition. Parahippocampal hypoactivation is consistent with effortless eidetic scene processing, in line with the notion of peaks and valleys of neural recruitment in individuals with ASD. These hyperactivation/hypoactivation patterns in daily life tasks provide a circuit-level signature of neural diversity in ASD, a possible intervention target.
Lara Merken; Maarten Schelles; Frederik Ceyssens; Michael Kraft; Peter Janssen
Thin flexible arrays for long-term multi-electrode recordings in macaque primary visual cortex Journal Article
In: Journal of Neural Engineering, pp. 1–14, 2022.
Objective. Basic, translational and clinical neuroscience are increasingly focusing on large-scale invasive recordings of neuronal activity. However, in large animals such as nonhuman primates and humans – in which the larger brain size with sulci and gyri imposes additional challenges compared to rodents, there is a huge unmet need to record from hundreds of neurons simultaneously anywhere in the brain for long periods of time. Here, we tested the electrical and mechanical properties of thin, flexible multi-electrode arrays inserted into the primary visual cortex of two macaque monkeys, and assessed their Magnetic Resonance Imaging (MRI) compatibility and their capacity to record extracellular activity over a period of 1 year. Approach. To allow insertion of the floating arrays into the visual cortex, the 20 by 100 µm2 shafts were temporarily strengthened by means of a resorbable poly(lactic-co-glycolic acid) (PLGA) coating. Main results. After manual insertion of the arrays, the ex vivo and in vivo MRI compatibility of the arrays proved to be excellent. We recorded clear single-unit activity (SUA) from up to 50% of the electrodes, and multi-unit activity (MUA) on 60-100% of the electrodes, which allowed detailed measurements of the receptive fields and the orientation selectivity of the neurons. Even 1 year after insertion, we obtained significant MUA responses on 70-100% of the electrodes, while the receptive fields remained remarkably stable over the entire recording period. Significance. Thus, the thin and flexible multielectrode arrays we tested offer several crucial advantages compared to existing arrays, most notably in terms of brain tissue compliance, scalability, and brain coverage. Future brain-machine interface applications in humans may strongly benefit from this new generation of chronically implanted multi-electrode arrays.
Lisa Kunkel genannt Bode; Anna Sophie Schulte; Björn Hauptmann; Thomas F. Münte; Andreas Sprenger; Björn Machner
Gaze-contingent display technology can help to reduce the ipsilesional attention bias in hemispatial neglect following stroke Journal Article
In: Journal of NeuroEngineering and Rehabilitation, vol. 19, pp. 1–15, 2022.
Background: Hemispatial neglect results from unilateral brain damage and represents a disabling unawareness for objects in the hemispace opposite the brain lesion (contralesional). The patients' attentional bias for ipsilesional hemispace represents a hallmark of neglect, which results from an imbalanced attentional priority map in the brain. The aim of this study was to investigate whether gaze-contingent display (GCD) technology, reducing the visual salience of objects in ipsilesional hemispace, is able to rebalance this map and increase awareness and exploration of objects in the neglected contralesional hemispace. Methods: Using remote eye-tracking, we recorded gaze positions in 19 patients with left hemispatial neglect following right-hemisphere stroke and 22 healthy control subjects, while they were watching static naturalistic scenes. There were two task conditions, free viewing (FV) or goal-directed visual search (VS), and four modification conditions including the unmodified original picture, a purely static modification and two differently strong modifications with an additional gaze-contingent mask (GC-LOW, GC-HIGH), that continuously reduced color saturation and contrast of objects in the right hemispace. Results: The patients' median gaze position (Center of Fixation) in the original pictures was markedly deviated to the right in both tasks (FV: 6.8° ± 0.8; VS: 5.5° ± 0.7), reflecting the neglect-typical ipsilesional attention bias. GC modification significantly reduced this bias in FV (GC-HIGH: d = − 3.2 ± 0.4°; p < 0.001). Furthermore, in FV and VS, GC modification increased the likelihood to start visual exploration in the (neglected) left hemifield by about 20%. This alleviation of the ipsilesional fixation bias was not associated with an improvement in detecting left-side targets, in contrast, the GC mask even decreased and slowed the detection of right-side targets. Subjectively, patients found the intervention pleasant and most of the patients did not notice any modification. Conclusions: GCD technology can be used to positively influence visual exploration patterns in patients with hemispatial neglect. Despite an alleviation of the neglect-related ipsilesional fixation bias, a concomitant functional benefit (improved detection of contralesional targets) was not achieved. Future studies may investigate individualized GCD-based modifications as augmented reality applications during the activities of daily living.
Haiyan Wang; Matthew Walenski; Kaitlyn Litcofsky; Jennifer E. Mack; M. Marsel Mesulam; Cynthia K. Thompson
Verb production and comprehension in primary progressive aphasia Journal Article
In: Journal of Neurolinguistics, vol. 64, pp. 1–18, 2022.
Studies of word class processing have found verb retrieval impairments in individuals with primary progressive aphasia (Bak et al., 2001; Cappa et al., 1998; Cotelli et al., 2006; Hillis, Heidler-Gary, et al., 2006; Hillis, Oh, & Ken, 2004; Marcotte et al., 2014; Rhee, Antiquena, & Grossman, 2001; Silveri & Ciccarelli, 2007; Thompson, Lukic, et al., 2012) associated primarily with the agrammatic variant. However, fewer studies have focused on verb comprehension, with inconsistent results. Because verbs are critical to both production and comprehension of clauses and sentences, we investigated verb processing across domains in agrammatic, logopenic, and semantic PPA and a group of age-matched healthy controls. Participants completed a confrontation naming task for verb production and an eye-tracking word-picture matching task for online verb comprehension. All PPA groups showed impaired verb production and comprehension relative to healthy controls. Most notably, the PPA-S group performed more poorly than the other two PPA variants in both domains. Overall, the results indicate that semantic deficits in the PPA-S extend beyond object knowledge to verbs as well, adding to our knowledge concerning the nature of the language deficits in the three variants of primary progressive aphasia.
Constanze Schmitt; Milosz Krala; Frank Bremmer
Neural signatures of actively controlled self-motion and the subjective encoding of distance Journal Article
In: eNeuro, vol. 9, no. 6, pp. 1–18, 2022.
Navigating through an environment requires knowledge about one's direction of self-motion (heading) and traveled distance. Behavioral studies showed that human participants can actively reproduce a previously observed travel distance purely based on visual information. Here, we employed electroencephalography (EEG) to investigate the underlying neural processes. We measured, in human observers, event-related potentials (ERPs) during visually simulated straight-forward self-motion across a ground plane. The participants' task was to reproduce (active condition) double the distance of a previously seen self-displacement (passive condition) using a gamepad. We recorded the trajectories of self-motion during the active condition and played it back to the participants in a third set of trials (replay condition). We analyzed EEG activity separately for four electrode clusters: frontal (F), central (C), parietal (P), and occipital (O). When aligned to self-motion onset or offset, response modulation of the ERPs was stronger, and several ERP components had different latencies in the passive as compared with the active condition. This result is in line with the concept of predictive coding, which implies modified neural activation for self-induced versus externally induced sensory stimulation. We aligned our data also to the times when subjects passed the (objective) single distance d_obj and the (subjective) single distance d_sub. Remarkably, wavelet-based temporal-frequency analyses revealed enhanced theta-band activation for F, P, and O-clusters shortly before passing d_sub. This enhanced activation could be indicative of a navigation related representation of subjective distance. More generally, our study design allows to investigate subjective perception without interfering neural activation because of the required response action.
Xuling Li; Man Zeng; Lei Gao; Shan Li; Zibei Niu; Danhui Wang; Tianzhi Li; Xuejun Bai; Xiaolei Gao
The mechanism of word satiation in Tibetan reading: Evidence from eye movements Journal Article
In: Journal of Eye Movement Research, vol. 15, no. 5, 2022.
Two eye-tracking experiments were used to investigate the mechanism of word satiation in Tibetan reading. The results revealed that, at a low repetition level, gaze duration and total fixation duration in the semantically unrelated condition were significantly longer than in the semantically related condition; at a medium repetition level, reaction time in the semantically related condition was significantly longer than in the semantically unrelated condition; at a high repetition level, the total fixation duration and reaction time in the semantically related condition were significantly longer than in the semantically unrelated condition. However, fixation duration and reaction time showed no significant difference between the similar and dissimilar orthography at any repetition level. These findings imply that there are semantic priming effects in Tibetan reading at a low repetition level, but semantic satiation effects at greater repetition levels, which occur in the late stage of lexical processing.
Yang Zhou; Krithika Mohan; David J. Freedman
Abstract encoding of categorical decisions in medial superior temporal and lateral intraparietal cortices Journal Article
In: Journal of Neuroscience, vol. 42, no. 48, pp. 9069–9081, 2022.
Categorization is an essential cognitive and perceptual process for decision making and recognition. The posterior parietal cortex (PPC), particularly the lateral intraparietal (LIP) area has been suggested to transform visual feature encoding into abstract categorical representations. By contrast, areas closer to sensory input, such as the middle temporal (MT) area, encode stimulus features but not more abstract categorical information during categorization tasks. Here, we compare the contributions of the medial superior temporal (MST) and LIP areas in category computation by recording neuronal activity in both areas from two male rhesus macaques trained to perform a visual motion categorization task. MST is a core motion processing area interconnected with MT, and often considered an intermediate processing stage between MT and LIP. Here we show that MST exhibits robust decision-correlated motion category encoding and working memory encoding similar to LIP, suggesting that MST plays a substantial role in cognitive computation, extending beyond its widely recognized role in visual motion processing. SIGNIFICANCE STATEMENT: Categorization requires assigning incoming sensory stimuli into behaviorally relevant groups. Previous work found that parietal area LIP shows a strong encoding of the learned category membership of visual motion stimuli, while visual area MT shows strong direction tuning but not category tuning during a motion direction categorization task. Here we show that area MST, a visual motion processing area interconnected with both LIP and MT, shows strong visual category encoding similar to that observed in LIP. This suggests that MST plays a greater role in abstract cognitive functions, extending beyond it well known role in visual motion processing.
Poppy Sharp; Tjerk Gutteling; David Melcher; Clayton Hickey
Spatial attention tunes temporal processing in early visual cortex by speeding and slowing alpha oscillations Journal Article
In: Journal of Neuroscience, vol. 42, no. 41, pp. 7824–7832, 2022.
The perception of dynamic visual stimuli relies on two apparently conflicting perceptual mechanisms: rapid visual input must sometimes be integrated into unitary percepts but at other times must be segregated or parsed into separate objects or events. Though they have opposite effects on our perceptual experience, the deployment of spatial attention benefits both operations. Little is known about the neural mechanisms underlying this impact of spatial attention on temporal perception. Here, we record magnetoencephalography (MEG) in male and female humans to demonstrate that the deployment of spatial attention for the purpose of segregating or integrating visual stimuli impacts prestimulus oscillatory activity in retinotopic visual brain areas where the attended location is represented. Alpha band oscillations contralateral to an attended location are therefore faster than ipsilateral oscillations when stimuli appearing at this location will need to be segregated, but slower in expectation of the need for integration, consistent with the idea that a frequency is linked to perceptual sampling rate. These results demonstrate a novel interaction between temporal visual processing and the allocation of attention in space.
Charlotte Moore; Elika Bergelson
Examining the roles of regularity and lexical class in 18–26-month-olds' representations of how words sound Journal Article
In: Journal of Memory and Language, vol. 126, pp. 1–17, 2022.
By around 12 months, infants have well-specified phonetic representations for the nouns they understand, for instance looking less at a car upon hearing ‘cur' than ‘car' (Swingley and Aslin, 2002). Here we test whether such high-fidelity representations extend to irregular nouns, and regular and irregular verbs. A corpus analysis confirms the intuition that irregular verbs are far more common than irregular nouns in speech to young children. Two eyetracking experiments then test whether toddlers are sensitive to mispronunciation in regular and irregular nouns (Experiment 1) and verbs (Experiment 2). For nouns, we find a mispronunciation effect and no regularity effect in 18-month-olds. For verbs, in Experiment 2a, we find only a regularity effect and no mispronunciation effect in 18-month-olds, though toddlers' poor comprehension overall limits interpretation. Finally, in Experiment 2b we find a mispronunciation effect and no regularity effect in 26-month-olds. The interlocking roles of lexical class and regularity for wordform representations and early word learning are discussed.
Brice Olivier; Anne Guérin-Dugué; Jean-Baptiste Durand
Hidden semi-Markov models to segment reading phases from eye movements Journal Article
In: Journal of Eye Movement Research, vol. 15, no. 4, pp. 1–19, 2022.
Our objective is to analyze scanpaths acquired through participants achieving a reading task aiming at answering a binary question: Is the text related or not to some given target topic? We propose a data-driven method based on hidden semi-Markov chains to segment scanpaths into phases deduced from the model states, which are shown to represent different cognitive strategies: normal reading, fast reading, information search, and slow confirmation. These phases were confirmed using different external covariates, among which semantic information extracted from texts. Analyses highlighted some strong preference of specific participants for specific strategies and more globally, large individual variability in eye-movement characteristics, as accounted for by random effects. As a perspective, the possibility of improving reading models by accounting for possible heterogeneity sources during reading is discussed.
Mahboubeh Habibi; Wolfgang H. Oertel; Brian J. White; Donald C. Brien; Brian C. Coe; Heidi C. Riek; Julia Perkins; Rachel Yep; Laurent Itti; Lars Timmermann; Christoph Best; Elisabeth Sittig; Annette Janzen; Douglas P. Munoz
Eye tracking identifies biomarkers in $alpha$-synucleinopathies versus progressive supranuclear palsy Journal Article
In: Journal of Neurology, vol. 269, pp. 4920–4938, 2022.
Objectives: This study (1) describes and compares saccade and pupil abnormalities in patients with manifest alpha-synucleinopathies ($alpha$SYN: Parkinson's disease (PD), Multiple System Atrophy (MSA)) and a tauopathy (progressive supranuclear palsy (PSP)); (2) determines whether patients with rapid-eye-movement sleep behaviour disorder (RBD), a prodromal stage of $alpha$SYN, already have abnormal responses that may indicate a risk for developing PD or MSA. Methods: Ninety (46 RBD, 27 PD, 17 MSA) patients with an $alpha$SYN, 10 PSP patients, and 132 healthy age-matched controls (CTRL) were examined with a 10-min video-based eye-tracking task (Free Viewing). Participants were free to look anywhere on the screen while saccade and pupil behaviours were measured. Results: PD, MSA, and PSP spent more time fixating the centre of the screen than CTRL. All patient groups made fewer macro-saccades (> 2◦ amplitude) with smaller amplitude than CTRL. Saccade frequency was greater in RBD than in other patients. Following clip change, saccades were temporarily suppressed, then rebounded at a slower pace than CTRL in all patient groups. RBD had distinct, although discrete saccade abnormalities that were more marked in PD, MSA, and even more in PSP. The vertical saccade rate was reduced in all patients and decreased most in PSP. Clip changes produced large increases or decreases in screen luminance requiring pupil constriction or dilation, respectively. PSP elicited smaller pupil constriction/dilation responses than CTRL, while MSA elicited the opposite. Conclusion: RBD patients already have discrete but less pronounced saccade abnormalities than PD and MSA patients. Vertical gaze palsy and altered pupil control differentiate PSP from $alpha$SYN.
Jordana S. Wynn; Zhong-Xu Liu; Jennifer D. Ryan
Neural correlates of subsequent memory-related gaze reinstatement Journal Article
In: Journal of Cognitive Neuroscience, vol. 34, no. 9, pp. 1547–1562, 2022.
Mounting evidence linking gaze reinstatement—the recapitulation of encoding-related gaze patterns during retrieval—to behavioral measures of memory suggests that eye movements play an important role in mnemonic processing. Yet, the nature of the gaze scanpath, including its informational content and neural correlates, has remained in question. In this study, we examined eye movement and neural data from a recognition memory task to further elucidate the behavioral and neural bases of functional gaze reinstatement. Consistent with previous work, gaze reinstatement during retrieval of freely viewed scene images was greater than chance and predictive of recognition memory performance. Gaze reinstatement was also associated with viewing of informationally salient image regions at encoding, suggesting that scanpaths may encode and contain high-level scene content. At the brain level, gaze reinstatement was predicted by encoding-related activity in the occipital pole and BG, neural regions associated with visual processing and oculomotor control. Finally, cross-voxel brain pattern similarity analysis revealed overlapping subsequent memory and subsequent gaze reinstatement modulation effects in the parahippocampal place area and hippocampus, in addition to the occipital pole and BG. Together, these findings suggest that encoding-related activity in brain regions associated with scene processing, oculomotor control, and memory supports the formation, and subsequent recapitulation, of functional scanpaths. More broadly, these findings lend support to Scanpath Theory's assertion that eye movements both encode, and are themselves embedded in, mnemonic representations.
Michael G. Cutter; Ruth Filik; Kevin B. Paterson
Do readers maintain word-level uncertainty during reading? A pre-registered replication study Journal Article
In: Journal of Memory and Language, vol. 125, pp. 1–14, 2022.
We present a replication of Levy, Bicknell, Slattery, and Rayner (2009). In this prior study participants read sentences in which a perceptually confusable preposition (at; confusable with as) or non-confusable preposition (toward) was followed by a verb more likely to appear in the syntactic structure formed by replacing at with as (e.g. tossed) or a verb that was not more likely to appear in this structure (e.g. thrown). Readers experienced processing difficulty upon fixating verbs like tossed following at, but not toward. Levy et al. argued that this suggests readers maintained uncertainty about previously fixated words' identities. We argue that this finding has wide-ranging implications for language processing theories, and that a replication is required. On the basis of a Bayes Factor Design Analysis we conducted a replication study with 56 items and 72 participants in order to determine whether Levy et al.'s effects are replicable. Using Bayesian statistical techniques we show that in our dataset there is evidence against the existence of the interaction Levy et al. found, and thus conclude that this study is non-replicable.
M. Antúnez; P. J. López-Pérez; J. Dampuré; H. A. Barber
Frequency-based foveal load modulates semantic parafoveal-on-foveal effects Journal Article
In: Journal of Neurolinguistics, vol. 63, 2022.
During reading, we can process words allocated to the parafoveal visual region. Our ability to extract parafoveal information is determined by the availability of attentional resources, and by how these are distributed among words in the visual field. According to the foveal load hypothesis, a greater difficulty in processing the foveal word would result in less attentional resources being allocated to the parafoveal word, thereby hindering its processing. However, contradictory results have raised questions about which foveal load manipulations may affect the processing of parafoveal words at different levels. We explored whether the semantic processing of parafoveal words can be modulated by variations in a frequency-based foveal load. When participants read word triads, modulations in the N400 component indicated that, while parafoveal words were semantically processed when foveal load was low, their meaning could not be accessed if the foveal word was more difficult to process. Therefore, a frequency-based foveal load modulates semantic parafoveal processing and a semantic preview manipulation may be a suitable baseline to test the foveal load hypothesis.
Christoph Helmchen; Philipp J. Koch; Gabriel Girard; Norbert Brüggemann; Björn Machner; Andreas Sprenger
NPTX1-related oculomotor apraxia: An intra-hemispheric disconnection disorder Miscellaneous
Oculomotor apraxia (OMA) is a rare and heavily disabling neurological disorder causing severe difficulties in the initia- tion and maintenance of voluntary eye movements when the head is stationary. If patients try to initiate saccades, they are grossly delayed and hypometric (stair-case). .. The aim of this study was to test competing pathophysiological hypotheses by functional and structural MRI, stating that OMA is related to either abnormal (i) inter-hemispheric or (ii) intra-hemispheric connectivity between the FEF and related oculomotor structures (oculomotor network) or (iii) both mechanisms.
Elio Balestrieri; Niko A. Busch
Spontaneous alpha-band oscillations bias subjective contrast perception Journal Article
In: Journal of Neuroscience, pp. 1–31, 2022.
Perceptual decisions depend both on the features of the incoming stimulus and on the ongoing brain activity at the moment the stimulus is received. Specifically, trial-to-trial fluctuations in cortical excitability have been linked to fluctuations in the amplitude of prestimulus $alpha$ oscillations ($sim$8-13 Hz), which are in turn are associated with fluctuations in subjects' tendency to report the detection of a stimulus. It is currently unknown whether $alpha$ oscillations bias postperceptual decision-making, or even bias subjective perception itself. To answer this question, we used a contrast discrimination task in which both male and female human subjects reported which of two gratings (one in each hemifield) was perceived as having a stronger contrast. Our EEG analysis showed that subjective contrast was reduced for the stimulus in the hemifield represented in the hemisphere with relatively stronger prestimulus $alpha$ amplitude, reflecting reduced cortical excitability. Furthermore, the strength of this spontaneous hemispheric lateralization was strongly correlated with the magnitude of individual subjects' biases, suggesting that the spontaneous patterns of $alpha$ lateralization play a role in explaining the intersubject variability in contrast perception. These results indicate that spontaneous fluctuations in cortical excitability, indicated by patterns of prestimulus $alpha$ amplitude, affect perceptual decisions by altering the phenomenological perception of the visual world.
Angela Radetz; Markus Siegel
Spectral fingerprints of cortical neuromodulation Journal Article
In: Journal of Neuroscience, vol. 42, no. 18, pp. 3836–3846, 2022.
Pupil size has been established as a versatile marker of noradrenergic and cholinergic neuromodulation, which has profound effects on neuronal processing, cognition, and behavior. However, little is known about the cortical control and effects of pupil-linked neuromodulation. Here, we show that pupil dynamics are tightly coupled to temporally, spectrally, and spatially specific modulations of local and large-scale cortical population activity in the human brain. We quantified the dynamics of band-limited cortical population activity in resting human subjects using magnetoencephalography and investigated how neural dynamics were linked to simultaneously recorded pupil dynamics. Our results show that pupil-linked neuromodulation does not merely affect cortical population activity in a stereotypical fashion. Instead, we identified three frontal, precentral, and occipitoparietal networks, in which local population activity with distinct spectral profiles in the theta, beta, and alpha bands temporally preceded and followed changes in pupil size. Furthermore, we found that amplitude coupling at;16 Hz in a large-scale frontoparietal network predicted pupil dynamics. Our results unravel network-specific spectral fingerprints of cortical neuromodulation in the human brain that likely reflect both the causes and effects of neuromodulation.
Kumari Liza; Supratim Ray
Local interactions between steady-state visually evoked potentials at nearby flickering frequencies Journal Article
In: Journal of Neuroscience, vol. 42, no. 19, pp. 3965–3974, 2022.
Steady-state visually evoked potentials (SSVEPs) are widely used to index top-down cognitive processing in human electroencephalogram (EEG) studies. Typically, two stimuli flickering at different temporal frequencies (TFs) are presented, each producing a distinct response in the EEG at its flicker frequency. However, how SSVEP responses in EEGs are modulated in the presence of a competing flickering stimulus just because of sensory interactions is not well understood. We have previously shown in local field potentials (LFPs) recorded from awake monkeys that when two overlapping full-screen gratings are counterphased at different TFs, there is an asymmetric SSVEP response suppression, with greater suppression from lower TFs, which further depends on the relative orientations of the gratings (stronger suppression and asymmetry for parallel compared with orthogonal gratings). Here, we first confirmed these effects in both male and female human EEG recordings. Then, we mapped the response suppression of one stimulus (target) by a competing stimulus (mask) over a much wider range than the previous study. Surprisingly, we found that the suppression was not stronger at low frequencies in general, but systematically varied depending on the target TF, indicating local interactions between the two competing stimuli. These results were confirmed in both human EEG and monkey LFP and electrocorticogram (ECoG) data. Our results show that sensory interactions between multiple SSVEPs are more complex than shown previously and are influenced by both local and global factors, underscoring the need to cautiously interpret the results of studies involving SSVEP paradigms.SIGNIFICANCE STATEMENT Steady-state visually evoked potentials (SSVEPs) are extensively used in human cognitive studies and brain-computer interfacing applications where multiple stimuli flickering at distinct frequencies are concurrently presented in the visual field. We recently characterized interactions between competing flickering stimuli in animal recordings and found that stimuli flickering slowly produce larger suppression. Here, we confirmed these in human EEGs, and further characterized the interactions by using a much wider range of target and competing (mask) frequencies in both human EEGs and invasive animal recordings. These revealed a new "local" component, whereby the suppression increased when competing stimuli flickered at nearby frequencies. Our results highlight the complexity of sensory interactions among multiple SSVEPs and underscore the need to cautiously interpret studies involving SSVEP paradigms.
Arno Libert; Arne Van Den Kerchove; Benjamin Wittevrongel; Marc M. Van Hulle
Analytic beamformer transformation for transfer learning in motion-onset visual evoked potential decoding Journal Article
In: Journal of Neural Engineering, vol. 19, pp. 1–16, 2022.
Objective. While decoders of electroencephalography-based event-related potentials (ERPs) are routinely tailored to the individual user to maximize performance, developing them on populations for individual usage has proven much more challenging. We propose the analytic beamformer transformation (ABT) to extract phase and/or magnitude information from spatiotemporal ERPs in response to motion-onset stimulation. Approach. We have tested ABT on 52 motion-onset visual evoked potential (mVEP) datasets from 26 healthy subjects and compared the classification accuracy of support vector machine (SVM), spatiotemporal beamformer (stBF) and stepwise linear discriminant analysis (SWLDA) when trained on individual subjects and on a population thereof. Main results. When using phase- and combined phase/magnitude information extracted by ABT, we show significant improvements in accuracy of population-trained classifiers applied to individual users (p < 0.001). We also show that 450 epochs are needed for a correct functioning of ABT, which corresponds to 2 min of paradigm stimulation. Significance. We have shown that ABT can be used to create population-trained mVEP classifiers using a limited number of epochs. We expect this to pertain to other ERPs or synchronous stimulation paradigms, allowing for a more effective, population-based training of visual BCIs. Finally, as ABT renders recordings across subjects more structurally invariant, it could be used for transfer learning purposes in view of plug-and-play BCI applications.
Nicola Grossheinrich; Julia Schaeffer; Christine Firk; Thomas Eggermann; Lynn Huestegge; Kerstin Konrad
Childhood adversity and approach/avoidance-related behaviour in boys Journal Article
In: Journal of Neural Transmission, vol. 129, pp. 421–429, 2022.
Childhood adversity has been suggested to affect the vulnerability for developmental psychopathology, including both externalizing and internalizing symptoms. This study examines spontaneous attention biases for negative and positive emotional facial expressions as potential intermediate phenotypes. In detail, typically developing boys (6–13 years) underwent an eye-tracking paradigm displaying happy, angry, sad and fearful faces. An approach bias towards positive emotional facial expressions with increasing childhood adversity levels was found. In addition, an attention bias away from negative facial expressions was observed with increasing childhood adversity levels, especially for sad facial expressions. The results might be interpreted in terms of emotional regulation strategies in boys at risk for reactive aggression and depressive behaviour.
Noam Tal-Perry; Shlomit Yuval-Greenberg
The spatiotemporal link of temporal expectations: Contextual temporal expectation is independent of spatial attention Journal Article
In: Journal of Neuroscience, vol. 42, no. 12, pp. 2516–2523, 2022.
Temporal expectation is the ability to construct predictions regarding the timing of events, based on previously experienced temporal regularities of different types. For example, cue-based expectations are constructed when a cue validly indicates when a target is expected to occur. However, in the absence of such cues, expectations can be constructed based on contextual temporal information, including the onset distribution of the event and recent prior experiences, both providing implicit probabilistic information regarding the timing of the event. It was previously suggested that cue-based temporal expectation is exerted via synchronization of spatially specific neural activity at a predictable time of a target, within receptive fields corresponding to the expected location of the target. Here, we tested whether the same theoretical model holds for contextual temporal effects. Participants (n = 40, 25 females) performed a speeded spatial-cuing detection task with two-thirds valid spatial cues. The hazard-rate function of the target was modulated by varying the foreperiod—the interval between the spatial cue and the target—among trials and was manipulated between groups by changing the interval distribution. Reaction times were analyzed using both frequentist and Bayesian generalized linear mixed models, accounting for hazard and sequential effects. Results showed that the effects of contextual temporal structures on reaction times were independent of spatial attention. This suggests that the spatiotemporal mechanisms, thought to account for cue-based expectation, cannot explain other sources of temporal expectations. We conclude that expectations based on contextual structures have different characteristics than cue-based temporal expectation, suggesting reliance on distinct neural mechanisms.
Antonio Fernández; Sara Okun; Marisa Carrasco
Differential effects of endogenous and exogenous attention on sensory tuning Journal Article
In: Journal of Neuroscience, vol. 42, no. 7, pp. 1316–1327, 2022.
Covert spatial attention (without concurrent eye movements) improves performance in many visual tasks (e.g., orientation discrimination and visual search). However, both covert attention systems—endogenous (voluntary) and exogenous (involuntary)—exhibit differential effects on performance in tasks mediated by spatial and temporal resolution suggesting an underlying mechanistic difference. We investigated whether these differences manifest in sensory tuning by assessing whether and how endogenous and exogenous attention differentially alter the representation of two basic visual dimensions—orientation and spatial frequency (SF). The same human observers detected a grating embedded in noise in two separate experiments (with endogenous or exogenous attention cues). Reverse correlation was used to infer the underlying neural representation from behavioral responses, and we linked our results to established neural computations via a normalization model of attention. Both endogenous and exogenous attention similarly improved performance at the attended location by enhancing the gain of all orientations without changing tuning width. In the SF dimension, endogenous attention enhanced the gain of SFs above and below the target SF, whereas exogenous attention only enhanced those above. Additionally, exogenous attention shifted peak sensitivity to SFs above the target SF, whereas endogenous attention did not. Both covert attention systems modulated sensory tuning via the same computation (gain changes). However, there were differences in the strength of the gain. Compared with endogenous attention, exogenous attention had a stronger orientation gain enhancement but a weaker overall SF gain enhancement. These differences in sensory tuning may underlie differential effects of endogenous and exogenous attention on performance.
Ruth E. Corps; Charlotte Brooke; Martin J. Pickering
Prediction involves two stages: Evidence from visual-world eye-tracking Journal Article
In: Journal of Memory and Language, vol. 122, pp. 1–20, 2022.
Comprehenders often predict what they are going to hear. But do they make the best predictions possible? We addressed this question in three visual-world eye-tracking experiments by asking when comprehenders consider perspective. Male and female participants listened to male and female speakers producing sentences (e.g., I would like to wear the nicełdots) about stereotypically masculine (target: tie; distractor: drill) and feminine (target: dress, distractor: hairdryer) objects. In all three experiments, participants rapidly predicted semantic associates of the verb. But participants also predicted consistently-that is, consistent with their beliefs about what the speaker would ultimately say. They predicted consistently from the speaker's perspective in Experiment 1, their own perspective in Experiment 2, and the character's perspective in Experiment 3. This consistent effect occurred later than the associative effect. We conclude that comprehenders consider perspective when predicting, but not from the earliest moments of prediction, consistent with a two-stage account.
Xibo Zuo; Ying Ling; Todd Jackson
Testing links between pain-related biases in visual attention and recognition memory: An eye-tracking study based on an impending pain paradigm Journal Article
In: Quarterly Journal of Experimental Psychology, pp. 1–15, 2022.
Although separate lines of research have evaluated pain-related biases in attention or memory, laboratory studies examining links between attention and memory for pain-related information have received little consideration. In this eye-tracking experiment, we assessed relations between pain-related attention biases (ABs) and recognition memory biases (MBs) among 122 pain-free adults randomly assigned to impending pain (n = 59) versus impending touch (n = 63) conditions, wherein offsets of trials that included pain images were followed by subsequent possibly painful and non-painful somatosensory stimulation, respectively. Gaze biases of participants were assessed during presentations of pain-neutral (P-N) and happy-neutral (H-N) face image pairs within these conditions. Subsequently, condition differences in recognition accuracy for previously viewed versus novel pained and happy face images were examined. Overall gaze durations were significantly longer for pain (vs. neutral) faces that signalled impending pain than impending non-painful touch, particularly among the less resilient in the former condition. Impending pain cohorts also exhibited comparatively better recognition accuracy for both pained and happy face images. Finally, longer gaze durations on pain faces that signalled potential pain, but not potential touch, were related to more accurate recognition of previously viewed pain faces. In sum, pain cues that signal potential personal discomfort maintain visual attention more fully and are subsequently recognised more accuracy than are pain cues that signal non-painful touch stimulation.
Ting Zou; Yutong Liu; Huiting Zhong
The roles of consonant, rime, and tone in Mandarin spoken word recognition: An eye-tracking study Journal Article
In: Frontiers in Psychology, vol. 12, pp. 1–19, 2022.
This study investigated the relative role of sub-syllabic components (initial consonant, rime, and tone) in spoken word recognition of Mandarin Chinese using an eye-tracking experiment with a visual world paradigm. Native Mandarin speakers (all born and grew up in Beijing) were presented with four pictures and an auditory stimulus. They were required to click the picture according to the sound stimulus they heard, and their eye movements were tracked during this process. For a target word (e.g., tang2 “candy”), nine conditions of competitors were constructed in terms of the amount of their phonological overlap with the target: consonant competitor (e.g., ti1 “ladder”), rime competitor (e.g., lang4 “wave”), tone competitor (e.g., niu2 “cow”), consonant plus rime competitor (e.g., tang1”soup”), consonant plus tone competitor (e.g., tou2 “head”), rime plus tone competitor (e.g., yang2 “sheep”), cohort competitor (e.g., ta3 “tower”), cohort plus tone competitor (e.g., tao2 “peach”), and baseline competitor (e.g., xue3 “snow”). A growth curve analysis was conducted with the fixation to competitors, targets, and distractors, and the results showed that (1) competitors with consonant or rime overlap can be adequately activated, while tone overlap plays a weaker role since additional tonal information can strengthen the competitive effect only when it was added to a candidate that already bears much phonological similarity with the target. (2) Mandarin words are processed in an incremental way in the time course of word recognition since different partially overlapping competitors could be activated immediately; (3) like the pattern found in English, both cohort and rime competitors were activated to compete for lexical activation, but these two competitors were not temporally distinctive and mainly differed in the size of their competitive effects. Generally, the gradation of activation based on the phonological similarity between target and candidates found in this study was in line with the continuous mapping models and may reflect a strategy of native speakers shaped by the informative characteristics of the interaction among different sub-syllabic components.