EyeLink Reading and Language Eye-Tracking Publications
All EyeLink reading and language research publications up until 2023 (with some early 2024s) are listed below by year. You can search the publications using keywords such as Visual World, Comprehension, Speech Production, etc. You can also search for individual author names. If we missed any EyeLink reading or language articles, please email us!
2022 |
Charlotte Moore; Elika Bergelson Examining the roles of regularity and lexical class in 18–26-month-olds' representations of how words sound Journal Article In: Journal of Memory and Language, vol. 126, pp. 1–17, 2022. @article{Moore2022a, By around 12 months, infants have well-specified phonetic representations for the nouns they understand, for instance looking less at a car upon hearing ‘cur' than ‘car' (Swingley and Aslin, 2002). Here we test whether such high-fidelity representations extend to irregular nouns, and regular and irregular verbs. A corpus analysis confirms the intuition that irregular verbs are far more common than irregular nouns in speech to young children. Two eyetracking experiments then test whether toddlers are sensitive to mispronunciation in regular and irregular nouns (Experiment 1) and verbs (Experiment 2). For nouns, we find a mispronunciation effect and no regularity effect in 18-month-olds. For verbs, in Experiment 2a, we find only a regularity effect and no mispronunciation effect in 18-month-olds, though toddlers' poor comprehension overall limits interpretation. Finally, in Experiment 2b we find a mispronunciation effect and no regularity effect in 26-month-olds. The interlocking roles of lexical class and regularity for wordform representations and early word learning are discussed. |
Brice Olivier; Anne Guérin-Dugué; Jean-Baptiste Durand Hidden semi-Markov models to segment reading phases from eye movements Journal Article In: Journal of Eye Movement Research, vol. 15, no. 4, pp. 1–19, 2022. @article{Olivier2022, Our objective is to analyze scanpaths acquired through participants achieving a reading task aiming at answering a binary question: Is the text related or not to some given target topic? We propose a data-driven method based on hidden semi-Markov chains to segment scanpaths into phases deduced from the model states, which are shown to represent different cognitive strategies: normal reading, fast reading, information search, and slow confirmation. These phases were confirmed using different external covariates, among which semantic information extracted from texts. Analyses highlighted some strong preference of specific participants for specific strategies and more globally, large individual variability in eye-movement characteristics, as accounted for by random effects. As a perspective, the possibility of improving reading models by accounting for possible heterogeneity sources during reading is discussed. |
M. Antúnez; P. J. López-Pérez; J. Dampuré; H. A. Barber Frequency-based foveal load modulates semantic parafoveal-on-foveal effects Journal Article In: Journal of Neurolinguistics, vol. 63, 2022. @article{Antunez2022, During reading, we can process words allocated to the parafoveal visual region. Our ability to extract parafoveal information is determined by the availability of attentional resources, and by how these are distributed among words in the visual field. According to the foveal load hypothesis, a greater difficulty in processing the foveal word would result in less attentional resources being allocated to the parafoveal word, thereby hindering its processing. However, contradictory results have raised questions about which foveal load manipulations may affect the processing of parafoveal words at different levels. We explored whether the semantic processing of parafoveal words can be modulated by variations in a frequency-based foveal load. When participants read word triads, modulations in the N400 component indicated that, while parafoveal words were semantically processed when foveal load was low, their meaning could not be accessed if the foveal word was more difficult to process. Therefore, a frequency-based foveal load modulates semantic parafoveal processing and a semantic preview manipulation may be a suitable baseline to test the foveal load hypothesis. |
Michael G. Cutter; Ruth Filik; Kevin B. Paterson Do readers maintain word-level uncertainty during reading? A pre-registered replication study Journal Article In: Journal of Memory and Language, vol. 125, pp. 1–14, 2022. @article{Cutter2022, We present a replication of Levy, Bicknell, Slattery, and Rayner (2009). In this prior study participants read sentences in which a perceptually confusable preposition (at; confusable with as) or non-confusable preposition (toward) was followed by a verb more likely to appear in the syntactic structure formed by replacing at with as (e.g. tossed) or a verb that was not more likely to appear in this structure (e.g. thrown). Readers experienced processing difficulty upon fixating verbs like tossed following at, but not toward. Levy et al. argued that this suggests readers maintained uncertainty about previously fixated words' identities. We argue that this finding has wide-ranging implications for language processing theories, and that a replication is required. On the basis of a Bayes Factor Design Analysis we conducted a replication study with 56 items and 72 participants in order to determine whether Levy et al.'s effects are replicable. Using Bayesian statistical techniques we show that in our dataset there is evidence against the existence of the interaction Levy et al. found, and thus conclude that this study is non-replicable. |
Ruth E. Corps; Charlotte Brooke; Martin J. Pickering Prediction involves two stages: Evidence from visual-world eye-tracking Journal Article In: Journal of Memory and Language, vol. 122, pp. 1–20, 2022. @article{Corps2022, Comprehenders often predict what they are going to hear. But do they make the best predictions possible? We addressed this question in three visual-world eye-tracking experiments by asking when comprehenders consider perspective. Male and female participants listened to male and female speakers producing sentences (e.g., I would like to wear the nice…) about stereotypically masculine (target: tie; distractor: drill) and feminine (target: dress, distractor: hairdryer) objects. In all three experiments, participants rapidly predicted semantic associates of the verb. But participants also predicted consistently-that is, consistent with their beliefs about what the speaker would ultimately say. They predicted consistently from the speaker's perspective in Experiment 1, their own perspective in Experiment 2, and the character's perspective in Experiment 3. This consistent effect occurred later than the associative effect. We conclude that comprehenders consider perspective when predicting, but not from the earliest moments of prediction, consistent with a two-stage account. |
Jon W. Carr; Valentina N. Pescuma; Michele Furlan; Maria Ktori; Davide Crepaldi Algorithms for the automated correction of vertical drift in eye-tracking data Journal Article In: Behavior Research Methods, vol. 54, no. 1, pp. 287–310, 2022. @article{Carr2022, A common problem in eye-tracking research is vertical drift—the progressive displacement of fixation registrations on the vertical axis that results from a gradual loss of eye-tracker calibration over time. This is particularly problematic in experiments that involve the reading of multiline passages, where it is critical that fixations on one line are not erroneously recorded on an adjacent line. Correction is often performed manually by the researcher, but this process is tedious, time-consuming, and prone to error and inconsistency. Various methods have previously been proposed for the automated, post hoc correction of vertical drift in reading data, but these methods vary greatly, not just in terms of the algorithmic principles on which they are based, but also in terms of their availability, documentation, implementation languages, and so forth. Furthermore, these methods have largely been developed in isolation with little attempt to systematically evaluate them, meaning that drift correction techniques are moving forward blindly. We document ten major algorithms, including two that are novel to this paper, and evaluate them using both simulated and natural eye-tracking data. Our results suggest that a method based on dynamic time warping offers great promise, but we also find that some algorithms are better suited than others to particular types of drift phenomena and reading behavior, allowing us to offer evidence-based advice on algorithm selection. |
Sarah Chabal; Sayuri Hayakawa; Viorica Marian Language is activated by visual input regardless of memory demands or capacity Journal Article In: Cognition, vol. 222, pp. 1–17, 2022. @article{Chabal2022, In the present study, we provide compelling evidence that viewing objects automatically activates linguistic labels and that this activation is not due to task-specific memory demands. In two experiments, eye-movements of English speakers were tracked while they identified a visual target among an array of four images, including a phonological competitor (e.g., flower-flag). Experiment 1 manipulated the capacity to subvocally rehearse the target label by imposing linguistic, spatial, or no working memory load. Experiment 2 manipulated the need to encode target objects by presenting target images either before or concurrently with the search display. While the timing and magnitude of competitor activation varied across conditions, we observed consistent evidence of language activation regardless of the capacity or need to maintain object labels in memory. We propose that language activation is automatic and not contingent upon working memory capacity or demands, and conclude that objects' labels influence visual search. |
Suphasiree Chantavarin; Emily Morgan; Fernanda Ferreira Robust processing advantage for binomial phrases with variant conjunctions Journal Article In: Cognitive Science, vol. 46, no. 9, pp. 1–43, 2022. @article{Chantavarin2022, Prior research has shown that various types of conventional multiword chunks are processed faster than matched novel strings, but it is unclear whether this processing advantage extends to variant multiword chunks that are less formulaic. To determine whether the processing advantage of multiword chunks accommodates variations in the canonical phrasal template, we examined the robustness of the processing advantage (i.e., predictability) of binomial phrases with non-canonical conjunctions (e.g., salt and also pepper; salt as well as pepper). Results from the cloze study (Experiment 1) showed that there was a high tendency of producing the canonical conjunct (pepper), even in the binomials that contained non-formulaic conjunctions. Consistent with these findings, results from two eye tracking studies (Experiments 2a and 2b) showed that canonical conjuncts were read faster than novel conjuncts that were matched on word length (e.g., paprika), even in the binomials with variant conjunctions. This robust online processing advantage was replicated in a self-paced reading study that compared all three conjunction types (Experiment 3). Taken together, these findings show that binomials with variant function words also receive facilitated processing relative to matched novel strings, even though both types of strings are neither conventional nor relatively frequent. Exploratory analyses revealed that this processing speed advantage was driven by the lexical–semantic association between the canonical conjuncts (salt–pepper), rather than lexical and phrasal frequency. Overall, these results highlight flexibility in the processing of multiword chunks that current models of multiword storage and processing must take into account. |
Xianglan Chen; Hulin Ren; Xiao Ying Yan Metonymy processing in Chinese: A linguistic context-sensitive eye-tracking preliminary study Journal Article In: Frontiers in Psychology, vol. 13, pp. 1–10, 2022. @article{Chen2022g, Current cognitively oriented research on metaphor proposes that understanding metaphorical expressions is a process of building embodied simulations, which are constrained by past and present bodily experiences. However, it has also been shown that metaphor processing is also constrained by the linguistic context but, to our knowledge, there is no comparable work in the domain of metonymy. As an initial attempt to fill this gap, the present study uses eye-tracking experimentation to explore this aspect of Chinese metonymy processing. It complements previous work on how the length of preceding linguistic context influences metonymic processing by focusing on: (1) the contextual information of both the preceding target words; (2) the immediate spillover after the target words; and (3) whether the logical relationship between the preceding contextual information and the target word is strong or weak (a 2 × 2 between-subject experiment with target words of literal/metonymy and logic of strong/weak). Results show that readers take longer to arrive at a literal interpretation than at a metonymic one when the preceding information is in a weak logic relationship with target words, although this disparity can disappear when the logic is strong. Another finding is that both the preceding and the spillover contextual information contribute to metonymy processing when the spillover information does more to the metonymy than it does to the literal meaning. This study further complements cognitive and pragmatic approaches to metonymy, which are centered on its conceptual nature and its role in interpretation, by drawing attention to how the components of sentences contribute to the metonymic processing of target words. Based on an experiment, a contextual model of Chinese metonymy processing is proposed. |
Yi-Ting Ting Chen; Ming-Chou Chou Ho Eye movement patterns differ while watching captioned videos of second language vs. mathematics lessons Journal Article In: Learning and Individual Differences, vol. 93, pp. 1–9, 2022. @article{Chen2022, Background: Extant eye-tracking studies suggest that foreign-language learners tend to read the native language captions while watching foreign-language videos. However, it remains unclear how the captions affect the learners' eye movements when watching Math videos. Purpose: While watching teaching videos, we seek to determine how the lesson type (English or Math), cognitive load (high or low), and caption type (meaningful, no captions, or meaningless) affect the dwell times and fixation counts on the captions. Methods: One hundred and eighty undergraduate students were randomly and equally assigned to six (2 lesson type × 3 caption type) conditions. Each participant watched two short teaching videos (one low load and one high load). After watching each video, a comprehension test and three self-reported items (fatigue, effort, and difficulty) regarding this particular video were given. Results: We reported more dwell times and fixation counts on the meaningful captions, compared to the meaningless captions and no captions. In the high-load condition, viewers watching an English lesson relied more on the meaningful captions than they did when watching a Math lesson. In the low-load condition, the dwell times and fixation counts on the captions were similar between the English and Math lessons. Finally, the captions did not affect the comprehension test performances after ruling out individual differences in the prior performances of English and Math. Conclusions: English language learning may rely more on the captions than is the case in learning Math. This study provides the direction for designing multimedia teaching materials in the current trend of multimedia teaching. In |
Sun-Joo Cho; Sarah Brown-Schmidt; Paul De Boeck; Matthew Naveiras Space-time modeling of intensive binary time series eye-tracking data using a generalized additive logistic regression model Journal Article In: Psychological Methods, vol. 27, no. 3, pp. 307–346, 2022. @article{Cho2022, Eye-tracking has emerged as a popular method for empirical studies of cognitive processes across multiple substantive research areas. Eye-tracking systems are capable of automatically generating fixation-location data over time at high temporal resolution. Often, the researcher obtains a binary measure of whether or not, at each point in time, the participant is fixating on a critical interest area or object in the real world or in a computerized display. Eye-tracking data are characterized by spatial-temporal correlations and random variability, driven by multiple fine-grained observations taken over small time intervals (e.g., every 10 ms). Ignoring these data complexities leads to biased inferences for the covariates of interest such as experimental condition effects. This article presents a novel application of a generalized additive logistic regressionmodel for intensive binary time series eye-tracking data from a between and within-subjects experimental design. The model is formulated as a generalized additive mixed model (GAMM) and implemented in the mgcv R package. The generalized additive logistic regression model was illustrated using an empirical data set aimed at understanding the accommodation of regional accents in spoken language processing. Accuracy of parameter estimates and the importance of modeling the spatial-temporal correlations in detecting the experimental condition effects were shown in conditions similar to our empirical data set via a simulation study. |
Derya Cokal; Patrick Sturt The real-time status of strong and weak islands Journal Article In: PLoS ONE, vol. 17, no. 2, pp. 1–25, 2022. @article{Cokal2022, In two eye-tracking reading experiments, we used a variant of the filled gap technique to investigate how strong and weak islands are processed on a moment-to-moment basis during comprehension. Experiment 1 provided a conceptual replication of previous studies showing that real time processing is sensitive to strong islands. In the absence of an island, readers experienced processing difficulty when a pronoun appeared in a position of a predicted gap, but this difficulty was absent when the pronoun appeared inside a strong island. Experiment 2 showed an analogous effect for weak islands: A processing cost was seen for a pronoun in the position of a predicted gap in a that-complement clause, but this cost was absent in a matched whether clause, which constitutes a weak island configuration. Overall, our results are compatible with the claim that active dependency formation is suspended, or reduced, in both weak and strong island structures. |
Fengjiao Cong; Baoguo Chen The letter position coding mechanism of second language words during sentence reading: Evidence from eye movements Journal Article In: Quarterly Journal of Experimental Psychology, vol. 75, no. 10, pp. 1932–1947, 2022. @article{Cong2022a, We conducted three eye movement experiments to investigate the mechanism for coding letter positions in a person's second language during sentence reading; we also examined the role of morphology in this process with a more rigorous manipulation. Given that readers obtain information not only from currently fixated words (i.e., the foveal area) but also from upcoming words (i.e., the parafoveal area) to guide their reading, we examined both when the targets were fixated (Exp. 1) and when the targets were seen parafoveally (Exps. 2 and 3). First, we found the classic transposed letter (TL) effect in Exp. 1, but not in Exp. 2 or 3. This implies that flexible letter position coding exists during sentence reading. However, this was limited to words located in the foveal area, suggesting that L2 readers whose L2 proficiency is not as high as skilled native readers are not able to extract and utilise the parafoveal letter identity and position information of a word, whether the word length is long (Exp. 2) or short (Exp. 3). Second, we found morphological information to influence the magnitude of the TL effect in Exp. 1. These results provide new eye movement evidence for the flexibility of L2 letter position coding during sentence reading, as well as the interactions between the different internal representations of words in this process. Future L2 reading frameworks should integrate word recognition and eye movement control models. |
Fengjiao Cong; Baoguo Chen Parafoveal orthographic processing in bilingual reading Journal Article In: International Journal of Bilingual Education and Bilingualism, vol. 25, pp. 3698–3710, 2022. @article{Cong2022, Reading is a very complex task in which readers obtain information to promote reading from not only the fixated word located in the foveal area but also non-fixated words located in the parafoveal area. We aimed to investigate the second language (L2) parafoveal orthographic (letter identity and letter position) processing mechanism adopting the eye-tracking technique and boundary paradigm. We set up four previews for each target: (1) the identity preview (e.g. reporter → reporter), (2) the transposed-letter preview (e.g. repotrer → reporter), (3) the substituted-letter condition (e.g. repokcer → reporter), and (4) the unrelated preview (e.g. chemaful → reporter). There are three main findings. First, L2 readers could extract and utilize the parafoveal orthographic information shared by the preview and the target to affect the late L2 processing stage. Second, when there was only a small difference between the preview and the target, L2 readers did not notice the subtle difference in the parafovea. Third, the identity and position of an internal single letter have little effect on L2 reading compared with the similarity of the whole word in the parafoveal area. Future L2 reading frameworks should be developed to explain these new findings. |
Lei Cui; Chuanli Zang; Xiaochen Xu; Wenxin Zhang; Yuhan Su; Simon P. Liversedge Predictability effects and parafoveal processing of compound words in natural Chinese reading Journal Article In: Quarterly Journal of Experimental Psychology, vol. 75, no. 1, pp. 18–29, 2022. @article{Cui2022, We report a boundary paradigm eye movement experiment to investigate whether the predictability of the second character of a two-character compound word affects how it is processed prior to direct fixation during reading. The boundary was positioned immediately prior to the second character of the target word, which itself was either predictable or unpredictable. The preview was either a pseudocharacter (nonsense preview) or an identity preview. We obtained clear preview effects in all conditions, but more importantly, skipping probability for the second character of the target word and the whole target word from pretarget was greater when it was predictable than when it was not predictable from the preceding context. Interactive effects for later measures on the whole target word (gaze duration and go-past time) were also obtained. These results demonstrate that predictability information from preceding sentential context and information regarding the likely identity of upcoming characters are used concurrently to constrain the nature of lexical processing during natural Chinese reading. |
Xiaohui Cui; Fabio Richlan; Wei Zhou Fixation-related fMRI analysis reveals the neural basis of parafoveal processing in self-paced reading of Chinese words Journal Article In: Brain Structure and Function, vol. 227, no. 8, pp. 2609–2621, 2022. @article{Cui2022a, While parafoveal word processing plays an important role in natural reading, the underlying neural mechanism remains unclear. The present study investigated the neural basis of parafoveal processing during Chinese word reading with the co-registration of eye-tracking and functional magnetic resonance imaging (fMRI) using fixation-related fMRI analysis. In the gaze-contingent boundary paradigm, preview conditions (words that are identical, orthographically similar, and unrelated to target words), pre-target word frequency and target word frequency were manipulated. When fixating the pre-target word, the identical preview condition elicited lower brain activation in the left fusiform gyrus relative to unrelated and orthographically similar preview conditions and there were significant interactions of preview condition and pre-target word frequency on brain activation of the left middle frontal gyrus, left fusiform gyrus and supplementary motor area. When fixating the target word, there was a significant main effect of preview condition on brain activation of the right fusiform gyrus and a significant interaction of preview condition and pre-target word frequency on brain activation of the left middle frontal gyrus. These results suggest that fixation-related brain activation provides immediate measures and new perspectives to understand the mechanism of parafoveal processing in self-paced reading. |
Emmelien Merchie; Sofie Heirweg; Hilde Van Keer Mind maps: Processed as intuitively as thought? Investigating late elementary students' eye-tracked visual behavior patterns in-depth Journal Article In: Frontiers in Psychology, vol. 13, pp. 1–18, 2022. @article{Merchie2022, In this study, 44 late elementary students' visual behavior patterns when reading mind maps were investigated, more particularly, the intuitive processing nature of their visual characteristics, reading sequence and presentation mode (i.e., mind map before or after text). Eye-tracked data were investigated by means of static early attention and dynamic educational process mining (EPM) analysis and combined with learning performance and retrospective interview data. All students seem to struggle with the map's radial structure during initial reading. Also, the picture's position in the map diverts students from consecutively reading interconnected branches. EPM analyses revealed different reading patterns in proceeding reading behavior. Students receiving a text first, seem to grasp the radial structure slightly more and show higher information integration attempts. They also attained higher free recall and coherence scores. The study concludes with instructional design principles for urgently needed explicit visual literacy instruction in elementary grades. |
Sara V. Milledge; Simon P. Liversedge; Hazel I. Blythe The importance of the first letter in children's parafoveal preprocessing in English: Is it phonologically or orthographically driven? Journal Article In: Journal of Experimental Psychology: Human Perception and Performance, vol. 48, no. 5, pp. 427–442, 2022. @article{Milledge2022, For both adult and child readers of English, the first letter of a word plays an important role in lexical identification. Using the boundary paradigm during silent sentence reading, we examined whether the first-letter bias in parafoveal preprocessing is phonologically or orthographically driven and whether this differs between skilled adult and beginner child readers. Participants read sentences that contained either a correctly spelled word in preview (identity; e.g., “circus”), a preview letter string that maintained the phonology but manipulated the orthography of the first letter (P+O-preview; e.g., “sircus”), or a preview letter string that manipulated both the phonology and the orthography of the first letter (P-O-preview; e.g., “wircus”). There was a cost associated with manipulating the first letter of the target words in preview for both adults and children. Critically, during first-pass reading, both adult and child readers displayed similar reading times between P+O- and P-O-previews. This shows that the first-letter bias is driven by orthographic encoding and that the first letter's orthographic code in preview is crucial for efficient, early processing of phonology |
Sara V. Milledge; Chuanli Zang; Simon P. Liversedge; Hazel I. Blythe Phonological parafoveal pre-processing in children reading English sentences Journal Article In: Cognition, vol. 225, pp. 1–13, 2022. @article{Milledge2022a, Although previous research has shown that, in English, both adult and teenage readers parafoveally pre-process phonological information during silent reading, to date, no research has been conducted to investigate such processing in children. Here we used the boundary paradigm during silent sentence reading, to ascertain whether typically developing English children, like adults, parafoveally process words phonologically. Participants' eye movements (adults: n = 48; children: n = 48) were recorded as they read sentences which contained, in preview, correctly spelled words (e.g., cheese), pseudohomophones (e.g., cheeze), or spelling controls (e.g., cheene). The orthographic similarity of the target words available in preview was also manipulated to be similar (e.g., cheese/cheeze/cheene) or dissimilar (e.g., queen/kween/treen). The results indicate that orthographic similarity facilitated both adults' and children's pre-processing. Moreover, children parafoveally pre-processed words phonologically very early in processing. The children demonstrated a pseudohomophone advantage from preview that was broadly similar to the effect displayed by the adults, although the orthographic similarity of the pseudohomophone previews was more important for the children than the adults. Overall, these results provide strong evidence for phonological recoding during silent English sentence reading in 8–9-year-old children. |
Sara Milligan; Martín Antúnez; Horacio A. Barber; Elizabeth R. Schotter Are eye movements and EEG on the same page?: A coregistration study on parafoveal preview and lexical frequency Journal Article In: Journal of Experimental Psychology: General, pp. 1–23, 2022. @article{Milligan2022, Readers extract visual and linguistic information not only from fixated words but also upcoming parafoveal words to introduce new input efficiently into the language processing pipeline. The lexical frequency of upcoming words and similarity with subsequent foveal information both influence the amount of time people spend once they fixate the word foveally. However, it is unclear from eye movements alone the extent to which parafoveal word processing, and the integration of that word with foveally obtained information, continues after saccade plans have been initiated. To investigate the underlying neural processes involved in word recognition after saccade planning, we coregistered electroencephalogram (EEG) and eye movements during a gaze-contingent display change paradigm. We orthogonally manipulated the frequency of the parafoveal and foveal words and measured fixation related potentials (FRPs) upon foveal fixation. Eye movements showed primarily an effect of preview frequency, suggesting that saccade planning is based on the familiarity of the parafoveal input. FRPs, on the other hand, demonstrated a disruption in downstream processing when parafoveal and foveal input differed, but only when the parafoveal word was high frequency. These findings demonstrate that lexical processing continues after the eyes have moved away from a word and that eye movements and FRPs provide distinct but complementary accounts about oculomotor behavior and neural processing that cannot be obtained from either method in isolation. Furthermore, these findings put constraints on models of reading by suggesting that lexical processes that occur before an eye movement program is initiated are qualitatively different from those that occur afterward. |
Sanako Mitsugi Polarity adverbs facilitate predictive processing in L2 Japanese Journal Article In: Second Language Research, vol. 38, no. 4, pp. 869–892, 2022. @article{Mitsugi2022, This study examines whether second language (L2) learners predict upcoming language prior to the verb in Japanese. Taking the dependency involving negative polarity adverbs – zenzen ‘at all' and amari ‘(not) very' – as a test case, this study examined whether Japanese native speakers and L2 learners of Japanese, aided by these adverbs, generate predictions of the polarity of the sentence-final verb. The visual-world paradigm experiment revealed that both native-speaker and L2-learner groups looked progressively more at the target picture before the negated verb when the information from adverbs was available than when it was not. The pattern of results underscores the usefulness of adverbial polarity items as predictive cues and indicates that they expedite the processing of negation in Japanese. Learners successfully exploited this information to generate nativelike predictions on sentence meaning. |
Lyndall Murray; Signy Wegener; Hua-Chen Wang; Rauno Parrila; Anne Castles Children processing novel irregular and regular words during reading: An eye tracking study Journal Article In: Scientific Studies of Reading, vol. 26, no. 5, pp. 417–431, 2022. @article{Murray2022a, Children may link words in their oral vocabulary with novel printed word forms through a process termed mispronunciation correction, which enables them to adjust an imperfect phonological decoding. Additional evidence suggests that sentence context may play a role in helping children to make link between a word in oral vocabulary and its irregular written form. Four groups of children were orally trained on a set of novel words but received no training on a second set. Half the trained words were designated irregular spellings and half regular spellings. Children later read the words in contextually supportive or neutral sentences while their eye movements were monitored. Fixations on untrained words were longer than on trained regular words but were similar to trained irregular words. Fixations on regular words were shorter than on irregular words, and there were larger differences between irregular and regular words viewed in contextually supportive sentences. Subsequently, children were able to read irregular words more accurately when they had previously appeared in a supportive context. These results suggest that orally known irregular words undergo additional processing when first viewed in text, which is consistent with the online operation of a mispronunciation correction mechanism. |
Shingo Nahatame Causal and semantic relations in L2 text processing: An eye-tracking study Journal Article In: Reading in a Foreign Language, vol. 34, no. 1, pp. 91–115, 2022. @article{Nahatame2022, This study is an extension of Nahatame's (2018) research that demonstrated the effects of causal and semantic relations between sentences on second language (L2) text processing. Employing eye tracking, this study aimed to examine whether these effects appear during more natural, uninterrupted reading processes and to identify the time course of the effects. In the experiment, Japanese learners of English read two-sentence texts that varied in their causal and semantic relatedness, as evaluated by crowdsourced human judgments and via a computational approach (latent semantic analysis), respectively. Two eye-movement measures were collected and analyzed: first-pass reading times for the second sentence and lookbacks from the second to the first sentence. The results indicated that causal relatedness had a robust impact on both reading times and lookbacks. However, semantic relatedness impacted only reading times, and its effects were modulated by causal relatedness. Theoretical, pedagogical, and methodological implications of this finding were discussed. |
Chie Nakamura; Manabu Arai; Yuki Hirose; Suzanne Flynn L2 learners do not ignore verb's subcategorization information in real-time syntactic processing Journal Article In: Frontiers in Psychology, vol. 12, pp. 1–13, 2022. @article{Nakamura2022, This study addressed the question of whether L2 learners are able to utilize verb's argument structure information in online structural analysis. Previous L2 research has shown that L2 learners have difficulty in using verb's intransitive information to guide online syntactic processing. This is true even though L2 learners have grammatical knowledge that is correct and similar to that of native speakers. In the present study, we contrasted three hypotheses, the initial inaccessibility account, the intransitivity overriding account, and the fuzzy subcategorization frame account, to investigate whether L2 learner's knowledge of intransitive verbs is in fact ignored in L2 online structural analysis. The initial inaccessibility account and the fuzzy subcategorization frame account predicted that L2 learners cannot access intransitivity information in building syntactic structures in any situation. The intransitivity overriding account predicted that intransitivity information is accessed in L2 parsing, but this process is overridden by the strong transitivity preference when a verb is followed by a noun phrase. Importantly, the intransitivity overriding account specifically predicted that L2 learners would be able to use intransitive information in online syntactic processing when a noun phrase does not appear immediately following a verb. We tested the three accounts in an eye-tracking reading experiment using filler-gap dependency structures. We manipulated verb's transitivity information and lexically based plausibility information and tested English native speakers as a control L1 group (N = 29) and Japanese-English L2 participants (N = 32). The results showed that L2 learners as well as native speakers processed sentences differently depending on the subcategorization information of the verb, and adopted transitive analysis only when the verb was optionally transitive, providing support for the intransitivity overriding. The results further demonstrated that L2 learners had strong expectations for the transitive structure, which is consistent with the view proposed by the hyper-active gap-filling hypothesis. In addition, the results showed that the semantic mismatch in the incorrect transitive analysis facilitated native speaker's processing but caused difficulty for L2 learners. Together, the current study provides evidence that L2 learners use intransitive information of the verbs to guide their structural analysis when there are no overriding constraints. |
Ádám Nárai; Zsuzsanna Nemecz; Zoltán Vidnyánszky; Béla Weiss Lateralization of orthographic processing in fixed-gaze and natural reading conditions Journal Article In: Cortex, vol. 157, pp. 99–116, 2022. @article{Narai2022, Lateralized processing of orthographic information is a hallmark of proficient reading. However, how this finding obtained for fixed-gaze processing of orthographic stimuli translates to ecologically valid reading conditions remained to be clarified. To address this shortcoming, here we assessed the lateralization of early orthographic processing in fixed-gaze and natural reading conditions using concurrent eye-tracking and EEG data recorded from young adults without reading difficulties. Sensor-space analyses confirmed the well-known left-lateralized negative-going deflection of fixed-gaze EEG activity throughout the period of early orthographic processing. At the same time, fixation-related EEG activity exhibited left-lateralized followed by right-lateralized processing of text stimuli during natural reading. A strong positive relationship was found between the early leftward lateralization in fixed-gaze and natural reading conditions. Using source-space analyses, early left-lateralized brain activity was obtained in lateraloccipital and posterior ventral occipito-temporal cortices reflecting letter-level processing in both conditions. In addition, in the same time interval, left-lateralized source activity was found also in premotor and parietal brain regions during natural reading. While brain activity remained left-lateralized in later stages representing word-level processing in posterior and middle ventral temporal regions in the fixed-gaze condition, fixation-related source activity became stronger in the right hemisphere in medial and more anterior ventral temporal brain regions indicating higher-level processing of orthographic information. Although our results show a strong positive relationship between the lateralization of letter-level processing in the two reading modes and suggest lateralized brain activity as a general marker for processing of orthographic information, they also clearly indicate the need for reading research in ecologically valid conditions to identify the neural basis of visuospatial attentional, oculomotor and higher-level processes specific to natural reading. |
Lama Nassif; Elizabeth Huntley; Ayman Mohamed Attention to verbal morphology in L2 Arabic reading: An eye-movement study Journal Article In: Foreign Language Annals, vol. 55, pp. 769–792, 2022. @article{Nassif2022, Attention is believed to help facilitate learning. Godfroid and Uggen found that attention to irregular verb morphology motivated the learning of novel second language (L2) German forms. The current study explored the generalizability of these findings to geminate and sound verbs in Arabic, a typologically different language with a novel writing system. Eleven fourth-semester learners of Arabic participated in the experiment. Participants completed a language learning background survey, took a fill-in-the blank pretest, read 20 sentence pairs while an Eyelink 1000 recorded their eye movements, and answered true/false comprehension questions that appeared on-screen following each sentence. A posttest, identical to the pretest, and a prior vocabulary knowledge scale task were then conducted. Learners' reflections were recorded in a subsequent recall task and a follow-up semistructured interview. Descriptive analyses of the eye-tracking metrics reveal generally equivalent reading times between verb types, although participants made more direct visual comparisons between geminate- than between sound-verb conjugations. Participants did not report awareness of geminate verbs, but noticed other aspects of input, and, on average, improved their written productive knowledge by 2% after only one exposure. Pedagogical implications are discussed in terms of input enhancement in a communicative L2 classroom. |
Keiyu Niikuni; Ming Wang; Michiru Makuuchi; Masatoshi Koizumi; Sachiko Kiyama Pupil dilation reflects emotional arousal via poetic language Journal Article In: Perceptual and Motor Skills, pp. 1–18, 2022. @article{Niikuni2022, We investigated pupillary responses to the world's shortest fixed verses, Japanese haiku as aesthetic poetry (AP) and senryu as comic poetry (CP), in comparison with non-poetry control stimuli (NP) comprised of slogans that had the same rhythm patterns. Native Japanese speakers without literary training listened to these stimuli while we recorded their pupil diameters. We found that participants' pupils were significantly dilated for CP compared to NP in an early time window. While AP also evoked larger dilations than NP, the latency for AP-related pupil dilation was relatively long. Thus, lay people experience quick and intense arousal in response to funny and humorous words, while aesthetic properties of words may also elicit intense but slower changes in listeners' arousal levels, presumably because they evoke more implicit and subtle emotional effects. This study is the first to provide evidence that poetic language elicits human pupillary dilation. A better understanding of the cognitive and neural substrates for the sensitive awareness of pleasures expressed via poetic language will provide insights for improving mental and physical health. Hence, pupillometry can act as a useful convenient measurement to delineate the sympathetic activation of emotional contexts via language. |
Kelly Nisbet; Raymond Bertram; Charlotte Erlinghagen; Aleks Pieczykolan; Victor Kuperman Quantifying the difference in reading fluency between L1 and L2 readers of English Journal Article In: Studies in Second Language Acquisition, vol. 44, pp. 407–434, 2022. @article{Nisbet2022, This study is a comparative examination of reading behavior of first-language (L1) Canadian and second-language (L2) Finnish and German readers of English. We measured eye-movement patterns during reading the same set of English sentences and administered tests of English vocabulary, spelling, and exposure to print. The core of our study is a novel method of statistical prediction used to generate hypothetical Finnish and German participants with maximum observed L1 scores in all component skills. We found that with L1-like component skills, hypothetical German readers can show the same reading speed as the L1 group. We hypothesize this advantage comes from the small linguistic distance to English. Conversely, hypothetical Finnish readers remain disadvantaged even with maximum component skills, likely due to a larger linguistic distance. We discuss theoretical and applied implications of our method for L2 acquisition research. |
Gal Nitsan; Shai Baharav; Dalith Tal-Shir; Vered Shakuf; Boaz M. Ben-David In: JMIR Serious Games, vol. 10, no. 3, pp. 1–15, 2022. @article{Nitsan2022, Background: The number of serious games for cognitive training in aging (SGCTAs) is proliferating in the market and attempting to combat one of the most feared aspects of aging-cognitive decline. However, the efficacy of many SGCTAs is still questionable. Even the measures used to validate SGCTAs are up for debate, with most studies using cognitive measures that gauge improvement in trained tasks, also known as near transfer. This study takes a different approach, testing the efficacy of the SGCTA-Effectivate-in generating tangible far-transfer improvements in a nontrained task-the Eye tracking of Word Identification in Noise Under Memory Increased Load (E-WINDMIL)-which tests speech processing in adverse conditions. Objective: This study aimed to validate the use of a real-time measure of speech processing as a gauge of the far-transfer efficacy of an SGCTA designed to train executive functions. Methods: In a randomized controlled trial that included 40 participants, we tested 20 (50%) older adults before and after self-administering the SGCTA Effectivate training and compared their performance with that of the control group of 20 (50%) older adults. The E-WINDMIL eye-tracking task was administered to all participants by blinded experimenters in 2 sessions separated by 2 to 8 weeks. Results: Specifically, we tested the change between sessions in the efficiency of segregating the spoken target word from its sound-sharing alternative, as the word unfolds in time. We found that training with the SGCTA Effectivate improved both early and late speech processing in adverse conditions, with higher discrimination scores in the training group than in the control group (early processing: F1 |
Kole A. Norberg; Charles Perfetti; Anne Helder Word-to-text integration and antecedent accessibility: Eye-tracking evidence extends results of event-related potentials (ERPs) Journal Article In: Journal of Experimental Psychology: Learning, Memory, and Cognition, vol. 48, no. 4, pp. 598–617, 2022. @article{Norberg2022, Eye tracking and event-related potentials (ERPs) have complementary advantages in the study of reading processes. We used eye tracking to extend ERP evidence of Helder et al. (2020) that word-to-text integration at the beginnings and ends of sentences is primarily determined by local text factors (antecedents in a previous sentence) but that global factors (central theme) may make these antecedents more accessible in memory and thus facilitate their integration. The ERP evidence for these conclusions comes from the N400 on a target noun, which varied with the appearance of an antecedent in the previous sentence and whether that antecedent was related to the passage theme. Here, using the same materials, we report eye tracking evidence that reflects not only integration processes indexed by fixation on target words and words following, but also regressions to the antecedent, a measure not possible with ERPs. Interestingly, conclusions from eye-tracking measures align generally with those from Helder et al., but reading times did not consistently correspond to reduced N400s. A distinctive eye-tracking result is that when antecedents were not related to the central theme of the passage (thus less accessible in memory) there was a greater likelihood of return to the antecedent from the regions beyond the target word. These findings demonstrate an independent influence of both local and global context on reading patterns that are unique to eye-tracking measurement, thus both converging with ERP conclusions and adding new ones. |
Marissa Ogren; Scott P. Johnson Nonverbal emotion perception and vocabulary in late infancy Journal Article In: Infant Behavior and Development, vol. 68, pp. 1–10, 2022. @article{Ogren2022, Language has been proposed as a potential mechanism for young children's developing understanding of emotion. However, much remains unknown about this relation at an individual difference level. The present study investigated 15- to 18-month-old infants' perception of emotions across multiple pairs of faces. Parents reported their child's productive vocabulary, and infants participated in a non-linguistic emotion perception task via an eye tracker. Infant vocabulary did not predict nonverbal emotion perception when accounting for infant age, gender, and general object perception ability (β = −0.15 |
Henri Olkoniemi; Raymond Bertram; Johanna K. Kaakinen Knowledge is a river and education is like a stairway: An eye movement study on how L2 speakers process metaphors and similes Journal Article In: Bilingualism, vol. 25, no. 2, pp. 307–320, 2022. @article{Olkoniemi2022, Very little is known about the processes underlying second language (L2) speakers' understanding of written metaphors and similes. Moreover, most of the theories on figurative language comprehension do not consider reader-related factors. In the study, we used eye-Tracking to examine how native Finnish speakers (N = 63) read written English nominal metaphors (education is a stairway) and similes (education is like a stairway). Identical topic-vehicle pairs were used in both conditions. After reading, participants evaluated familiarity of each pair. English proficiency was measured using the Bilingual-language Profile Questionnaire and the Lexical Test for Advanced Learners of English. The results showed that readers were more likely to regress within metaphors than within similes, indicating that processing metaphors requires more processing effort than processing similes. The familiarity of a metaphor and L2 English proficiency modulated this effect. The results are discussed in the light of current theories on figurative language processing. |
Eric Failes; Mitchell S. Sommers Using eye-tracking to investigate an activation-based account of false hearing in younger and older adults Journal Article In: Frontiers in Psychology, vol. 13, pp. 1–15, 2022. @article{Failes2022, Several recent studies have demonstrated context-based, high-confidence misperceptions in hearing, referred to as false hearing. These studies have unanimously found that older adults are more susceptible to false hearing than are younger adults, which the authors have attributed to an age-related decline in the ability to inhibit the activation of a contextually predicted (but incorrect) response. However, no published work has investigated this activation-based account of false hearing. In the present study, younger and older adults listened to sentences in which the semantic context provided by the sentence was either unpredictive, highly predictive and valid, or highly predictive and misleading with relation to a sentence-final word in noise. Participants were tasked with clicking on one of four images to indicate which image depicted the sentence-final word in noise. We used eye-tracking to investigate how activation, as revealed in patterns of fixations, of different response options changed in real-time over the course of sentences. We found that both younger and older adults exhibited anticipatory activation of the target word when highly predictive contextual cues were available. When these contextual cues were misleading, younger adults were able to suppress the activation of the contextually predicted word to a greater extent than older adults. These findings are interpreted as evidence for an activation-based model of speech perception and for the role of inhibitory control in false hearing. |
Xi Fan; Ronan G. Reilly Eye movement control in reading Chinese: A matter of strength of character? Journal Article In: Acta Psychologica, vol. 230, pp. 1–13, 2022. @article{Fan2022a, This paper explores the processes underlying eye movement control in Chinese reading among a population of young 4th and 5th grade readers. Various proposals to explain the underlying mechanisms involved in eye movement control are examined and the paper concludes that the most likely account is of a two-factor process whereby the character is the main driver for longer saccades and that the word plays a role in shorter ones. A computational model is proposed to provide an integrated account of the dynamic interaction of these two factors. |
Claudia Felser; Janna Deborah Drummer Binding out of relative clauses in native and non-native sentence comprehension Journal Article In: Journal of Psycholinguistic Research, vol. 51, pp. 763–788, 2022. @article{Felser2022, Pronouns can sometimes covary with a non c-commanding quantifier phrase (QP). To obtain such 'telescoping' readings, a semantic representation must be computed in which the QP's semantic scope extends beyond its surface scope. Non-native speakers have been claimed to have more difficulty than native speakers deriving such non-isomorphic syntax-semantics mappings, but evidence from processing studies is scarce. We report the results from an eye-movement monitoring experiment and an offline questionnaire investigating whether native and non-native speakers of German can link personal pronouns to non c-commanding QPs inside relative clauses. Our results show that both participant groups were able to obtain telescoping readings offline, but only the native speakers showed evidence of forming telescoping dependencies during incremental parsing. During processing the non-native speakers focused on a discourse-prominent, non-quantified alternative antecedent instead. The observed group differences indicate that non-native comprehenders have more difficulty than native comprehenders computing scope-shifted representations in real time. |
Sara Fernández Cuenca; Jill Jegerski A role for verb regularity in the L2 processing of the Spanish subjunctive mood: Evidence from eye-tracking Journal Article In: Studies in Second Language Acquisition, pp. 1–30, 2022. @article{FernandezCuenca2022, The present study investigated the second language processing of grammatical mood in Spanish. Eye-movement data from a group of advanced proficiency second language users revealed nativelike processing with irregular verb stimuli but not with regular verb stimuli. A comparison group of native speakers showed the expected effect with both types of stimuli, but these were slightly more robust with irregular verbs than with regular verbs. We propose that the role of verb form regularity was due to the greater visual salience of Spanish subjunctive forms with irregular verbs versus regular verbs and possibly also due to less efficient processing of rule-based regular inflectional morphology versus whole irregular word forms. In any case, the results suggest that what appeared to be difficulty with sentence processing could be traced back to word-level processes, which appeared to be the primary area of difficulty. This outcome seems to go against theories that suggest that L2 sentence processing is shallow. |
Danil Fokin; Stefan Blohm; Elena Riekhakaynen Reading Russian poetry: An expert-novice study Journal Article In: Journal of Eye Movement Research, vol. 13, no. 3, pp. 1–13, 2022. @article{Fokin2022, Studying the role of expertise in poetry reading, we hypothesized that poets' expe11 knowledge comprises genre-appropriate reading-and comprehension straregies that are reflected in distinct patterns of reading behavior. We recorded eye movements while two groups of native speakers (n= 1 O each) read selected Russian poetty: an expert group of professional poets who read poetry daily, and a control group ofnovices who read poet1y less than once a 111011th. We conducted mixed-effects regression analyses to test for effects of group 011 first-fixation durations, first-pass gaze durations , and total reading times per word while controlling for lexical-and text variables. First-fü.:ation durations exclusively reflected lexical features. and total reading times re-tlected both lexical-and rexr variables; only firsr-pass gaze durarions were additionally modulated by readers' level of expertise. Whereas gaze durations of novice readers became faster as they progressed through the poems, and differed between line-final words and non-final ones, poets retained a steady pace of firsr-pass reading throughout the poems and within verse lines. Additionally, poets' gaze durations were less sensitive to word length. We conclude that readers' level of expertise modulates the way they read poetry. Our findings support theories of litera1y comprehension that assume distinct processing modes which emerge from prior experience with litera1y texts. |
Max R. Freeman; Viorica Marian Visual word recognition in bilinguals Journal Article In: Studies in Second Language Acquisition, vol. 44, no. 3, pp. 1–29, 2022. @article{Freeman2022, A bilingual's language system is highly interactive. When hearing a second language (L2), bilinguals access native-language (L1) words that share sounds across languages. In the present study, we examine whether input modality and L2 proficiency moderate the extent to which bilinguals activate L1 phonotactic constraints (i.e., rules for combining speech sounds) during L2 processing. Eye movements of English monolinguals and Spanish-English bilinguals were tracked as they searched for a target English word in a visual display. On critical trials, displays included a target that conflicted with the Spanish vowel-onset rule (e.g., spa), as well as a competitor containing the potentially activated e onset (e.g., egg). The rule violation was processed either in the visual modality (Experiment 1) or audio-visually (Experiment 2). In both experiments, bilinguals with lower L2 proficiency made more eye movements to competitors than fillers. Findings suggest that bilinguals who have lower L2 proficiency access L1 phonotactic constraints during L2 visual word processing with and without auditory input of the constraint-conflicting structure (e.g., spa). We conclude that the interactivity between a bilingual's two languages is not limited to words that share form across languages, but also extends to sublexical, rule-based structures. |
Cheryl Frenck-Mestre; Hyeree Choo; Ana Zappa; Julia Herschensohn; Seung Kyung Kim; Alain Ghio; Sungryung Koh The online processing of Korean case by native Korean speakers and second language learners as revealed by eye movements Journal Article In: Brain Sciences, vol. 12, pp. 1–28, 2022. @article{FrenckMestre2022, Previous experimental studies have reported clear differences between native speakers and second language (L2) learners as concerns their capacity to extract and exploit morphosyntactic information during online processing. We examined the online processing of nominal case morphology in Korean by native speakers and L2 learners by contrasting canonical (SOV) and scrambled (OSV) structures, across auditory (Experiment 1) and written (Experiment 2) formats. Moreover, we compared different instances of nominal case marking: accusative (NOM-ACC) and dative (NOM-DAT). During auditory processing, Koreans showed incremental processing based on case information, with no effect of scrambling or specific case marking. In contrast, the L2 group showed no evidence of predictive processing and was negatively impacted by scrambling, especially for the accusative. During reading, both Koreans and the L2 group showed a cost of scrambling on first pass reading times, specifically for the dative. Lastly, L2 learners showed better comprehension for scrambled dative than accusative structures across formats. The current set of results show that format, the specific case marking, and word order all affect the online processing of nominal case morphology. |
Mathilda Froesel; Maëva Gacoin; Simon Clavagnier; Marc Hauser; Quentin Goudard; Suliann Ben Hamed Socially meaningful visual context either enhances or inhibits vocalisation processing in the macaque brain Journal Article In: Nature Communications, vol. 13, no. 1, pp. 1–17, 2022. @article{Froesel2022, Social interactions rely on the interpretation of semantic and emotional information, often from multiple sensory modalities. Nonhuman primates send and receive auditory and visual communicative signals. However, the neural mechanisms underlying the association of visual and auditory information based on their common social meaning are unknown. Using heart rate estimates and functional neuroimaging, we show that in the lateral and superior temporal sulcus of the macaque monkey, neural responses are enhanced in response to species-specific vocalisations paired with a matching visual context, or when vocalisations follow, in time, visual information, but inhibited when vocalisation are incongruent with the visual context. For example, responses to affiliative vocalisations are enhanced when paired with affiliative contexts but inhibited when paired with aggressive or escape contexts. Overall, we propose that the identified neural network represents social meaning irrespective of sensory modality. |
Sónia Frota; Jovana Pejovic; Marisa Cruz; Cátia Severino; Marina Vigário Early word segmentation behind the mask Journal Article In: Frontiers in Psychology, vol. 13, pp. 1–19, 2022. @article{Frota2022, Infants have been shown to rely both on auditory and visual cues when processing speech. We investigated the impact of COVID-related changes, in particular of face masks, in early word segmentation abilities. Following up on our previous study demonstrating that, by 4 months, infants already segmented targets presented auditorily at utterance-edge position, and, using the same visual familiarization paradigm, 7–9-month-old infants performed an auditory and an audiovisual word segmentation experiment in two conditions: without and with an FFP2 face mask. Analysis of acoustic and visual cues showed changes in face-masked speech affecting the amount, weight, and location of cues. Utterance-edge position displayed more salient cues than utterance-medial position, but the cues were attenuated in face-masked speech. Results revealed no evidence for segmentation, not even at edge position, regardless of mask condition and auditory or visual speech presentation. However, in the audiovisual experiment, infants attended more to the screen during the test trials when familiarized with without mask speech. Also, the infants attended more to the mouth and less to the eyes in without mask than with mask. In addition, evidence for an advantage of the utterance-edge position in emerging segmentation abilities was found. Thus, audiovisual information provided some support to developing word segmentation. We compared 7–9-monthers segmentation ability observed in the Butler and Frota pre-COVID study with the current auditory without mask data. Mean looking time for edge was significantly higher than unfamiliar in the pre-COVID study only. Measures of cognitive and language development obtained with the CSBS scales showed that the infants of the current study scored significantly lower than the same-age infants from the CSBS (pre-COVID) normative data. Our results suggest an overall effect of the pandemic on early segmentation abilities and language development, calling for longitudinal studies to determine how development proceeds. |
Conor I. Frye; Sarah C. Creel Perceptual flexibility in word learning: Preschoolers learn words with speech sound variability Journal Article In: Brain and Language, vol. 226, pp. 1–11, 2022. @article{Frye2022, Children's language input is rife with acoustic variability. Much of this variability may facilitate learning by highlighting unvarying, criterial speech attributes. But in many cases, learners experience variation in those criterial attributes themselves, as when hearing speakers with different accents. How flexible are children in the face of this variability? The current study taught 3–5-year-olds new words containing speech-sound variability: a single picture might be labeled both deev and teev. After learning, children's knowledge was tested by presenting two pictures and asking them to point to one. Picture-pointing accuracy and eye movements were tracked. While children pointed less accurately and looked less rapidly to dual-label than single-label words, they robustly exceeded chance. Performance was weaker when children learned two distinct labels, such as vayfe and fosh, for a single object. Findings suggest moderate learning even with speech-sound variability. One implication is that neural representations of speech contain rich gradient information. |
Zuzanna Fuchs Facilitative use of grammatical gender in Heritage Spanish Journal Article In: Linguistic Approaches to Bilingualism, vol. 12, no. 6, pp. 845–871, 2022. @article{Fuchs2022a, This paper presents an eye-tracking study using the Visual World Paradigm that tests whether participants are able to access gender information on definite articles and deploy it to facilitate lexical retrieval of subsequent nouns. A comparison of heritage speakers of Spanish with control monolingual speakers of Spanish suggests that the heritage speakers' performance on this task is qualitatively similar to that of the baseline. This suggests that, despite non-target-like performance in offline tasks targeting gender production and comprehension, heritage speakers of Spanish can use gender in a target-like manner in online tasks. In line with proposals put forth by Grüter et al. (2012) and Montrul et al. (2014) , a preliminary comparison with previous work on L2 learners ( Lew-Williams & Fernald, 2010 ; Grüter et al., 2012 ; Dussias et al., 2013 ) provides tentative support for the idea that the nature of early language learning is crucial in developing the ability to use grammatical gender to facilitate lexical retrieval ( Grüter et al., 2012 ; Montrul et al., 2014 ). |
Zuzanna Fuchs Eyetracking evidence for heritage speakers' access to abstract syntactic agreement features in real-time processing Journal Article In: Frontiers in Psychology, vol. 13, pp. 1–17, 2022. @article{Fuchs2022, This paper presents the results of an eyetracking study that uses the Visual World Paradigm to determine whether heritage speakers of Polish can use grammatical gender cues to facilitate lexical retrieval of the subsequent noun during real time processing. Previous work has investigated this question for heritage speakers of Spanish with gender cues located on definite articles, which are highly frequent in Spanish; the results are therefore consistent both with a grammatical account, wherein heritage speakers access abstract syntactic gender features during processing, and a probabilistic account, wherein facilitation is due to transition probabilities between frequently co-occurring elements. In Polish, gender cues appear on adjectives, which are optional and infrequent. Results of the present study show that heritage speakers of Polish can use gender on inflected adjectives to fixate on the target noun faster in trials where that gender cue uniquely identifies the target noun. This finding supports a grammatical rather than probabilistic account of the facilitative use of grammatical gender in this population: heritage speakers are able to access abstract syntactic information in real time to aid word recognition in a target-like manner. |
Hiroki Fujita; Ian Cunnings Interference and filler-gap dependency formation in native and non-native language comprehension Journal Article In: Journal of Experimental Psychology: Learning, Memory, and Cognition, vol. 48, no. 5, pp. 702–716, 2022. @article{Fujita2022, The mechanisms underlying native (L1) and non-native (L2) sentence processing have been widely debated. One account of potential L1/L2 differences is that L2 sentence processing underuses syntactic information and relies heavily on semantic and surface cues. Recently, an alternative account has been proposed, which argues that the source of L1/L2 differences lies in how susceptible L1 and L2 speakers are to interference during memory retrieval operations. The present study tested these two accounts by investigating filler-gap dependency formation and susceptibility to similarity-based interference in L1 and L2 language comprehension. The results demonstrated that L1 and L2 speakers recover the information of the filler upon encountering a gap and are susceptible to similarity-based interference during filler-gap dependency formation. However, there was no significant evidence of L1/L2 differences. These findings suggest that L1 and L2 speakers similarly engage in cue-based memory retrieval operations during filler-gap dependency formation. |
Kumiko Fukumura; Maria Nella Carminati Overspecification and incremental referential processing: An eye-tracking study Journal Article In: Journal of Experimental Psychology: Learning, Memory, and Cognition, vol. 48, no. 5, pp. 680–701, 2022. @article{Fukumura2022, Using eye-tracking, we examined whether overspecification hinders or facilitates referent selection and the extent to which this depends on the properties of the attribute mentioned in the referring expressions and the underpinning processing mode. Following spoken instructions, participants selected the referent in a visual dis- play while their eye movements were monitored. The referring expressions were presented either simultaneously with the displays, so the attributes could be incrementally processed in sequence,or before the display presentation, so the attributes could be processed in parallel from the outset of search. Experiment 1 showed that when the attributes were processed incrementally, how quickly an earlier-mentioned attribute discriminated determined whether a late-mentioned, overspecified attribute contributed to discrimination: When color was mentioned first and was fully discriminating, the referent was selected fast regardless of the second-mentioned pattern, whereas when pattern was mentioned first and fully discriminating, the second-mentioned color facilitated discrimination. Experiment 2 found that under incremental processing, color mention after a fully discriminating pattern increased fixations but delayed referent selection relative to a pattern-only description; under parallel processing, however, color mention immediately eliminated alternatives and sped up referent selection. Experiment 3 showed that pattern mention after a fully discriminating color delayed referent selec- tion and tended to reduce fixations relative to a color-only description in both processing modes. Hence, additional attributes can speed up referent selection but only when they can discriminate much faster than alternative attributes mentioned in a more concise description and,critically,when they can be used early for referent search. |
Benjamin Gagl; Klara Gregorova; Julius Golch; Stefan Hawelka; Jona Sassenhagen; Alessandro Tavano; David Poeppel; Christian J. Fiebach Eye movements during text reading align with the rate of speech production Journal Article In: Nature Human Behaviour, vol. 6, no. 3, pp. 429–442, 2022. @article{Gagl2022, Across languages, the speech signal is characterized by a predominant modulation of the amplitude spectrum between about 4.3 and 5.5 Hz, reflecting the production and processing of linguistic information chunks (syllables and words) every ~200 ms. Interestingly, ~200 ms is also the typical duration of eye fixations during reading. Prompted by this observation, we demonstrate that German readers sample written text at ~5 Hz. A subsequent meta-analysis of 142 studies from 14 languages replicates this result and shows that sampling frequencies vary across languages between 3.9 Hz and 5.2 Hz. This variation systematically depends on the complexity of the writing systems (character-based versus alphabetic systems and orthographic transparency). Finally, we empirically demonstrate a positive correlation between speech spectrum and eye movement sampling in low-skilled non-native readers, with tentative evidence from post hoc analysis suggesting the same relationship in low-skilled native readers. On the basis of this convergent evidence, we propose that during reading, our brain's linguistic processing systems imprint a preferred processing rate—that is, the rate of spoken language production and perception—onto the oculomotor system. |
Juan J. Garrido-Pozú Predictive processing of grammatical gender: Using gender cues to facilitate processing in Spanish Journal Article In: Lingua, vol. 278, pp. 1–19, 2022. @article{GarridoPozu2022, The present study investigated whether Spanish monolinguals and English learners of Spanish use grammatical gender information in articles and nouns to facilitate the processing of upcoming adjectives. Specifically, this study examined the role of noun transparency, the number of gender cues, and L2 proficiency. The results of an eye-tracking task revealed that Spanish monolinguals predicted the gender of upcoming adjectives much sooner than L2 learners. L2 learners used grammatical gender predictively, but their predictions were delayed compared to monolinguals. Advanced L2 learners predicted upcoming adjectives faster than intermediate learners. Notably, transparent nouns and a higher number of gender cues increased target fixations in all groups. Target fixations of advanced learners increased especially with transparent nouns, and target fixations of intermediate learners were higher with more gender cues present. The results showed that L2 learners can integrate L2 morphosyntactic information during rapid speech processing, and that transparently marked nouns and more gender cues aid gender processing. |
Carolina A. Gattei; Federico Alvarez; Luis París; Alejandro Wainselboim; Yamila Sevilla; Diego Shalom Why bother? What our eyes tell about psych verb (non) causative constructions Journal Article In: Glossa Psycholinguistics, vol. 1, no. 1, pp. 1–38, 2022. @article{Gattei2022, We present an eyetracking study that investigates how linking is achieved during real-time comprehension of Spanish sentences with causative psych verbs and alternative case marking. This group of verbs lead to verbs' argument structures that require direct or inverse syntax-to-semantics linking according to the type of case marking assigned to their object. The study aimed at disentangling whether processing inverse linking was more costly than direct linking, and exploring how incremental argument interpretation takes place when lexemes that accept several case markings are used. Results showed that during incremental comprehension, inverse linking is more difficult than direct linking, irrespective of word order. As for argument interpretation, the current study partially replicated the results of previous studies conducted in this language using different verb types. Findings are discussed under the light of different psycholinguistic models addressing case marking processing and incremental linking. |
Floor Berg; Jelle Brouwer; Thomas B. Tienkamp; Josje Verhagen; Merel Keijzer Language entropy relates to behavioral and pupil indices of executive control in young adult bilinguals Journal Article In: Frontiers in Psychology, vol. 13, pp. 1–17, 2022. @article{Berg2022, Introduction: It has been proposed that bilinguals' language use patterns are differentially associated with executive control. To further examine this, the present study relates the social diversity of bilingual language use to performance on a color- shape switching task (CSST) in a group of bilingual university students with diverse linguistic backgrounds. Crucially, this study used language entropy as a measure of bilinguals' language use patterns. This continuous measure reflects a spectrum of language use in a variety of social contexts, ranging from compartmentalized use to fully integrated use. Methods: Language entropy for university and non-university contexts was calculated from questionnaire data on language use. Reaction times (RTs) were measured to calculate global RT and switching and mixing costs on the CSST, representing conflict monitoring, mental set shifting, and goal maintenance, respectively. In addition, this study innovatively recorded a potentially more sensitive measure of set shifting abilities, namely, pupil size during task performance. Results: Higher university entropy was related to slower global RT. Neither university entropy nor non-university entropy were associated with switching costs as manifested in RTs. However, bilinguals with more compartmentalized language use in non-university contexts showed a larger difference in pupil dilation for switch trials in comparison with non-switch trials. Mixing costs in RTs were reduced for bilinguals with higher diversity of language use in non-university contexts. No such effects were found for university entropy. Discussion: These results point to the social diversity of bilinguals' language use as being associated with executive control, but the direction of the effects may depend on social context (university vs. non-university). Importantly, the results also suggest that some of these effects may only be detected by using more sensitive measures, such as pupil dilation. The paper discusses theoretical and practical implications regarding the language entropy measure and the cognitive effects of bilingual experiences more generally, as well as as how methodological choices can advance our understanding of these effects. |
Yevgeni Berzak; Chie Nakamura; Amelia Smith; Emily Weng; Boris Katz; Suzanne Flynn; Roger Levy CELER : A 365-participant corpus of eye movements in L1 and L2 English reading Journal Article In: Open Mind: Discoveries in Cognitive Science, vol. 6, pp. 41–50, 2022. @article{Berzak2022, We present CELER (Corpus of Eye Movements in L1 and L2 English Reading), a broad coverage eye-tracking corpus for English. CELER comprises over 320,000 words, and eye-tracking data from 365 participants. Sixty-nine participants are L1 (first language) speakers, and 296 are L2 (second language) speakers from a wide range of English proficiency levels and five different native language backgrounds. As such, CELER has an order of magnitude more L2 participants than any currently available eye movements dataset with L2 readers. Each participant in CELER reads 156 newswire sentences from the Wall Street Journal (WSJ), in a new experimental design where half of the sentences are shared across participants and half are unique to each participant. We provide analyses that compare L1 and L2 participants with respect to standard reading time measures, as well as the effects of frequency, surprisal, and word length on reading times. These analyses validate the corpus and demonstrate some of its strengths. We envision CELER to enable new types of research on language processing and acquisition, and to facilitate interactions between psycholinguistics and natural language processing (NLP). |
Isha Bhutada; Peggy Skelly; Jonathan Jacobs; Jordan Murray; Aasef G. Shaikh; Fatema F. Ghasia Reading difficulties in amblyopia: Consequence of visual sensory and oculomotor dysfunction Journal Article In: Journal of the Neurological Sciences, vol. 442, pp. 1–13, 2022. @article{Bhutada2022, Introduction: Reading is a vision-reliant task, requiring sequential eye movements. Binocularly discordant input results in visual sensory and oculomotor dysfunction in amblyopia, which may contribute to reading difficulties. This study aims to determine the contributions of fixation eye movement (FEM) abnormalities, clinical type and severity of amblyopia to reading performance under binocular and monocular viewing conditions. Methods: Twenty-three amblyopic patients and nine healthy controls were recruited. Eye movements elicited during fixation and reading of preselected passages were collected for each subject using infrared video- oculography. Subjects were classified as having no nystagmus (n = 9), fusion maldevelopment nystagmus (FMN |
Nicoletta Biondo; Marielena Soilemezidi; Simona Mancini In: Journal of Experimental Psychology: Learning, Memory, and Cognition, vol. 48, no. 7, pp. 1001–1018, 2022. @article{Biondo2022, The ability to think about nonpresent time is a crucial aspect of human cognition. Both the past and future imply a temporal displacement of an event outside the “now.” They also intrinsically differ: The past refers to inalterable events; the future to alterable events, to possible worlds. Are the past and future processed similarly or differently? In this study, we addressed this question by investigating how Spanish speakers process past/future time reference violations during sentence processing, while recording eye movements. We also investigated the role of verbs (in isolation; within sentences) and adverbs (deictic; nondeictic) during time processing. Existing accounts propose that past processing, which requires a link to discourse, is more complex than future processing, which—like the present—is locally bound. Our findings show that past and future processing differs, especially at early stages of verb processing, but this difference is not limited to the presence/absence of discourse linking. We found earlier mismatch effects for past compared to future time reference in incongruous sentences, in line with previous studies. Interestingly, it took longer to categorize the past than the future tense when verbs were presented in isolation. However, it took longer to categorize the future than the past when verbs were presented in congruous sentences, arguably because the future implies alterable worlds. Finally, temporal adverbs were found to play an important role in reinspection and reanalysis triggered by the presence of undefined time frames (nondeictic adverbs) or incongruences (mismatching verbs). |
Stefan Blohm; Stefano Versace; Sanja Methner; Valentin Wagner; Matthias Schlesewsky; Winfried Menninghaus Reading poetry and prose: Eye movements and acoustic evidence Journal Article In: Discourse Processes, vol. 59, no. 3, pp. 159–183, 2022. @article{Blohm2022, We examined genre-specific reading strategies for literary texts and hypothesized that text categorization (literary prose vs. poetry) modulates both how readers gather information from a text (eye movements) and how they realize its phonetic surface form (speech production). We recorded eye movements and speech while college students (N = 32) orally read identical texts that we categorized and formatted as either literary prose or poetry. We further varied the text position of critical regions (text-initial vs. text-medial) to compare how identical information is read and articulated with and without context; this allowed us to assess whether genre-specific reading strategies make differential use of identical context information. We observed genre-dependent differences in reading and speaking tempo that reflected several aspects of reading and articulation. Analyses of regions of interests revealed that word-skipping increased particularly while readers progressed through the texts in the prose condition; speech rhythm was more pronounced in the poetry condition irrespective of the text position. Our results characterize strategic poetry and prose reading, indicate that adjustments of reading behavior partly reflect differences in phonetic surface form, and shed light onto the dynamics of genre-specific literary reading. They generally support a theory of literary comprehension that assumes distinct literary processing modes and incorporates text categorization as an initial processing step. |
Liam P. Blything; Maialen Iraola Azpiroz; Shanley Allen; Regina Hert; Juhani Järvikivi; Juhani Jarvikivi The influence of prominence cues in 7- to 10-year-olds' pronoun resolution: Disentangling order of mention, grammatical role, and semantic role Journal Article In: Journal of Child Language, vol. 49, no. 5, pp. 930–958, 2022. @article{Blything2022, In two visual world experiments we disentangled the influence of order of mention (first vs. second mention), grammatical role (subject vs object), and semantic role (proto-agent vs proto-patient) on 7- to 10-year-olds' real-time interpretation of German pronouns. Children listened to SVO or OVS sentences containing active accusative verbs (küssen "to kiss") in Experiment 1 (N = 72), or dative object-experiencer verbs (gefallen "to like") in Experiment 2 (N = 64). This was followed by the personal pronoun er or the demonstrative pronoun der. Interpretive preferences for er were most robust when high prominence cues (first mention, subject, proto-agent) were aligned onto the same entity; and the same applied to der for low prominence cues (second mention, object, proto-patient). These preferences were reduced in conditions where cues were misaligned, and there was evidence that each cue independently influenced performance. Crucially, individual variation in age predicted adult-like weighting preferences for semantic cues (Schumacher, Roberts & Järvikivi, 2017). |
Arielle Borovsky Developmental changes in how children generalize from their experience to support predictive linguistic processing Journal Article In: Journal of Experimental Child Psychology, vol. 219, pp. 1–21, 2022. @article{Borovsky2022, Prediction is posited to support fluent comprehension of speech—but how and when do young listeners, who encounter unfamiliar and novel events with high frequency, learn to deploy predictive processing strategies in these unfamiliar circumstances? The current work used a discourse-based event teaching paradigm to explore how English-speaking school-aged children (aged 5;0–8;11 [years;months]; N = 92) generalize from their (experimentally controlled) experience to generate real-time linguistic predictions about novel events during an eye-tracked sentence recognition task. The findings reveal developmental differences in how the initial structure of event exposure supports generalization. Specifically, real-time extension was supported by viewing multiple instances of events involving varied agents in the younger children (5–6 years), whereas older children (7–8 years) extended when they experienced repetition of events with identical agents. The findings support accounts of predictive processing suggesting that learners generate predictions in a variety of less predictable circumstances and suggest practical directions to support early learning and language processing skills. |
Emily A. Burg; Tanvi D. Thakkar; Ruth Y. Litovsky Interaural speech asymmetry predicts bilateral speech intelligibility but not listening effort in adults with bilateral cochlear implants Journal Article In: Frontiers in Neuroscience, vol. 16, pp. 1–13, 2022. @article{Burg2022, Introduction: Bilateral cochlear implants (BiCIs) can facilitate improved speech intelligibility in noise and sound localization abilities compared to a unilateral implant in individuals with bilateral severe to profound hearing loss. Still, many individuals with BiCIs do not benefit from binaural hearing to the same extent that normal hearing (NH) listeners do. For example, binaural redundancy, a speech intelligibility benefit derived from having access to duplicate copies of a signal, is highly variable among BiCI users. Additionally, patients with hearing loss commonly report elevated listening effort compared to NH listeners. There is some evidence to suggest that BiCIs may reduce listening effort compared to a unilateral CI, but the limited existing literature has not shown this consistently. Critically, no studies to date have investigated this question using pupillometry to quantify listening effort, where large pupil sizes indicate high effort and small pupil sizes indicate low effort. Thus, the present study aimed to build on existing literature by investigating the potential benefits of BiCIs for both speech intelligibility and listening effort. Methods: Twelve BiCI adults were tested in three listening conditions: Better Ear, Poorer Ear, and Bilateral. Stimuli were IEEE sentences presented from a loudspeaker at 0° azimuth in quiet. Participants were asked to repeat back the sentences, and responses were scored by an experimenter while changes in pupil dilation were measured. Results: On average, participants demonstrated similar speech intelligibility in the Better Ear and Bilateral conditions, and significantly worse speech intelligibility in the Poorer Ear condition. Despite similar speech intelligibility in the Better Ear and Bilateral conditions, pupil dilation was significantly larger in the Bilateral condition. Discussion: These results suggest that the BiCI users tested in this study did not demonstrate binaural redundancy in quiet. The large interaural speech asymmetries demonstrated by participants may have precluded them from obtaining binaural redundancy, as shown by the inverse relationship between the two variables. Further, participants did not obtain a release from effort when listening with two ears versus their better ear only. Instead, results indicate that bilateral listening elicited increased effort compared to better ear listening, which may be due to poor integration of asymmetric inputs. |
Alexander P. Burgoyne; Sari Saba-Sadiya; Lauren Julius Harris; Mark W. Becker; Jan W. Brascamp; David Z. Hambrick Revisiting the self-generation effect in proofreading Journal Article In: Psychological Research, pp. 1–16, 2022. @article{Burgoyne2022, The self-generation effect refers to the finding that people's memory for information tends to be better when they generate it themselves. Counterintuitively, when proofreading, this effect may make it more difficult to detect mistakes in one's own writing than in others' writing. We investigated the self-generation effect and sources of individual differences in proofreading performance in two eye-tracking experiments. Experiment 1 failed to reveal a self-generation effect. Experiment 2 used a studying manipulation to induce overfamiliarity for self-generated text, revealing a weak but non-significant self-generation effect. Overall, word errors (i.e., wrong words) were detected less often than non-word errors (i.e., misspellings), and function word errors were detected less often than content word errors. Fluid intelligence predicted proofreading performance, whereas reading comprehension, working memory capacity, processing speed, and indicators of miserly cognitive processing did not. Students who made more text fixations and spent more time proofreading detected more errors. |
Isabelle Dautriche; Louise Goupil; Kenny Smith; Hugh Rabagliati Two-year-olds' eye movements reflect confidence in their understanding of words Journal Article In: Psychological Science, vol. 33, no. 11, pp. 1842–1856, 2022. @article{Dautriche2022, We studied the fundamental issue of whether children evaluate the reliability of their language interpretation, that is, their confidence in understanding words. In two experiments, 2-year-olds (Experiment 1: N = 50; Experiment 2: N = 60) saw two objects and heard one of them being named; both objects were then hidden behind screens and children were asked to look toward the named object, which was eventually revealed. When children knew the label used, they showed increased postdecision persistence after a correct compared with an incorrect anticipatory look, a marker of confidence in word comprehension (Experiment 1). When interacting with an unreliable speaker, children showed accurate word comprehension but reduced confidence in the accuracy of their own choice, indicating that children's confidence estimates are influenced by social information (Experiment 2). Thus, by the age of 2 years, children can estimate their confidence during language comprehension, long before they can talk about their linguistic skills. |
Catherine Davies; Vincent Porretta; Kremena Koleva; Ekaterini Klepousniotou Speaker-specific cues influence semantic disambiguation Journal Article In: Journal of Psycholinguistic Research, vol. 51, pp. 933–955, 2022. @article{Davies2022, Addressees use information from specific speakers' previous discourse to make predictions about incoming linguistic material and to restrict the choice of potential interpretations. In this way, speaker specificity has been shown to be an influential factor in language processing across several domains e.g., spoken word recognition, sentence processing, and pragmatics. However, its influence on semantic disambiguation has received little attention to date. Using an exposure-test design and visual world eye tracking, we examined the effect of speaker-specific literal vs. nonliteral style on the disambiguation of metaphorical polysemes such as ‘fork', ‘head', and ‘mouse'. Eye movement data revealed that when interpreting polysemous words with a literal and a nonliteral meaning, addressees showed a late-stage preference for the literal meaning in response to a nonliteral speaker. We interpret this as reflecting an indeterminacy in the intended meaning in this condition, as well as the influence of meaning dominance cues at later stages of processing. Response data revealed that addressees then ultimately resolved to the literal target in 90% of trials. These results suggest that addressees consider a range of senses in the earlier stages of processing, and that speaker style is a contextual determinant in semantic processing. |
Charles P. Davis; Inge Marie Eigsti; Roisin Healy; Gitte H. Joergensen; Eiling Yee Autism-spectrum traits in neurotypicals predict the embodiment of manipulation knowledge about object concepts: Evidence from eyetracking Journal Article In: PLoS ONE, vol. 17, no. 7, pp. 1–20, 2022. @article{Davis2022, Sensorimotor-based theories of cognition predict that even subtle developmental motor differences, such as those characterizing autism spectrum disorder (ASD), impact how we represent the meaning of manipulable objects (e.g., faucet). Here, we test 85 neurotypical participants, who varied widely on the Adult Autism Spectrum Quotient (AQ), a measure intended to capture variability in ASD characteristics in the general adult population (participant scores were all below the clinical threshold for autism). Participants completed a visual world eyetracking task designed to assess the activation of conceptual representations of manipulable objects. Participants heard words referring to manually manipulable objects (e.g., faucet) while we recorded their eye movements to arrays of four objects: the named object, a related object typically manipulated similarly (e.g., jar), and two unrelated objects. Consistent with prior work, we observed more looks to the related object than to the unrelated ones (i.e., a manipulation-relatedness effect). This effect likely reflects the overlapping conceptual representations of objects sharing manipulation characteristics (e.g., faucet and jar), due to embodied sensorimotor properties being part of their representations. Critically, we observed—among typically developed young adults—that as AQ scores increased, manipulation-relatedness effects decreased. In contrast, in a visual control condition, in which a target object was paired with related objects of a similar shape (e.g., snake and rope), relatedness effects increased with AQ scores. The results show that AQ scores can predict variation in how object-concept representations are activated for typically developed individuals. More speculatively, they are consistent with the hypothesis that in individuals with ASD, differences in object-concept representations emerge at least in part via differences in sensorimotor experience. |
Megan Elizabeth Deibel; Jocelyn R. Folk Are there individual differences in learning homophones during silent reading? Journal Article In: Journal of Psycholinguistic Research, vol. 51, pp. 1121–1142, 2022. @article{Deibel2022, The present study evaluated if lexical expertise, defined as the quality and quantity of a reader's word representations, influenced college students' ability to learn novel homophones while reading. In two experiments novel homophones (e.g. ‘brale') and novel nonhomophones (e.g. ‘gloobs') were embedded in sentences. In Experiment 1, novel homophones had low-frequency familiar word mates, and in Experiment 2 they had high-frequency familiar word mates. Learning was assessed with meaning and spelling recognition post-tests. Although eye movements during reading did not differ between the word types, participants had more difficulty learning the spellings of the novel homophones compared to the novel nonhomophones in Experiments 1 and 2. In contrast, participants only had difficulty learning the meaning of novel homophones when it had a low-frequency mate. Higher levels of lexical expertise were related to higher learning rates of novel homophone spellings only when the novel homophones had a high-frequency mate. Phonology is activated when novel words are encountered and can interfere with learning under certain circumstances. |
Xizi Deng; Ashley Farris-Trimble; H. Henny Yeung Contextual effects on spoken word processing: An eye-tracking study of the time course of tone and vowel activation in Mandarin Journal Article In: Journal of Experimental Psychology: Learning, Memory, and Cognition, vol. 49, no. 7, pp. 1–16, 2022. @article{Deng2022, Lexical access is highly contextual. For example, vowel (rime) information is prioritized over tone in the lexical access of isolated words in Mandarin Chinese, but these roles are flipped in constraining contexts. The time course of these contextual effects remains unclear, and so here we tracked the real-time eye gaze of native Mandarin speakers in a visual-world paradigm. While listening to a noun classifier, before the target noun was even uttered, gaze to the target noun was already greater than looking to phonologically unrelated distractors. Critically, there was also more distraction from a cohort competitor (tone information) than a segmental competitor (vowel information) in more semantically constraining contexts. Results confirm that phonological activation in Mandarin lexical access is highly sensitive to context, with tone taking priority over vowel information even before a target word is heard. Results suggest that phonological activation in real-time lexical access may be highly context-specific across languages |
Félix Desmeules-Trudel; Marc F. Joanisse Learning unfamiliar words and perceiving non-native vowels in a second language: Insights from eye tracking Journal Article In: Acta Psychologica, vol. 226, pp. 1–11, 2022. @article{DesmeulesTrudel2022, One of the challenges in second-language learning is learning unfamiliar word forms, especially when this involves novel phoneme contrasts. The present study examines how real-time processing of newly-learned words and phonemes in a second language is impacted by the structure of learning (discrimination training) and whether asking participants to complete the same task after a 16–21 h delay favours subsequent word recognition. Specifically, using a visual world eye tracking paradigm, we assessed how English listeners processed newly-learned words containing non-native French front-rounded [y] compared to native-sounding vowels, both immediately after training and the following day. Some learners were forced to discriminate between vowels that are perceptually similar for English listeners, [y]-[u], while others were not. We found significantly better word-level processing on a variety of indices after an overnight delay. We also found that training [y] words paired with [u] words (vs. [y]-Control pairs) led to a greater decrease in reaction times during the word recognition task over the two testing sessions. Discrimination training using perceptually similar sounds had facilitative effects on second language word learning with novel phonemic information, and real-time processing measures such as eyetracking provided valuable insights into how individuals learn words and phonemes in a second language. |
Sara Dhaene; Nicolas Dirix; Hélène Van Marcke; Evy Woumans Moses or Noah? A case of 'potato-potahto' when using a foreign language Journal Article In: Bilingualism: Language and Cognition, vol. 25, pp. 444–458, 2022. @article{Dhaene2022, Research among bilinguals suggests a foreign language effect for various tasks requiring a more systematic processing style. For instance, bilinguals seem less prone to heuristic reasoning when solving problem statements in their foreign (FL) as opposed to their native (NL) language. The present study aimed to determine whether such an effect might also be observed in the detection of semantic anomalies. Participants were presented NL and FL ques- tions with and without anomalies while their eye movements were recorded. Overall, they failed to detect the anomaly in more than half of the trials. Furthermore, more illusions occurred for questions presented in the FL, indicating an FL disadvantage. Additionally, eye movement analyses suggested that reading patterns for anomalies are predominantly similar across languages. Our results therefore substantiate theories suggesting that FL use induces cognitive load, causing increased susceptibility to illusions due to partial semantic processing. |
Denis Drieghe; Robert Chan Seem Parafoveal processing of repeated words during reading Journal Article In: Psychonomic Bulletin & Review, vol. 29, pp. 1451–1460, 2022. @article{Drieghe2022, In an eye-tracking experiment during reading, we examined the repetition effect, whereby words that are repeated in the same paragraph receive shorter fixation durations. Target words that were either high-frequency or low-frequency words and of which the parafoveal preview was either correct or with all letters replaced were embedded three times in the same paragraph. Shorter fixation times and higher skipping rates were observed for high-frequency compared to low-frequency words, words for which the parafoveal preview was correct versus incorrect, and as the word was being repeated more often. An interaction between frequency and repetition indicated that the reduction in fixation times due to repetition was more pronounced for low-frequency words. We also observed influences of word repetition on parafoveal processing, as repeated words were skipped more often. An interaction between parafoveal preview and repetition indicated an absent repetition effect when the preview was incorrect, but this effect was short lived, as it was restricted to the first fixation duration on the target word. |
Camille L. Grasso; Johannes C. Ziegler; Jennifer T. Coull; Marie Montant Space-time congruency effects using eye movements during processing of past- and future-related words Journal Article In: Experimental psychology, vol. 69, no. 4, pp. 210–217, 2022. @article{Grasso2022, In Western cultures where people read and write from left to right, time is represented along a spatial continuum that goes from left to right (past to future), known as the mental timeline (MTL). In language, this MTL was supported by space-time congruency effects: People are faster to make lexical decisions to words conveying past or future information when left/right manual responses are compatible with the MTL. Alternatively, in cultures where people read from right to left, space-time congruency effects go in the opposite direction. Such cross-cultural differences suggest that repeated writing and reading dynamic movements are critically involved in the spatial representation of time. In most experiments on the space-time congruency effect, participants use their hand for responding, an effector that is associated to the directionality of writing. To investigate the role of the directionality of reading in the space-time congruency effect, we asked participants to make lateralized eye movements (left or right saccades) to indicate whether stimuli were real words or not (lexical decision). Eye movement responses were slower and higher in amplitude for responses incompatible with the direction of the MTL. These results reinforce the claim that repeated directional reading and writing movements promote the embodiment of time-related words. |
Tuomo Häikiö; Tinja Luotojärvi The effect of syllable-level hyphenation on novel word reading in early Finnish readers: Evidence from eye movements Journal Article In: Scientific Studies of Reading, vol. 26, no. 1, pp. 38–46, 2022. @article{Haeikioe2022, In early Finnish reading instruction, hyphens are used to denote syllable boundaries. However, this practice slows down reading already during the 1st grade. It has been hypothesized that hyphenation forces readers to rely more on phonology than orthography. Since hyphenation highlights the phonology of the word, it may facilitate reading during the very first encounters of the word. To assess whether this is the case, Finnish 1st and 2nd graders read stories about fictional animals while their eye movements were registered. Each story included four occurrences of a novel target (pseudo)word, hyphenated at the syllable level in half of the stories. Target words were read faster with repeated exposure but there were no effects regarding grade or hyphenation. The use of hyphenation does not give rise to enhanced processing of phonology in novel words and is likely to hinder the processes connected to the use of orthography. |
Jarkko Hautala; Ladislao Salmerón; Asko Tolvanen; Otto Loberg; Paavo Leppänen Task-oriented reading efficiency: Interplay of general cognitive ability, task demands, strategies and reading fluency Journal Article In: Reading and Writing, vol. 35, no. 8, pp. 1787–1813, 2022. @article{Hautala2022, The associations among readers' cognitive skills (general cognitive ability, reading skills, and attentional functioning), task demands (easy versus difficult questions), and process measures (total fixation time on relevant and irrelevant paragraphs) was investigated to explain task-oriented reading accuracy and efficiency (number of scores in a given time unit). Structural equation modeling was applied to a large dataset collected with sixth-grade students, which included samples of dysfluent readers and those with attention difficulties. The results are in line with previous findings regarding the dominant role of general cognitive ability in the accuracy of task-oriented reading. However, efficiency in task-oriented reading was mostly explained by the shorter viewing times of both paragraph types (i.e., relevant and irrelevant), which were modestly explained by general cognitive ability and reading fluency. These findings suggest that high efficiency in task orientation is obtained by relying on a selective reading strategy when reading both irrelevant and relevant paragraphs. The selective reading strategy seems to be specifically learned, and this potentially applies to most students, even those with low cognitive abilities. |
Andrea Helo; Ernesto Guerra; Carmen Julia Coloma; María Antonia Reyes; Pia Rämä Objects shape activation during spoken word recognition in preschoolers with typical and atypical language development: An eye-tracking study Journal Article In: Language Learning and Development, vol. 18, no. 3, pp. 324–351, 2022. @article{Helo2022a, Visually situated spoken words activate phonological, visual, and semantic representations guiding overt attention during visual exploration. We compared the activation of these representations in children with and without developmental language disorder (DLD) across four eye-tracking experiments, with a particular focus on visual (shape) representations. Two types of trials were presented in each experiment. In Experiment 1, participants heard a word while seeing (1) an object visually associated with the spoken word (i.e., shape competitor) together with a phonologically related object (i.e., cohort competitor), or (2) a shape competitor with an unrelated object. In Experiment 2 and 3, participants heard a word while seeing (1) a shape competitor with an object semantically related to the spoken word (i.e., semantic competitor), or (2) a shape competitor with an unrelated object. In Experiment 4, children heard a word while seeing a semantic competitor with (1) the visual referent of the spoken or (2) with an unrelated object. The visual context was previewed for three seconds before the spoken word, except for Experiment 2, where it appeared at the onset of the spoken word (i.e., no preview). The results showed that when a preview was provided both groups were equally attracted by cohort and semantic competitors and preferred the shape competitors over the unrelated objects. However, shape preference disappeared in the DLD group when no preview was provided and when the shape competitor was presented with a semantic competitor. Our results indicate that children with DLD have a less efficient retrieval of shape representation during word recognition compared to typically developing children. |
Kristi Hendrickson; Keith Apfelbaum; Claire Goodwin; Christina Blomquist; Kelsey Klein; Bob McMurray The profile of real-time competition in spoken and written word recognition: More similar than different Journal Article In: Quarterly Journal of Experimental Psychology, vol. 75, no. 9, pp. 1653–1673, 2022. @article{Hendrickson2022a, Word recognition occurs across two sensory modalities: auditory (spoken words) and visual (written words). While each faces different challenges, they are often described in similar terms as a competition process by which multiple lexical candidates are activated and compete for recognition. While there is a general consensus regarding the types of words that compete during spoken word recognition, there is less consensus for written word recognition. The present study develops a novel version of the Visual World Paradigm (VWP) to examine written word recognition and uses this to assess the nature of the competitor set during word recognition in both modalities using the same experimental design. For both spoken and written words, we found evidence for activation of onset competitors (cohorts, e.g., cat, cap) and words that contain the same phonemes or letters in reverse order (anadromes, e.g., cat, tack). We found no evidence of activation for rhymes (e.g., cat, hat). The results across modalities were quite similar, with the exception that for spoken words, cohorts were more active than anadromes, whereas for written words activation was similar. These results suggest a common characterisation of lexical similarity across spoken and written words: temporal or spatial order is coarsely coded, and onsets may receive more weight in both systems. However, for spoken words, temporary ambiguity during the moment of processing gives cohorts an additional boost during real-time recognition. |
Kristi Hendrickson; Danielle Ernest The recognition of whispered speech in real-time Journal Article In: Ear & Hearing, vol. 43, no. 2, pp. 554–562, 2022. @article{Hendrickson2022, Objectives: Whispered speech offers a unique set of challenges to speech perception and word recognition. The goals of the present study were twofold: First, to determine how listeners recognize whispered speech. Second, to inform major theories of spoken word recognition by considering how recognition changes when major cues to phoneme identity are reduced or largely absent compared with normal voiced speech. Design: Using eye tracking in the Visual World Paradigm, we examined how listeners recognize whispered speech. After hearing a target word (normal or whispered), participants selected the corresponding image from a display of four - a target (e.g., money), a word that shares sounds with the target at the beginning (cohort competitor, e.g., mother), a word that shares sounds with the target at the end (rhyme competitor, e.g., honey), and a phonologically unrelated word (e.g., whistle). Eye movements to each object were monitored to measure (1) how fast listeners process whispered speech, and (2) how strongly they consider lexical competitors (cohorts and rhymes) as the speech signal unfolds. Results: Listeners were slower to recognize whispered words. Compared with normal speech, listeners displayed slower reaction times to click the target image, were slower to fixate the target, and fixated the target less overall. Further, we found clear evidence that the dynamics of lexical competition are altered during whispered speech recognition. Relative to normal speech, words that overlapped with the target at the beginning (cohorts) displayed slower, reduced, and delayed activation, whereas words that overlapped with the target at the end (rhymes) exhibited faster, more robust, and longer lasting activation. Conclusion: When listeners are confronted with whispered speech, they engage in a "wait-and-see" approach. Listeners delay lexical access, and by the time they begin to consider what word they are hearing, the beginning of the word has largely come and gone, and activation for cohorts is reduced. However, delays in lexical access actually increase consideration of rhyme competitors; the delay pushes lexical activation to a point later in processing, and the recognition system puts more weight on the word-final overlap between the target and the rhyme. |
Inga Hennecke Prepositional constituents in multi-word units: An experimental reading study of the French preposition de Journal Article In: Linguistics, vol. 60, no. 6, pp. 1785–1810, 2022. @article{Hennecke2022, The processing of multi-word units and complex words has been one of the main issues of psycholinguistic research in the last decades. However, there is still no mutual consent on how multi-word units, complex words, and their internal constituents are accessed in language processing. Current models of linguistic theory and language processing generally assume that there is no interconnection between the morphosyntactic information of a lexical unit and itws phonetic realization. Recent studies challenge this assumption and suggest a relationship between the morphosyntactic, lexical, and pragmatic information of specific lexemes or morphemes and the phonetic signal. The present study adds to these current studies in psycholinguistics and morphophonetics by investigating the French preposition de 'of' as a constituent in different construction types. While de occurs regularly as a free lexeme in syntactic structures, it also appears as a bound constituent in lexicalized and grammaticalized constructions. First, this study presents an analysis of French de in eye-tracking data from a reading task with French native speakers. Second, this study presents a statistical analysis of acoustic durations of de from an experimental reading task. The results suggest that the constituent de shows certain peculiarities in its processing and acoustic realization as a constituent in a certain construction type. The results are discussed with regard to current theoretical approaches to the processing of multi-word units, n-grams, and complex words. |
Annina K. Hessel; Sascha Schroeder Word processing difficulty and executive control interactively shape comprehension monitoring in a second language: An eye-tracking study Journal Article In: Reading and Writing, vol. 35, pp. 2287–2312, 2022. @article{Hessel2022, Successful reading comprehension—especially in a second language (L2)—relies on the ability to monitor one's comprehension, that is, to notice comprehension breaks and make repairs. Comprehension monitoring may be limited given effortful word processing but may also be supported through active reading. The current study addresses to what extent word processing difficulty reduces adolescents' ability to monitor their comprehension in their L2, and whether readers can compensate limitations given sufficient executive control. We conducted an eye-tracking experiment in which 34 adolescent L2 learners (aged 13–17 years) read short expository texts containing two within-subject manipulations. First, comprehension monitoring was tested through inconsistencies, for example, when the topic changed from Spanish to Russian vis-à-vis consistent controls. Second, word processing difficulty was altered by inserting either shorter and higher-frequency words such as want, or longer and lower-frequency words such as prefer. We additionally measured participants' executive control. Outcome variables were reading times on the whole texts and the words manipulated for inconsistency and word processing difficulty. We found evidence of successful moment-to-moment monitoring, as visible in adolescents' increased rereading of inconsistent compared to consistent information. We also found that adolescents adapted their monitoring differently to word processing difficulty, depending on their executive control: while adolescents with weaker control reduced their monitoring given higher word processing difficulty, adolescents with stronger control monitored their comprehension more (instead of less) on difficult texts. These findings provide insights into how L2 comprehension monitoring arises in the interplay of lower-level processing load and active reading processes. |
Isabel Orenes; Orlando Espino; Ruth M. J. Byrne Similarities and differences in understanding negative and affirmative counterfactuals and causal assertions: Evidence from eye-tracking Journal Article In: Quarterly Journal of Experimental Psychology, vol. 75, no. 4, pp. 633–651, 2022. @article{Orenes2022, Two eye-tracking experiments compared affirmative and negative counterfactuals, “if she had (not) arrived early, she would (not) have bought roses” and affirmative and negative causal assertions, “Because she arrived (did not arrive) early, she bought (did not buy) roses.” When participants heard a counterfactual, they looked on screen at words corresponding to its conjecture (“roses”), and its presupposed facts (“no roses”), whereas for a causal assertion, they looked only at words corresponding to the facts. For counterfactuals, they looked at the conjecture first, and later the presupposed facts, and at the latter more than the former. The effect was more pronounced for negative counterfactuals than affirmative ones because the negative counterfactual's presupposed facts identify a specific item (“she bought roses”), whereas the affirmative counterfactual's presupposed facts do not (“she did not buy roses”). Hence, when participants were given a binary context, “she did not know whether to buy roses or carnations,” they looked primarily at the presupposed facts for both sorts of counterfactuals. We discuss the implications for theories of negation, the dual meaning of counterfactuals, and their relation to causal assertions. |
Rana Abu-Zhaya; Inbal Arnon; Arielle Borovsky Do children use multi-word information in real-time sentence comprehension? Journal Article In: Cognitive Science, vol. 46, no. 3, pp. 1–32, 2022. @article{AbuZhaya2022, Meaning in language emerges from multiple words, and children are sensitive to multi-word frequency from infancy. While children successfully use cues from single words to generate linguistic predictions, it is less clear whether and how they use multi-word sequences to guide real-time language processing and whether they form predictions on the basis of multi-word information or pairwise associations. We address these questions in two visual-world eye-tracking experiments with 5- to 8-year-old children. In Experiment 1, we asked whether children generate more robust predictions for the sentence-final object of highly frequent sequences (e.g.,“Throw the ball”), compared to less frequent sequences (e.g., “Throw the book”). We further examined if gaze patterns reflect event knowledge or phrasal frequency by comparing the processing of phrases that have the same event structure but differ in multi-word content (e.g.,“Brush your teeth” vs. “Brush her teeth”). In the second study, we employed a training paradigm to ask if children are capable of generating predictions from novel multi-word associations while controlling for the overall frequency of the sequences. While the results of Experiment 1 suggested that children primarily relied on event associations to generate real-time predictions, those of Experiment 2 showed that the same children were able to use recurring novel multi-word sequences to generate real-time linguistic predictions. Together, these findings suggest that children can draw on multi-word information to generate linguistic predictions, in a context-dependent fashion, and highlight the need to account for the influence of multi-word sequences in models of language processing. |
Victoria I. Adedeji; Martin R. Vasilev; Julie A. Kirkby; Timothy J. Slattery Return‑sweep saccades in oral reading Journal Article In: Psychological Research, vol. 86, no. 6, pp. 1804–1815, 2022. @article{Adedeji2022, Recent research on return-sweep saccades has improved our understanding of eye movements when reading paragraphs. However, these saccades, which take our gaze from the end of one line to the start of the next line, have been studied only within the context of silent reading. Articulatory demands and the coordination of the eye–voice span (EVS) at line bounda- ries suggest that the execution of this saccade may be different in oral reading. We compared launch and landing positions of return-sweeps, corrective saccade probability and fixations adjacent to return-sweeps in skilled adult readers while reading paragraphs aloud and silently. Compared to silent reading, return-sweeps were launched from closer to the end of the line and landed closer to the start of the next line when reading aloud. The probability of making a corrective saccade was higher for oral reading than silent reading. These indicate that oral reading may compel readers to rely more on foveal processing at the expense of parafoveal processing. We found an interaction between reading modality and fixation type on fixation durations. The reading modality effect (i.e., increased fixation durations in oral compared to silent reading) was greater for accurate line-initial fixations and marginally greater for line-final fixations compared to intra-line fixations. This suggests that readers may use the fixations adjacent to return-sweeps as natural pause locations to modulate the EVS. |
Miriam Aguilar; Pilar Ferré; José A. Hinojosa; José M. Gavilán; Josep Demestre Locality and attachment preferences in preverbal versus post-verbal Relative Clauses Journal Article In: Language, Cognition and Neuroscience, vol. 37, no. 10, pp. 1303–1310, 2022. @article{Aguilar2022, The universality of locality is a long-standing debate that has endured in psycholinguistics in spite of the challenges. The non-local preference of attachment in Relative Clauses (RCs) with double antecedent (DP1-of-DP2-RC) reported in a subset of languages (i.e. Spanish) represented an important challenge that locality-based accounts had to address. The forces responsible for attachment preferences turned out to be multifactorial, with relevant roles for prosody, referentiality, lexical semantics and Pseudo-Relative availability. In the present eye-tracking study, we explore the timing of disambiguation in Spanish DP1-of-DP2-RC structures placed in preverbal and post-verbal positions, while also controlling for the previously mentioned influencing factors. Our results are straightforward: an early processing cost arises when the RC is disambiguated non-locally, irrespective of the position. The implications of this work contribute to a better understanding of parsing processes and suggest that locality is at the centre of the forces that influence RC attachment. |
Rania Al-Aqarbeh; Mohammed Al-Malahmeh Grammatical resumption and dependency processing in Southern Jordanian Arabic Journal Article In: Brill's Journal of Afroasiatic Languages and Linguistics, vol. 14, no. 2, pp. 153–193, 2022. @article{AlAqarbeh2022, This study investigates the sensitivity of grammatical resumption to islands in wh-interrogative and relative clause dependencies in Southern Jordanian Arabic (JA). An offline acceptability judgment task and an eye-tracking reading experiment were conducted. The results reveal that resumption in southern JA exhibits sensitivity to strong islands, such as adjunct islands, in both dependencies. The findings also suggest that the southern JA parser posits a resumptive pronoun (RP) inside islands that allow resumption. However, the parser does not predict an RP inside islands that disallow resumption. Furthermore, quantitative data show that wh-interrogative and relative clause dependencies pattern similarly in their sensitivity to islands. |
Svetlana Alexeeva; Vladislav Zubov; Alena Konina The effect of a dyslexia-specific Cyrillic font, LexiaD, on reading speed: further exploration in adolescents with and without dyslexia Journal Article In: Primenjena Psihologija, vol. 15, no. 2, pp. 199–236, 2022. @article{Alexeeva2022, The current study aims to test the assumption that a specially designed Cyrillic font, LexiaD, can assist adolescents with reading problems and facilitate their reading experience. LexiaD was compared with the widely used Arial font. Two groups of adolescents with dyslexia (N = 34) and without dyslexia (N = 28) silently read 144 sentences from the Russian Sentence Corpus (Laurinavichyute et al., 2019), some of which were presented in LexiaD, and others in Arial, while their eye movements were recorded. LexiaD did not show the desired effect for adolescents at the beginning of the experiment: Arial outperformed it in reading speed in both participant groups. However, by the end of the experiment, LexiaD showed a better performance. Although the speed of the higher-level cognitive processing (e.g., lexical access) in both fonts did not differ significantly, the feature extraction was found to be better in LexiaD than in Arial. Thus, we found some positive effect of LexiaD when participants with and without dyslexia got accustomed to it. A follow-up study with an explicit exposure session is needed to confirm this conclusion. |
Maryam A. AlJassmi; Kayleigh L. Warrington; Victoria A. McGowan; Sarah J. White; Kevin B. Paterson Effects of word predictability on eye movements during Arabic reading Journal Article In: Attention, Perception, and Psychophysics, vol. 84, no. 1, pp. 10–24, 2022. @article{AlJassmi2022, Contextual predictability influences both the probability and duration of eye fixations on words when reading Latinate alphabetic scripts like English and German. However, it is unknown whether word predictability influences eye movements in reading similarly for Semitic languages like Arabic, which are alphabetic languages with very different visual and linguistic characteristics. Such knowledge is nevertheless important for establishing the generality of mechanisms of eye-movement control across different alphabetic writing systems. Accordingly, we investigated word predictability effects in Arabic in two eye-movement experiments. Both produced shorter fixation times for words with high compared to low predictability, consistent with previous findings. Predictability did not influence skipping probabilities for (four- to eight-letter) words of varying length and morphological complexity (Experiment 1). However, it did for short (three- to four-letter) words with simpler structures (Experiment 2). We suggest that word-skipping is reduced, and affected less by contextual predictability, in Arabic compared to Latinate alphabetic reading, because of specific orthographic and morphological characteristics of the Arabic script. |
Nadja Althaus; Sandra Kotzor; Swetlana Schuster; Aditi Lahiri Distinct orthography boosts morphophonological discrimination: Vowel raising in Bengali verb inflections Journal Article In: Cognition, vol. 222, pp. 1–22, 2022. @article{Althaus2022, This study is concerned with how vowel alternation, in combination with and without orthographic reflection of the vowel change, affects lexical access and the discrimination of morphologically related forms. Bengali inflected verb forms provide an ideal test case, since present tense verb forms undergo phonologically conditioned, predictable vowel raising. The mid-to-high alternations, but not the low-to-mid ones, are represented in the orthography. This results in three different cases: items with no change (NoDiff), items with a phonological change not represented in the orthography (PronDiff) and items for which both phonology and orthography change (OrthPronDiff). To determine whether these three cases differ in terms of lexical access and discrimination, we conducted two experiments. Experiment 1 was a cross-modal lexical decision task with auditory primes (1st PERSON and 3rd PERSON forms, e.g. [lekhe] or [likhi]) and visual targets (verbal noun; e.g. [lekha]). Experiment 2 uses eye tracking in a fragment completion task, in which auditory fragments (first syllable of 1st or 3rd PERSON form, e.g. [le-] from [lekhe]) were to be matched to one of two visual targets (full 1st and 3rd PERSON forms, [lekhe] vs. [likhi] in Bengali script). While the lexical decision task, a global measure of lexical access, did not show a difference between the cases, the eye-tracking experiment revealed effects of both phonology and orthography. Discrimination accuracy in the OrthPronDiff condition (vowel alternation represented in the orthography) was high. In the PronDiff condition, where phonologically differing forms are represented by the same graphemes, manual responses were at chance, although eye movements revealed that match and non-match were discriminated. Thus, our results indicate that phonological alternations which are not represented in spelling are difficult to process, whereas having orthographically distinct forms boosts discrimination performance, implying orthographically influenced mental phonological representations. |
Rhona M. Amos; Kilian G. Seeber; Martin J. Pickering Prediction during simultaneous interpreting: Evidence from the visual-world paradigm Journal Article In: Cognition, vol. 220, pp. 1–16, 2022. @article{Amos2022, We report the results of an eye-tracking study which used the Visual World Paradigm (VWP) to investigate the time-course of prediction during a simultaneous interpreting task. Twenty-four L1 French professional conference interpreters and twenty-four L1 French professional translators untrained in simultaneous interpretation listened to sentences in English and interpreted them simultaneously into French while looking at a visual scene. Sentences contained a highly predictable word (e.g., The dentist asked the man to open his mouth a little wider). The visual scene comprised four objects, one of which depicted either the target object (mouth; bouche), an English phonological competitor (mouse; souris), a French phonological competitor (cork; bouchon), or an unrelated word (bone; os). We considered 1) whether interpreters and translators predict upcoming nouns during a simultaneous interpreting task, 2) whether interpreters and translators predict the form of these nouns in English and in French and 3) whether interpreters and translators manifest different predictive behaviour. Our results suggest that both interpreters and translators predict upcoming nouns, but neither group predicts the word-form of these nouns. In addition, we did not find significant differences between patterns of prediction in interpreters and translators. Thus, evidence from the visual-world paradigm shows that prediction takes place in simultaneous interpreting, regardless of training and experience. However, we were unable to establish whether word-form was predicted. |
Filip Andras; Marta Rivera; Teresa Bajo; Paola E. Dussias; Daniela Paolieri Cognate facilitation effect during auditory comprehension of a second language: A visual world eye-tracking study Journal Article In: International Journal of Bilingualism, vol. 26, no. 4, pp. 405–425, 2022. @article{Andras2022, Aims and Objectives: The cognate facilitation effect (CFE) is a robust effect in language production and visual word comprehension, but evidence for CFE during auditory comprehension is still scarce. This study aimed to explore the CFE during auditory comprehension of a second language (L2) while manipulating proficiency in the L2 and cognate type. These two variables are known to influence the CFE. Methodology: Low and highly proficient Spanish–English bilinguals listened to individual words in their L2, English, that shared high, low, and no phonological overlap (PO) with their native language Spanish. We designed a visual world paradigm task that consisted of selecting an image shown as a spoken word unfolded in time while eye movements were recorded. Data and Analysis: Response times revealed a clear CFE in low proficiency bilinguals, while this effect was absent in highly proficient bilinguals. The eye-tracking (ET) data showed late coactivation of low-PO words and, surprisingly, no coactivation of high-PO words in low proficiency bilinguals. Highly proficient bilinguals showed no clear pattern of language coactivation in the ET data. The English monolingual control group showed no effects during the critical time window. Conclusions: These results are interpreted within the framework of L2 processing models. At low levels of proficiency, the PO between translations facilitates access to meaning. On the other hand, highly proficient bilinguals no longer benefit from the PO between translations, at least for concrete and simple nouns. Originality: The findings demonstrate a clear CFE in auditory comprehension. Proficiency in L2 and PO modulated the effect, as shown in both the response time and in the ET data, respectively. Implications: These findings suggest that at low levels of L2 proficiency, learners more easily access the conceptual information if the auditory input is similar to their native language. Nevertheless, as proficiency increases, this facilitation disappears. |
Martín Antúnez; Sara Milligan; Juan Andrés Hernández-Cabrera; Horacio A. Barber; Elizabeth R. Schotter Semantic parafoveal processing in natural reading: Insight from fixation-related potentials & eye movements Journal Article In: Psychophysiology, vol. 59, no. 4, pp. 1–19, 2022. @article{Antunez2022a, Prior research suggests that we may access the meaning of parafoveal words during reading. We explored how semantic-plausibility parafoveal processing takes place in natural reading through the co-registration of eye movements (EM) and fixation-related potentials (FRPs), using the boundary paradigm. We replicated previous evidence of semantic parafoveal processing from highly controlled reading situations, extending their findings to more ecologically valid reading scenarios. Additionally, and exploring the time-course of plausibility preview effects, we found distinct but complementary evidence from EM and FRPs measures. FRPs measures, showing a different trend than EM evidence, revealed that plausibility preview effects may be long-lasting. We highlight the importance of a co-registration set-up in ecologically valid scenarios to disentangle the mechanisms related to semantic-plausibility parafoveal processing. |
Emma Axelsson; Nur Najihah Othman; Nayantara Kansal Temperament and children's accuracy and attention during word learning Journal Article In: Infant Behavior and Development, vol. 69, pp. 1–16, 2022. @article{Axelsson2022a, When hearing a novel word, children typically rule out familiar objects and assume a speaker is referring to a novel object. This strategy is known as fast mapping, and young children use this with a high degree of accuracy. However, not all children engage in fast mapping to the same extent and temperament can play a role. Shyness is associated with poorer fast mapping and less attention to target objects, which is associated with poorer retention (Hilton et al., 2019; Hilton & Westermann, 2017). We further investigated the relationship between temperament and fast mapping by presenting 2.5-year-old children with 8 familiar target fast mapping trials and 4 novel target trials presented twice. We considered two temperamental dimensions: approachability due to its similarity to shyness; and reactivity, which could predict children's capacity to engage during fast mapping. We found an association between approachability and fast mapping accuracy the second time children fast-mapped novel targets, and approachability was associated with greater retention accuracy. Reactivity predicted proportions of target looking during fast mapping with less reactive temperament scores associated with greater focus on targets. This provides support for a relationship between two dimensions of temperament and fast mapping and retention. Approachability may be associated with a further opportunity to fast map and memory for novel words, and/or how willing children are to guess the targets. Reactivity may be associated with the capacity to focus during word learning situations. Different aspects of temperament could have implications for children's capacity to disambiguate and learn words. |
Najla Azaiez; Otto Loberg; Jarmo A. Hämäläinen; Paavo H. T. Leppänen Brain source correlates of speech perception and reading processes in children with and without reading difficulties Journal Article In: Frontiers in Neuroscience, vol. 16, pp. 1–18, 2022. @article{Azaiez2022, Neural correlates in reading and speech processing have been addressed extensively in the literature. While reading skills and speech perception have been shown to be associated with each other, their relationship remains debatable. In this study, we investigated reading skills, speech perception, reading, and their correlates with brain source activity in auditory and visual modalities. We used high-density event-related potentials (ERPs), fixation-related potentials (FRPs), and the source reconstruction method. The analysis was conducted on 12–13-year-old schoolchildren who had different reading levels. Brain ERP source indices were computed from frequently repeated Finnish speech stimuli presented in an auditory oddball paradigm. Brain FRP source indices were also computed for words within sentences presented in a reading task. The results showed significant correlations between speech ERP sources and reading scores at the P100 (P1) time range in the left hemisphere and the N250 time range in both hemispheres, and a weaker correlation for visual word processing N170 FRP source(s) in the posterior occipital areas, in the vicinity of the visual word form areas (VWFA). Furthermore, significant brain-to-brain correlations were found between the two modalities, where the speech brain sources of the P1 and N250 responses correlated with the reading N170 response. The results suggest that speech processes are linked to reading fluency and that brain activations to speech are linked to visual brain processes of reading. These results indicate that a relationship between language and reading systems is present even after several years of exposure to print. |
Chiara Barattieri di San Pietro; Giovanni Girolamo; Claudio Luzzatti; Marco Marelli Agency of subjects and eye movements in schizophrenia spectrum disorders Journal Article In: Journal of Psycholinguistic Research, vol. 51, no. 6, pp. 1371–1391, 2022. @article{BarattieridiSanPietro2022, People with schizophrenia spectrum disorders (SSD) show anomalies in language processing with respect to “who is doing what” in an action. This linguistic behavior is suggestive of an atypical representation of the formal concepts of “Agent” in the lexical representation of a verb, i.e., its thematic grid. To test this hypothesis, we administered a silent-reading task with sentences including a semantic violation of the animacy trait of the grammatical subject to 30 people with SSD and 30 healthy control participants (HCs). When the anomalous grammatical subject was the Agent of the event, a significant increase of Gaze Duration was observed in HCs, but not in SSDs. Conversely, when the anomalous subject was a Theme, SSDs displayed an increased probability of go-back movements, unlike HCs. These results are suggestive of a higher tolerability for anomalous Agents in SSD compared to the normal population. The fact that SSD participants did not show a similar tolerability for anomalous Themes rules out the issue of an attention deficit. We suggest that general communication abilities in SSD might benefit from explicit training on deep linguistic structures. |
Alisa Baron; Katrina Connell; Zenzi M. Griffin Grammatical gender in spoken word recognition in school-age Spanish-English bilingual children Journal Article In: Frontiers in Psychology, vol. 13, pp. 1–10, 2022. @article{Baron2022, This study investigated grammatical gender processing in school-age Spanish-English bilingual children using a visual world paradigm with a 4-picture display where the target noun was heard with a gendered article that was either in a context where all distractor images were the same gender as the target noun (same gender; uninformative) or in a context where all distractor images were the opposite gender than the target noun (different gender; informative). We investigated 32 bilingual children (ages 5;6–8;6) who were exposed to Spanish since infancy and began learning English by school entry. Along with the eye-tracking experiment, all children participated in a standardized language assessment and told narratives in English and Spanish, and parents reported on their child's current Spanish language use. The differential proportion fixations to target (target − averaged distractor fixations) were analyzed in two time regions with linear mixed-effects models (LME). Results show that prior to the target word being spoken, these bilingual children did not use the gendered articles to actively anticipate upcoming nouns. In the subsequent time region (during the noun), it was shown that there are differences in the way they use feminine and masculine articles, with a lack of use of the masculine article and a potential facilitatory use of the feminine article for children who currently use more Spanish than English. This asymmetry in the use of gendered articles in processing is modulated by current Spanish language use and trends with results found for bilingual and second-language learning adults. |
Jan-Louis Kruger; Natalia Wisniewska; Sixin Liao Why subtitle speed matters: Evidence from word skipping and rereading Journal Article In: Applied Psycholinguistics, vol. 43, pp. 211–236, 2022. @article{Kruger2022, High subtitle speed undoubtedly impacts the viewer experience. However, little is known about how fast subtitles might impact the reading of individual words. This article presents new findings on the effect of subtitle speed on viewers' reading behavior using word-based eye-tracking measures with specific attention to word skipping and rereading. In multimodal reading situations such as reading subtitles in video, rereading allows people to correct for oculomotor error or comprehension failure during linguistic processing or integrate words with elements of the image to build a situation model of the video. However, the opportunity to reread words, to read the majority of the words in the subtitle and to read subtitles to completion, is likely to be compromised when subtitles are too fast. Participants watched videos with subtitles at 12, 20, and 28 characters per second (cps) while their eye movements were recorded. It was found that comprehension declined as speed increased. Eye movement records also showed that faster subtitles resulted in more incomplete reading of subtitles. Furthermore, increased speed also caused fewer words to be reread following both horizontal eye movements (likely resulting in reduced lexical processing) and vertical eye movements (which would likely reduce higher-level comprehension and integration). |
Elke B. Lange; Dominik Thiele; Moniek M. Kuijpers Narrative aesthetic absorption in audiobooks is predicted by blink rate and acoustic features Journal Article In: Psychology of Aesthetics, Creativity, and the Arts, vol. 16, no. 1, pp. 110–124, 2022. @article{Lange2022, Narrative aesthetic absorption describes a state in which we focus on the story world of a narrative while becoming less aware of our surroundings and ourselves. It is characterized by dimensions such as focused attention, vivid mental imagery, and emotional engagement. In our study, we investigate narrative aesthetic absorption in the context of listening to audiobooks. We asked participants to evaluate their absorption experience during listening to 56 excerpts of audiobooks, and we recorded their eyes to measure saccadic, pupil, and blinking activity. In addition, we analyzed the acoustic features of the audiobook excerpts. To understand the relationships between absorption in audiobooks, eye movement behavior, trait absorption, and acoustic signatures of audiobooks, we fitted linear mixed effect models predicting the subjective experience of absorption. Our results show that absorption was predicted by decreased blink rate, increased articulation rate of the narrator, and trait absorption. Blink rate and trait absorption also predicted valence and liking of the audiobooks. Articulation rate predicted liking and pitch predicted arousal. Being absorbed during audiobook listening shows high similarity with being absorbed during literary reading but less similarity with feeling absorbed while listening to music. |
Sidney Evaldo Leal; Katerina Lukasova; Maria Teresa Carthery-Goulart; Sandra Maria Aluísio In: Language Resources and Evaluation, vol. 56, pp. 1333–1372, 2022. @article{Leal2022, This article presents RastrOS, a new eye-tracking corpus of eye movement data from university students during silent reading of paragraphs of texts in Brazilian Portuguese (BP). The article shows the potential of the corpus for natural language processing (NLP) using it to evaluate the sentence complexity prediction task in BP and it also focuses on the description of NLP resources and methods developed to create the corpus. Specifically, we present: (i) the method used to select the corpus paragraphs from large corpora, using linguistic metrics and clustering algorithms; (ii) the platform for collecting the Cloze test, which is also responsible for creating the project datasets, and (iii) the hybrid semantic similarity method, based on word embedding models and contextualised word representations, used to generate semantic predictability norms. RastrOS can be downloaded from the open science framework repository with the computational infrastructure mentioned above. Datasets with predictability norms of 393 participants and eye-tracking data of 37 participants are available in the OSF repository for this work (https://osf.io/9jxg3/). |
Sungyoon Lee; Steven Woltering; Christopher Prickett; Qinxin Shi; Huilin Sun; Julie L. Thompson Exploring the associations between reading skills and eye movements in elementary children's silent sentence reading Journal Article In: Reading Psychology, vol. 43, no. 1, pp. 85–103, 2022. @article{Lee2022d, The purpose of this study was to investigate the associations between elementary students' reading skills and their online reading (i.e., real-time reading) behaviors during silent sentence processing. Thirty-five students participated in this study and their eye movements were recorded during sentence reading tasks. The effects of students' reading skills measured by traditional standardized measures were investigated for widely-used eye tracking measures such as first fixation duration, gaze duration, regression path duration, total duration, word skipping, fixation count, and regression frequency. The eye tracking measures were chosen to represent early/late cognitive processes and temporal/spatial gaze behaviors. Linear mixed-effects regression analyses revealed that children's performances in reading skills predict most of the eye tracking measures. |
Hui Li; Kevin B. Paterson; Kayleigh L. Warrington; Xiaolu Wang Insights into the processing of collocations during L2 English reading: Evidence from eye movements Journal Article In: Frontiers in Psychology, vol. 13, pp. 1–8, 2022. @article{Li2022a, We report an eye movement experiment that investigates the effects of collocation strength and contextual predictability on the reading of collocative phrases by L2 English readers. Thirty-eight Chinese English as foreign language learners (EFL) read 40 sentences, each including a specific two-word phrase that was either a strong (e.g., black coffee) or weak (e.g., bitter coffee) adjective-noun collocation and was either highly predictable or unpredictable from the previous sentence context. Eye movement measures showed that L2 reading times for the collocative phrases were sensitive to both collocation strength and contextual predictability. However, an interaction effect between these factors, which appeared relatively late in the eye movement record, additionally revealed that contextual predictability more strongly influenced time spent reading weak compared with strong collocations. This was most likely because the greater familiarity of strong collocations facilitated their integration, even in the absence of strong contextual constraint. We discuss the findings in terms of the value of collocations in second language learning. |
Nan Li; Olaf Dimigen; Werner Sommer; Suiping Wang Parafoveal words can modulate sentence meaning: Electrophysiological evidence from an RSVP-with-flanker task Journal Article In: Psychophysiology, vol. 59, pp. 1–18, 2022. @article{Li2022d, During natural reading, readers can take up some visual information from not-yet-fixated words to the right of the current fixation and it is well-established that this parafoveal preview facilitates the subsequent foveal processing of the word. However, the extraction and integration of word meaning from parafoveal words and their possible influence on the semantic content of the sentence are controversial. In the current study, we recorded event-related potentials (ERPs) in the RSVP-with-flankers paradigm to test whether and how updates of sentential meaning, based only on parafoveal information, may influence the subsequent foveal processing. In Chinese sentences, the congruency of parafoveal and foveal target words with the sentence was orthogonally manipulated. In contrast to previous research, we also controlled for potentially confounding effects of parafoveal-to-foveal repetition priming (identity preview effects) on the N400. Crucially, we found that the classic effect of foveal congruency on the N400 component only appeared when the word in preview had been congruent with sentence meaning; in contrast, there was no N400 as a function of foveal incongruency when the preview word had also been incongruent. These results indicate that sentence meaning rapidly adapts to parafoveal preview, altering the semantic context for the subsequently fixated word. We also show that correct parafoveal preview generally attenuates the N400 once a word is fixated, regardless of congruency. Taken together, our findings underline the highly generative and adaptive framework of language comprehension. |
Sainan Li; Yongsheng Wang; Zebo Lan; Xiaoyuan Yuan; Li Zhang; Guoli Yan Effects of word spacing on children's reading: Evidence from eye movements Journal Article In: Reading and Writing, vol. 35, pp. 1019–1033, 2022. @article{Li2022e, Word is important in Chinese reading. However, when inter-word spaces are inserted into Chinese text, there is no facilitation or disruption to adults' reading. Researchers argued that there was a trade-off between word segmentation facilita- tion and disruption due to format unfamiliarity. To assess the trade-off hypothesis, in Experiment 1, we tested Grade 1, 2 and 3 children who have less reading experi- ence than adults and manipulated four spacing conditions: normal unspaced, word spaced, character spaced, and nonword spaced text. In Experiment 2, we collected data from Grade 1 and 3 with the word spaced condition and normal unspaced con- dition. Overall, global analyses from both Experiments consistently showed that word spacing facilitated Grade 1 reading, with much reduced facilitation for higher grade readers; local measures (total reading time and second-pass reading time) replicated the same pattern in which word spacing effects were more pronounced among younger readers. In summary, we obtained greater facilitatory effects of word spacing for younger compared with elder readers, which provides strong evidence for the trade-off hypothesis. |
Wei Li; Hannah Rohde; Martin Corley Veritable untruths: Autistic traits and the processing of deception Journal Article In: Journal of Autism and Developmental Disorders, vol. 52, pp. 4921–4930, 2022. @article{Li2022g, How do we decide whether a statement is literally true? Here, we contrast participants' eventual evaluations of a speaker's meaning with the real-time processes of comprehension. We record participants' eye movements as they respond to potentially misleading instructions to click on one of two objects which might be concealing treasure (the treasure is behind thee, uh, hat). Participants are less likely to click on the named object when the instructions are disfluent. However, when hearing disfluent utterances, a tendency to fixate the named object early increases with participants' autism quotient scores. This suggests that, even where utterances are equivalently understood, the processes by which interpretations are achieved vary across individuals. |
Xiao-Wei Li; Shan Li; Lei Gao; Zi-Bei Niu; Dan-Hui Wang; Man Zeng; Tian-Zhi Li; Xue-Jun Bai; Xiao-Lei Gao Eye movement control in Tibetan reading: The roles of word length and frequency Journal Article In: Brain Sciences, vol. 12, pp. 1–19, 2022. @article{Li2022i, We investigated the effects of word length and frequency on eye movement control during Tibetan reading through two experiments. A preliminary experiment examined the predictive effect of word length and frequency on fixation duration and landing position using multiple linear regression analysis. In the formal experiment, we manipulated the length and frequency of target words simultaneously to investigate the effects of word length and frequency on fixation duration and landing position in Tibetan reading. In this study, we found that: (1) there were significant word-length and word-frequency effects affecting all lexical processing in Tibetan reading; (2) there are preferred viewing locations in Tibetan reading; specifically, for short words, it is the end, whereas for long words, it spans from the center to the beginning of the word; (3) word frequency does not affect preferred viewing location in Tibetan reading; (4) the preferred viewing position and the interaction of word length and viewing position found in this study supported the “strategy-tactics” approach. |
Xinjing Li; Xingshan Li; Qingqing Qu Predicting phonology in language comprehension: Evidence from the visual world eye-tracking task in Mandarin Chinese Journal Article In: Journal of Experimental Psychology: Human Perception and Performance, vol. 48, no. 5, pp. 531–547, 2022. @article{Li2022j, Whereas a growing body of research demonstrates that people may predict conceptual representations during language comprehension, evidence for phonological prediction is less straightforward. Moreover, existing findings of phonological prediction come largely from studies conducted with languages with alphabetic scripts, making it difficult to dissociate the effects of phonology from orthography. In two experiments, we used the visual world paradigm to investigate whether comprehenders predict phonological information during comprehension of Chinese sentences, where phonology and orthography are largely dissociable. Participants listened to sentences containing a highly predictable word while viewing a visual display consisting of a critical object and three distractors (target object, a semantic competitor object, a phonological competitor object, or an unrelated object). We manipulated preview time (i.e., 2 s in Experiment 1 and 1 s in Experiment 2) to investigate how preview time influences the phonological prediction effect. In addition, we used different stimuli to test the robustness of the results. Results showed anticipatory eye movements for semantic competitors: participants fixated more on the semantic competitors than unrelated objects before the onset of predictable target words. Critically, in the two experiments, the results showed anticipatory fixations on phonological competitor objects. These findings provide clear evidence for the preactivation of both semantic and phonological information in sentence comprehension. |
Sixin Liao; Lili Yu; Jan-Louis Kruger; Erik D. Reichle The impact of audio on the reading of intralingual versus interlingual subtitles: Evidence from eye movements Journal Article In: Applied Psycholinguistics, vol. 43, pp. 237–269, 2022. @article{Liao2022, This study investigated how semantically relevant auditory information might affect the reading of subtitles, and if such effects might be modulated by the concurrent video content. Thirty-four native Chinese speakers with English as their second language watched video with English subtitles in six conditions defined by manipulating the nature of the audio (Chinese/L1 audio vs. English/L2 audio vs. no audio) and the presence versus absence of video content. Global eye-movement analyses showed that participants tended to rely less on subtitles with Chinese or English audio than without audio, and the effects of audio were more pronounced in the presence of video presentation. Lexical processing of subtitles was not modulated by the audio. However, Chinese audio, which presumably obviated the need to read the subtitles, resulted in more superficial post-lexical processing of the subtitles relative to either the English or no audio. On the contrary, English audio accentuated post-lexical processing of the subtitles compared with Chinese audio or no audio, indicating that participants might use English audio to support subtitle reading (or vice versa) and thus engaged in deeper processing of the subtitles. These findings suggest that, in multimodal reading situations, eye movements are not only controlled by processing difficulties associated with properties of words (e.g., their frequency and length) but also guided by metacognitive strategies involved in monitoring comprehension and its online modulation by different information sources. |