All 8000+ peer-reviewed EyeLink eye tracking publications (up to 2019) are listed below in alphabetical order based on the first author. You can search the publications library using key words such as Visual Search, Smooth Pursuit, Parkinsons, etc. You can also search for individual author names. Eye tracking studies grouped by research area can be found on the solutions pages. If we missed any EyeLink eye tracking paper, please email us!
Nahid Zokaei; Alexander G Board; Sanjay G Manohar; Anna C Nobre Modulation of the pupillary response by the content of visual working memory Journal Article Proceedings of the National Academy of Sciences, 115 (45), pp. 22802–22810, 2019. @article{Zokaei2019, title = {Modulation of the pupillary response by the content of visual working memory}, author = {Nahid Zokaei and Alexander G Board and Sanjay G Manohar and Anna C Nobre}, doi = {10.1073/pnas.1909959116}, year = {2019}, date = {2019-10-01}, journal = {Proceedings of the National Academy of Sciences}, volume = {115}, number = {45}, pages = {22802--22810}, abstract = {Studies of selective attention during perception have revealed modulation of the pupillary response according to the brightness of task-relevant (attended) vs. -irrelevant (unattended) stimuli within a visual display. As a strong test of top-down modulation of the pupil response by selective attention, we asked whether changes in pupil diameter follow internal shifts of attention to memoranda of visual stimuli of different brightness maintained in working memory, in the absence of any visual stimulation. Across 3 studies, we reveal dilation of the pupil when participants orient attention to the memorandum of a dark grating relative to that of a bright grating. The effect occurs even when the attention-orienting cue is independent of stimulus brightness, and even when stimulus brightness is merely incidental and not required for the working-memory task of judging stimulus orientation. Furthermore, relative dilation and constriction of the pupil occurred dynamically and followed the changing temporal expectation that 1 or the other stimulus would be probed across the retention delay. The results provide surprising and consistent evidence that pupil responses are under top-down control by cognitive factors, even when there is no direct adaptive gain for such modulation, since no visual stimuli were presented or anticipated. The results also strengthen the view of sensory recruitment during working memory, suggesting even activation of sensory receptors. The thought-provoking corollary to our findings is that the pupils provide a reliable measure of what is in the focus of mind, thus giving a different meaning to old proverbs about the eyes being a window to the mind.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Studies of selective attention during perception have revealed modulation of the pupillary response according to the brightness of task-relevant (attended) vs. -irrelevant (unattended) stimuli within a visual display. As a strong test of top-down modulation of the pupil response by selective attention, we asked whether changes in pupil diameter follow internal shifts of attention to memoranda of visual stimuli of different brightness maintained in working memory, in the absence of any visual stimulation. Across 3 studies, we reveal dilation of the pupil when participants orient attention to the memorandum of a dark grating relative to that of a bright grating. The effect occurs even when the attention-orienting cue is independent of stimulus brightness, and even when stimulus brightness is merely incidental and not required for the working-memory task of judging stimulus orientation. Furthermore, relative dilation and constriction of the pupil occurred dynamically and followed the changing temporal expectation that 1 or the other stimulus would be probed across the retention delay. The results provide surprising and consistent evidence that pupil responses are under top-down control by cognitive factors, even when there is no direct adaptive gain for such modulation, since no visual stimuli were presented or anticipated. The results also strengthen the view of sensory recruitment during working memory, suggesting even activation of sensory receptors. The thought-provoking corollary to our findings is that the pupils provide a reliable measure of what is in the focus of mind, thus giving a different meaning to old proverbs about the eyes being a window to the mind. |
Joshua Zonca; Giorgio Coricelli; Luca Polonio Gaze data reveal individual differences in relational representation processes Journal Article Journal of Experimental Psychology: Learning, Memory, and Cognition, 46 (2), pp. 257–279, 2020. @article{Zonca2020, title = {Gaze data reveal individual differences in relational representation processes}, author = {Joshua Zonca and Giorgio Coricelli and Luca Polonio}, doi = {10.1037/xlm0000723}, year = {2020}, date = {2020-01-01}, journal = {Journal of Experimental Psychology: Learning, Memory, and Cognition}, volume = {46}, number = {2}, pages = {257--279}, publisher = {American Psychological Association Inc.}, abstract = {In our everyday life, we often need to anticipate the potential occurrence of events and their consequences. In this context, the way we represent contingencies can determine our ability to adapt to the environment. However, it is not clear how agents encode and organize available knowledge about the future to react to possible states of the world. In the present study, we investigated the process of contingency representation with three eye-tracking experiments. In Experiment 1, we introduced a novel relational-inference task in which participants had to learn and represent conditional rules regulating the occurrence of interdependent future events. A cluster analysis on early gaze data revealed the existence of 2 distinct types of encoders. A group of (sophisticated) participants built exhaustive contingency models that explicitly linked states with each of their potential consequences. Another group of (unsophisticated) participants simply learned binary conditional rules without exploring the underlying relational complexity. Analyses of individual cognitive measures revealed that cognitive reflection is associated with the emergence of either sophisticated or unsophisticated representation behavior. In Experiment 2, we observed that unsophisticated participants switched toward the sophisticated strategy after having received information about its existence, suggesting that representation behavior was modulated by strategy generation mechanisms. In Experiment 3, we showed that the heterogeneity in representation strategy emerges also in conditional reasoning with verbal sequences, indicating the existence of a general disposition in building either sophisticated or unsophisticated models of contingencies.}, keywords = {}, pubstate = {published}, tppubtype = {article} } In our everyday life, we often need to anticipate the potential occurrence of events and their consequences. In this context, the way we represent contingencies can determine our ability to adapt to the environment. However, it is not clear how agents encode and organize available knowledge about the future to react to possible states of the world. In the present study, we investigated the process of contingency representation with three eye-tracking experiments. In Experiment 1, we introduced a novel relational-inference task in which participants had to learn and represent conditional rules regulating the occurrence of interdependent future events. A cluster analysis on early gaze data revealed the existence of 2 distinct types of encoders. A group of (sophisticated) participants built exhaustive contingency models that explicitly linked states with each of their potential consequences. Another group of (unsophisticated) participants simply learned binary conditional rules without exploring the underlying relational complexity. Analyses of individual cognitive measures revealed that cognitive reflection is associated with the emergence of either sophisticated or unsophisticated representation behavior. In Experiment 2, we observed that unsophisticated participants switched toward the sophisticated strategy after having received information about its existence, suggesting that representation behavior was modulated by strategy generation mechanisms. In Experiment 3, we showed that the heterogeneity in representation strategy emerges also in conditional reasoning with verbal sequences, indicating the existence of a general disposition in building either sophisticated or unsophisticated models of contingencies. |
Regine Zopf; Marina Butko; Alexandra Woolgar; Mark A Williams; Anina N Rich Representing the location of manipulable objects in shape-selective occipitotemporal cortex: Beyond retinotopic reference frames? Journal Article Cortex, 106 , pp. 132–150, 2018. @article{Zopf2018, title = {Representing the location of manipulable objects in shape-selective occipitotemporal cortex: Beyond retinotopic reference frames?}, author = {Regine Zopf and Marina Butko and Alexandra Woolgar and Mark A Williams and Anina N Rich}, doi = {10.1016/j.cortex.2018.05.009}, year = {2018}, date = {2018-01-01}, journal = {Cortex}, volume = {106}, pages = {132--150}, abstract = {When interacting with objects, we have to represent their location relative to our bodies. To facilitate bodily reactions, location may be encoded in the brain not just with respect to the retina (retinotopic reference frame), but also in relation to the head, trunk or arm (collectively spatiotopic reference frames). While spatiotopic reference frames for location encoding can be found in brain areas for action planning, such as parietal areas, there is debate about the existence of spatiotopic reference frames in higher-level occipitotemporal visual areas. In an extensive multi-voxel pattern analysis (MVPA) fMRI study using faces, headless bodies and scenes stimuli, Golomb and Kanwisher (2012) did not find evidence for spatiotopic reference frames in shape-selective occipitotemporal cortex. This finding is important for theories of how stimulus location is encoded in the brain. It is possible, however, that their failure to find spatiotopic reference frames is related to their stimuli: we typically do not manipulate faces, headless bodies or scenes. It is plausible that we only represent body-centred location when viewing objects that are typically manipulated. Here, we tested for object location encoding in shape-selective occipitotemporal cortex using manipulable object stimuli (balls and cups) in a MVPA fMRI study. We employed Bayesian analyses to determine sample size and evaluate the sensitivity of our data to test the hypothesis that location can be encoded in a spatiotopic reference frame in shape-selective occipitotemporal cortex over the null hypothesis of no spatiotopic location encoding. We found strong evidence for retinotopic location encoding consistent with previous findings that retinotopic reference frames are common neural representations of object location. In contrast, when testing for spatiotopic encoding, we found evidence that object location information for small manipulable objects is not decodable in relation to the body in shape-selective occipitotemporal cortex. Post-hoc exploratory analyses suggested that spatiotopic aspects might modulate retinotopic location encoding.}, keywords = {}, pubstate = {published}, tppubtype = {article} } When interacting with objects, we have to represent their location relative to our bodies. To facilitate bodily reactions, location may be encoded in the brain not just with respect to the retina (retinotopic reference frame), but also in relation to the head, trunk or arm (collectively spatiotopic reference frames). While spatiotopic reference frames for location encoding can be found in brain areas for action planning, such as parietal areas, there is debate about the existence of spatiotopic reference frames in higher-level occipitotemporal visual areas. In an extensive multi-voxel pattern analysis (MVPA) fMRI study using faces, headless bodies and scenes stimuli, Golomb and Kanwisher (2012) did not find evidence for spatiotopic reference frames in shape-selective occipitotemporal cortex. This finding is important for theories of how stimulus location is encoded in the brain. It is possible, however, that their failure to find spatiotopic reference frames is related to their stimuli: we typically do not manipulate faces, headless bodies or scenes. It is plausible that we only represent body-centred location when viewing objects that are typically manipulated. Here, we tested for object location encoding in shape-selective occipitotemporal cortex using manipulable object stimuli (balls and cups) in a MVPA fMRI study. We employed Bayesian analyses to determine sample size and evaluate the sensitivity of our data to test the hypothesis that location can be encoded in a spatiotopic reference frame in shape-selective occipitotemporal cortex over the null hypothesis of no spatiotopic location encoding. We found strong evidence for retinotopic location encoding consistent with previous findings that retinotopic reference frames are common neural representations of object location. In contrast, when testing for spatiotopic encoding, we found evidence that object location information for small manipulable objects is not decodable in relation to the body in shape-selective occipitotemporal cortex. Post-hoc exploratory analyses suggested that spatiotopic aspects might modulate retinotopic location encoding. |
Eirini Zormpa; Antje S Meyer; Laurel E Brehm Slow naming of pictures facilitates memory for their names Journal Article Psychonomic Bulletin & Review, 26 , pp. 1675–1682, 2019. @article{Zormpa2019, title = {Slow naming of pictures facilitates memory for their names}, author = {Eirini Zormpa and Antje S Meyer and Laurel E Brehm}, doi = {10.3758/s13423-019-01620-x}, year = {2019}, date = {2019-01-01}, journal = {Psychonomic Bulletin & Review}, volume = {26}, pages = {1675--1682}, publisher = {Psychonomic Bulletin & Review}, abstract = {Speakers remember their own utterances better than those of their interlocutors, suggesting that language production is beneficial to memory. This may be partly explained by a generation effect: The act of generating a word is known to lead to a memory advantage (Slamecka & Graf, 1978). In earlier work, we showed a generation effect for recognition of images (Zormpa, Brehm, Hoedemaker, & Meyer, 2019). Here, we tested whether the recognition of their names would also benefit from name generation. Testing whether picture naming improves memory for words was our primary aim, as it serves to clarify whether the representations affected by generation are visual or conceptual/lexical. A secondary aim was to assess the influence of processing time on memory. Fifty-one participants named pictures in three conditions: after hearing the picture name (identity condition), backward speech, or an unrelated word. A day later, recognition memory was tested in a yes/no task. Memory in the backward speech and unrelated conditions, which required generation, was superior to memory in the identity condition, which did not require generation. The time taken by participants for naming was a good predictor of memory, such that words that took longer to be retrieved were remembered better. Importantly, that was the case only when generation was required: In the no-generation (identity) condition, processing time was not related to recognition memory performance. This work has shown that generation affects conceptual/lexical representations, making an important contribution to the understanding of the relationship between memory and language.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Speakers remember their own utterances better than those of their interlocutors, suggesting that language production is beneficial to memory. This may be partly explained by a generation effect: The act of generating a word is known to lead to a memory advantage (Slamecka & Graf, 1978). In earlier work, we showed a generation effect for recognition of images (Zormpa, Brehm, Hoedemaker, & Meyer, 2019). Here, we tested whether the recognition of their names would also benefit from name generation. Testing whether picture naming improves memory for words was our primary aim, as it serves to clarify whether the representations affected by generation are visual or conceptual/lexical. A secondary aim was to assess the influence of processing time on memory. Fifty-one participants named pictures in three conditions: after hearing the picture name (identity condition), backward speech, or an unrelated word. A day later, recognition memory was tested in a yes/no task. Memory in the backward speech and unrelated conditions, which required generation, was superior to memory in the identity condition, which did not require generation. The time taken by participants for naming was a good predictor of memory, such that words that took longer to be retrieved were remembered better. Importantly, that was the case only when generation was required: In the no-generation (identity) condition, processing time was not related to recognition memory performance. This work has shown that generation affects conceptual/lexical representations, making an important contribution to the understanding of the relationship between memory and language. |
Heng Zou; Hermann J Muller; Zhuanghua Shi Non-spatial sounds regulate eye movements and enhance visual search Journal Article Journal of Vision, 12 (5), pp. 2–2, 2012. @article{Zou2012, title = {Non-spatial sounds regulate eye movements and enhance visual search}, author = {Heng Zou and Hermann J Muller and Zhuanghua Shi}, doi = {10.1167/12.5.2}, year = {2012}, date = {2012-01-01}, journal = {Journal of Vision}, volume = {12}, number = {5}, pages = {2--2}, abstract = {Spatially uninformative sounds can enhance visual search when the sounds are synchronized with color changes of the visual target, a phenomenon referred to as "pip-and-pop" effect (van der Burg, Olivers, Bronkhorst, & Theeuwes, 2008). The present study investigated the relationship of this effect to changes in oculomotor scanning behavior induced by the sounds. The results revealed sound events to increase fixation durations upon their occurrence and to decrease the mean number of saccades. More specifically, spatially uninformative sounds facilitated the orientation of ocular scanning away from already scanned display regions not containing a target (Experiment 1) and enhanced search performance even on target-absent trials (Experiment 2). Facilitation was also observed when the sounds were presented 100 ms prior to the target or at random (Experiment 3). These findings suggest that non-spatial sounds cause a general freezing effect on oculomotor scanning behavior, an effect which in turn benefits visual search performance by temporally and spatially extended information sampling.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Spatially uninformative sounds can enhance visual search when the sounds are synchronized with color changes of the visual target, a phenomenon referred to as "pip-and-pop" effect (van der Burg, Olivers, Bronkhorst, & Theeuwes, 2008). The present study investigated the relationship of this effect to changes in oculomotor scanning behavior induced by the sounds. The results revealed sound events to increase fixation durations upon their occurrence and to decrease the mean number of saccades. More specifically, spatially uninformative sounds facilitated the orientation of ocular scanning away from already scanned display regions not containing a target (Experiment 1) and enhanced search performance even on target-absent trials (Experiment 2). Facilitation was also observed when the sounds were presented 100 ms prior to the target or at random (Experiment 3). These findings suggest that non-spatial sounds cause a general freezing effect on oculomotor scanning behavior, an effect which in turn benefits visual search performance by temporally and spatially extended information sampling. |
Tianlong Zu; John Hutson; Lester C Loschky; Sanjay N Rebello Using eye movements to measure intrinsic, extraneous, and germane load in a multimedia learning environment Journal Article Journal of Educational Psychology, pp. 1–15, 2019. @article{Zu2019, title = {Using eye movements to measure intrinsic, extraneous, and germane load in a multimedia learning environment}, author = {Tianlong Zu and John Hutson and Lester C Loschky and Sanjay N Rebello}, doi = {10.1037/edu0000441}, year = {2019}, date = {2019-01-01}, journal = {Journal of Educational Psychology}, pages = {1--15}, abstract = {In a previous study, DeLeeuw and Mayer (2008) found support for the triarchic model of cognitive load (Sweller, Van Merriënboer, & Paas, 1998, 2019) by showing that three different metrics could be used to independently measure 3 hypothesized types of cognitive load: intrinsic, extraneous, and germane. However, 2 of the 3 metrics that the authors used were intrusive in nature because learning had to be stopped momentarily to complete the measures. The current study extends the design of DeLeeuw and Mayer (2008) by investigating whether learners' eye movement behavior can be used to measure the three proposed types of cognitive load without interrupting learning. During a 1-hr experiment, we presented a multimedia lesson explaining the mechanism of electric motors to participants who had low prior knowledge of this topic. First, we replicated the main results of DeLeeuw and Mayer (2008), providing further support for the triarchic structure of cognitive load. Second, we identified eye movement measures that differentiated the three types of cognitive load. These findings were independent of participants' working memory capacity. Together, these results provide further evidence for the triarchic nature of cognitive load (Sweller et al., 1998, 2019), and are a first step toward online measures of cognitive load that could potentially be implemented into computer assisted learning technologies.}, keywords = {}, pubstate = {published}, tppubtype = {article} } In a previous study, DeLeeuw and Mayer (2008) found support for the triarchic model of cognitive load (Sweller, Van Merriënboer, & Paas, 1998, 2019) by showing that three different metrics could be used to independently measure 3 hypothesized types of cognitive load: intrinsic, extraneous, and germane. However, 2 of the 3 metrics that the authors used were intrusive in nature because learning had to be stopped momentarily to complete the measures. The current study extends the design of DeLeeuw and Mayer (2008) by investigating whether learners' eye movement behavior can be used to measure the three proposed types of cognitive load without interrupting learning. During a 1-hr experiment, we presented a multimedia lesson explaining the mechanism of electric motors to participants who had low prior knowledge of this topic. First, we replicated the main results of DeLeeuw and Mayer (2008), providing further support for the triarchic structure of cognitive load. Second, we identified eye movement measures that differentiated the three types of cognitive load. These findings were independent of participants' working memory capacity. Together, these results provide further evidence for the triarchic nature of cognitive load (Sweller et al., 1998, 2019), and are a first step toward online measures of cognitive load that could potentially be implemented into computer assisted learning technologies. |
Sandrine Zufferey; Willem Mak; Liesbeth Degand; Ted Sanders Advanced learners' comprehension of discourse connectives: The role of L1 transfer across on-line and off-line tasks Journal Article Second Language Research, 31 (3), pp. 389–411, 2015. @article{Zufferey2015, title = {Advanced learners' comprehension of discourse connectives: The role of L1 transfer across on-line and off-line tasks}, author = {Sandrine Zufferey and Willem Mak and Liesbeth Degand and Ted Sanders}, doi = {10.1177/0267658315573349}, year = {2015}, date = {2015-01-01}, journal = {Second Language Research}, volume = {31}, number = {3}, pages = {389--411}, abstract = {Discourse connectives are important indicators of textual coherence, and mastering them is an essential part of acquiring a language. In this article, we compare advanced learners' sensitivity to the meaning conveyed by connectives in an off-line grammaticality judgment task and an on-line reading experiment using eye-tracking. We also assess the influence of first language (L1) transfer by comparing learners' comprehension of two non-native-like semantic uses of connectives in English, often produced by learners due to transfer from French and Dutch. Our results indicate that in an off-line task transfer is an important factor accounting for French-and Dutch-speaking learners' non-native-like comprehension of connectives. During on-line processing, however, learners are as sensitive as native speakers to the meaning conveyed by connectives. These results raise intriguing questions regarding explicit vs. implicit knowledge in language learners.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Discourse connectives are important indicators of textual coherence, and mastering them is an essential part of acquiring a language. In this article, we compare advanced learners' sensitivity to the meaning conveyed by connectives in an off-line grammaticality judgment task and an on-line reading experiment using eye-tracking. We also assess the influence of first language (L1) transfer by comparing learners' comprehension of two non-native-like semantic uses of connectives in English, often produced by learners due to transfer from French and Dutch. Our results indicate that in an off-line task transfer is an important factor accounting for French-and Dutch-speaking learners' non-native-like comprehension of connectives. During on-line processing, however, learners are as sensitive as native speakers to the meaning conveyed by connectives. These results raise intriguing questions regarding explicit vs. implicit knowledge in language learners. |
Wietske Zuiderbaan; Ben M Harvey; Serge O Dumoulin Modeling center – surround configurations in population receptive fields using fMRI Journal Article Journal of Vision, 12 (3), pp. 1–15, 2012. @article{Zuiderbaan2012, title = {Modeling center – surround configurations in population receptive fields using fMRI}, author = {Wietske Zuiderbaan and Ben M Harvey and Serge O Dumoulin}, doi = {10.1167/12.3.10.Introduction}, year = {2012}, date = {2012-01-01}, journal = {Journal of Vision}, volume = {12}, number = {3}, pages = {1--15}, abstract = {Antagonistic center–surround configurations are a central organizational principle of our visual system. In visual cortex, stimulation outside the classical receptive field can decrease neural activity and also decrease functional Magnetic Resonance Imaging (fMRI) signal amplitudes. Decreased fMRI amplitudes below baseline—0% contrast—are often referred to as “negative” responses. Using neural model-based fMRI data analyses, we can estimate the region of visual space to which each cortical location responds, i.e., the population receptive field (pRF). Current models of the pRF do not account for a center–surround organization or negative fMRI responses. Here, we extend the pRF model by adding surround suppression. Where the conventional model uses a circular symmetric Gaussian function to describe the pRF, the new model uses a circular symmetric difference-of-Gaussians (DoG) function. The DoG model allows the pRF analysis to capture fMRI signals below baseline and surround suppression. Comparing the fits of the models, an increased variance explained is found for the DoG model. This improvement was predominantly present in V1/2/3 and decreased in later visual areas. The improvement of the fits was particularly striking in the parts of the fMRI signal below baseline. Estimates for the surround size of the pRF show an increase with eccentricity and over visual areas V1/2/3. For the suppression index, which is based on the ratio between the volumes of both Gaussians, we show a decrease over visual areas V1 and V2. Using non-invasive fMRI techniques, this method gives the possibility to examine assumptions about center–surround receptive fields in human subjects.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Antagonistic center–surround configurations are a central organizational principle of our visual system. In visual cortex, stimulation outside the classical receptive field can decrease neural activity and also decrease functional Magnetic Resonance Imaging (fMRI) signal amplitudes. Decreased fMRI amplitudes below baseline—0% contrast—are often referred to as “negative” responses. Using neural model-based fMRI data analyses, we can estimate the region of visual space to which each cortical location responds, i.e., the population receptive field (pRF). Current models of the pRF do not account for a center–surround organization or negative fMRI responses. Here, we extend the pRF model by adding surround suppression. Where the conventional model uses a circular symmetric Gaussian function to describe the pRF, the new model uses a circular symmetric difference-of-Gaussians (DoG) function. The DoG model allows the pRF analysis to capture fMRI signals below baseline and surround suppression. Comparing the fits of the models, an increased variance explained is found for the DoG model. This improvement was predominantly present in V1/2/3 and decreased in later visual areas. The improvement of the fits was particularly striking in the parts of the fMRI signal below baseline. Estimates for the surround size of the pRF show an increase with eccentricity and over visual areas V1/2/3. For the suppression index, which is based on the ratio between the volumes of both Gaussians, we show a decrease over visual areas V1 and V2. Using non-invasive fMRI techniques, this method gives the possibility to examine assumptions about center–surround receptive fields in human subjects. |
Jan Zwickel; Hermann J Müller Eye movements as a means to evaluate and improve robots Journal Article International Journal of Social Robotics, 1 (4), pp. 357–366, 2009. @article{Zwickel2009, title = {Eye movements as a means to evaluate and improve robots}, author = {Jan Zwickel and Hermann J Müller}, doi = {10.1007/s12369-009-0033-3}, year = {2009}, date = {2009-01-01}, journal = {International Journal of Social Robotics}, volume = {1}, number = {4}, pages = {357--366}, abstract = {Abstract With an increase in their capabilities, robots start to play a role in everyday settings. This necessitates a step from a robot-centered (i.e., teaching humans to adapt to robots) to a more human-centered approach (where robots integrate naturally into human activities). Achieving this will increase the effectiveness of robot usage (e.g., shortening the time required for learning), reduce errors, and increase user acceptance. Robotic camera control will play an important role for a more natural and easier-to-interpret behavior, owing to the central importance of gaze in human communication. This study is intended to provide a first step towards improving camera control by a better understanding of human gaze behavior in social situations. To this end, we registered the eye movements of humans watching different types of movies. In all movies, the same two triangles moved around in a self-propelled fashion. However, crucially, some of the movies elicited the attribution of mental states to the triangles, while others did not. This permitted us to directly distinguish eye movement patterns relating to the attribution of mental states in (perceived) social situations, from the patterns in non-social situations. We argue that a better understanding of what characterizes human gaze patterns in social situations will help shape robotic behavior, make it more natural for humans to communicate with robots, and establish joint attention (to certain objects) between humans and robots. In addition, a better understanding of human gaze in social situations will provide a measure for evaluating whether robots are perceived as social agents rather than non-intentional machines. This could help decide which behaviors a robot should display in order to be perceived as a social interaction partner.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Abstract With an increase in their capabilities, robots start to play a role in everyday settings. This necessitates a step from a robot-centered (i.e., teaching humans to adapt to robots) to a more human-centered approach (where robots integrate naturally into human activities). Achieving this will increase the effectiveness of robot usage (e.g., shortening the time required for learning), reduce errors, and increase user acceptance. Robotic camera control will play an important role for a more natural and easier-to-interpret behavior, owing to the central importance of gaze in human communication. This study is intended to provide a first step towards improving camera control by a better understanding of human gaze behavior in social situations. To this end, we registered the eye movements of humans watching different types of movies. In all movies, the same two triangles moved around in a self-propelled fashion. However, crucially, some of the movies elicited the attribution of mental states to the triangles, while others did not. This permitted us to directly distinguish eye movement patterns relating to the attribution of mental states in (perceived) social situations, from the patterns in non-social situations. We argue that a better understanding of what characterizes human gaze patterns in social situations will help shape robotic behavior, make it more natural for humans to communicate with robots, and establish joint attention (to certain objects) between humans and robots. In addition, a better understanding of human gaze in social situations will provide a measure for evaluating whether robots are perceived as social agents rather than non-intentional machines. This could help decide which behaviors a robot should display in order to be perceived as a social interaction partner. |
Jan Zwickel; Melissa L -H Võ How the presence of persons biases eye movements Journal Article Psychonomic Bulletin & Review, 17 (2), pp. 257–262, 2010. @article{Zwickel2010, title = {How the presence of persons biases eye movements}, author = {Jan Zwickel and Melissa L -H V{õ}}, doi = {10.3758/PBR.17.2.257}, year = {2010}, date = {2010-01-01}, journal = {Psychonomic Bulletin & Review}, volume = {17}, number = {2}, pages = {257--262}, abstract = {We investigated modulation of gaze behavior of observers viewing complex scenes that included a person. To assess spontaneous orientation-following, and in contrast to earlier studies, we did not make the person salient via instruction or low-level saliency. Still, objects that were referred to by the orientation of the person were visited earlier, more often, and longer than when they were not referred to. Analysis of fixation sequences showed that the number of saccades to the cued and uncued objects differed only for saccades that started from the head region, but not for saccades starting from a control object or from a body region. We therefore argue that viewing a person leads to an increase in spontaneous following of the person's viewing direction even when the person plays no role in scene understanding and is not made prominent.}, keywords = {}, pubstate = {published}, tppubtype = {article} } We investigated modulation of gaze behavior of observers viewing complex scenes that included a person. To assess spontaneous orientation-following, and in contrast to earlier studies, we did not make the person salient via instruction or low-level saliency. Still, objects that were referred to by the orientation of the person were visited earlier, more often, and longer than when they were not referred to. Analysis of fixation sequences showed that the number of saccades to the cued and uncued objects differed only for saccades that started from the head region, but not for saccades starting from a control object or from a body region. We therefore argue that viewing a person leads to an increase in spontaneous following of the person's viewing direction even when the person plays no role in scene understanding and is not made prominent. |
Jan Zwickel; Mathias Hegele; Marc Grosjean Ocular tracking of biological and nonbiological motion: The effect of instructed agency Journal Article Psychonomic Bulletin & Review, 19 (1), pp. 52–57, 2012. @article{Zwickel2012, title = {Ocular tracking of biological and nonbiological motion: The effect of instructed agency}, author = {Jan Zwickel and Mathias Hegele and Marc Grosjean}, doi = {10.3758/s13423-011-0193-7}, year = {2012}, date = {2012-01-01}, journal = {Psychonomic Bulletin & Review}, volume = {19}, number = {1}, pages = {52--57}, abstract = {Recent findings suggest that visuomotor performance is modulated by people's beliefs about the agency (e.g., animate vs. inanimate) behind the events they perceive. This study investigated the effect of instructed agency on ocular tracking of point-light motions with biological and nonbiological velocity profiles. The motions followed either a relatively simple (ellipse) or a more complex (scribble) trajectory, and agency was manipulated by informing the participants that the motions they saw were either human or computer generated. In line with previous findings, tracking performance was better for biological than for nonbiological motions, and this effect was particularly pronounced for the simpler (elliptical) motions. The biological advantage was also larger for the human than for the computer instruction condition, but only for a measure that captured the predictive component of smooth pursuit. These results suggest that ocular tracking is influenced by the internal forward model people choose to adopt.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Recent findings suggest that visuomotor performance is modulated by people's beliefs about the agency (e.g., animate vs. inanimate) behind the events they perceive. This study investigated the effect of instructed agency on ocular tracking of point-light motions with biological and nonbiological velocity profiles. The motions followed either a relatively simple (ellipse) or a more complex (scribble) trajectory, and agency was manipulated by informing the participants that the motions they saw were either human or computer generated. In line with previous findings, tracking performance was better for biological than for nonbiological motions, and this effect was particularly pronounced for the simpler (elliptical) motions. The biological advantage was also larger for the human than for the computer instruction condition, but only for a measure that captured the predictive component of smooth pursuit. These results suggest that ocular tracking is influenced by the internal forward model people choose to adopt. |
Ariel Zylberberg; Pablo Barttfeld; Mariano Sigman The construction of confidence in a perceptual decision Journal Article Frontiers in Integrative Neuroscience, 6 (September), pp. 1–10, 2012. @article{Zylberberg2012, title = {The construction of confidence in a perceptual decision}, author = {Ariel Zylberberg and Pablo Barttfeld and Mariano Sigman}, doi = {10.3389/fnint.2012.00079}, year = {2012}, date = {2012-01-01}, journal = {Frontiers in Integrative Neuroscience}, volume = {6}, number = {September}, pages = {1--10}, abstract = {Decision-making involves the selection of one out of many possible courses of action. A decision may bear on other decisions, as when humans seek a second medical opinion before undergoing a risky surgical intervention. These "meta-decisions" are mediated by confidence judgments-the degree to which decision-makers consider that a choice is likely to be correct. We studied how subjective confidence is constructed from noisy sensory evidence. The psychophysical kernels used to convert sensory information into choice and confidence decisions were precisely reconstructed measuring the impact of small fluctuations in sensory input. This is shown in two independent experiments in which human participants made a decision about the direction of motion of a set of randomly moving dots, or compared the brightness of a group of fluctuating bars, followed by a confidence report. The results of both experiments converged to show that: (1) confidence was influenced by evidence during a short window of time at the initial moments of the decision, and (2) confidence was influenced by evidence for the selected choice but was virtually blind to evidence for the non-selected choice. Our findings challenge classical models of subjective confidence-which posit that the difference of evidence in favor of each choice is the seed of the confidence signal.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Decision-making involves the selection of one out of many possible courses of action. A decision may bear on other decisions, as when humans seek a second medical opinion before undergoing a risky surgical intervention. These "meta-decisions" are mediated by confidence judgments-the degree to which decision-makers consider that a choice is likely to be correct. We studied how subjective confidence is constructed from noisy sensory evidence. The psychophysical kernels used to convert sensory information into choice and confidence decisions were precisely reconstructed measuring the impact of small fluctuations in sensory input. This is shown in two independent experiments in which human participants made a decision about the direction of motion of a set of randomly moving dots, or compared the brightness of a group of fluctuating bars, followed by a confidence report. The results of both experiments converged to show that: (1) confidence was influenced by evidence during a short window of time at the initial moments of the decision, and (2) confidence was influenced by evidence for the selected choice but was virtually blind to evidence for the non-selected choice. Our findings challenge classical models of subjective confidence-which posit that the difference of evidence in favor of each choice is the seed of the confidence signal. |
Ariel Zylberberg; Manuel Oliva; Mariano Sigman Pupil dilation: A fingerprint of temporal selection during the "Attentional Blink" Journal Article Frontiers in Psychology, 3 (AUG), pp. 1–6, 2012. @article{Zylberberg2012a, title = {Pupil dilation: A fingerprint of temporal selection during the "Attentional Blink"}, author = {Ariel Zylberberg and Manuel Oliva and Mariano Sigman}, doi = {10.3389/fpsyg.2012.00316}, year = {2012}, date = {2012-01-01}, journal = {Frontiers in Psychology}, volume = {3}, number = {AUG}, pages = {1--6}, abstract = {Pupil dilation indexes cognitive events of behavioral relevance, like the storage of information to memory and the deployment of attention. Yet, given the slow temporal response of the pupil dilation, it is not known from previous studies whether the pupil can index cognitive events in the short time scale of ∼100 ms. Here we measured the size of the pupil in the Attentional Blink (AB) experiment, a classic demonstration of attentional limitations in processing rapidly presented stimuli. In the AB, two targets embedded in a sequence have to be reported and the second stimulus is often missed if presented between 200 and 500 ms after the first. We show that pupil dilation can be used as a marker of cognitive processing in AB, revealing both the timing and amount of cognitive processing. Specifically, we found that in the time range where the AB is known to occur: (i) the pupil dilation was delayed, mimicking the pattern of response times in the Psychological Refractory Period (PRP) paradigm, (ii) the amplitude of the pupil was reduced relative to that of larger lags, even for correctly identified targets, and (iii) the amplitude of the pupil was smaller for missed than for correctly reported targets. These results support two-stage theories of the Attentional Blink where a second processing stage is delayed inside the interference regime, and indicate that the pupil dilation can be used as a marker of cognitive processing in the time scale of ∼100 ms. Furthermore, given the known relation between the pupil dilation and the activity of the locus coeruleus, our results also support theories that link the serial stage to the action of a specific neuromodulator, norepinephrine.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Pupil dilation indexes cognitive events of behavioral relevance, like the storage of information to memory and the deployment of attention. Yet, given the slow temporal response of the pupil dilation, it is not known from previous studies whether the pupil can index cognitive events in the short time scale of ∼100 ms. Here we measured the size of the pupil in the Attentional Blink (AB) experiment, a classic demonstration of attentional limitations in processing rapidly presented stimuli. In the AB, two targets embedded in a sequence have to be reported and the second stimulus is often missed if presented between 200 and 500 ms after the first. We show that pupil dilation can be used as a marker of cognitive processing in AB, revealing both the timing and amount of cognitive processing. Specifically, we found that in the time range where the AB is known to occur: (i) the pupil dilation was delayed, mimicking the pattern of response times in the Psychological Refractory Period (PRP) paradigm, (ii) the amplitude of the pupil was reduced relative to that of larger lags, even for correctly identified targets, and (iii) the amplitude of the pupil was smaller for missed than for correctly reported targets. These results support two-stage theories of the Attentional Blink where a second processing stage is delayed inside the interference regime, and indicate that the pupil dilation can be used as a marker of cognitive processing in the time scale of ∼100 ms. Furthermore, given the known relation between the pupil dilation and the activity of the locus coeruleus, our results also support theories that link the serial stage to the action of a specific neuromodulator, norepinephrine. |
Ariel Zylberberg; Daniel M Wolpert; Michael N Shadlen Counterfactual reasoning underlies the learning of priors in decision making Journal Article Neuron, 99 (5), pp. 1083–1097, 2018. @article{Zylberberg2018, title = {Counterfactual reasoning underlies the learning of priors in decision making}, author = {Ariel Zylberberg and Daniel M Wolpert and Michael N Shadlen}, doi = {10.1016/j.neuron.2018.07.035}, year = {2018}, date = {2018-01-01}, journal = {Neuron}, volume = {99}, number = {5}, pages = {1083--1097}, publisher = {The Authors}, abstract = {Accurate decisions require knowledge of prior probabilities (e.g., prevalence or base rate), but it is unclear how prior probabilities are learned in the absence of a teacher. We hypothesized that humans could learn base rates from experience making decisions, even without feedback. Participants made difficult decisions about the direction of dynamic random dot motion. Across blocks of 15–42 trials, the base rate favoring left or right varied. Participants were not informed of the base rate or choice accuracy, yet they gradually biased their choices and thereby increased accuracy and confidence in their decisions. They achieved this by updating knowledge of base rate after each decision, using a counterfactual representation of confidence that simulates a neutral prior. The strategy is consistent with Bayesian updating of belief and suggests that humans represent both true confidence, which incorporates the evolving belief of the prior, and counterfactual confidence, which discounts the prior. Zylberberg et al. show that human decision makers can learn environmental biases from sequences of difficult decisions, without feedback about accuracy, by calculating the belief that the decisions would have been correct in an unbiased environment—a form of counterfactual confidence.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Accurate decisions require knowledge of prior probabilities (e.g., prevalence or base rate), but it is unclear how prior probabilities are learned in the absence of a teacher. We hypothesized that humans could learn base rates from experience making decisions, even without feedback. Participants made difficult decisions about the direction of dynamic random dot motion. Across blocks of 15–42 trials, the base rate favoring left or right varied. Participants were not informed of the base rate or choice accuracy, yet they gradually biased their choices and thereby increased accuracy and confidence in their decisions. They achieved this by updating knowledge of base rate after each decision, using a counterfactual representation of confidence that simulates a neutral prior. The strategy is consistent with Bayesian updating of belief and suggests that humans represent both true confidence, which incorporates the evolving belief of the prior, and counterfactual confidence, which discounts the prior. Zylberberg et al. show that human decision makers can learn environmental biases from sequences of difficult decisions, without feedback about accuracy, by calculating the belief that the decisions would have been correct in an unbiased environment—a form of counterfactual confidence. |