12-17-2023, 09:46 AM
Unity is a popular video game/virtual world programming environment that is sometimes used in scientific research, including in research involving eye trackers. For such research, it is often important for there to be a data collection/data analysis pipeline that provides the following:
1) the ability to set up and calibrate the eye tracker for each experimental session
2) the ability, at the data analysis stage, to replay a video screen recording of the experimental sessions with gaze data overlaid (so you can see where the participants had been looking while interacting with the Unity-based experimental application)
3) a mechanism for the Unity application to log the position/size of visual stimuli as dynamic interest area information that can be used in the data analysis stage (i.e., so that gaze data can be compared to the dynamic interest areas and so that dependent measures characterizing this gaze/interest area matching can be output in data reports)
4) if needed, a way to access eye tracking data in real time for potential use in the Unity application
The use of WebLink in conjunction with a) a bit of custom C# programming in Unity and b) the use of the Data Viewer analysis software, provides mechanisms for these research requirements (and much more). WebLink natively provides mechanisms for the eye tracker setup/calibration and the video screen recording, and, with its ability to allow other applications, like Unity, to open sockets with it, provides a pipeline of communication with the EyeLink system/Host PC.
The communication pipeline between Unity and the EyeLink system (via WebLink) is bidirectional. Unity can send messages to the EyeLink data file (including event-marking messages and interest area messages) and can send EyeLink commands to control eye tracker settings and perform special Host PC functions. And Unity can receive eye data, in real time, from the Host PC (via WebLink). This communication is all handled by opening sockets with WebLink from C# code in a Unity project. Once those sockets are opened, the Unity application can send/receive information to/from the Host PC, with WebLink providing the direct connection with the EyeLink system and with the Unity application communicating with the EyeLink system, via its open sockets to WebLink.
During a WebLink/Unity session, both WebLink and Unity will run on the stimulus Display PC. You first run your WebLink project to allow eye tracker setup/calibration. Once that is complete, WebLink will give the screen back to the operating system and initiate video screen recording and eye tracker recording and provide synchronization between the screen recording video files and eye tracking data file (EDF). You then launch/run your Unity experiment (it can also be automatically launched by WebLink), which will open sockets with the WebLink software to allow communication between Unity and the EyeLink system to begin. Throughout the session, the Unity script will send event-marking messages, log dynamic interest area information, and/or access the gaze data in real-time (for potential use by the Unity script in gameplay).
A zip file with the following resources can be downloaded from the link below:
EyeLinkWebLinkUnity_Brickbreaker.zip
a) EyeLinkWebLinkUnity_Brickbreaker_WL -- the WebLink example project
b) EyeLinkWebLinkUnity_Brickbreaker_UN -- a modified version of the Brick Breaker Unity game that includes EyeLink/WebLink integration (see below for links to the original, non-EyeLink version, and tutorials covering how it works)
c) GettingStartedWithWebLinkAndUnity.pdf -- a Getting Started guide describing how EyeLink/WebLink/Unity integration works in the context of the modified version of the Brick Breaker example
The Getting Started guide first covers how to setup and run the Unity/WebLink Brick Breaker game included with this guide. It then discusses WebLink project setup and recommended WebLink/Unity project folder structuring. Finally, it covers the specifics of EyeLink communication in the context of the modified version of the Brick Breaker game.
Note, the original, non-EyeLink/WebLink version of the version of the Brick Breaker game is credited to the authors at the links below. The makers of the original game provide an excellent video tutorial describing how it works, so if you are unfamiliar with Unity (or just want to see how the non-EyeLink parts of the game work), that would be a great place to start.
Youtube Tutorial Video (for non-EyeLink aspects of the script) at the following link:
https://youtu.be/RYG8UExRkhA?si=n6mrfCiNWYX03_0C
GitHub page for the original (non-EyeLink) source code:
https://github.com/zigurous/unity-brick-...r-tutorial
This modified EyeLink/Unity version of the game allows for paddle control with the eyes (e.g., if gaze position is to the left of the paddle, the paddle moves left) and logs dynamic interest area information for the paddle and the ball (which can be seen/used in Data Viewer). The Unity project includes an EyeLinkWebLinkUtil class that can be dropped into other Unity projects. That class provides generic functions that can handle details of opening sockets, sending messages/commands, and accessing the EyeLink gaze data. The walkthrough of this EyeLink version of the Brick Breaker game will cover how these functions work generically and how they are used in the context of that game.
1) the ability to set up and calibrate the eye tracker for each experimental session
2) the ability, at the data analysis stage, to replay a video screen recording of the experimental sessions with gaze data overlaid (so you can see where the participants had been looking while interacting with the Unity-based experimental application)
3) a mechanism for the Unity application to log the position/size of visual stimuli as dynamic interest area information that can be used in the data analysis stage (i.e., so that gaze data can be compared to the dynamic interest areas and so that dependent measures characterizing this gaze/interest area matching can be output in data reports)
4) if needed, a way to access eye tracking data in real time for potential use in the Unity application
The use of WebLink in conjunction with a) a bit of custom C# programming in Unity and b) the use of the Data Viewer analysis software, provides mechanisms for these research requirements (and much more). WebLink natively provides mechanisms for the eye tracker setup/calibration and the video screen recording, and, with its ability to allow other applications, like Unity, to open sockets with it, provides a pipeline of communication with the EyeLink system/Host PC.
The communication pipeline between Unity and the EyeLink system (via WebLink) is bidirectional. Unity can send messages to the EyeLink data file (including event-marking messages and interest area messages) and can send EyeLink commands to control eye tracker settings and perform special Host PC functions. And Unity can receive eye data, in real time, from the Host PC (via WebLink). This communication is all handled by opening sockets with WebLink from C# code in a Unity project. Once those sockets are opened, the Unity application can send/receive information to/from the Host PC, with WebLink providing the direct connection with the EyeLink system and with the Unity application communicating with the EyeLink system, via its open sockets to WebLink.
During a WebLink/Unity session, both WebLink and Unity will run on the stimulus Display PC. You first run your WebLink project to allow eye tracker setup/calibration. Once that is complete, WebLink will give the screen back to the operating system and initiate video screen recording and eye tracker recording and provide synchronization between the screen recording video files and eye tracking data file (EDF). You then launch/run your Unity experiment (it can also be automatically launched by WebLink), which will open sockets with the WebLink software to allow communication between Unity and the EyeLink system to begin. Throughout the session, the Unity script will send event-marking messages, log dynamic interest area information, and/or access the gaze data in real-time (for potential use by the Unity script in gameplay).
A zip file with the following resources can be downloaded from the link below:
EyeLinkWebLinkUnity_Brickbreaker.zip
a) EyeLinkWebLinkUnity_Brickbreaker_WL -- the WebLink example project
b) EyeLinkWebLinkUnity_Brickbreaker_UN -- a modified version of the Brick Breaker Unity game that includes EyeLink/WebLink integration (see below for links to the original, non-EyeLink version, and tutorials covering how it works)
c) GettingStartedWithWebLinkAndUnity.pdf -- a Getting Started guide describing how EyeLink/WebLink/Unity integration works in the context of the modified version of the Brick Breaker example
The Getting Started guide first covers how to setup and run the Unity/WebLink Brick Breaker game included with this guide. It then discusses WebLink project setup and recommended WebLink/Unity project folder structuring. Finally, it covers the specifics of EyeLink communication in the context of the modified version of the Brick Breaker game.
Note, the original, non-EyeLink/WebLink version of the version of the Brick Breaker game is credited to the authors at the links below. The makers of the original game provide an excellent video tutorial describing how it works, so if you are unfamiliar with Unity (or just want to see how the non-EyeLink parts of the game work), that would be a great place to start.
Youtube Tutorial Video (for non-EyeLink aspects of the script) at the following link:
https://youtu.be/RYG8UExRkhA?si=n6mrfCiNWYX03_0C
GitHub page for the original (non-EyeLink) source code:
https://github.com/zigurous/unity-brick-...r-tutorial
This modified EyeLink/Unity version of the game allows for paddle control with the eyes (e.g., if gaze position is to the left of the paddle, the paddle moves left) and logs dynamic interest area information for the paddle and the ball (which can be seen/used in Data Viewer). The Unity project includes an EyeLinkWebLinkUtil class that can be dropped into other Unity projects. That class provides generic functions that can handle details of opening sockets, sending messages/commands, and accessing the EyeLink gaze data. The walkthrough of this EyeLink version of the Brick Breaker game will cover how these functions work generically and how they are used in the context of that game.