Unity is a popular programming environment for creating video games and virtual worlds. It is sometimes used in scientific research, including research involving eye trackers. For such research, it is often important for there to be a pipeline for data collection and analysis that provides the following:
The communication pipeline between Unity and the EyeLink system (via WebLink) is bidirectional. Unity can send messages to the EyeLink data file (including event-marking messages and interest area messages) and can send EyeLink commands to control eye tracker settings and perform special Host PC functions. In the other direction, Unity can receive eye data, in real time, from the Host PC (via WebLink). This communication is all handled by opening sockets with WebLink from C# code in a Unity project. Once those sockets are opened, the Unity application can send and/or receive information to or from the Host PC, with WebLink providing the direct connection with the EyeLink system and with the Unity application communicating with the EyeLink system via its open sockets to WebLink.
During a WebLink/Unity session, both WebLink and Unity will run on the stimulus Display PC. You first run your WebLink project to allow eye tracker setup/calibration. Once that is complete, WebLink will give the screen back to the operating system, initiate video screen recording and eye tracker recording, and provide synchronization between the screen recording video files and the resulting eye-tracking data file (EDF). You then launch/run your Unity experiment (it can also be automatically launched by WebLink), which will open sockets with the WebLink software to allow communication between Unity and the EyeLink system to begin. Throughout the session, the Unity script will send event-marking messages, log dynamic interest area information, and/or access the gaze data in real-time (for potential use by the Unity script in gameplay).
The EyeLinkWebLinkUnity_Brickbreaker.zip example contains the following resources:
Note that the original, non-EyeLink/WebLink version of the Brick Breaker game is credited to the authors at the links below. The makers of the original game provide an excellent video tutorial describing how it works, so if you are unfamiliar with Unity (or just want to see how the non-EyeLink parts of the game work), that would be a great place to start. Additionally their GitHub page contains the original (non-EyeLink) source code and resources.
This modified EyeLink/Unity version of the game allows for paddle control with the eyes (e.g., if gaze position is to the left of the paddle, the paddle moves left) and logs dynamic interest area information for the paddle and the ball (which can be seen and/or used in Data Viewer). The Unity project includes an EyeLinkWebLinkUtil class that can be dropped into other Unity projects. That class provides generic functions that can handle details of opening sockets, sending messages/commands, and accessing the EyeLink gaze data. The walkthrough of this EyeLink version of the Brick Breaker game will cover how these functions work generically and how they are used in the context of that game.
- The ability to set up and calibrate the eye tracker for each experimental session.
- The ability, at the data analysis stage, to replay a video screen recording of the experimental session with gaze data overlaid (so you can see where participants were looking while interacting with the Unity-based experimental application).
- A mechanism for the Unity application to log the position and/or size of visual stimuli as dynamic interest area information that can be used in the data analysis stage (i.e., so that gaze data can be compared to the dynamic interest areas and so that dependent measures characterizing this gaze-to-interest area matching can be output in data reports).
- If needed, a way to access eye-tracking data in real time for potential use in the Unity application.
The communication pipeline between Unity and the EyeLink system (via WebLink) is bidirectional. Unity can send messages to the EyeLink data file (including event-marking messages and interest area messages) and can send EyeLink commands to control eye tracker settings and perform special Host PC functions. In the other direction, Unity can receive eye data, in real time, from the Host PC (via WebLink). This communication is all handled by opening sockets with WebLink from C# code in a Unity project. Once those sockets are opened, the Unity application can send and/or receive information to or from the Host PC, with WebLink providing the direct connection with the EyeLink system and with the Unity application communicating with the EyeLink system via its open sockets to WebLink.
During a WebLink/Unity session, both WebLink and Unity will run on the stimulus Display PC. You first run your WebLink project to allow eye tracker setup/calibration. Once that is complete, WebLink will give the screen back to the operating system, initiate video screen recording and eye tracker recording, and provide synchronization between the screen recording video files and the resulting eye-tracking data file (EDF). You then launch/run your Unity experiment (it can also be automatically launched by WebLink), which will open sockets with the WebLink software to allow communication between Unity and the EyeLink system to begin. Throughout the session, the Unity script will send event-marking messages, log dynamic interest area information, and/or access the gaze data in real-time (for potential use by the Unity script in gameplay).
The EyeLinkWebLinkUnity_Brickbreaker.zip example contains the following resources:
- EyeLinkWebLinkUnity_Brickbreaker_WL -- the WebLink example project
- EyeLinkWebLinkUnity_Brickbreaker_UN -- a modified version of the Brick Breaker Unity game that includes EyeLink/WebLink integration (see below for links to the original, non-EyeLink version, and tutorials covering how it works)
- GettingStartedWithWebLinkAndUnity.pdf -- a Getting Started guide describing how EyeLink/WebLink/Unity integration works in the context of the modified version of the Brick Breaker example
Note that the original, non-EyeLink/WebLink version of the Brick Breaker game is credited to the authors at the links below. The makers of the original game provide an excellent video tutorial describing how it works, so if you are unfamiliar with Unity (or just want to see how the non-EyeLink parts of the game work), that would be a great place to start. Additionally their GitHub page contains the original (non-EyeLink) source code and resources.
This modified EyeLink/Unity version of the game allows for paddle control with the eyes (e.g., if gaze position is to the left of the paddle, the paddle moves left) and logs dynamic interest area information for the paddle and the ball (which can be seen and/or used in Data Viewer). The Unity project includes an EyeLinkWebLinkUtil class that can be dropped into other Unity projects. That class provides generic functions that can handle details of opening sockets, sending messages/commands, and accessing the EyeLink gaze data. The walkthrough of this EyeLink version of the Brick Breaker game will cover how these functions work generically and how they are used in the context of that game.