MRGazer: decoding eye gaze points from functional magnetic resonance imaging in individual space.

Researchers

Journal

Modalities

Models

Abstract

Eye-tracking research has proven valuable in understanding numerous cognitive functions. Recently, Frey et al. provided an exciting deep learning method for learning eye movements from functional magnetic resonance imaging (fMRI) data. It employed the multi-step co-registration of fMRI into the group template to obtain eyeball signal, and thus required additional templates and was time consuming. To resolve this issue, in this paper, we propose a framework named MRGazer for predicting eye gaze points from fMRI in individual space. The MRGazer consists of an eyeball extraction module and a residual network-based eye gaze prediction module. Compared to the previous method, the proposed framework skips the fMRI co-registration step, simplifies the processing protocol, and achieves end-to-end eye gaze regression. The proposed method achieved superior performance in eye fixation regression (Euclidean error, EE=2.04°) than the co-registration-based method (EE=2.89°)， and delivered objective results within a shorter time (~0.02 second/volume) than prior method (~0.3 second/volume). The code is available at https://github.com/ustc-bmec/MRGazer.© 2024 IOP Publishing Ltd.