gaze-model: Kilohertz 2D Eye Tracking Technology Based on Multimodal Particle Filter

gaze-model is an open-source research project that uses scanning laser ophthalmoscopy and multimodal particle filter technology to achieve kilohertz-level 2D eye gaze reconstruction. It solves the spatial aliasing problem of traditional single-hypothesis trackers at high sampling rates and can be applied in fields such as ophthalmic diagnosis, neuroscience research, and human-computer interaction.

Eye tracking technology is widely used in ophthalmic diagnosis, neuroscience research, and human-computer interaction. Traditional methods based on single-hypothesis models (e.g., Kalman filter) are prone to spatial aliasing at high sampling rates, leading to reduced accuracy. Scanning laser ophthalmoscopy can capture high-speed eye images, providing a data foundation for kilohertz tracking, but extracting accurate gaze positions from high-speed scanning data remains a challenge.

Multimodal Particle Filter

The project adopts an innovative multimodal particle filter approach, implemented via an analysis-synthesis framework:

1. Particle Rendering: Each particle renders scan lines from the most recent frame
2. Eye Movement Prior: Propagate the particle cloud
3. 2D Registration: Re-anchor estimates when appearance information is insufficient

Technical Component Architecture

Module	Function Description
filter.py	Multimodal particle filter (prediction/weighting/estimation/resampling)
dynamics.py	Interactive multiple model prior (tracking saccades + saccade main sequence)
decoder.py	Frozen differentiable line renderer (atlas ↔ lines)
likelihood.py	Physical appearance likelihood (alias: perceptual score)
khz2d_methods.py	M0–M5 candidate methods and benchmarking framework
losses.py, train.py	Self-supervised loss + optional learned likelihood

The project includes a complete experimental framework:

• docs/make_figures.py: Regenerate document figures
• results/: Contains reports and figures for each gaze point
• The benchmarking framework supports comparison of multiple candidate methods from M0 to M5

Current Status

The project is in the research preprint stage. Real data figures are verified through self-consistency and consistency with independent trackers (necessary but not sufficient). Absolute accuracy verification needs to be completed with human eyes (future direction).

Data Description

Raw collected data, per-person atlas, cache, and result videos are not included in the repository (see .gitignore). The repository only contains source code, project pages, and lightweight result summaries/figures.

Practical Application Value

• Ophthalmic Diagnosis: Capture subtle eye movements such as microsaccades, provide precise eye movement indicators, and support fine visual assessment
• Neuroscience: Study the relationship between eye movements and cognition, analyze reading gaze patterns, and explore attention allocation
• Human-Computer Interaction: Ultra-low-latency gaze interaction, natural interface control, and gaze input for assistive technologies

Technical Insights and Outlook

The project demonstrates the potential of combining classical signal processing (particle filter) with ophthalmic imaging. It is open-source, providing a reproducible foundation, a benchmark for method comparison, and expansion possibilities. After completing absolute accuracy verification, it is expected to be applied in clinical ophthalmology and high-end human-computer interaction systems.