# gaze-model: Kilohertz 2D Eye Tracking Technology Based on Multimodal Particle Filter

> An open-source research project that uses scanning laser ophthalmoscopy and particle filter technology to achieve high-speed 2D eye gaze reconstruction, solving the spatial aliasing problem of traditional single-hypothesis trackers at high sampling rates.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-06-11T23:55:09.000Z
- 最近活动: 2026-06-12T00:21:29.717Z
- 热度: 139.6
- 关键词: gaze tracking, particle filter, ophthalmoscope, eye tracking, computer vision, signal processing, neuroscience
- 页面链接: https://www.zingnex.cn/en/forum/thread/gaze-model
- Canonical: https://www.zingnex.cn/forum/thread/gaze-model
- Markdown 来源: floors_fallback

---

## gaze-model: Guide to Kilohertz 2D Eye Tracking Technology Based on Multimodal Particle Filter

gaze-model is an open-source research project that uses scanning laser ophthalmoscopy and multimodal particle filter technology to achieve kilohertz-level 2D eye gaze reconstruction. It solves the spatial aliasing problem of traditional single-hypothesis trackers at high sampling rates and can be applied in fields such as ophthalmic diagnosis, neuroscience research, and human-computer interaction.

## Research Background: Technical Challenges of High-Sampling-Rate Eye Tracking

Eye tracking technology is widely used in ophthalmic diagnosis, neuroscience research, and human-computer interaction. Traditional methods based on single-hypothesis models (e.g., Kalman filter) are prone to spatial aliasing at high sampling rates, leading to reduced accuracy. Scanning laser ophthalmoscopy can capture high-speed eye images, providing a data foundation for kilohertz tracking, but extracting accurate gaze positions from high-speed scanning data remains a challenge.

## Core Technical Innovations and Component Architecture

### Multimodal Particle Filter
The project adopts an innovative multimodal particle filter approach, implemented via an analysis-synthesis framework:
1. Particle Rendering: Each particle renders scan lines from the most recent frame
2. Eye Movement Prior: Propagate the particle cloud
3. 2D Registration: Re-anchor estimates when appearance information is insufficient

### Technical Component Architecture
| Module | Function Description |
|--------|---------------------|
| `filter.py` | Multimodal particle filter (prediction/weighting/estimation/resampling) |
| `dynamics.py` | Interactive multiple model prior (tracking saccades + saccade main sequence) |
| `decoder.py` | Frozen differentiable line renderer (atlas ↔ lines) |
| `likelihood.py` | Physical appearance likelihood (alias: perceptual score) |
| `khz2d_methods.py` | M0–M5 candidate methods and benchmarking framework |
| `losses.py`, `train.py` | Self-supervised loss + optional learned likelihood |

## Experimental Framework and Results

The project includes a complete experimental framework:
- `docs/make_figures.py`: Regenerate document figures
- `results/`: Contains reports and figures for each gaze point
- The benchmarking framework supports comparison of multiple candidate methods from M0 to M5

## Project Status and Limitations

### Current Status
The project is in the research preprint stage. Real data figures are verified through self-consistency and consistency with independent trackers (necessary but not sufficient). Absolute accuracy verification needs to be completed with human eyes (future direction).

### Data Description
Raw collected data, per-person atlas, cache, and result videos are not included in the repository (see .gitignore). The repository only contains source code, project pages, and lightweight result summaries/figures.

## Application Value and Future Outlook

### Practical Application Value
- **Ophthalmic Diagnosis**: Capture subtle eye movements such as microsaccades, provide precise eye movement indicators, and support fine visual assessment
- **Neuroscience**: Study the relationship between eye movements and cognition, analyze reading gaze patterns, and explore attention allocation
- **Human-Computer Interaction**: Ultra-low-latency gaze interaction, natural interface control, and gaze input for assistive technologies

### Technical Insights and Outlook
The project demonstrates the potential of combining classical signal processing (particle filter) with ophthalmic imaging. It is open-source, providing a reproducible foundation, a benchmark for method comparison, and expansion possibilities. After completing absolute accuracy verification, it is expected to be applied in clinical ophthalmology and high-end human-computer interaction systems.