# NEO-ov: A Native Unified Vision Model for End-to-End Learning of Pixel-to-Word Correspondences

> This article proposes NEO-ov, a native vision-language model that learns cross-frame pixel-to-word correspondences end-to-end without external encoders or adapters. Experiments show that the native architecture performs excellently in fine-grained visual perception, verifying the feasibility of a single vision architecture for large-scale applications.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-05-27T17:59:55.000Z
- 最近活动: 2026-05-28T04:54:14.449Z
- 热度: 138.1
- 关键词: 原生视觉模型, 端到端学习, 视觉语言模型, 多图像理解, 视频理解, 像素-词语对应, 开源模型
- 页面链接: https://www.zingnex.cn/en/forum/thread/neo-ov
- Canonical: https://www.zingnex.cn/forum/thread/neo-ov
- Markdown 来源: floors_fallback

---

## [Introduction] NEO-ov: End-to-End Breakthrough of a Native Unified Vision Model

This article introduces NEO-ov, a native vision-language model whose core is to learn cross-frame pixel-to-word correspondences end-to-end without external encoders or adapters. Experiments verify the advantages of this native architecture in fine-grained visual perception and the feasibility of a single vision architecture for large-scale applications.

## Research Background: Limitations of Modular Architectures and Gaps in Native VLM Exploration

Current mainstream vision-language models (VLMs) adopt modular designs, which have limitations such as fragmented pixel-level signals, lack of early interaction, and complex architectures. While native VLMs perform well in single-image tasks, their exploration in complex scenarios like multi-image understanding and video understanding is still insufficient.

## NEO-ov Design: A Native End-to-End Unified Vision Architecture

NEO-ov adheres to the concept of native end-to-end learning, completely eliminating module boundaries and not relying on external encoders, auxiliary adapters, or post-hoc fusion. Its technical features include end-to-end learning of cross-frame and pixel-to-word correspondences, with fine-grained and unified spatiotemporal modeling emerging natively within the model.

## Performance: Fine-Grained Advantages and Multi-Scenario Adaptation

Experimental results show that NEO-ov approaches or reaches the level of modular models in multiple benchmark tests; it performs excellently in fine-grained visual perception scenarios; as a native architecture, it can naturally extend to multi-image and video understanding without additional adaptation mechanisms.

## Training Details and Open-Source Contributions

The training recipe includes large-scale image-text pairs, multi-frame videos, and spatial localization annotation data, divided into three stages: pre-training, multi-task fine-tuning, and reinforcement learning optimization, using techniques such as progressive resolution enhancement. The project has been open-sourced (GitHub link: https://github.com/EvolvingLMMs-Lab/NEO), providing model weights, code, documentation, etc., to support community research.

## Theoretical Significance and Application Prospects

Theoretically, it verifies the large-scale feasibility of the native end-to-end architecture, reveals that fine-grained spatiotemporal modeling can emerge naturally, and that pixel-to-word correspondences can be effectively learned end-to-end. Application prospects include scenarios such as video understanding, spatial intelligence, multi-image analysis, and real-time deployment.

## Limitations and Future Research Directions

Currently, NEO-ov has limitations such as limited scale verification, gaps compared to dedicated models in specific tasks, high computational costs, and strict data requirements. Future directions include expanding model scale, exploring efficient training methods, adapting to more modalities, and hardware optimization.