STRIDE: Active Activation Decision and Sequence Denoising in Streaming Video Understanding

This article introduces the STRIDE framework, which models active activation in streaming video as a structured sequence problem. By iteratively optimizing activation signals through a lightweight masked diffusion module, it significantly improves the quality of "when to respond" decisions, providing a new technical path for real-time video interaction systems. The following sections will discuss background challenges, modeling perspectives, framework details, experimental results, insights, and future directions.

Video Large Language Models (Video-LLMs) already have strong offline reasoning capabilities, but real-world deployment requires streaming perception and active interaction: when video frames arrive online, the system needs to decide "when to respond". Responding too early based on incomplete information is prone to errors, while responding too late misses the optimal timing. Existing systems mostly focus on offline scenarios and lack an active activation mechanism for streaming scenarios, making it impossible to determine whether sufficient information has been accumulated to make reliable responses.

The research team models active activation in streaming video as a structured sequence problem: related events often last for a period of time (e.g., actions, scene transitions), so activation signals should be regarded as sequence patterns across multiple time steps rather than independent frame-level decisions. STRIDE jointly models activation signals on a sliding time window and iteratively updates them when new frames arrive to capture temporal context and improve decision coherence and reliability.

The core of STRIDE (Structured Temporal Refinement with Iterative DEnoising) is a lightweight masked diffusion module, and its architecture includes three components:

1. Video Encoder: Extracts visual features (e.g., TimeSformer, Video Swin Transformer);
2. Activation Interface: Uses a masked diffusion model to generate/optimize activation signal sequences (binary sequences. During training, it denoises from noise to real patterns; during inference, it starts from random noise and performs multi-step denoising, handling unobserved frames via attention masks);
3. Language Decoder: Generates text responses when activated. Iterative Optimization: When new frames arrive, incremental updates are made based on previous predictions to maintain temporal consistency and adjust predictions.

Experiments were conducted on three benchmark tests: StreamingEgo4D (first-person daily activities), StreamingCharades (streaming action recognition), and LiveStreamingQA (real-time question answering). Key results: Activation accuracy is 8-12% higher than the best baseline, activation signals have better temporal coherence, response accuracy (e.g., in StreamingCharades) is about 5% higher, and latency is lower. Ablation studies: Removing the diffusion model → accuracy drops by 10%; disabling iterative optimization → activation signals become incoherent; reducing denoising steps → speed increases but accuracy decreases, with 4-8 steps being the optimal balance.

STRIDE brings three technical insights:

1. Activation decisions should be modeled as sequence problems rather than independent classifications;
2. Generative models (e.g., diffusion models) can be used for discriminative tasks, and progressive optimization is suitable for streaming scenarios;
3. Iterative updates are more suitable for streaming environments than independent predictions, improving efficiency and decision stability.

Limitations: High inference cost of diffusion models limits deployment on resource-constrained devices; only targets the visual modality; activation decisions are based on heuristic thresholds. Future directions: Explore diffusion model acceleration techniques (distillation/quantization); expand to multi-modal streaming scenarios; dynamically adjust activation strategies to achieve fine-grained control.