PRMStream: Implementation of a Streaming Process Reward Model Based on State Space Models

The PRMStream project aims to address the memory bottleneck of traditional Process Reward Models (PRM). It implements a streaming PRM using State Space Models (SSM) and linear attention mechanisms, providing real-time feedback for large language model reasoning. The original author of the project is hinanohart, source is GitHub (link: https://github.com/hinanohart/prmstream), update time: 2026-05-26T18:12:21Z.

Process Reward Models (PRM) can evaluate the intermediate steps of LLM reasoning, solving issues like credit assignment, low learning efficiency, and lack of interpretability in outcome reward models. They are suitable for scenarios such as mathematical reasoning and code generation. However, traditional PRMs need to store the complete reasoning state, and the self-attention mechanism of the Transformer architecture has a complexity of O(n²), leading to a sharp increase in memory consumption for long sequences.

PRMStream uses State Space Models (SSM) as its backbone, which has linear complexity, fixed state size, and causal guarantee. It introduces linear attention (codenamed "lattica"), converting attention computation to linear complexity via kernel tricks while retaining the ability to capture long-range dependencies.

Streaming Process: Initialize pre-trained SSM → Incrementally update state → Real-time scoring → Pass state. Bounded Memory: Fixed state dimension, no historical cache, incremental updates. Comparison with Traditional PRM: Memory complexity O(1) vs O(n²), supports infinite sequences, streaming scoring, and native long text processing.

SSM Backbone: Based on modern SSM architectures like Mamba. Reward Head: A lightweight network maps the state to a scalar reward and outputs uncertainty estimation. Training Strategy: Addresses temporal consistency and gradient propagation issues, using human-annotated process supervision data.

Suitable for real-time reasoning monitoring (deployed in inference services to monitor generation quality), edge device deployment (resource-constrained environments like mobile devices), and ultra-long text processing (scenarios such as document analysis and codebase understanding).

Current Limitations: Immature SSM ecosystem, lack of process supervision datasets, no unified evaluation benchmark. Future Directions: Multimodal expansion, adaptive state adjustment, SSM-specific hardware optimization.