ARCHITECT: Building a Real-Time Consistency Engine for LLM Conversations

ARCHITECT is a serverless, installation-free browser-side runtime consistency monitoring system designed to address consistency drift and hallucination issues in large language models (LLMs) during long conversations. Using technologies like TF-IDF+JSD scoring, Kalman filtering, GARCH variance modeling, and Monte Carlo SDE uncertainty bands, the system enables real-time detection and correction of LLM conversations, reducing deployment costs and technical barriers.

Traditional LLM applications rely on single prompt engineering or post-processing validation, but as conversation turns increase, issues like over-accommodation, topic hijacking, and hallucinations tend to arise. Most existing solutions require independent servers or complex configurations, whereas ARCHITECT, as a single-file React component, runs entirely on the client side with zero deployment costs.

ARCHITECT uses multi-layered mathematical modeling to evaluate conversation consistency:

1. TF-IDF+JSD five-dimensional weighted scoring (semantic coherence, topic relevance, etc.);
2. Kalman filtering to smooth consistency trajectories and distinguish between normal fluctuations and trend declines;
3. GARCH(1,1) model to capture volatility clustering and identify phases of sharp fluctuations in response quality;
4. Monte Carlo SDE to generate 50-path uncertainty bands and trigger early warning mechanisms.

The system comes with 7 pre-configured industry scenario templates (default, technology, creativity, research, medical, circuit, custom) to optimize variance tolerance thresholds. Automatic correction mechanisms include:

• Pipeline injection: dynamically append correction instructions;
• Drift gate: limit context length;
• Mute mode: enforce brief responses;
• Session backtracking: restore session state using a 20-turn rolling buffer.

It has 9 built-in signal detection agents: 6 for behavioral anomalies (over-accommodation, topic hijacking, etc.) and 3 for hallucination detection (factual consistency, logical coherence, evidence support). The RAG memory system retrieves historically relevant content and automatically prunes context to maintain relevance.

Experimental features include alternative SDE models (CIR, Heston), custom guardrails, stability panels, etc. The TypeScript SDK has no UI dependencies, provides core functions like computeCoherence and kalmanStep, and multiple modules, supporting use in Node.js or browser environments.

ARCHITECT introduces financial econometrics and control theory into AI conversation management, shifting from passive post-validation to active in-process monitoring. However, this tool is for research purposes and not suitable for clinical/legal scenarios; its metrics are mathematical proxies and require parameter tuning by domain experts.

ARCHITECT demonstrates the application of rigorous mathematical methods in LLM conversation management, with zero deployment costs lowering technical barriers. As LLMs are increasingly applied in critical fields, the importance of such runtime monitoring tools will become more prominent.