# CacheProbe: A Study on Audit Attacks Against Prompt Cache Isolation in LLM Gateway APIs

> This article provides an in-depth analysis of the CacheProbe research, revealing that API gateway architectures like OpenRouter may bypass the prompt cache isolation mechanisms of LLM providers, leading to cross-user data leakage risks. The study proposes a timing attack method targeting prompt caches, which has important warning implications for AI security in multi-tenant environments.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-05-28T22:06:54.000Z
- 最近活动: 2026-06-01T02:48:07.116Z
- 热度: 88.0
- 关键词: LLM安全, 提示缓存, API网关, 时序攻击, 数据隔离, OpenRouter, 侧信道攻击, 多租户安全
- 页面链接: https://www.zingnex.cn/en/forum/thread/cacheprobe-llmapi
- Canonical: https://www.zingnex.cn/forum/thread/cacheprobe-llmapi
- Markdown 来源: floors_fallback

---

## CacheProbe Research Guide: Revealing Prompt Cache Isolation Vulnerabilities in LLM Gateway APIs

### Core of the CacheProbe Research
This article focuses on the prompt cache isolation security issue in LLM gateway APIs, published by Gu et al. at ICML 2025 (arXiv source, released in May 2026). The research reveals that third-party API gateways like OpenRouter may bypass the prompt cache isolation mechanisms of LLM providers, leading to cross-user data leakage risks. The proposed CacheProbe audit method is based on timing attacks, which has important warning implications for AI security in multi-tenant environments.

## Research Background: The Rise of Prompt Caching and Its Security Concerns

Over the past year, prompt caching technology in LLM inference APIs has rapidly gained popularity, reducing latency and cutting costs by reusing KV caches. However, behind this technical convenience lie security risks: when multiple tenants share a cache, time differences between cache hits and misses (timing attacks) or metadata leaks may become channels for data theft.

## Core Issue: Gateway Architecture Undermines Isolation Commitments

Mainstream LLM providers (e.g., OpenAI) implement cache isolation by account/organization, but third-party gateways (e.g., OpenRouter) use shared organizational credentials to access underlying providers, which may lead to global cache sharing—user A's prompt cache may be hit by user B, causing cross-user cache pollution and violating data isolation expectations.

## Attack Principle: Technical Mechanism of CacheProbe

The CacheProbe audit method is based on two key observations:
1. **Timing Side Channel**: Cache hits result in faster responses, while misses are slower; attackers can infer other users' prompt prefixes through time differences.
2. **Metadata Leakage**: Some implementations leak cache status (e.g., hit or miss) via response headers or error messages.

## Experimental Findings: Risk Assessment in OpenRouter Scenarios

Experiments targeting OpenRouter validated three hypotheses:
1. Shared credentials cause the underlying provider to treat all gateway traffic as from the same organization.
2. Insufficient isolation of end-users in cache key logic leads to cross-user sharing.
3. Attackers can detect isolation vulnerabilities via timing analysis. The results show that gateway architectures may weaken or invalidate underlying isolation mechanisms.

## Security Implications: Risks to Be Noted by Stakeholders

- **End Users**: When using LLMs via gateways, prompt prefixes may be probed; sensitive information should not be placed in prefixes, and adding random prefixes is recommended.
- **Gateway Operators**: Need to audit isolation mechanisms, assign independent credentials, implement additional isolation layers, and disclose risks.
- **LLM Providers**: Strengthen cache key logic, provide fine-grained control, and accept third-party audits.

## Defense Recommendations: Multi-Layer Mitigation Measures and Best Practices

### Architecture Level
1. Credential Isolation: Assign independent credentials to users.
2. Cache Namespace Isolation: Embed user identifiers in cache keys.
3. Gateway-Level Caching: Use an independent caching system.
### Application Level
1. Prompt Prefix Randomization.
2. Sensitive Information Postponement.
3. Response Timing Obfuscation.
### Audit and Monitoring
1. Regular audits using CacheProbe.
2. Anomaly query pattern detection.
3. Public disclosure of cache security practices.

## Conclusion and Outlook: Insights for AI Infrastructure Security

CacheProbe proves that timing analysis can detect isolation flaws in multi-tenant environments, reminding us that performance optimization must be considered alongside security. Future directions: Expand to more gateways/proxies, develop automated audit tools, and explore privacy-preserving caching mechanisms (e.g., differential privacy caching).