LLM Logger：零API密钥的实时大模型推理监控工具

This post introduces LLM Logger, an open-source full-stack developer tool supporting real-time inference monitoring for over 15 mainstream large language models. Key features include no API key configuration needed, tracking of latency, token usage, request status, and a visual dashboard. It aims to solve LLM application debugging pain points and lower the barrier for developers.

With LLM's wide adoption in app development, developers face challenges like lack of visibility into the full lifecycle of requests (latency, token consumption, response quality), need for self-built logging systems or expensive third-party APM services, and complex API key management for multiple providers. These issues hinder rapid prototyping and teaching. LLM Logger was created to address these by offering a zero-config, open-source solution.

LLM Logger integrates three core modules:

1. Real-time Chat Interface: React-based, supports streaming responses and 15+ models (GPT-4o, Claude, Gemini, Llama3.3, Grok, Mistral, DeepSeek R1/V3) via Puter.js AI gateway (no API keys needed).
2. Auto Logging: Captures each model call and stores in MongoDB (latency, token estimates, request status, input/output preview, request ID; supports filtering/pagination).
3. Visual Dashboard: Provides real-time metrics (success rate, average latency trend, total tokens, requests per minute, model usage distribution).
4. Conversation History: Persists full history for review, continuation, or deletion.
5. PII Desensitization: Optional client-side脱敏 for sensitive info (emails, phone numbers, etc.) in stored previews.

Frontend: React19 + TypeScript6 (code quality), Vite8 (fast dev), Redux Toolkit2 (state management), shadcn/ui + Radix UI + Tailwind CSS v4 (UI), Recharts3 (visualization). Backend: Node.js + Express4 (RESTful API), MongoDB + Mongoose8 (storage/queries), Zod3 (parameter validation). AI Gateway: Integrated Puter.js AI gateway (unified interface for multiple models, zero config—just log in to Puter account).

Environment Prep: Node.js18+, MongoDB instance (local or Atlas free cluster). Backend Setup:

1. cd server → npm install.
2. Create .env: MONGODB_URI (connection string), PORT=3001, NODE_ENV=development.
3. npm run dev (API at http://localhost:3001/api). Frontend Setup:
4. cd client → npm install → npm run dev (runs at http://localhost:5173; log in to Puter account first). Production Deployment: Use npm run build (optimized package) + npm start; frontend can be hosted on Vercel/Netlify, backend needs Node.js env.

LLM Logger applies to:

• Prototype Dev: Compare model response quality, latency, cost to choose suitable ones.
• Prompt Engineering: Analyze prompt-effect to optimize outputs.
• Performance Diagnosis: Use dashboard/logs to locate slow responses or abnormal token consumption.
• Teaching/Demo: No API key config for students to experience multiple models.
• Multi-model A/B Testing: Switch models in same conversation for comparison.

Current limitations:

1. Token Estimation: Uses character count (4 chars ≈1 token) since Puter SDK doesn't expose actual token data (estimates may deviate from real billing).
2. PII Desensitization: Only applies to stored previews; full content still sent to models (avoid sensitive info in prompts).
3. Provider Support: Only Puter.js now (future plans for native provider APIs).
4. Streaming Cancel: Depends on model implementation (some may continue processing after abort).

LLM Logger provides a lightweight yet full-featured monitoring solution for LLM apps. Its zero API key design lowers the trial barrier, allowing developers to focus on app logic instead of key management. As the LLM ecosystem grows, such tools will become essential in the development toolchain to build reliable, efficient AI-driven apps.