InferenceAI: A Production-Grade Full-Stack AI Development Assistant Based on React and FastAPI

Core Overview

InferenceAI is a production-grade AI development assistant built with React + Vite frontend and FastAPI backend. It supports real-time streaming responses, code generation/interpretation/fix, and is compatible with OpenRouter and OpenAI-format APIs. This project aims to provide an out-of-the-box, fully functional, and easy-to-deploy solution for integrating AI into developers' workflows.

With the rapid advancement of LLM capabilities, developers' demand for integrating AI into daily programming workflows is growing. However, existing AI programming assistants are either too simple (only basic Q&A) or too complex (requiring tedious deployment and configuration).

InferenceAI was created to address this gap: it's a production-grade full-stack AI development assistant that combines modern frontend interaction with backend AI capabilities, supporting real-time streaming responses for instant feedback.

Frontend Stack

• React19: Uses concurrency features and automatic batching to optimize rendering.
• TypeScript: Full type coverage for better maintainability.
• Vite: Fast build tool with HMR for efficient development.
• Tailwind CSS: Utility-first framework for modern UIs.
• Zustand: Lightweight state management avoiding Redux complexity.

Backend Stack

• FastAPI: High-performance Python framework with async support and auto API docs.
• httpx: Async HTTP client for LLM API communication.
• Pydantic Settings: Type-safe config management (env vars/config files).
• Uvicorn: High-performance ASGI server (HTTP/2 & WebSocket support).

LLM Integration Default integration with OpenRouter, compatible with any OpenAI API-format services, allowing flexible model provider choices.

Three Interaction Modes

1. Code Generation: AI generates standard code snippets based on user requirements.
2. Code Explanation: Explains selected code's working principle, intent, and notes (useful for learning/reading others' code).
3. Code Fix: Analyzes error code/problems and provides fixes/optimizations.

Real-Time Streaming Response Uses SSE (Server-Sent Events) for streaming: backend via FastAPI's StreamingResponse, frontend via EventSource API. Provides ChatGPT-like smooth experience, allowing users to interrupt during generation.

Session Management & UI

• Left sidebar: Session history (view, switch, delete/rename sessions).
• Theme switch: Dark/light modes.
• Code syntax highlighting for mainstream languages.

Production-Grade Features Rate limiting (anti-abuse), structured logs (monitoring/troubleshooting), type-safe request/response (Pydantic), retry mechanism (network stability), centralized error handling (unified response format).

RESTful API with /api/v1 prefix for versioning. Key endpoints:

Endpoint	Function
POST /api/v1/generate	Code generation
POST /api/v1/explain	Code explanation
POST /api/v1/fix-code	Code fix
POST .../generate/stream	Streaming generation
GET /api/v1/health	Health check

Unified response format: {"status": "success"/"error", "data", "error", "error_code"}.

Local Development

• Backend: cd backend → venv setup → install requirements → copy .env.example → configure OPENROUTER_API_KEY → run.py.
• Frontend: cd frontend → npm install → copy .env.example → npm run dev. Default ports: backend (8000), frontend (5173).

Production Notes

1. CORS: Ensure ALLOWED_ORIGINS includes frontend domain.
2. Env Vars: Vite embeds VITE_API_BASE_URL; use runtime-config.js for dynamic config.
3. API Key Security: Safeguard LLM API keys (use key management services).

Future Plans

• Automated testing: Frontend E2E and backend contract tests.
• Non-stream client: Traditional request-response mode for specific scenarios.
• Auth & multi-tenant: User authentication and multi-tenant API key management.
• Docker Compose: One-click local environment setup.

Summary InferenceAI demonstrates how to build a modern AI app: clear tech selection, reasonable architecture, good UX, and production considerations. It's an excellent reference for developers building similar tools. Licensed under MIT, it's free to use/modify/distribute, and will play an increasingly important role in the developer ecosystem as LLM tech evolves.