llm-p: A Practical Project for LLM Security API in Python Development Courses

This article introduces the llm-p project, a teaching project designed for Python development courses. It aims to help learners understand security issues in AI service deployment by building a secure LLM API. The project covers core security practices such as authentication and authorization, input validation, and rate limiting, while also teaching web development techniques and fostering developers' security awareness.

Against the backdrop of the booming development of AI applications, more and more developers need to integrate LLM services. However, directly exposing LLM services poses risks such as API key leakage, prompt injection, and abuse. As the first project in Python development courses, llm-p teaches the construction of a secure and controllable LLM API middleware layer through practical methods, helping learners master the key points of secure deployment of AI services in production environments.

The llm-p project implements multi-layer security protection:

1. Authentication and Authorization: API key-based authentication (generation, rotation, revocation, encrypted hash storage), role-based access control (quota limits, endpoint permissions);
2. Input Security: Strict validation (length, format), sensitive word filtering, prompt injection protection;
3. Output Filtering: Desensitization of sensitive information (email, phone number, ID card), content moderation;
4. Rate Limiting: Tiered rate limiting based on IP/API key, supporting burst traffic handling and different user quotas.

Technology Stack: FastAPI (asynchronous web framework), Pydantic (data validation), SQLAlchemy (ORM), Redis (caching/rate limiting); Security Components: JWT (token authentication), bcrypt (password hashing), slowapi (rate limiting); LLM Integration: OpenAI (GPT series), Anthropic (Claude), local models (Ollama); Core API Endpoints: Text generation, chat completion, streaming response, key management; Middleware: Authentication middleware (key verification), rate limiting middleware (Redis counting).

Best Practices:

• Key Management: Use the secrets module to generate random keys, store only SHA-256 hashes, and use secure comparison to prevent timing attacks;
• Input Processing: Remove control characters, limit length, and use regex to detect injection patterns;
• Output Filtering: Use regex to identify PII and desensitize it; Testing: Unit tests (core functions), integration tests (end-to-end flow), security tests (attack scenarios such as brute force cracking and injection).

Containerization: Docker deployment (based on Python 3.11 image, running as non-root user); Environment Configuration: Sensitive information (database connection, keys) managed via environment variables, with sample configurations provided; Monitoring and Logging: Structured JSON logs, integrated with Prometheus metrics (request volume, response time, token generation volume).

Educational Value: Helps learners master core web development concepts (routing, middleware, dependency injection), fosters security awareness for AI applications, and has a clear code structure suitable for learning; Limitations and Improvements: Can add support for more LLM providers, complex conversation management, and content moderation API integration; implement a user management interface for non-technical personnel to operate.

The llm-p project successfully combines LLM API development and security practices, providing an excellent hands-on project for Python learners. It not only teaches technical skills but also emphasizes the importance of security in AI applications, which aligns with the development needs in the current context of AI popularization.