Mario Avolio's Machine Learning Microservices Portfolio: Engineering Practice of Agentic AI and Generative AI

Mario Avolio's open-source portfolio showcases an elegant solution for integrating generative AI and Agentic AI services via an API gateway pattern. Built on FastAPI, it aggregates independent microservices (like RAG chatbots and multi-agent research tools) into a unified entry point. Key highlights include multi-LLM support, modern Python tooling, and a scalable architecture—making it a valuable reference for AI engineering practices.

The portfolio uses a classic API gateway/service registry pattern. The FastAPI gateway acts as a single entry point, maintaining a service registry to route requests to decoupled microservices (e.g., portfolio-assistant, deep-research). It handles health checks and graceful degradation (returning 503 if services are down). The unified request endpoint is POST /gateway/api/v1/services/{name}/query, which forwards requests to target services and relays responses.

Portfolio Assistant (RAG)

A RAG chatbot for answering career-related questions, using LangChain and Chroma DB. It supports multiple LLM providers: OpenAI (gpt-4.1-nano), Google (gemini-2.0-flash), HuggingFace (zephyr-7b-beta), and Ollama (llama3.2:1b, local inference).

Deep Research (Multi-Agent)

Built with OpenAI Agents SDK, this service enables autonomous deep network research and structured report generation via multi-agent collaboration.

Market Sentinel (Planned)

A future service using CrewAI and SQLite for SWOT competitive intelligence analysis, showing the project's forward-looking roadmap.

Unified Project Structure

Each service follows src/<package> layout with independent pyproject.toml, uv lockfiles, tests, and Dockerfiles—ensuring consistency and ease of maintenance.

Modern Toolchain

Uses uv (high-performance Python package manager) alongside FastAPI, Pydantic, pytest, Docker, and GitHub Actions for CI/CD.

Full Tech Stack

Backend: FastAPI, Pydantic, httpx; AI/ML: LangChain, ChromaDB, OpenAI Agents SDK, CrewAI; Dev Tools: uv, pytest, Docker; Cloud: Google Cloud Run, Cloudflare R2 (Azure planned); Storage: SQLite (DuckDB planned); CI/CD: GitHub Actions.

Development Roadmap

Incremental stages:

1. Gateway, uv workspace, Docker, CI
2. Migrate existing projects to microservices
3. Multi-LLM provider support
4. Cloud storage (S3-compatible)
5. Cloud deployment (Google Cloud Run)
6. Agentic AI services
7. Multi-cloud + data lake (Azure, Parquet, DuckDB)

Practical Value

• Learners: Full AI app lifecycle reference (local to cloud).
• Developers: Scalable API gateway pattern for AI service portfolios.
• Teams: Standardized structure reduces collaboration costs.

Mario Avolio's portfolio exemplifies pragmatic AI engineering—focusing on reliable, maintainable, and scalable delivery of AI capabilities rather than complex algorithms. It serves as an excellent reference for individual developers learning modern AI workflows and teams seeking microservices architecture best practices for AI applications.