llm_composer: A New Option for LLM Integration in the Elixir Ecosystem

This article introduces doofinder's open-source llm_composer library, which aims to seamlessly integrate large language models like OpenAI and Ollama into Elixir applications, filling the gap in AI integration within the Elixir ecosystem. Key highlights include multi-backend support, functional API design, and adaptation to BEAM virtual machine features, helping developers introduce AI capabilities while retaining Elixir's advantages.

Elixir, built on the BEAM virtual machine, is known for high concurrency and fault tolerance, but lags behind in AI integration. Maintained by doofinder, llm_composer is an HTTP client library designed specifically for Elixir, providing an extensible and configurable abstraction layer to meet AI integration needs in production environments, rather than being a simple API wrapper.

Key designs of llm_composer:

1. HTTP Communication Model: Compatible with mainstream LLM services, leveraging Elixir's mature HTTP libraries to achieve efficient concurrency;
2. Multi-backend Support: Natively supports OpenAI (commercial) and Ollama (local open-source), with reserved extension interfaces;
3. Functional API: Follows the principles of immutable data and pure functions, supports integration with OTP components, and makes code easy to test and maintain.

Typical application scenarios:

1. Real-time Chat/Customer Service: Combine with Phoenix framework's real-time capabilities to push responses incrementally via streaming;
2. Content Processing Pipeline: Collaborate with Flow/Broadway to build high-throughput text processing (summarization, classification, etc.);
3. Local AI Development: Integrate local models via Ollama to enable zero-cost prototype validation and development for privacy-sensitive scenarios.

The Elixir community's demand for AI is growing, and llm_composer benefits from refinement through business scenarios. Comparison with similar libraries:

• llm_composer: Focuses on multi-backend support and extensibility;
• instructor_ex: Specializes in structured output and function calls;
• openai_ex: Fully covers the OpenAI API. Developers can choose or combine them as needed.

Recommendations for developers:

1. Evaluation Phase: Use Ollama for local prototype validation;
2. Integration Testing: Verify compatibility with OTP architecture (supervision trees, fault recovery);
3. Production Planning: Choose backend combinations and connection pool configurations based on load;
4. Monitoring and Optimization: Use Telemetry to achieve observability of LLM calls. Future features may include automatic retries and rate limiting.