Nagarro Maturity Assessment Portal: An Enterprise DevOps Capability Evaluation Platform Based on RAG and Multi-Agent

The Nagarro Maturity Assessment Portal is an enterprise-level AI-driven DevOps and software engineering maturity assessment platform that integrates RAG-based question generation, weighted DAG traversal, multi-agent workflows, real-time telemetry, and multi-tenant analysis to enable dynamic and adaptive capability evaluation. This project is maintained by itshivams, sourced from the GitHub platform, with the original title Nagarro-Maturity-Assessment-Portal, and was released on June 1, 2026.

Enterprise DevOps and software engineering maturity assessments typically use static questionnaires or fixed checklists, which have three major flaws: 1. One-size-fits-all questionnaires cannot adapt to the technical stacks and business characteristics of different teams; 2. Traditional assessments are one-time snapshots and cannot capture the evolution of team capabilities; 3. Assessment results remain at the report level and cannot be converted into actionable recommendations.

The core innovations of the Nagarro Maturity Assessment Portal include:

• RAG-driven question generation: Dynamically generate questions based on the knowledge base with reference sources attached;
• Weighted DAG traversal: Intelligently select assessment paths to achieve efficient diagnosis;
• Multi-agent collaborative workflow: Different agents are responsible for tasks such as question generation and answer evaluation;
• Real-time telemetry and confusion detection: Monitor user interaction behaviors to identify confusion or understanding deviations.

The platform adopts a microservices architecture, divided into:

• Frontend layer: Client portal built with Next.js, supporting WebSocket real-time updates;
• API gateway layer: Unified entry implemented with Go/Kratos, responsible for routing, authentication, etc.;
• Core service layer: 11 components including assessment service, AI orchestrator, RAG service, etc.;
• Data layer: Storage systems such as PostgreSQL, Redis, Kafka, Qdrant, and MongoDB.

The platform adopts a hybrid LLM strategy:

• Groq as the managed path: Provides high-performance cloud inference capabilities;
• Ollama as the local fallback: Automatically switches in network-restricted or sensitive scenarios. Configuration examples support the hybrid mode, balancing performance, cost, and reliability.

Assessment process: User initiation → System generates questions → User answers → AI analysis → Dynamic path planning → Auxiliary information generation → Iteration → Report generation. Insights are stored in MongoDB, including maturity levels, confidence levels, telemetry data, etc.; visualization supports personal dashboards, management backend monitoring, and analysis APIs.

Current limitations:

• The quality of the knowledge base affects RAG effectiveness;
• Models may have implicit biases;
• DAG traversal is not transparent enough for non-technical users. Improvement directions: Introduce user feedback to optimize the knowledge base, add manual review mechanisms, and provide more intuitive path visualization.

This project demonstrates a typical paradigm for enterprise-level AI applications: building a complete business system around AI capabilities. Core insights:

• RAG+Agent collaboration improves output reliability;
• Production-grade AI systems need to consider enterprise requirements such as tenant isolation and auditability;
• Hybrid deployment strategies balance performance, cost, and data privacy. It provides a valuable reference implementation for enterprise AI assessment systems.