Multi-User Large Language Model Agents: When AI Needs to Serve Multiple "Masters" Simultaneously

Researchers from MIT and other institutions have proposed the first systematic multi-user LLM agent research framework, revealing key flaws in current models in multi-user scenarios such as privacy leaks and coordination failures, and open-sourced a complete evaluation benchmark (MUSES Bench) and training pipeline. This research marks a key step in the evolution of AI systems from personal assistants to team collaborators.

Most current LLM agent systems implicitly assume a single user, with models designed to serve a single entity. However, in real-world scenarios, AI needs to serve multiple users simultaneously (e.g., medical AI connecting doctors/nurses/patients, enterprise assistants coordinating cross-departmental employees), facing technical challenges such as interest conflicts, information asymmetry, and privacy constraints: How to meet the needs of each user in a multi-agent environment?

Researchers proposed a formal theoretical framework for multi-user interaction, defining it as a "multi-agent decision problem" (a single agent needs to consider the constraints of multiple potentially conflicting users and overall coordination). They also designed the MUSES Bench evaluation benchmark and open-sourced all code and datasets.

1. Privacy Protection and Access Control

Tested permission enforcement, privacy-aware summarization, and resistance to social engineering attacks, finding that cutting-edge LLMs exhibit a "privacy leak escalation" trend (prone to disclosing sensitive information after multi-turn conversations).

2. Sequential Coordination and Meeting Scheduling

Simulated meeting coordination, examining preference elicitation, conflict resolution, and context management; existing models have efficiency bottlenecks (requiring too many conversational turns to complete scheduling).

3. Resource Optimization for Shared LLM Inference Queues

Tested fairness and efficiency of resource scheduling, corresponding to real-world LLM service deployment scenarios.

4. Multi-User Instruction Following

Tested the ability to simultaneously satisfy conflicting instruction preferences (e.g., concise/detailed/formal language).

1. Unstable priorities under conflicting goals: Models cannot maintain consistent decisions and tend to waver.
2. Privacy leaks escalate with conversation turns: "Memory contamination" in multi-turn conversations leads to sensitive information disclosure.
3. Coordination efficiency bottleneck: Excessive interaction turns when iteratively collecting information, leading to low efficiency.

The research team open-sourced a full training pipeline, including:

• Data generation tool (teacher model generates synthetic multi-user conversations)
• Data aggregation script (formats training data)
• SFT training code (supports fine-tuning base models)
• vLLM inference support (efficient inference for trained models) Researchers can use this framework to evaluate existing models or train multi-user optimized models.

• Developers: Need to explicitly design multi-user architectures, add training data for multi-user scenarios, and deploy permission/privacy boundary mechanisms.
• Researchers: MUSES Bench provides a standardized evaluation platform, promoting fair comparison of multi-user capabilities and domain development. This research provides a theoretical foundation and practical tools for multi-user AI system design.

Multi-user LLM agent research breaks through single-user limitations and reveals current technical flaws. With the development of the field, it is expected that intelligent collaborators capable of handling complex interpersonal environments will emerge in the future, promoting the evolution of AI from personal tools to team collaboration tools.