Adversarial Coevolution: A Game Training Framework for RL Agents and LLMs in Gin Rummy

This project explores an innovative training paradigm: enabling PPO reinforcement learning agents and large language models (LLMs) to mutually improve through adversarial coevolution, achieving high-performance decision-making capabilities in the Gin Rummy card game. The project was released by Nikelroid on GitHub on 2026-05-29 (link: https://github.com/Nikelroid/adversarial-coevolution). Core keywords include reinforcement learning, large language models, adversarial training, PPO, curriculum learning, knowledge distillation, game theory, and Gin Rummy.

Reinforcement Learning (RL) learns optimal strategies through trial-and-error interaction with the environment, excelling at precise decision-making in structured environments; LLMs gain strong reasoning and generalization abilities through massive text pre-training, enabling them to understand complex contexts. Traditionally, the two have developed independently. This project attempts to explore: Can RL agents and LLMs engage in adversarial games to mutually promote and coevolve?

The project uses Gin Rummy as an experimental field to build an adversarial training framework:

• PPO Agent: Uses the Proximal Policy Optimization algorithm to improve card skills through self-play and adversarial learning
• LLM Opponent: Uses reasoning abilities to analyze game states, predict intentions, and formulate strategies In a dynamically evolving environment, RL needs to deal with LLM's non-traditional strategies, while LLMs need to adapt to RL's optimized tactics, forming a mutually promoting situation.

1. Curriculum Learning: Gradually increase training difficulty—master basics with weak opponents in the early stage, and force learning of complex skills (card counting, probability calculation, etc.) with strong opponents in the later stage to avoid training collapse or bottlenecks.
2. Knowledge Distillation: Extract decision-making knowledge from LLMs and transfer it to RL policy networks to accelerate convergence, allowing RL to gain LLM's intuitive judgment while retaining the advantage of precise optimization.
3. Adversarial Coevolution Cycle: Evaluation (collect data from RL vs. LLM games) → Learning (RL updates strategies, LLMs adjust prompts) → Evolution (both sides improve synchronously) → Iteration, avoiding local optima.

• State Representation: Design a compact and information-rich state encoding for Gin Rummy, preserving key decision-making information to adapt to neural networks.
• Reward Shaping: Set intermediate rewards (successful meld formation, preventing opponents from going gin, etc.) in addition to win/loss rewards to guide fine-grained strategy learning.
• LLM Integration: Use optimizations like caching, batch processing, and asynchronous calls to ensure real-time decision efficiency.
• Scalable Architecture: A general design that adapts to other two-player game scenarios, providing a foundation for future research.

• Game AI: Provides new ideas for the development of complex card/board game AI, especially suitable for games with large state spaces and incomplete information.
• RL Training Enhancement: Proves that LLMs can serve as high-quality opponents/coaches, helping RL break through the limitations of self-play and learn diverse strategies.
• LLM Capability Evaluation: Objectively assesses LLM's capabilities such as strategic reasoning and long-term planning through competitive confrontation, providing a new evaluation dimension.
• Hybrid Intelligent Systems: Explores the possibility of combining symbolic reasoning (LLMs) and numerical optimization (RL), paving the way for powerful hybrid systems in the future.

The project demonstrates the huge potential of AI technology integration: the synergistic effect of RL's precise optimization and LLM's flexible reasoning far exceeds their individual capabilities. Future AI systems may be an organic combination of multiple intelligent paradigms, and adversarial coevolution is both a training method and a design philosophy. With the improvement of LLMs and RL, this framework is expected to be applied in fields such as autonomous driving, financial trading, robot control, and scientific discovery, and a new paradigm of human-machine/machine-machine collaboration is taking shape.