Training Battleship Game AI with Deep Reinforcement Learning: From Random Bombing to Intelligent Hunting

This article introduces the open-source project battleship_rl, which trains battleship game AI using algorithms like Deep Q-Network (DQN). It supports battles between multiple agents (Random, Hunt, Bayesian, Deep Q-Learning), explores the evolutionary path from random strategies to intelligent hunting, and analyzes the decision-making mechanisms and practical performance of different strategies. The project serves both as a benchmark platform for RL research and an excellent learning case for beginners.

The battleship game is a classic strategy board game born in the early 20th century. Its rules are simple but decision-making is complex—both sides hide their fleets and take turns bombing coordinates to sink each other's ships. The core challenge is to efficiently locate the enemy fleet with limited hit information. In recent years, deep reinforcement learning (DRL) has developed rapidly, and researchers have tried to use AI to solve such imperfect information game problems. battleship_rl is a practical project in this field.

battleship_rl is a complete multi-agent battle framework that supports multiple AI players:

• Random Agent: Randomly selects bombing coordinates (baseline control group)
• Hunt Agent: Heuristic strategy—prioritizes searching adjacent areas after a hit
• Bayes Agent: Uses probabilistic reasoning to calculate the cells most likely to hide ships
• Q-Agent: Learns optimal strategies through neural networks This framework allows intuitive comparison of different algorithm performances and provides a benchmark platform for RL research.

The core highlight of the project is the DQN implementation. DQN approximates the Q-value function via neural networks, solving the curse of dimensionality in traditional Q-learning. In the battleship game, the state space is the current board pattern (hit, miss, unknown), and the action space is the un-bombed coordinates. The training process includes:

1. Experience Replay: Stores historical data to break sample correlation
2. Target Network: Uses an independent target network to calculate Q-values, improving stability
3. ε-Greedy Exploration: Dynamically balances exploration and exploitation Through interaction with the environment, the agent learns to extract features from the board to predict hit coordinates.

The project implements a Bayesian agent, whose core is to calculate the posterior probability of each unknown cell based on known hit/miss information: enumerate all fleet layouts that match the observations, count the frequency of each cell being occupied, and the higher the frequency, the higher the priority. This strategy is nearly optimal in information utilization, providing a theoretical upper limit for evaluating neural network agents. Comparative experiments show that a fully trained DQN agent can approach or even surpass the performance of the Bayesian strategy, indicating that the neural network implicitly learns probabilistic reasoning and captures tactical patterns.

The project supports multiple operation modes:

• Headless Mode: Runs purely in the background, suitable for batch training and automated testing
• WebSocket Mode: Humans can play against AI in real time via WebSocket
• Terminal Mode: Command-line interaction, convenient for debugging and demonstration In addition, the project focuses on observability and reproducibility:
• Logging System: Multi-level log output, recording the board state of each game
• Checkpoint Mechanism: Regularly saves model parameters, supporting resume training from breakpoints
• Training Log Directory: Archives metrics by timestamp, facilitating analysis and visualization These details are key to moving from a toy project to serious research.

The battleship game belongs to imperfect information games, which have wide real-world applications: radar search, medical diagnosis, resource exploration, etc. The technical framework of battleship_rl can be migrated to these scenarios, demonstrating the combination of classical game theory and deep learning (Bayesian provides theoretical guidance, neural networks learn approximate optimal strategies). For RL beginners, this project has simple rules, a moderate state space, clear code, and complete documentation—it is an excellent starting point. It is recommended to clone the project and train a battleship AI commander yourself.