Unity Genetic Algorithm Neural Network Autonomous Driving Simulation: Training AI Racers from Scratch

An open-source project based on the Unity engine, combining genetic algorithms and neural networks to enable virtual race cars to autonomously learn driving skills from scratch. The core mechanism is: no manual data labeling required; neural network weights are optimized via genetic algorithms, allowing the AI to evolve through "survival of the fittest" natural selection over multiple generations, mastering driving abilities such as obstacle avoidance and navigation.

The core idea of the project is to apply biological evolution theory to the field of machine learning, using genetic algorithms to optimize neural network weights. Unlike traditional supervised learning, no manual data labeling is needed. The technical architecture includes three key components:

• CarController.cs: Responsible for vehicle physics control, sensor data collection, and fitness calculation
• NeuralNetwork.cs: Implements forward propagation inference for feedforward neural networks
• GeneticManager.cs: Manages population evolution, selection, crossover, and mutation operations

Each race car is equipped with three sets of ray sensors (left-front, front, right-front) that continuously detect the distance to the track walls, providing environmental perception input for the neural network. The sensor principle is similar to bat ultrasonic positioning:

• When a ray hits a wall, it returns a normalized distance value between 0 and 1; the closer the distance, the smaller the value (higher risk)
• The three sensors combine to form a fan-shaped spatial perception of the front This design avoids complex visual processing and allows the AI to quickly understand the geometric structure of the environment.

A classic feedforward neural network structure is used:

• Input layer: 3 neurons (left, middle, right sensor distances)
• Hidden layers: Configurable multiple layers (number of neurons adjustable)
• Output layer: 2 neurons (acceleration and steering values, range from -1 to 1) Forward propagation uses the MathNet Numerics library for matrix operations to ensure efficiency and stability. Weight matrices and bias vectors are initially generated randomly and optimized via genetic algorithms. Backpropagation is not used; learning occurs at the population level.

The genetic algorithm is the core learning mechanism. Each generation contains multiple neural networks (genomes). After calculating fitness based on driving performance, evolution operations are performed:

1. Fitness evaluation: Integrates three dimensions—driving distance, average speed, and sensor safety distance—with flexibly adjustable parameters
2. Selection mechanism: Elite retention (copying the best networks), diversity retention (some poorer networks enter the gene pool), and crossover breeding
3. Crossover and mutation: Parent networks exchange weights to produce offspring; weights are randomly perturbed at a mutation rate to introduce new variations, simulating the core mechanisms of biological evolution.

The training process forms a closed loop:

1. Initialization: Create the first generation of random neural network population
2. Evaluation: Each network controls a vehicle starting from the origin until it hits a wall or times out
3. Scoring: Calculate fitness based on driving distance, speed, and obstacle avoidance performance
4. Evolution: Select excellent individuals, perform crossover and mutation to generate the next generation
5. Repeat: Cycle through the above steps for multiple generations of training Supports the timeScale parameter to accelerate training (up to 20x speed) and shorten training time.

Optimization suggestions for adjustable parameters:

• Population size: It is recommended to start with 50-100; a larger population increases diversity but extends evaluation time
• Network complexity: 1-2 hidden layers are sufficient for simple tracks; more layers/neurons are needed for complex behaviors
• Mutation rate: Increase when learning stagnates (to escape local optima), decrease when behavior is random (to stabilize convergence)
• Fitness weights: Increase sensorMultiplier for narrow curves, and avgSpeedMultiplier for long straight roads

Educational value of the project: It intuitively demonstrates core ideas such as genetic algorithm optimization of complex systems, neural network perception-decision mapping, exploration-exploitation trade-off in evolutionary computing, and trial-and-error learning in reinforcement learning. It is an ideal teaching tool for beginners in machine learning, game AI, and autonomous driving. Application prospects: Provides a basic algorithm reference for autonomous driving technology; AI evolution on virtual tracks may inspire the development of future real-world intelligent driving systems.