First Aid Decision Engine: An AI Reinforcement Learning Environment for Non-Professional First Responders

This project addresses the pain point that non-professionals lack systematic first aid capabilities in emergency medical scenarios. It builds a deterministic OpenEnv-compatible reinforcement learning environment for training and evaluating AI agents. The environment simulates step-by-step decision-making processes in emergency scenarios such as cardiac arrest and severe bleeding, trains strategies via tabular Q-learning, and aims to guide non-professionals to perform correct first aid operations, filling the golden rescue gap before professional help arrives.

In emergency medical scenarios, the golden rescue time is only a few minutes, so on-site response before professional help arrives is crucial. However, most ordinary people lack systematic first aid training and often feel at a loss or make wrong operations when facing cardiac arrest, severe bleeding, etc. This project aims to address this social need through an AI decision support system, building a reinforcement learning environment to train agents that guide non-professionals in correct handling.

The environment simulates real first aid processes, with core features including:

1. Step-by-Step Reasoning: First aid operations must follow strict sequences (e.g., confirm breathing before CPR);
2. Delayed Consequences: Delays or wrong operations result in negative rewards;
3. Partial Observability: Key information (e.g., pulse) needs to be actively assessed and obtained. The action space includes 12 first aid operations (such as CALL_EMERGENCY, START_CPR, etc.), and observation information is designed to be partially visible to simulate the process of gradual information collection in real scenarios.

The environment uses a dense reward mechanism:

• Positive Rewards: Correct operations, reasonable sequences, stable patient conditions, etc.;
• Negative Rewards: Repeated operations, delayed interventions, unsafe behaviors, etc.;
• Final Rewards: Rewards or penalties based on overall performance. The agent is implemented based on tabular Q-learning, learning only through environment interaction. It uses deterministic state encoding to ensure reproducibility, and the trained strategy can be persisted as a JSON file.

Three typical scenarios and optimal operation sequences are predefined:

1. Airport Cardiac Arrest: CALL_EMERGENCY → CHECK_BREATHING → START_CPR → USE_AED → MONITOR_PATIENT;
2. Severe Bleeding in Kitchen: CHECK_SCENE_SAFETY → CALL_EMERGENCY → APPLY_PRESSURE → CHECK_PULSE → MONITOR_PATIENT;
3. Traffic Accident with Multiple Injuries: CHECK_SCENE_SAFETY → CALL_EMERGENCY → APPLY_PRESSURE → CHECK_BREATHING → CONTROL_AIRWAY → CHECK_PULSE → MONITOR_PATIENT. The scenarios increase in difficulty, comprehensively testing the agent's decision-making ability.

The project uses FastAPI to build a web service, providing RESTful APIs and a React front-end interface, and supports Docker containerized deployment. Evaluation verification points include: successful Docker build, valid JSON returned by interfaces, complete task list, Q-table generated by training scripts, inference scripts completed on time, etc., to ensure the system is stable and reproducible.

This project is not only an RL research platform but also has practical application potential:

• Training System: Used for first aid simulation training;
• Decision Support: Integrated into first aid apps to guide non-professionals;
• Research Platform: Test the performance of different AI algorithms in high-risk scenarios. Through reinforcement learning technology, it is expected to develop an intelligent decision-making assistant that saves lives at critical moments.