Titanic Survival Prediction: A Practical Case of Ensemble Learning and Feature Engineering

This project is a practical case of predicting Titanic passengers' survival rate. Using ensemble learning methods (stacking Random Forest, Gradient Boosting, and SVM models) combined with feature engineering, it achieved a score of 0.77990 in the Kaggle competition. The project source is from a GitHub repository (Author: bayudwimulyadi, Link: https://github.com/bayudwimulyadi/Titanic-Survival-Prediction, Release Date: 2026-05-24). The following floors will detail the background, feature engineering, model construction, results, and experience summary.

Project Background

The Titanic sank on its maiden voyage in 1912, with 1502 out of 2224 passengers and crew losing their lives. The Titanic dataset provided by Kaggle is a classic introductory competition, and this project aims to predict survival rates using ensemble learning.

Dataset Overview

Key features include: PassengerId (unique identifier), Pclass (ticket class), Name, Sex, Age, SibSp (number of siblings/spouses aboard), Parch (number of parents/children aboard), Ticket, Fare (ticket price), Cabin (cabin number), Embarked (port of embarkation); the target variable is Survived (whether survived: 0 = No, 1 = Yes).

Missing Value Handling

• Age: Filled with the median of Pclass + Sex groups
• Embarked: Filled with the mode
• Fare: Filled with the median of the corresponding Pclass
• Cabin: Extract the first letter; missing values marked as Unknown

Feature Creation

• FamilySize: SibSp + Parch +1
• IsAlone: 1 if FamilySize is 1, else 0
• Title: Extract titles from names (e.g., Mr, Mrs)
• AgeGroup: Age binning (infant, child, etc.)
• FareCategory: Fare binning

Feature Encoding

• Ordinal variables (e.g., Pclass) use label encoding
• Nominal variables (e.g., Embarked, Title) use one-hot encoding

Ensemble Strategy

Using Stacking:

1. First layer: Train and predict with Random Forest, Gradient Boosting, and SVM respectively
2. Second layer: Train a meta-learner using the prediction results of base models
3. Final prediction: Meta-learner outputs the comprehensive result

Hyperparameter Tuning

• Grid search: Tune parameters for each model (e.g., n_estimators for RF, learning_rate for GB, C for SVM)
• K-fold cross-validation: Ensure model generalization ability

Base Model Advantages

• Random Forest: Handles high-dimensional data, resists overfitting
• Gradient Boosting: High-precision fitting
• SVM: Performs well in high-dimensional spaces

Performance Metrics

Kaggle test set accuracy is 0.77990; confusion matrix can analyze error patterns

Feature Importance

Top5: Sex (most critical), Pclass, Age, Fare, Title

Technical Highlights

• Comprehensive feature engineering (e.g., title extraction)
• Stacking ensemble strategy
• Systematic tuning process

Experience

1. Data quality first: Feature engineering is more important than complex models
2. Domain knowledge guidance: e.g., "Women and children first" principle
3. Ensemble learning improves performance

Expansion Directions

• Feature engineering: Analyze ticket patterns, combine external data (distance between cabin and lifeboat)
• Models: Try XGBoost, LightGBM, or neural networks
• Interpretability: Use SHAP values and interactive visualization to explain predictions