Machine Learning from Scratch: A Complete Survival Guide for AI Algorithms

This open-source learning resource from BRAC University's CSE422 course, maintained by azaynul10 on GitHub, connects theory and practice in the form of a "survival guide". It covers everything from classic AI search algorithms to core concepts of modern machine learning, providing comprehensive guidance on mathematical derivations, code implementations, and exam tips to help learners bridge the gap between theory and hands-on practice.

Resource Background

• Original author/maintainer: azaynul10
• Source platform: GitHub
• Original link: https://github.com/azaynul10/machine-learning-fundamentals-from-scratch
• Release date: June 4, 2026

Three-Stage Course Structure

1. Classical AI and Search Algorithms: Uninformed/informed search (BFS, A*, etc.), local search (hill climbing, simulated annealing, genetic algorithms), adversarial search (Minimax, Alpha-Beta pruning)
2. Probability Theory and Bayesian Inference: Joint/marginal/conditional probability, Naive Bayes classifier
3. Modern AI and Machine Learning: Decision trees, linear/logistic regression, neural networks

A* Search Algorithm

The core is maintaining a priority queue, selecting nodes based on f(n)=g(n)+h(n) (actual cost + heuristic estimate), and paying attention to neighbor update logic to ensure optimality.

Genetic Algorithm

Key components: fitness function (with penalty terms), mutation operation (safe gene flipping), chromosome encoding/decoding (avoiding boundary issues), and modular structure to improve readability.

Neural Networks

• Forward propagation: input layer → hidden layer → output layer, with weighted sum + activation function propagation
• Backpropagation: propagate error back from the output layer and update weights; formula for number of parameters: (input dimension × number of neurons) + number of biases

1. Binary Cross-Entropy Loss: Loss = -[y·ln(A) + (1-y)·ln(1-A)] (y = true label, A = predicted probability)
2. Sigmoid Derivative: σ'(x)=A×(1-A)
3. Linear Regression Gradient Descent: m_new = m_old - α×(2/n×Σ(ŷ-y)×x)
4. Logistic Regression Gradient Descent: w_new = w_old - α×((A-y)×x)

Practical Tips

• File processing: Use basic string splitting and type conversion; avoid over-reliance on advanced functions
• Manual practice: Naive Bayes posterior probability, gradient descent updates, Sigmoid output
• Video recommendations: 3Blue1Brown neural network visualization, StatQuest statistical explanations, official course videos
• Colab projects: Model evaluation, K-fold cross-validation, exploratory data analysis

Exam Pitfalls

• Simulated annealing: High temperature → high randomness (easy to accept suboptimal solutions), low temperature → converges to hill climbing
• A* search: Optimality in graph search requires consistency (monotonicity); mere admissibility is insufficient
• Problem identification: Single-layer perceptron = logistic regression; conditional independence → adjust joint probability; continuous target → linear regression; 0/1 target → logistic regression

1. Beginners: Follow the three stages (classical search → probability → modern ML) step by step, with manual calculations and code implementations
2. Exam Preparation: Focus on mastering core formulas and common pitfalls; practice repeatedly to form memory
3. Project Practice: Refer to Colab notebooks and modular code structures to quickly set up an experimental environment

There are no shortcuts to learning AI, but this guide can reduce detours and increase learning certainty.