Machine learning can feel larger than it really is.
There are hundreds of model names, libraries, papers, and courses. New terms appear every week. It is easy to learn them separately and still not understand how they connect.
This article is a map.
It does not replace the mathematics or practice behind each concept. Its purpose is to show the whole field in one structure, so you know what to learn and where each idea belongs.
1. Start with the problem
Machine learning means learning a useful pattern from data instead of writing every rule by hand.
Before thinking about an algorithm, define:
- the input available at prediction time;
- the output the system should produce;
- the unit of observation, such as a patient, image, user, or transaction;
- the metric that represents useful performance;
- and the cost of different mistakes.
A model can optimise the wrong problem perfectly.
2. The main learning settings
Supervised learning
The data contains inputs and known targets.
- Classification predicts a category.
- Regression predicts a number.
- Ranking orders items by relevance.
- Structured prediction produces outputs such as sequences, boxes, or masks.
Unsupervised learning
There is no target label.
- Clustering groups similar examples.
- Dimensionality reduction creates a smaller representation.
- Density estimation models how data is distributed.
- Anomaly detection finds unusual observations.
Semi-supervised and self-supervised learning
Semi-supervised learning combines a small labelled set with a larger unlabelled set.
Self-supervised learning creates a training signal from the data itself—for example, hiding part of an input and asking the model to reconstruct or predict it.
Reinforcement learning
An agent takes actions in an environment and receives rewards. The goal is to learn a policy that maximises long-term reward.
Its main ideas are states, actions, rewards, policies, value functions, exploration, and credit assignment.
3. Data concepts
Features and targets
A feature is information given to the model. A target is what it should learn to predict.
Features may be numbers, categories, text tokens, pixels, audio samples, or learned embeddings.
Training, validation, and test sets
- The training set fits the parameters.
- The validation set guides model and hyperparameter choices.
- The test set estimates final performance on unseen data.
The test set should not influence training decisions.
Data leakage
Leakage happens when training uses information that would not truly be available at prediction time.
Examples include preprocessing the full dataset before splitting, including a future measurement as a feature, or placing near-duplicate images in training and testing.
Leakage creates impressive results that disappear in reality.
Preprocessing
Common steps include:
- missing-value handling;
- scaling and normalisation;
- categorical encoding;
- text tokenisation;
- image resizing and augmentation;
- outlier inspection;
- and class balancing.
Every preprocessing operation learned from data must be fitted on the training set and then applied to validation and test sets.
4. Core mathematical ideas
You do not need all of mathematics before beginning, but these areas matter:
- Linear algebra: vectors, matrices, dot products, eigenvalues, and tensor operations.
- Calculus: derivatives, partial derivatives, gradients, and the chain rule.
- Probability: random variables, conditional probability, expectation, likelihood, and common distributions.
- Statistics: sampling, estimation, bias, variance, confidence, and hypothesis testing.
- Optimisation: objective functions, constraints, convexity, and gradient methods.
The mathematics explains why an algorithm behaves as it does. Code only shows that it ran.
5. Classical machine-learning models
Linear and logistic regression
Linear regression predicts a continuous value from a weighted combination of features.
Logistic regression estimates the probability of a class. Both are strong baselines because they are fast and easier to interpret.
k-nearest neighbours
KNN predicts from nearby training examples. It is simple, but prediction can become expensive and distance becomes less meaningful in high dimensions.
Naive Bayes
Naive Bayes uses probability with a strong independence assumption between features. It can work surprisingly well for text and small datasets.
Decision trees
A tree divides data using a sequence of rules. Trees are easy to inspect but can overfit.
Ensembles
Ensembles combine several models.
- Random forests average many varied decision trees.
- Gradient boosting builds trees sequentially to correct earlier errors.
- Bagging reduces variance by training on sampled data.
- Boosting combines weak learners into a stronger one.
- Stacking learns how to combine predictions from different models.
XGBoost, LightGBM, and CatBoost are common choices for structured tabular data.
Support vector machines
SVMs find a separating boundary with a large margin. Kernels allow nonlinear boundaries, though training can become expensive on large datasets.
Clustering and dimensionality reduction
K-means, hierarchical clustering, and density-based methods such as DBSCAN find groups using different assumptions.
PCA creates linear low-dimensional representations. t-SNE and UMAP are often used for visualisation, but their plots should not be treated as proof of true clusters.
6. Generalisation
The real goal is not to perform well on training data. It is to work on new data.
Underfitting and overfitting
Underfitting means the model is too limited or insufficiently trained to capture the useful pattern.
Overfitting means it learns details of the training set that do not generalise.
Bias and variance
High bias often causes underfitting. High variance makes a model sensitive to the particular training sample.
More data, regularisation, simpler models, augmentation, and ensembling can change this balance.
Regularisation
Regularisation discourages unnecessary complexity.
Common methods include L1 and L2 penalties, weight decay, dropout, early stopping, data augmentation, label smoothing, and noise injection.
Cross-validation
Cross-validation repeats training across different folds to produce a more stable estimate when data is limited.
The split strategy must match the problem. Time series need temporal splits. Medical data may need patient-level splits. Geographic data may need spatial splits.
7. Evaluation
Classification metrics
- Accuracy: fraction of correct predictions.
- Precision: how many predicted positives were correct.
- Recall: how many real positives were found.
- F1 score: harmonic balance of precision and recall.
- ROC-AUC: ranking across classification thresholds.
- PR-AUC: often more informative for rare positive classes.
- Log loss: evaluates probability quality, not only final labels.
A confusion matrix shows which classes are confused.
Regression metrics
- MAE measures average absolute error.
- MSE and RMSE penalise large errors more strongly.
- R² measures improvement over predicting the mean.
Calibration and uncertainty
A model can be accurate but overconfident.
Calibration asks whether predictions made with 80% confidence are correct about 80% of the time. Uncertainty matters when predictions support medical, financial, or safety decisions.
Error analysis
Do not stop at one metric.
Inspect failures by class, subgroup, data source, time, geography, image quality, and confidence. The next useful improvement often comes from understanding the errors rather than choosing a larger model.
8. Neural-network foundations
A neural network is a sequence of parameterised transformations.
Neurons, layers, and activations
A neuron computes a weighted sum, adds a bias, and applies an activation function.
Common activations include ReLU, GELU, sigmoid, and tanh. Without nonlinear activations, many layers would still behave like one linear transformation.
Forward pass, loss, and backpropagation
- The forward pass produces a prediction.
- A loss function measures the error.
- Backpropagation uses the chain rule to compute gradients.
- An optimiser updates the parameters.
Optimisers
Stochastic gradient descent updates parameters using mini-batches. Momentum smooths updates. Adam adapts learning rates for individual parameters.
The learning rate is often the most important hyperparameter. Schedulers change it during training.
Batch size, epochs, and initialisation
A batch is a subset processed before an update. An epoch is one pass through the training data. Initialisation sets starting weights so signals and gradients can flow through the network.
Normalisation layers and residual connections help train deeper networks.
9. Major deep-learning architectures
Multilayer perceptrons
MLPs use fully connected layers. They are the basic neural-network form and can work well on fixed-size feature vectors.
Convolutional neural networks
CNNs use local filters with shared weights. They are efficient for images because nearby pixels have meaningful structure.
Important ideas include kernels, feature maps, receptive fields, stride, padding, pooling, and skip connections.
Recurrent networks
RNNs process sequences step by step. LSTMs and GRUs use gates to preserve useful information over longer ranges.
They remain relevant, although transformers now handle many sequence problems.
Attention and transformers
Attention lets each token or patch weigh information from other positions.
A transformer combines attention, feed-forward layers, residual connections, normalisation, and positional information.
Transformers support language models, vision transformers, multimodal models, and many modern foundation models.
Autoencoders
An encoder compresses an input into a representation; a decoder reconstructs it.
Variants are used for representation learning, denoising, compression, anomaly detection, and generation.
Generative models
- Autoregressive models predict the next element.
- Variational autoencoders learn a structured latent distribution.
- GANs train a generator against a discriminator.
- Diffusion models learn to reverse a gradual noising process.
Generative quality does not guarantee factuality or useful understanding.
Graph neural networks
GNNs operate on nodes and edges. They are useful for molecules, networks, recommendation, knowledge graphs, and relational data.
10. Representation and transfer learning
Deep learning often works because the model learns useful representations.
Transfer learning starts from weights trained on a large dataset.
- Feature extraction keeps most pretrained weights frozen.
- Fine-tuning updates some or all of them on the new task.
- Parameter-efficient fine-tuning changes a smaller set of parameters.
Self-supervised pretraining, contrastive learning, masked modelling, and foundation models reduce the need to learn every task from scratch.
11. Practical training concepts
Important ideas include:
- reproducible random seeds and recorded configurations;
- dataset and experiment versioning;
- mixed-precision training;
- gradient accumulation and clipping;
- checkpointing and early stopping;
- hyperparameter search;
- distributed training;
- class weighting and sampling;
- choosing losses that match the task;
- and tracking both metrics and qualitative examples.
A training run without saved settings is difficult to reproduce and harder to trust.
12. Deployment and MLOps
The model lifecycle continues after training:
- package preprocessing with the model;
- export or serve it;
- test latency, throughput, memory, and cost;
- version data, code, and weights;
- monitor inputs and predictions;
- detect data and concept drift;
- collect feedback carefully;
- retrain, validate, and roll back safely.
Batch inference, real-time APIs, mobile devices, browsers, and edge hardware all have different constraints.
13. Responsible machine learning
Ask who is represented in the data and who may be harmed by an error.
Important concerns include privacy, consent, security, bias, accessibility, explainability, energy use, dual use, and human oversight.
Interpretability methods can help inspect a model, but an attractive explanation is not proof that the model reasoned correctly.
A learning order that works
- Python, NumPy, pandas, and visualisation.
- Basic probability, statistics, and linear algebra.
- Data splitting, preprocessing, and evaluation.
- Linear models, trees, ensembles, and clustering.
- One full scikit-learn project.
- Neural networks and backpropagation.
- PyTorch training loops.
- CNNs, sequence models, and transformers.
- Transfer learning and one domain project.
- Deployment, monitoring, and responsible evaluation.
Do not wait to finish every theory chapter before building. Learn one concept, use it, inspect what failed, and then return to the theory with a real question.
The map in one sentence
Data defines what can be learned. The objective defines what the model tries to learn. Optimisation finds parameters. Evaluation tests whether the pattern generalises. Deployment discovers whether any of it remains useful in the real world.
That is the field beneath the changing model names.