A camera records pixels. It does not know what those pixels mean.
It does not know that a bright region is a lung, that a small shape is a person, or that two nearly identical satellite images show a flooded road.
Computer vision is the work of turning visual data into useful information.
That simple idea connects phone cameras, medical scans, self-driving systems, factory inspection, agriculture, robotics, and satellite imaging.
Why computer vision matters
Images contain a huge amount of information, but humans cannot inspect all of them.
A hospital may produce thousands of scans. A satellite can repeatedly photograph entire regions. A factory camera can see every product moving through a line. A person can review some of this data. A computer can help process it at scale.
Computer vision is useful when we need to:
- recognise what is present in an image;
- find where an object is;
- measure its shape or size;
- follow it through video;
- compare images taken at different times;
- or detect a pattern that is easy to miss.
The goal is not always to replace a human decision. In many important systems, the better goal is to help a person notice, measure, or prioritise something.
Image processing and computer vision are related, but different
Image processing changes an image.
It may resize it, remove noise, improve contrast, correct colour, or find edges. The output is usually another image.
Computer vision tries to understand something from the image.
It may produce a class label, a bounding box, a segmentation mask, a depth map, or a description.
In practice, we use both. Image processing prepares the data; computer vision extracts meaning from it.
The main computer-vision tasks
Image classification
Classification gives one or more labels to an image.
Examples:
- normal or abnormal X-ray;
- cat, dog, or bird;
- forest, water, farmland, or urban land.
The model answers what is in this image?, but not where it is.
Object detection
Detection finds objects and draws a bounding box around each one.
Examples include locating pedestrians, vehicles, cells, ships, or defects.
The output contains both a class and a position.
Semantic segmentation
Segmentation assigns a class to every pixel.
This is useful when the exact shape matters: separating a tumour from surrounding tissue, mapping roads in satellite imagery, or identifying cracks in concrete.
Instance segmentation
Semantic segmentation treats every object of one class as the same region. Instance segmentation separates individual objects.
If an image contains five people, instance segmentation produces five separate masks.
Keypoint estimation
Keypoint models locate important points such as body joints, facial landmarks, or corners of an object.
This supports pose estimation, motion analysis, alignment, and augmented reality.
Tracking
Tracking follows an object across video frames.
Detection tells us where an object is now. Tracking helps determine whether it is the same object seen a moment earlier.
Depth, motion, and 3D vision
Some tasks estimate how far objects are from the camera, how they move, or what their three-dimensional structure may be.
These ideas are important in robotics, autonomous navigation, mapping, and mixed-reality systems.
The libraries worth learning
No single library does everything. A useful stack has different tools for loading data, processing images, training models, evaluating results, and deploying them.
OpenCV: the image-processing toolbox
OpenCV is usually the first library to learn.
It handles:
- image and video reading;
- resizing, cropping, and colour conversion;
- filtering and noise removal;
- thresholding and morphology;
- edge, contour, and feature detection;
- geometric transforms and camera calibration;
- video capture and classical tracking.
import cv2
image = cv2.imread("image.jpg")
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
edges = cv2.Canny(gray, 100, 200)
OpenCV is not only for older, non-deep-learning methods. It remains valuable inside modern pipelines because data still needs to be decoded, transformed, measured, and displayed.
NumPy: how images become arrays
An image in Python is often a NumPy array.
Its shape may look like (height, width, channels). Pixel values are numbers,
which means slicing, masking, normalisation, and numerical operations all depend
on understanding NumPy.
If OpenCV is the toolbox, NumPy is the language many tools use to exchange data.
Pillow: simple image input and output
Pillow is a clean option for opening, saving, resizing, and drawing on common image formats.
It is often enough for web applications and dataset scripts. OpenCV becomes more useful when the work needs advanced image processing or video.
Matplotlib: seeing what the pipeline did
Computer vision should not be developed blindly.
Matplotlib helps display images, masks, training curves, class distributions, and errors. Visual inspection often reveals a wrong colour order, broken mask, or bad augmentation before a metric does.
PyTorch and torchvision: building learning systems
PyTorch provides tensors, automatic differentiation, neural-network layers, optimisation, and GPU training.
torchvision adds common vision datasets, image transforms, pretrained models, and utilities for classification, detection, segmentation, keypoints, video, and optical flow.
from torchvision.models import resnet18, ResNet18_Weights
weights = ResNet18_Weights.DEFAULT
model = resnet18(weights=weights)
model.eval()
Learn PyTorch when you want to train or fine-tune a model rather than only call an existing one.
Albumentations: reliable data augmentation
Albumentations applies random changes during training so a model does not memorise only the exact images it has seen.
It can transform an image while keeping its masks, bounding boxes, and keypoints aligned. That detail is essential. A flipped image with an unflipped annotation teaches the model the wrong answer.
Useful augmentations include crops, flips, rotation, blur, noise, and brightness changes—but every augmentation is an assumption. A vertical flip may be valid for satellite imagery and completely wrong for a chest X-ray.
scikit-image: algorithms for scientific images
scikit-image provides accessible implementations of segmentation, filtering, morphology, feature extraction, measurement, and restoration methods.
It is especially useful for experiments and scientific-image analysis where a clear classical method may solve the problem without a large neural network.
MONAI: medical-imaging workflows
MONAI is built on PyTorch for healthcare imaging.
It adds medical-image transforms, losses, metrics, networks, and workflows for 2D and 3D data such as CT and MRI volumes. It is useful because medical images carry concerns that ordinary JPEG pipelines do not: voxel spacing, orientation, multiple modalities, and large 3D volumes.
Ultralytics: fast object-detection experiments
Ultralytics provides an approachable way to train and run YOLO models for detection, segmentation, pose, classification, and tracking.
It is useful for prototypes. Still, learning only one high-level command can hide important choices about data quality, evaluation, and deployment.
Hugging Face Transformers and timm: pretrained models
Transformers includes vision transformers and multimodal models. timm offers a broad collection of image backbones and pretrained weights.
These libraries become helpful when comparing architectures or using transfer learning.
ONNX and OpenVINO: moving beyond the notebook
A trained model is not yet a useful product.
ONNX provides a portable model format. OpenVINO optimises inference for supported hardware. Other targets may use TensorRT, Core ML, or platform-specific runtimes.
Deployment introduces new questions: latency, memory, batch size, precision, input validation, monitoring, and hardware support.
A sensible order for learning
You do not need to learn every library at once.
- Learn Python and NumPy arrays.
- Use OpenCV or Pillow to load and change images.
- Learn colour spaces, filters, thresholding, contours, and morphology.
- Understand classification, detection, and segmentation.
- Learn PyTorch tensors, datasets, models, losses, and training loops.
- Fine-tune a pretrained torchvision model.
- Add Albumentations and evaluate whether it actually helps.
- Build one complete project, including error analysis and deployment.
What matters more than the library
A library can make code shorter. It cannot decide whether the problem is defined well.
Before choosing a model, ask:
- What decision will this prediction support?
- Who created the labels, and how reliable are they?
- Does the test set represent the real world?
- Which errors are expensive?
- Could lighting, camera type, hospital, geography, or season change the result?
- How will a human inspect uncertain predictions?
Computer vision is not just about making a model recognise an image.
It is about building a trustworthy path from pixels to a decision.