A topic in the Open Knowledge Graph — a free, open map of 15,290 topics and the order to learn them in.
Representation learning automatically discovers useful feature representations from raw data through deep learning. Learned representations are more informative and generalizable than hand-crafted features. Autoencoders, GANs, and self-supervised methods learn representations; downstream tasks leverage these compact, meaningful embeddings.
From your study of neural networks, you know that each layer transforms its input into a new set of values, and that training adjusts these transformations to minimize a loss function. Representation learning is the insight that those intermediate transformations are not just computational plumbing — they are *learned features* that can be more valuable than the final output itself. The hidden layers of a neural network are automatically discovering how to represent raw data in a form that makes the task easier, and those representations turn out to be remarkably powerful and reusable.
Consider the traditional machine learning pipeline: a human expert examines the data (images, text, audio) and manually designs features — edge detectors for images, n-grams for text, spectral coefficients for audio. This feature engineering requires deep domain knowledge, is labor-intensive, and often fails to capture the most informative patterns. Representation learning replaces this manual step. When you train a convolutional neural network on images, the early layers learn to detect edges and textures, middle layers learn to recognize parts like eyes or wheels, and later layers compose these into object-level representations. Nobody told the network to look for edges — it discovered that edges are useful because they help minimize the classification loss.
The power of learned representations becomes most apparent when you consider embeddings: compact vector representations that capture semantic relationships. Word embeddings like Word2Vec map words into a space where "king" minus "man" plus "woman" lands near "queen." Image embeddings from a pretrained network place visually similar images near each other in vector space. These embeddings work because the training process forces the network to organize its internal representations so that inputs with similar meanings or functions end up with similar representations — a structure that emerges naturally from the learning objective.
Autoencoders make representation learning explicit by training a network to compress input into a small bottleneck layer and then reconstruct the original input from that compressed representation. Whatever information survives the bottleneck must be the most essential features of the data. Self-supervised methods go further: they create their own training signal from unlabeled data (predicting masked words, matching augmented views of the same image) to learn representations without any human-provided labels. The result is that a model trained on vast unlabeled data can produce representations that transfer effectively to many downstream tasks — a foundation model's hidden layers become a general-purpose feature extractor, and fine-tuning on a small labeled dataset is often all that is needed for a specific application.