MATHEMATICS · COMPUTATION · INFERENCE

Understanding how
machines learn.

A study space for the mathematical ideas behind modern machine learning—from linear models and neural networks to convolution, recurrence and attention.

01 / FOUNDATIONS

Learning as optimisation

Representations, probability, loss functions, regularisation and generalisation.

θ* = arg minθ E[L(Y, f(X; θ))]
02 / NEURAL NETWORKS

Composing functions

Layers transform inputs through learned affine maps and nonlinear activation functions.

h(l) = σ(W(l)h(l−1) + b(l))
03 / CNN

Learning spatial features

Convolutions share parameters across an image to detect local patterns and structures.

Y(i,j) = Σm,n X(i−m,j−n)K(m,n)
04 / RNN

Learning sequences

Recurrent state carries information across time; training commonly uses backpropagation through time.

ht = φ(Wxxt + Whht−1 + b)
05 / TRANSFORMERS

Attention as content-dependent computation

Queries compare with keys to form weights that combine value vectors. This allows each position to use information from other positions.

Attention(Q,K,V) = softmax(QKT / √dk)V