Learning as optimisation
Representations, probability, loss functions, regularisation and generalisation.
θ* = arg minθ E[L(Y, f(X; θ))]
MATHEMATICS · COMPUTATION · INFERENCE
A study space for the mathematical ideas behind modern machine learning—from linear models and neural networks to convolution, recurrence and attention.
Representations, probability, loss functions, regularisation and generalisation.
Layers transform inputs through learned affine maps and nonlinear activation functions.
Convolutions share parameters across an image to detect local patterns and structures.
Recurrent state carries information across time; training commonly uses backpropagation through time.
Queries compare with keys to form weights that combine value vectors. This allows each position to use information from other positions.