Glossary

• logit: The inverse of the sigmoid. For probability \(p\), \(\mathrm{logit}(p)=\log\frac{p}{1-p}\).
• sigmoid: The logistic function \(\sigma(z)=\frac{1}{1+e^{-z}}\), mapping \(\mathbb{R}\to(0,1)\).
• softmax: Normalizes a vector \(z\) into a categorical distribution: \(\mathrm{softmax}(z_i)=\frac{e^{z_i}}{\sum_j e^{z_j}}\).