Brier Score¶
Formula¶
\[
\operatorname{BS} = \frac{1}{N}\sum_{i=1}^N (p_i - y_i)^2
\]
Plot¶
fn: (1-x)^2
xmin: 0
xmax: 1
ymin: 0
ymax: 1.05
height: 280
title: Brier score for y=1 vs predicted p
Parameters¶
- \(p_i\): predicted probability for positive class
- \(y_i\in\{0,1\}\): true label
What it means¶
Mean squared error of probabilistic predictions.
What it's used for¶
- Evaluating probabilistic forecasts (calibration + sharpness).
- Comparing predicted probabilities to outcomes.
Key properties¶
- Lower is better; 0 is perfect
- Decomposes into calibration and refinement components
Common gotchas¶
- Extends to multiclass via vector probabilities and squared error.
- Not a proper scoring rule for some multi-label setups.
Example¶
For a single example with \(y=1\) and \(p=0.8\), \((p-y)^2=(0.8-1)^2=0.04\).
How to Compute (Pseudocode)¶
Input: predicted probabilities p[1..N], labels y[1..N]
Output: Brier score
sum_sq <- 0
for i from 1 to N:
sum_sq <- sum_sq + (p[i] - y[i])^2
return sum_sq / N
Complexity¶
- Time: \(O(N)\) for \(N\) predictions
- Space: \(O(1)\) extra space
- Assumptions: Binary Brier score shown; multiclass versions use vector probabilities per sample