13. Metrics for Model evaluation¶

Methods commonly used to evaluate model performance, include:

Mean absolute error (MAE)

\[\mathrm{MAE}=\frac{1}{N} \sum_{i=1}^{N}\left|y_{i}-\hat{y}_{i}\right|\]

where \(N\) is number of observations, \(y_i\) the actual expected output and \(\hat{y}_{i}\) the model’s prediction (same notations below if not indicated otherwise).

Mean bias error (MBE)

\[\mathrm{MBE}=\frac{1}{N} \sum_{i=1}^{N}\left(y_{i}-\hat{y}_{i}\right)\]

Mean square error (MSE)

\[\mathrm{MSE}=\frac{1}{N} \sum_{i=1}^{N}\left(y_{i}-\hat{y}_{i}\right)^{2}\]

Root mean square error (RMSE)

\[\mathrm{RMSE}=\sqrt{\frac{1}{N} \sum_{i=1}^{N}\left(y_{i}-\hat{y}_{i}\right)^{2}}\]

Coefficient of determination (\(R^2\))

\[ \begin{align}\begin{aligned}R^{2}= 1-\frac{\mathrm{MSE}(\text { model })} {\mathrm{MSE}(\text { baseline })}\\\mathrm{MSE}(\text { baseline })= \frac{1}{N} \sum_{i=1}^{N}\left(y_{i}-\overline{y}\right)^{2}\end{aligned}\end{align} \]

where \(\overline{y}\) is mean of observed \(y_i\).

These presented with plots (e.g. scatter, time series) allow identification of periods where model perform well/poorly relative to observations. It should be remembered that both the model (e.g. parameters, forcing data) and the evaluation observations have errors.