Model evaluation error metrics

TSCC.assessment.model_error_metrics.NSE(y_pred, y_true)

Measure the Nash-Sutcliffe Efficiency (NSE)

Parameters:

y_gtpandas series

Pandas series of the ground truth dataset.

y_predpandas series

Pandas series of your prediction.

Returns:

float

The Nash-Sutcliffe Efficiency.

TSCC.assessment.model_error_metrics.abs_vol_error(y_gt, y_pred)

Measure the absolute volume error (AVE) assuming equal weighting (equidistant time steps) for all observations

Parameters:

y_gtpandas series

Pandas series of the ground truth dataset.

y_predpandas series

Pandas series of your prediction.

Returns:

avefloat

The absolute volume error.

Examples

>>> gt_series = pd.Series([1, 2])
>>> pred_series = pd.Series([3, 4])
>>> ave = TSCC.assessment.abs_vol_error(gt_series, pred_series)
>>> ave
XY

TSCC.assessment.model_error_metrics.indicator_error_relevance(y_gt, y_pred, subset_boundary, relevance_function, *extraArgs)

Measure the indicator error and relevance of each value.

Parameters:

y_gtpandas series

Pandas series of the ground truth dataset.

y_predpandas series

Pandas series of your prediction.

thresholdint or float

A threshold for the difference of gt-value and pred-value. If the difference is lower than the threshold, then this values become irrelevant for the indicator.

relevance_functionfunction

A function you can use to set a threshold, so lower gt-values won’t be included in this measurement.

*extraArgsint or float

A threshold for relevance_function.

Returns:

ind_relint or float

The indicator error.

Examples

>>> gt_series = pd.Series([1, 2])
>>> pred_series = pd.Series([3, 4])
>>> ind_rel = indicator_error_relevance(gt_series, pred_series, relevance_fct_indicator, 1)
>>> ind_rel
1.0

TSCC.assessment.model_error_metrics.peak_error(y_gt, y_pred)

Measure the peak error assuming equal weighting (equidistant time steps) for all observations

Parameters:

y_gtpandas series

Pandas series of the ground truth dataset with datetime index.

y_predpandas series

Pandas series of your prediction with datetime index.

Returns:

apefloat

Absolute Peak Error (APE), absolute deviation from peak value.

rpefloat

Relative Peak Error (RPE), relative deviation from peak value.

ptefloat

Peak Time Error (PTE) , time deviation from peak time.

Examples

>>> gt_series = pd.Series([1, 2])# index must be datetime
>>> pred_series = pd.Series([3, 4])# index must be datetime
>>> ape, rpe, pte = peak_error(gt_series, pred_series)
>>> rve
XY

TSCC.assessment.model_error_metrics.rate_error_relevance(y_gt, y_pred, relevance_function, *extraArgs)

Measure the error rate and relevance of each value.

Parameters:

y_gtpandas series

Pandas series of the ground truth dataset.

y_predpandas series

Pandas series of your prediction.

relevance_functionfunction

A function you can use to set a threshold, so lower gt-values won’t be included in this measurement.

*extraArgsint or float

A threshold for relevance_function.

Returns:

rate_relint or float

The error rate.

Examples

>>> gt_series = pd.Series([1, 2])
>>> pred_series = pd.Series([3, 4])
>>> rate_rel = rate_error_relevance(gt_series, pred_series, relevance_fct_indicator, 0)
>>> rate_rel
1.5

TSCC.assessment.model_error_metrics.rel_vol_error(y_gt, y_pred)

Measure the relative volume error (RVE) assuming equal weighting (equidistant time steps) for all observations

Parameters:

y_gtpandas series

Pandas series of the ground truth dataset.

y_predpandas series

Pandas series of your prediction.

Returns:

rvefloat

The relative volume error.

Examples

>>> gt_series = pd.Series([1, 2])
>>> pred_series = pd.Series([3, 4])
>>> ave = rel_vol_error(gt_series, pred_series)
>>> rve
XY

TSCC.assessment.model_error_metrics.relevance_fct_indicator(series, subset_boundary=(- inf, inf))

Indicator function

Parameters:

seriespandas series

Pandas series of which entries are assessed regarding its relevance.

subset_boundarytuple

Lower and upper boundary of subset receiving value one, and all other elements to zero

Returns:

seriespandas series

Pandas series of boolean dtype.

TSCC.assessment.model_error_metrics.relevance_fct_root(series, addend=1e-05)

Root function

Parameters:

seriespandas series

Pandas series of which entries are assessed regarding its relevance.

addendfloat

Addend of root function

Returns:

seriespandas series

Pandas series of boolean dtype.

TSCC.assessment.model_error_metrics.squared_error_relevance_area(y_gt, y_pred, y_relevance=None, interval_size=0.001)

Measure the squared error relevance and its area proposed by Ribeiro and Moniz, 2020

Parameters:

y_gtpandas series

Pandas series of the ground truth dataset.

y_predpandas series

Pandas series of your prediction.

y_relevancepandas series

Pandas series of the obervations relevances.

interval_sizefloat, optional

An interval size between 0 and 1. The default is 0.001.

Returns:

serafloat

The squared error relevance area.

df_serpandas dataframe

The squared error relevances for each phi(y).

Examples

>>> gt_series = pd.Series([1, 2])
>>> pred_series = pd.Series([3, 4])
>>> sera, ser = squared_error_relevance_area(gt_series, pred_series, relevance_fct_root(gt_series))
>>> sera
1.5
>>> plt.plot(ser["phi"], ser["ser_phi"])