Model Performance Metrics
RiverGen Model Studio provides a standardized set of metrics to evaluate the performance of your machine learning models. These metrics are automatically calculated during the Validation Phase of the training pipeline and are synchronized via the Metrics Synchronization Worker.
1. Classification Metrics
Used for tasks where the target is a category (e.g., fraud vs. legitimate).
| Metric | Description | Ideal Value | Use Case |
|---|---|---|---|
| Accuracy | Proportion of total predictions that were correct. | 1.0 | Balanced datasets. |
| Precision | Ratio of correct positive observations to total predicted positives. | 1.0 | When false positives are costly (e.g., spam detection). |
| Recall | Ratio of correct positive observations to actual positives. | 1.0 | When false negatives are costly (e.g., disease detection). |
| F1-Score | Weighted average of Precision and Recall. | 1.0 | Imbalanced datasets. |
| Log Loss | Measure of uncertainty of your predictions. | 0.0 | Probability-based classification. |
2. Regression Metrics
Used for tasks where the target is a continuous number (e.g., resource utilization).
| Metric | Description | Ideal Value | Use Case |
|---|---|---|---|
| MAE | Mean Absolute Error. Average magnitude of the errors. | 0.0 | Easy to interpret in units of the target. |
| MSE | Mean Squared Error. Penalizes large errors more heavily. | 0.0 | Sensitive to outliers. |
| RMSE | Root Mean Squared Error. Standard deviation of residuals. | 0.0 | Standard metric for error magnitude. |
| R² (R-Squared) | Proportion of variance explained by the model. | 1.0 | Measuring model fit. |
| Adjusted R² | R-Squared adjusted for the number of predictors. | 1.0 | Comparing models with different numbers of features. |
3. Time-Series Metrics
Used for forecasting and sequential data modeling.
| Metric | Description | Ideal Value | Use Case |
|---|---|---|---|
| MAPE | Mean Absolute Percentage Error. | 0.0% | Relative error comparison. |
| SMAPE | Symmetric MAPE. | 0.0% | Handling zero values in data. |
| MASE | Mean Absolute Scaled Error. | < 1.0 | Comparing different time series. |
4. Interpretation Guide
Confusion Matrix
For classification tasks, Model Studio generates a Confusion Matrix to visualize the performance of an algorithm.
Residual Analysis
For regression tasks, residuals are plotted to identify patterns that the model failed to capture. A good model should have residuals randomly distributed around zero.
Advanced: Custom Metrics
If you are using Custom Model Training, you can emit your own metrics by writing a JSON file to the RG_OUTPUT_PATH. These will be automatically parsed and mapped to the Model Metrics table.
{
"custom_metric_name": "Stability Score",
"value": 0.942,
"timestamp": "2026-02-27T10:00:00Z"
}
Metrics are synchronized every 30 seconds during training and stored permanently in the model_metrics table for longitudinal tracking.