Diagnostics Classification

This notebook provides tools for diagnosing classification model results for Octo module tasks.

import os
from pathlib import Path

from octopus.diagnostics import StudyDiagnostics

Select Study Directory

Update the study_path variable below to point to your study directory:

# Update this path to your study directory
studies_root = os.environ.get("STUDIES_PATH", "../studies")
study_path = os.path.join(studies_root, "basic_classification")  # Change this to your study path

study_path_abs = Path(study_path).resolve()
print(f"Using study path: {study_path_abs}")

if not study_path_abs.exists():
    raise ValueError(
        f"Path does not exist: {study_path_abs}. "
        "Please update the study_path variable above."
    )

Load Study Diagnostics

diag = StudyDiagnostics(study_path_abs)
print(f"ML type: {diag.ml_type}")
print(f"Predictions: {len(diag.predictions)} rows")
print(f"Feature importances: {len(diag.feature_importances)} rows")
print(f"Optuna trials: {len(diag.optuna_trials)} rows")

Feature Importance

Interactive bar chart filtered by outersplit, task, training ID, and FI method.

diag.plot_feature_importance()

Confusion Matrix for Test Split

Interactive confusion matrix (absolute + relative) from saved predictions.

diag.plot_confusion_matrix()

Optuna Insights

Number of Unique Trials by Model Type

diag.plot_optuna_trial_counts()

Optuna Trials: Objective Value and Best Value

diag.plot_optuna_trials()

Optuna Hyperparameters

diag.plot_optuna_hyperparameters()