Skip to content

[ENH] Allow TorchExperiment metric to set "higher is better" #215

Description

@SimonBlanke

The TorchExperiment assumes all metrics should be minimized (lower is better) by default. This can be seen here in this line. However, many metrics should be maximized (accuracy, F1, AUC), requiring users to manually negate them or wrap them in custom logic. The integration doesn't auto-detect metric direction from common naming patterns.

Sklearn's OptCV uses _coerce_to_scorer_and_sign (from _skl_metrics.py) to automatically detect metric direction. Maybe we can write something similar for the TorchExperiment.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions