How to Use Decision Models for Evals

This Hrih News piece argues that the default 0.50 decision threshold used by new decision models for automated evals often leaves performance on the table, and that tuning thresholds, weighing cost against accuracy, and calibrating scores can make these cheap classifiers far more useful.