Anthropic says its AI safety scores depend heavily on how they’re measured
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
Anthropic published a guide on building AI safety tests, arguing that a model's safety score depends heavily on how the test is set up. It reports that Claude Fable 5.1 behaved appropriately in 94% of self-harm test conversations in the Claude app, but only 60% through the API, where a…