Tests as Data
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
Matteo Vaccari, writing on martinfowler.com, argues that AI-written tests are hard to trust because they are voluminous, hard to read, coupled to implementation and often just confirm current behavior. He proposes customer-style black-box tests kept as data (tables, YAML, or small text DSLs) run by a short runner an agent can generate, making changes easier to spot in review. His support is interviews with 26 Thoughtworks developers and worked examples; the quality-decline reports he cites are not sourced, and he offers no measured results.