The story’s score and claims

Comments on No. 16 of 106 · martinfowler.com · edition of Thursday 12 November 2026

Tests as Data

Summary

Matteo Vaccari, writing on martinfowler.com, argues that AI-written tests are hard to trust because they are voluminous, hard to read, coupled to implementation and often just confirm current behavior. He proposes customer-style black-box tests kept as data (tables, YAML, or small text DSLs) run by a short runner an agent can generate, making changes easier to spot in review. His support is interviews with 26 Thoughtworks developers and worked examples; the quality-decline reports he cites are not sourced, and he offers no measured results.

Comments

Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules