Audit test value with Codex without dropping important regressions
I asked Codex to make ChatPRD’s CI faster, then asked a more basic question: did we need all of those tests? I specifically wanted to cut assertions about exact prompts and other details that could change without breaking the product.
Codex inspected the tests and recorded what each proposed removal was protecting. We removed nine low-value or redundant tests and trimmed exact wording, editorial-heading, CSS-class, and duplicate rendering assertions. The audit kept the important checks around permissions, privacy, persistence, publication, and recovery that prevents repeated paid calls.
The remaining suite passed locally with the existing opt-in skips unchanged. The useful deliverable was the audit ledger as much as the smaller suite: it explained each removal and pointed to the behavior still covered elsewhere. I could review what protection remained instead of judging the change by a test count.
I would use the same request again when an AI-written test suite starts to freeze implementation details. Ask what failure each test catches, then make a small, reviewable change. A passing build alone does not replace tests for application behavior.
# Audit test value before deleting tests
Inspect the test suite and its recent failures. Identify tests that primarily assert exact prompt wording, cosmetic details, internal structure, or behavior already covered elsewhere.
For each candidate, explain the actual failure it catches and whether removing it would leave a gap. If the behavior matters, replace the brittle assertion with a behavior check rather than deleting the protection.
Preserve authorization, privacy, persistence, recovery, publication, concurrency, and safeguards against duplicate paid operations. Preserve existing opt-in boundaries; do not invoke paid services to run this audit.
Make a small change and keep an audit ledger with the removed assertion, rationale, and surviving coverage. Run the relevant tests and required repository checks. Report skips and distinguish local results from hosted CI. Do not change branch protections or merge the change.