- Automate in order of cost of failure: authentication, payment, then the core create and read paths.
- The API test pyramid is wide at the endpoint level, narrower for chained workflows, and thin at end-to-end.
- Three causes explain most flakiness: shared mutable data, fixed sleeps, and live third-party dependencies.
- Measure endpoint-and-status coverage, not line coverage. Most suites cover happy paths well and error branches badly.
What to automate first
Test automation projects fail by trying to cover everything at once. Sequence by the cost of a failure instead:
- Authentication. If login breaks, nothing else matters, and every other test depends on it.
- Payment and billing. Failures here cost money directly and are the hardest to detect after the fact.
- Core create and read paths for your primary resource — the endpoints touched by nearly every user session.
- Endpoints with a history of defects. Your bug tracker is a ranked list of where your code is weakest.
- Everything else, as it changes. New endpoints get tests when written; old stable ones can wait.
This ordering also front-loads the value. A suite covering the first three categories catches the majority of incidents while remaining small enough to run on every commit.
The API test pyramid
| Layer | Scope | Count | Runtime | Runs on |
|---|---|---|---|---|
| Endpoint tests | One endpoint, one behaviour | Hundreds | 10–80ms each | Every commit |
| Workflow tests | 3–8 chained endpoints | Dozens | 1–5s each | On merge |
| End-to-end | Full journey with real dependencies | A handful | 10s+ each | Pre-release |
Load, security, fuzz and chaos suites sit alongside this pyramid rather than inside it. They answer different questions, take far longer, and belong on a schedule rather than on the commit path.
The ratio matters more than the absolute numbers. When workflow tests outnumber endpoint tests, the suite becomes slow and diagnosis becomes guesswork — any of eight steps could be responsible for a failure.
Eliminating flakiness
A flaky suite is worse than no suite: it trains the team to ignore red builds. Three causes account for most of it.
Shared mutable test data
Tests that assume a specific record exists fail when another test modifies it, and fail differently depending on execution order. Fix: create the data each test needs at the start of that test and tear it down afterwards. Tests should be runnable in any order, in parallel, repeatedly.
Fixed sleeps
sleep(2) makes a suite slow and flaky simultaneously — too short when the system is loaded, wasted time when it isn't. Fix: poll for the expected condition with a timeout. Wait for the state you need, not for a duration.
Live third-party dependencies
A test calling a real payment sandbox fails when that sandbox has an outage, and the failure has nothing to do with your code. Fix: stub external services for functional tests and verify the real integration separately on a schedule.
Quarantine any test that fails intermittently. Move it out of the blocking suite, file it as a defect, and fix it deliberately. Tolerating a 2% failure rate across 300 tests means roughly one red build in every six for no reason — and the team stops reading results entirely.
Test data strategy
- Generate, don't fixture. Build records with a factory that produces unique values per run rather than loading a shared seed file.
- Namespace everything. Prefix generated records with the run ID so parallel runs never collide and cleanup is unambiguous.
- Clean up in teardown, not setup. Cleaning at the start leaves the environment full of debris when a run aborts.
- Never depend on production data. It changes, it's subject to access rules, and using it in tests is how personal data ends up in CI logs.
Coverage metrics that mean something
Line coverage is close to meaningless for API suites — a single happy-path call through a controller can execute most of the lines while testing almost none of the behaviour. Track these instead:
- Endpoint coverage. The percentage of documented endpoints with at least one passing test. Aim for 100%; anything less is an endpoint nobody is watching.
- Status-code coverage. The percentage of documented response codes actually exercised. This is where suites are weakest: the 200 is covered, the 409 never is.
- Authorization coverage. The percentage of endpoints tested with a second, unauthorized user. Usually near zero, and the highest-value gap to close — see security testing.
- Mean time to diagnosis. How long from a red build to knowing the cause. If it exceeds a few minutes, failure messages need work.
Where AI generation fits
The bottleneck in API test automation has never been running tests — it is writing and maintaining them. Deriving cases from an endpoint's shape addresses precisely that bottleneck: the happy path, the type violations, the missing-field cases and the boundary values are all mechanically derivable from a schema.
What generation does not replace is judgement about your domain. Whether a refund may exceed the original charge, or whether a cancelled order can still ship, is business logic no schema encodes. The productive division is to generate the mechanical bulk and spend human review on the rules that matter.
Test your API without writing the tests
Paste a URL. Flasqo discovers your endpoints, generates the suite and runs it — free, no credit card.
Start testing free