What is regression testing?
Regression testing confirms that recent code changes haven't broken existing features. For APIs, this means: **Does this endpoint still return the same response structure, status codes, and data after our latest deploy?**
The test works by comparing current behavior against a baseline ("golden snapshot") captured when the API was known to work correctly. Any unexpected difference triggers a failure.
Why regression testing matters
Feature development creates risk. A seemingly innocent change in one part of your codebase can cascade and break unrelated endpoints. Regression testing catches these accidents before users do.
Real-world regression failures
- Knight Capital ($440M loss): Deployment activated old code, breaking live trading systems.
- AWS S3 outage (2017): Typo in deployment script took down half the internet for 4 hours.
- Stripe API (2019): Breaking change in v2 API rollback broke thousands of integrations.
- GitHub (2020): Database schema change broke OAuth flow for 2 hours.
What regression tests detect
- Schema changes: Field removed, renamed, or type changed
- Status code changes: Endpoint now returns 500 instead of 200
- Response structure changes: Array became object, nested field moved
- Performance regressions: Response time jumped from 100ms to 2s
- Security regressions: Auth bypass reintroduced after refactor
- Business logic changes: Calculation formula accidentally modified
How to run regression tests in Flasqo
-
Navigate to Regression Testing
Go to Testing Types → Regression Testing → Launch -
Enter API base URL
Example:https://api.yourapp.com -
Select endpoints to test
Add critical endpoints:
•GET /users/:id
•POST /auth/login
•GET /products
Flasqo can auto-discover endpoints from OpenAPI spec -
Capture baseline snapshot
Click "Capture Baseline"
Flasqo calls each endpoint and saves:
• Response body (JSON structure)
• Status code
• Headers
• Response time
This becomes your "known-good" reference -
Make code changes and deploy
Update your code, deploy to staging/production -
Run regression test
Click "Run Regression Test"
Flasqo re-calls all endpoints and compares results to baseline -
Review differences
• ✅ No changes: API behavior unchanged
• ⚠️ Minor diff: Non-breaking change (new optional field)
• ❌ Breaking change: Field removed, type changed, status code different
View side-by-side diff of baseline vs current -
Update baseline or fix bug
• If change is intentional → Update baseline
• If change is a bug → Rollback deployment and fix
Types of regression testing
1. Response regression (most common)
Compares JSON response bodies. Catches schema changes, missing fields, type changes.
2. Visual regression
For HTML/rendered endpoints. Takes screenshots and compares pixel-by-pixel. Detects CSS/layout breaks.
3. Performance regression
Compares response times. Alerts if latency increased significantly (e.g., 200ms → 2s).
4. Security regression
Checks if auth requirements changed. Detects if endpoints became public accidentally.
Baseline management strategies
When to update baselines
- Intentional API changes: New field added, deprecating old field
- Version bumps: v1 → v2 API migration
- After fixing regression bugs: Re-baseline to lock in correct behavior
When NOT to update baselines
- Test failures: Don't update baseline to make tests pass!
- Production incidents: Baseline should reflect working state, not broken state
- Flaky responses: Fix flakiness first, then re-baseline
Best practices
1. Run regression tests before every production deploy
Make regression tests a deployment gate. If tests fail, stop deployment automatically.
2. Test against staging first
Run regression tests on staging environment after deploy. Catch issues before production.
3. Ignore non-deterministic fields
Exclude timestamps, UUIDs, and random data from comparison:
{
"id": "UUID-IGNORE",
"created_at": "TIMESTAMP-IGNORE",
"price": 29.99 // Compare this
}
4. Version your baselines
Store baselines in version control alongside code. When you create a new feature branch, you have a matching baseline.
5. Test both success and error paths
Baseline both 200 responses AND 404/400 errors. Ensure error messages don't change unexpectedly.
Prevent breaking changes
Automated baseline comparison for every deployment. Catch regressions in seconds.
Start Regression Testing FreeFrequently asked questions
How often should I update my baselines?
Update baselines only when you intentionally change API behavior. Treat baselines like unit test assertions — they define correct behavior. If tests fail, fix the bug, don't update the baseline to match broken behavior.
What's the difference between regression testing and functional testing?
Functional testing validates that features work correctly (does login accept valid credentials?). Regression testing validates that features STILL work after changes (does login still work after we refactored the database?). Run functional tests once; run regression tests after every deploy.
Can regression tests catch performance issues?
Yes, if you include response time in your baseline. Flasqo can alert if response time increases by more than a threshold (e.g., 50% slower than baseline). This catches performance regressions like N+1 queries or missing indexes.
Should I run regression tests in production?
Yes, but carefully. Run regression tests immediately after production deployment to verify the deploy succeeded. Use read-only endpoints or test accounts to avoid side effects.