We ran 16 pull requests over 3 days. Every single one of them passed. The CI pipeline was solid green.

The problem was that the code was fundamentally broken, and exactly zero tests were actually executing.

The Illusion of Safety

AMA is a mobile app with a shared Kotlin Multiplatform core and a native UI layer. We have strict CI gates. You cannot merge without passing the test suite.

For three days, developers pushed code, saw the green checkmark, and merged. The test suite reported success in seconds. It felt incredibly fast. It was fast because it was doing absolutely nothing.

A recent configuration change had updated the testMatch path. The goal was simple, but a typo crept in: the configuration instructed the runner to look for .spec.ts files instead of .test.ts.

How 0 Becomes True

When a test runner is given a path that matches zero files, it doesn’t crash. It faithfully reports exactly what it did: it ran 0 tests, and 0 tests failed.

Since no tests failed, the runner exits with code 0. Since the runner exits with code 0, the CI pipeline marks the job as a success. Since the job is a success, the pull request gets a green light.

The system isn’t broken. It is doing exactly what it is programmed to do. That is what makes it so dangerous.

The Cost of Silent Failures

We only caught it when a glaring logical bug—one that a core unit test explicitly checks for—made it to the staging environment.

We checked the test logs for that PR. Tests: 0 passed, 0 total

A test suite that lies to you is worse than having no tests at all. If you have no tests, you know you have to verify things manually. If you have a green CI, you merge with confidence. A false positive builds trust in a broken artifact.

The Fix

The fix isn’t just correcting the path back to .test.ts. That just solves today’s typo.

The real fix is enforcing that a test suite must execute a minimum number of tests (or strictly > 0) to exit cleanly. The absence of failure is not proof of success. The presence of confirmed, executed assertions is the only proof of success.

If your gate requires tests to pass, ensure the gate actually fails when no tests show up to work.

• #ci
• #testing
• #methodology