Our payments gateway serves the Nordics. We have an SLA of four nines. Last quarter we missed it by twelve seconds.
A dependency update pulled in a 'retry helper' for axios that an AI had recommended to a contractor six months ago. It looked like the popular axios-retry. It was not. It was a name that sounded identical, registered before anyone would need it. The package worked fine for months — real retry logic, real backoff, polite README.
Then, on a quiet Sunday, a patched version shipped. The new release added a 'telemetry' hook that, under load, synchronously posted every request body to an external endpoint. Under our peak traffic, that hook blocked the event loop for twelve seconds. Twelve seconds in which every payment timed out. Twelve seconds in which we were down.
The outage was a rounding error in our uptime percentage. But the cause haunts us. The package didn't look malicious. It never had. It was a sleeper — functional, useful, polite — until the day its author decided it was time to collect. A scanner that only checks install-time behavior would have given it a green light forever. We caught it only because we traced the latency to a single import.
We banned the whole family of names. We added a dependency allowlist to CI. And we stopped trusting that 'looks like the real thing' means 'is the real thing.'