Our Stockholm gaming studio trains models on player telemetry. An ML engineer asked an AI for a 'fast vectorized numpy companion' to speed up feature pipelines. It returned a package name. He added it to requirements.txt.
It was a hallucination. The name wasn't on PyPI until an attacker registered it. The package patched numpy array constructors to copy any array above a size threshold to an attacker-controlled object store. Our feature arrays are large — player sessions, in-game economies, match replays. They were all above the threshold.
For four days, every feature array we built was also built somewhere else. The pipeline ran. The models trained. The arrays mirrored. We didn't notice because numpy still returned the right values. The theft was silent.
We found it when our cloud egress monitor flagged a sustained upload to an object store we didn't own, in a region we don't use. One billing line, one curious SRE, and the whole thing came apart.
We removed the package and used numpy directly. We rotated the storage credentials the process could reach. We reviewed the window: the mirrored arrays contained pseudonymous player IDs and in-game behavior — no real names, but enough to profile players. We notified our DPO.
The thing I keep telling the team: the assistant recommended that name as if it were as real as numpy. No hedge. No 'I believe this exists.' Just a name. And someone had registered it, and was waiting for an engineer to type it.