IncidentAI infrastructure
Railway: attackers escalated privileges in build infrastructure and decrypted environment variables, apparently hunting AI provider keys
Railway's preliminary report says that on October 20 its monitoring flagged elevated latency and engineers found an attempted privilege escalation in build infrastructure. By October 21 the attacker had pulled secrets in encrypted form, reached CI credentials and decrypted artifacts that held more API access, and queried encrypted environment variables, a subset of which were decrypted. Railway invalidated user sessions, told vendors to revoke keys and says the attacker lost access on October 22; running customer services stayed online. Because the keys were envelope-encrypted, Railway could not tell exactly which variables were read; it says at most 1.85 percent of workspaces, about 100,000, had variables potentially exposed, and it notified upstream providers about every key it stored. Railway has not ruled out access to user source code, and did not explain the initial entry point or give exploit details while the investigation runs. It says the attacker's goal was obtaining AI provider API keys, possibly for distillation attacks, and believes it had access to abliterated models and substantial compute; it does not describe agents running the attack. Wiz lists the incident among nine at AI developer platforms in October and says one downstream case of stolen keys ran about 800,000 model calls in roughly 30 hours before the key was remediated.
Replay the attack step by step in the Learning Center
How it happened
- An attacker escalated privileges inside the build system, the part of a platform that handles every customer's secrets.
- From there they reached CI credentials and decrypted environment variables.
- Stolen AI provider keys are worth money immediately: they are validated within minutes and resold.
Impact
Environment variables for up to about 100,000 workspaces potentially exposed; Railway cannot refund spend made with compromised keys.
The fix
If you signed up before October 21, rotate every sensitive variable and key stored on Railway, remove public network access to databases, review GitHub app access and registry audit logs, redeploy the latest commit, and check AI provider accounts for spend you did not make.
Lessons
- A platform that stores your keys is a single point of loss; the build system is where it concentrates.
- AI provider keys are the loot of the month: set spend limits and per-key alerts so a stolen key shows up as a bill spike, not a surprise.
- Envelope encryption protected the keys at rest but not once the attacker held the build credentials.