Case study · Aviation

The CrowdStrike outage: when one update grounds the planes

Worldwide, 2024. A faulty security software update crashes 8.5 million Windows devices. Delta takes five days to recover.

In one sentenceIt was not an attack: one update deployed everywhere at once was enough. Your security tool is also a single point of failure.

Analysis based solely on public information. FZ Consultation was not involved in this incident and has no affiliation with the organizations named.

What happened

19 July 2024, 04:09 UTC
CrowdStrike releases a configuration update containing a logic error for its Falcon sensor on Windows.
05:27 UTC
The update is pulled, but machines already hit stay down and often have to be fixed one by one.
19 July
Airlines, banks, hospitals and emergency services are disrupted; thousands of flights are cancelled.
19–24 July
Delta says it cancelled about 7,000 flights affecting 1.3 million passengers, at a cost of about $500m.
October 2024
Delta sues CrowdStrike; CrowdStrike disputes responsibility.

According to the public sources listed below.

The update reached all affected machines at once, with no staged rollout to catch the error.

Many machines had to be booted into safe mode, sometimes with their BitLocker key, one by one.

Some airlines recovered in a day, others took several, depending on how well their operational systems could catch up.

1
Treat vendors as critical dependencies
A security tool installed everywhere is also a single point of failure.
Prepare: recovery that holds
2
Plan recovery at scale
Access to recovery keys, mass procedures and staff you can mobilize must be ready.
Prepare: recovery that holds
3
Test operational catch-up
Beyond systems, rehearse your ability to reschedule crews, stock or appointments after the outage.
Prove: rehearse before the real day

Three questions to put to your own organization this week. If you cannot answer one of them in five minutes, that is your first task.

Which software receives automatic updates across your whole estate at once, with no staged rollout?
Where are your recovery keys (BitLocker, for example), and could you reach them if your network were down?
Once systems are back, how long to reschedule crews, stock or appointments?