FAA Halts Every US Departure After Air Traffic Control Network Fails: What Happened and the Lesson for IT Teams

The US Federal Aviation Administration halted every departing flight in the country for several hours on Friday 13 November after a failure in the network that links its air traffic control facilities. The FAA lifted the ground stop at 8:15pm Eastern time, 01:15 UK time on Saturday, after testing its systems. Transportation Secretary Sean Duffy blamed old routers that “got overloaded with data” and began returning inconsistent data, and Homeland Security Secretary Markwayne Mullin said there was no indication of a cyberattack and no connection to the Microsoft 365The KBY LexiconMicrosoft 365Microsoft 365 defined at plain and technical depth, with architecture, operational relevance, an example and a common misunderstanding for practitioners. Read Full Context outage the same week. FlightAware listed more than 10,000 delays and over 4,000 cancellations on Friday, CBS News reported.

If you or your staff are flying to, from or within the US this weekend, check with the airline before going to the airport: Duffy said it would “take us some time” to get back to capacity.

#What officials have said caused it

• Overloaded legacy routers. Duffy said the routers “started to give us inconsistency in the data” and the FAA shut the system down. He described “old switches, old routers” and “1980s little copper lines”, and said none of the newer equipment went down, ABC News reported.
• A change to the network. Multiple sources told CBS News that an update to the private communications network connecting the air traffic control system caused it to reboot, leading to a crash. Mullin said new security patches introduced into the older system appeared to “overwhelm” its routers. CBS says the network is operated by L3Harris, which had not commented.
• Not an attack, so far. Mullin said there was no indication of a “nefarious act” and that there was “no connection” to Microsoft.
• The FAA’s new AI system. Earlier on Friday, Mullin said DHS was looking at whether the FAA’s new AI-based air traffic management system, SMART, played any part, KPRC reported. Officials have not established that it did, and we have not seen anything that says it did.

Duffy used the outage to ask Congress for $50 billion for a “complete overhaul” of air traffic control systems, warning: “You will have more of these.”

#Our view: two big outages, one lesson

We think the most useful detail is the one officials gave almost in passing: on the account so far, security patches pushed to old network equipment overloaded it. That is the same family of failure as the Microsoft 365 outage that started on Thursday, which Microsoft blames on an issue from a network capacity upgrade. Two of the biggest outages of the year, in the same week, both traced so far to changes made to network infrastructure.

Neither explanation is final, and we would wait for the investigations before drawing firm conclusions about either. But the pattern is familiar to anyone who runs a network: changes to routers and switches are often treated as routine and pushed widely, when they are among the riskiest changes an organisation makes, because one bad change can stop everything that depends on the network at once.

Duffy’s case for new money is reasonable, but we would not let it hide the operational question. Old equipment is a fact of life in most organisations; what keeps old equipment safe is how changes to it are tested and rolled out. We expect the FAA’s and L3Harris’s review to focus on why a patch reached the whole network before anyone saw it overloading a part of it. If it turns out the patch went everywhere at once, that is the lesson, whatever the age of the routers.

#What IT teams can take from it

1. Roll network changes out in stages, one site or segment first, with a pause long enough to see the effect under real load before the next.
2. Test patches for old equipment under realistic load, not just for whether the device boots. Older hardware often has far less headroom.
3. Have a rollback you have actually tried, and a way to reach the equipment if the network you would normally use to manage it is the thing that has failed.
4. Know which business processes stop if one network goes down, and what you do instead. The FAA’s answer was to stop everything safely, which is the right answer for aviation and an expensive one.