FAA issues nationwide flights ground stop ‘due to extraordinary circumstances' and ‘safety'

[AZ_HighCountry]

SecTrans Duffy reporting what I've suspected and shared here. Ignore the requests for funding:

Duffy sounds alarm over aging FAA system after nationwide air travel shutdown

Sean Duffy blamed outdated routers and 1980s-era copper lines for the FAA ground stop, demanding Congress approve $50 billion in emergency funding.

www.foxbusiness.com

I am shocked they are still running on copper for data services. Especially at major airports and ATCs. My employer was moving to fiber 15+ years ago now.

 

[Davey Crockett]

My truck stop is a five hour drive north of Dallas.

The mob should be passing through by now.

Travelers grounded in DFW scramble for a way out after nationwide flight shutdown

Some passengers could get home by air. Others would have to rebook, spend another night in a hotel or commit to hours behind the wheel.

www.cbsnews.com

 

[SolarWarden]

SecTrans Duffy reporting what I've suspected and shared here. Ignore the requests for funding:

Duffy sounds alarm over aging FAA system after nationwide air travel shutdown

Sean Duffy blamed outdated routers and 1980s-era copper lines for the FAA ground stop, demanding Congress approve $50 billion in emergency funding.

www.foxbusiness.com

I am shocked they are still running on copper for data services. Especially at major airports and ATCs. My employer was moving to fiber 15+ years ago now.

It's embarrassing and they are fortunate nobody was hurt or killed. Now the fun will begin by both useless parties on who's to blame. Fun times.

 

[MNZGamerX]

Things are just out of control today -

 

[Davey Crockett]

Blind aliens?

 

[AZ_HighCountry]

It's embarrassing and they are fortunate nobody was hurt or killed. Now the fun will begin by both useless parties on who's to blame. Fun times.

There's no one to blame but themselves. Antiquated infrastructure and lack of proper prioritization. It's a wonder there have not been more outages.

But these people will find a way, as they always due, the deflect the blame in hopes of scoring political points.

 

[Ali_Baba]

I'd be curious to know how data traffic is being managed and what that map looks like.

Catching up with the thread, and the news it does seem to have started with hardware failure/backlog which caused the scenario I thought had happened i.e. inconsistent information caused the split brain/data inconsistency reconciliation issues.

There is obviously an inherent design flaw in the dynamic failure and failover model within the air traffic control system if a switch/router can do this(there should be multiple banks of these switches/routers in a redundant configuration with dynamic failover for hardware issues etc). I guess the ATC hardware architecture and software architecture was designed(poorly) at a time where maybe there wasn't as strong a base in architectural design patterns for fault tolerance and failure & recovery to draw on that we have in our inventory if we do a modern system now.

The ATC really can do with an upgrade if they can just replace the messaging layer and associated physical infrastructure. It will cost many hundreds of millions if not lower billions but no way would that cost $50billion - that seems a stretch.

Given the nature of this type of system and its age, making architectural changes would be difficult given that the original SME knowledge won't be as good. The UK went through the growing pains of a new ATC, as did Europe so plenty to draw upon.

I wonder if the telecoms issue has caused a split brain scenario where you no longer know what the article of truth is on the real air picture is. Typically you can get a firestorm of attempts at data reconciliation in this scenario which can use a melt down. Safest in this scenario is to terminate all flights and reset.

I am extrapolating this from it be a comms(telecom) issue.

Good to hear that this all was resolved with issue with life or people thankfully. A rare time where the pilots showed what their worth was to bring the planes home safetly.

 

[AZ_HighCountry]

The US ATC system is in need of replacement. Those of us, including news sources, are not joking when we say that parts of it are still operating on floppy disks. As I posted earlier, part of the system are older than forum members posting here.

The system needs a complete overhaul from the ground up. IT systems, data and voice communications, get off copper and move to fiber including full redundancy, etc. Damn, I could go on and on about that.

How in the hell do you NOT have a DR (disaster recovery) process in place? I was tasked with that in the division I worked in at DHL. None of our critical systems had DR. Was like that for years. We designed a process and system, built a small model for POC, and once it worked, we began to roll it out and scale it. Took 10 months from start to finish. In fact, the final, and most critical piece of the system, went live while I was in Kenya for most of the month of October that year.

 

[RescueRanger]

SecTrans Duffy reporting what I've suspected and shared here. Ignore the requests for funding:

Duffy sounds alarm over aging FAA system after nationwide air travel shutdown

Sean Duffy blamed outdated routers and 1980s-era copper lines for the FAA ground stop, demanding Congress approve $50 billion in emergency funding.

www.foxbusiness.com

I am shocked they are still running on copper for data services. Especially at major airports and ATCs. My employer was moving to fiber 15+ years ago now.

whilst MPF offers greater redundancy and tolerance vs assured ethernet, I am still struggling to understand how they didn't have a business continuity plan for this shit show?

Hopefully this will now highlight why investment and overhaul are the need of the hour.

I mean we are in 2026 and reliance on tech from the 80s for Mission critical systems is simply dumbfounding:

Case in point:.

 

[AZ_HighCountry]

My truck stop is a five hour drive north of Dallas.

The mob should be passing through by now.

Travelers grounded in DFW scramble for a way out after nationwide flight shutdown

Some passengers could get home by air. Others would have to rebook, spend another night in a hotel or commit to hours behind the wheel.

www.cbsnews.com

Down in PHX, it was reported on the local news that hotels surrounding Sky Harbor took advantage of the situation as people tried to book rooms. Rates as high as $1000 / night.

 

[Ali_Baba]

whilst MPF offers greater redundancy and tolerance vs assured ethernet, I am still struggling to understand how they didn't have a business continuity plan for this shit show?

Hopefully this will now highlight why investment and overhaul are the need of the hour.

I mean we are in 2026 and reliance on tech from the 80s for Mission critical systems is simply dumbfounding:

Case in point:.
View attachment 222863

The problem with initiating, managing and delivering change successfully, especially in a safety critical environment like ATC is information, documentation and SME of the platform. Given how old the platform is, most of the people who know or worked on it have retired, and what is left are the operational people who know how to "run the platform" but not necessarily know how to change the platform on any scale. Who ever is left working the ATC will know how to tinker, and make minor changes right now and not to the point where i suspect that they know enough to be able to change the messaging layer and associated hardware. All of the stuff will be propriety I suspect as well.

 

[AZ_HighCountry]

whilst MPF offers greater redundancy and tolerance vs assured ethernet, I am still struggling to understand how they didn't have a business continuity plan for this shit show?

Hopefully this will now highlight why investment and overhaul are the need of the hour.

I mean we are in 2026 and reliance on tech from the 80s for Mission critical systems is simply dumbfounding:

Case in point:.
View attachment 222863

As I've already suggested: other priorities. And, both parties can be fully blamed.

In my own case at DHL, up until 15 years ago, we were still installing POTS lines and USRobotics 56k modems as backups into the routers in the event of an outage. 2010 and we're still deploying dial-up services? I lost my mind on that one. Even got reprimanded because "I didn't understand".

Yeah, I did.

When I was handed the DR program, it was wondered if I would be able to deliver. It had already failed twice. Led both times by Directors (I was not at that time) one of whom is now an SVP of Software Engineering. Our CIO thought it was a great idea to make it one of his KPIs for our CEO. Great!

So when it was assigned to me, part of my team included those two directors and one other. I told them all "this is the way it is going to be. Some things will not be open to debate since the boss (our CIO) has made this a Top 5 KPI. Failing a 3rd time is not an option."

As I said, we did not fail.

 

[AZ_HighCountry]

The problem with initiating, managing and delivering change successfully, especially in a safety critical environment like ATC is information, documentation and SME of the platform. Given how old the platform is, most of the people who know or worked on it have retired, and what is left are the operational people who know how to "run the platform" but not necessarily know how to change the platform on any scale. Who ever is left working the ATC will know how to tinker, and make minor changes right now and not to the point where i suspect that they know enough to be able to change the messaging layer and associated hardware. All of the stuff will be propriety I suspect as well.

You are spot on.

Don't ask how I know. :-(

 

[AZ_HighCountry]

My truck stop is a five hour drive north of Dallas.

The mob should be passing through by now.

Travelers grounded in DFW scramble for a way out after nationwide flight shutdown

Some passengers could get home by air. Others would have to rebook, spend another night in a hotel or commit to hours behind the wheel.

www.cbsnews.com

Donut racks full?

 

[Ali_Baba]

Duffy sounds alarm over aging FAA system after nationwide air travel shutdown

Sean Duffy blamed outdated routers and 1980s-era copper lines for the FAA ground stop, demanding Congress approve $50 billion in emergency funding.

www.foxbusiness.com

The FAA's previous modernization efforts have included a nearly $5 billion contract awarded to Rohde & Schwarz USA in April to replace up to 462 outdated voice communication switches.

More than 40 new switches have since been installed, according to FLYING magazine, while federal officials have also reported replacing roughly half of the aging copper wiring used to transmit radar and communications data with modern connections.

This being the USA, there is a lot of massive profiteering going on with the upgrade. That is ALOT of money to replace switching infrastructure, even if you put in multiple dedicated redundant fibre optic runs between switches imho ...

Projects like this are feasting time for companies like Rohde & Schwarz .. That is part of why nothing gets upgraded when it needs to be done.