No Way Out

Outmaneuver Complexity: AI Gold Rush 2.0 & Adaptive Capacity with David Woods, PhD

• Mark McGrath and Brian "Ponch" Rivera • Season 3 • Episode 122

Share episode

Send us a text

What happens when our increasingly interconnected systems face unexpected challenges? Dr. David Woods, pioneer of resilience engineering, explores how organizations can build the adaptive capacity needed to survive in an age of growing complexity.

Drawing from decades studying high-risk industries, Woods frames our current technological moment with historical perspective. The "second AI gold rush" unfolds with familiar patterns – promising seamless automation while overlooking the inevitable new complexities and vulnerabilities that emerge. Through compelling examples from Boeing's 737 MAX disasters to financial system collapses, he demonstrates how brittle systems eventually break when organizations prioritize short-term productivity over long-term resilience.

Woods introduces core principles of adaptive organizations – graceful extensibility, the capacity to reconfigure and reprioritize under pressure, and the critical ability to anticipate approaching saturation points before collapse occurs. He challenges the linear thinking that dominates most organizations, explaining why reframing – updating our mental models to match changing reality – proves so difficult yet essential for survival.

Whether you're navigating organizational challenges, interested in the future of human-AI collaboration, or seeking to understand resilience in an uncertain world, this episode provides essential frameworks for thinking differently about complexity, surprise, and adaptation when failure isn't an option.

David Woods on LinkedIn

Dept. of Integrated Systems Engineering, David Woods, PhD 

NWO Intro with Boyd

March 25, 2025

Flow Learning Lab

Find us on X. @NoWayOutcast
Substack: The Whirl of ReOrientation

Want to develop your organization’s capacity for free and independent action (Organic Success)? Learn more and follow us at:
https://www.aglx.com/
https://www.youtube.com/@AGLXConsulting
https://www.linkedin.com/company/aglx-consulting-llc/
https://www.linkedin.com/in/briandrivera
https://www.linkedin.com/in/markjmcgrath1
https://www.linkedin.com/in/stevemccrone

Stay in the Loop. Don't have time to listen to the podcast? Want to make some snowmobiles? Subscribe to our weekly newsletter to receive deeper insights on current and past episodes.
Recent podcasts where you’ll also find Mark and Ponch:

The No Bell Podcast Episode 24
...

David Woods, PhD:

The periods of change we're going through are not new, but every period that has these processes going on has differences in how they play out because the future is always open and how people interact and make decisions and how they combine, sometimes not at all, and sometimes surprisingly, in terms of what builds and leads to forward movement. So it is. You know, the story has been written before. This is a new version of the story. Yeah, so we will try not to step on landmines, but, yes, it is a world with landmines.

Brian "Ponch" Rivera:

Well, I appreciate that. So Dr David Woods is with us today on no Way Out. The last time you and I actually the first time you and I met was 2018. With landmines Well, I appreciate that. So Dr David Woods is with us today on no Way Out. The last time you and I actually the first time you and I met was 2018. We were at a complexity conference for the Navy. It was a great conference on understanding complexity and then since that time, nothing in the world has changed, correct? I mean, we didn't have a pandemic. But I guess my point there is, dr Woods, we've seen so much in the last five or six years, from the mishaps at Boeing, a pandemic. And then I think you're calling this the second age of AI. Is that correct?

David Woods, PhD:

It's the second gold rush. Tech gold rushes have happened before. Right, it can peter out, people still aren't going around with virtual reality glasses in 24-7 to do all activities. So that tech gold rush was hyped and while new technology and there's niche applications for it, some are big niches in gaming, but the gold rush never happened. This is the second AI gold rush.

David Woods, PhD:

I started my career in the first AI gold rush, so many of the patterns that are playing out now are things I lived through and studied and developed other kinds of designs back in the early 80s. There are differences because the nature of the algorithms being developed are different than the families of algorithms that were being developed and hyped Then. One of my jokes about this this gold rush is so much more powerful than the first one because they added one letter to it. It was AI, meaning this is a universal machine-based model of cognition that will substitute for people, and look at all the great things that will happen as we can substitute this new AI for them. And now we have the same rhetoric, only it's AGI. So adding one letter seems to have made so much more happen and make it so much more powerful, when in many ways it's the same story, but with some new families of algorithms.

Brian "Ponch" Rivera:

So I do want to talk about this connection to safety and AI AGI here in a little bit, but I think looking at your background and having met you and understanding your depth of knowledge in complex adaptive systems and other disciplines is a great place to start. Specifically, on this show we talk a lot about John Boyd's Observe Orient Decide Act loop, which he looked at multiple disciplines. When we look back at your growth and involvement with Three Mile Island and where you are today, hopefully you can take us back through the influences that got you to where you are with resilience, engineering, graceful extensibility and your thoughts going forward with this, as complexity seems to be a key to everything we're doing.

David Woods, PhD:

So you put a couple things together there. So one of them is the OODA loop that is so familiar in defense communities, and just that that wasn't a new invention then. Just that that wasn't a new invention, then it was a reinvention, positive to promulgate these ideas that have been around for a long time about cognition. For me it started in 1976 when the person who first coined cognitive psychology went whoops, I was wrong and said, hey, it doesn't work the way we thought it did. Now we can go back much further than him and his insight and switch.

David Woods, PhD:

But I tell that story because in some ways the history of AI is very much around people starting out and, oh, look, how powerful these algorithms are. And then, as they confront the real world, they start to go wait a minute, we're missing a lot. And the algorithmic power by itself does not translate into action power, reasoning power in the real world, given the complexities, variability, uncertainty, surprises. Given the complexities, variability, uncertainty, surprises. So the journey really started with a line that came from some of our early colleagues' papers of coping with complexity, and so what we started in AI Gold Rush 1 in the early 80s was studying how people adapt to cope with complexity and rather than new technology eliminating complexities, it actually did bring new capabilities.

David Woods, PhD:

New capabilities generated forms of growth. As people adapted those capabilities, growth creates new kinds and forms of complexity and the penalties that go with that. And generally what people did was pretend it was linear and with only a few little interactions and complexities we had to wrestle with, but mostly linear, and we kept going nope, it's nonlinear from the beginning. Right, it is. Time and dynamics matter. Always Stopping time or seeing it as a series of snapshots distorts what really goes on.

Brian "Ponch" Rivera:

So just to pause you there for a second, the landscape that you're talking about when you talk about new capabilities is that connected to the idea of the adjacent possible or affordances.

David Woods, PhD:

Sure. The issue is as we like to say. There are three basics that underlie the foundations of resilience engineering that are kind of should be non-controversial and easy to understand. One is there's always finite resources. Two, time never stops. Change is always going on Slower, apparently quiet, but change is always percolating somewhere. And the third is there are other people who are adapting because they are seeking advantage. They're seeking what works for them, given their position in the world, their goals, the constraints they operate under and looking for success.

David Woods, PhD:

So the idea of putting the difficulties that arise from technology in the context of success. In a sense, we really did create a new capability. People really did take advantage of it. That meant they created a new world, they created new roles, they created new interdependencies that led to things were operating at new scales. And so what happened with the linear stuff is nobody designed for any of these changes, right. And so what happens is messes arise, right, snafus arise, inevitably the old military coinage from the grunt American soldier in World War II. And so snafu catching is not a rarity, it is not a corner case, it is ubiquitous because this messiness keeps happening.

David Woods, PhD:

And again, given finite resources, change and others adapting, it turns out. That's fundamental. But the selling of and the excitement in the initial phase of a gold rush is always this projection into the future. That's very linear and they don't see any of these reverberations and transformations. So they don't prepare for the new vulnerabilities, they don't prepare for the new forms of coordination that's necessary, they don't prepare for the new kinds of models that you need in order to understand what's really going on, in order to handle surprise Right, because they think surprise is going away or will be less powerful, and the answer is no.

David Woods, PhD:

You get new surprises from new directions, which are therefore even more surprising in the sense of are you really prepared to handle them? Now, military history is full of this, and one of the great empirical sources for all of this is military history and the contrasts that go on in how do you prepare to be surprised? So one of the lines we used in 2000 when we started resilience engineering was, if you want to summarize what it's about, it's how do you prepare to be surprised? And that paradox actually is the insight that you can prepare even though you don't know what's going to happen.

Brian "Ponch" Rivera:

So I'm curious. And that's the key phrase right there Adaptive capacity. You write about, hall-nagel writes about it, decker writes about it, we've had Todd Conklin on here, we've had John Flack on here, we had Gary Klein on here. Can you help us understand, or help our listeners understand, what that actually means and how you might go about creating adaptive capacity for an organization?

David Woods, PhD:

Well, first, in the biological world, adaptive capacity is everywhere. We can see it in all kinds of examples and many of our success stories places where we really understand how this stuff works can come from different aspects of biology, including human physiology and the stress response reaction. So we have lots of examples, all the way up to human societal scale examples. Adaptive capacity is future oriented. It's not about what you're doing now. It's about the capacity to change what you're doing now. Change your model, change your relationships, change your activities, because what's happening you're sensitive to what's happening is different than what you were prepared to do before. So it's the potential to change, the potential to change Now. What's interesting in this is you know physics, basic physics classes we all took have this potential energy versus kinetic energy. We have energy we have blown off into the world and there is the energy potential to go off in the world. You know to think in military context. It's quite concrete and quite visible. The ammunition is potential. When the artillery piece fires, it just became kinetic. So we should be able to deal with this idea of potential. And ahead, what are we doing to prepare ahead Combat power? Let's take an example there. It's trying right. It needs to get more dynamic and more future oriented. But basically, combat power is a way to think about adaptive capacity. I have set this unit up at this scale of military operations to have certain capabilities to express in situations, tactically, operationally, in supportive strategic directions. Now, in the real world, what happens? Nobody deploys at 100% combat power, and even if you did, you wouldn't have it for very long. Right Finite resources too is what do you do with your units and combat power? They're not all the same. Two is what do you do with your units and combat power? They're not all the same. Right, you have different ones that you coordinate and how you put them together in the history of military operations, no matter which technologies are dominating the military sphere, how you put those different capabilities together in the context of developing challenges and changes at one scale or another. So our tactical, operational strategic is a simplification of the multiple scales, the layers that are going on. Who coordinates the different fundamental functions of military power wins all through history. And so you see this illustration of future adaptive capacity and building up this effectiveness.

David Woods, PhD:

Now notice you have to be able to translate potential into right, in a sense, kinetic adaptation. Let's coin something new here, kinetic adaptation, right, when we have to actually go do it because we're surprised, right, it didn't work the way we thought it would work. Now I have to what Reconfigure and reprioritize so we build a responsiveness. That responsiveness is called into action and we think of that very simply. We can think of that very simply as the deploy, mobilize, generate triad that operates in time. As you're challenged or surprised, what deployable stuff do I have? How do I reconfigure my deployable assets given the challenges that I am experiencing and are growing happens and I'm threatened with running out of capacity to keep pace with the demands of the situation. How do I reprioritize? And what do I do Then? I mobilize or signal processes for mobilizing capabilities that can come into the situation. There are challenges in inserting new capabilities into a dynamic situation. Why? Because of the paradox of overload, I most need help when it's least affordable to insert the help into the situation, because it takes workload capability out to bring other parties in other assets into the situation and make them effective in handling things at a time when I'm running out.

David Woods, PhD:

What's the key thing we see in the studies, whether they're new studies in emergency medicine or whether they're older studies in military history, even recent history, it is you need to anticipate that you're approaching saturation on some attribute If you can anticipate how you're approaching saturation, then you can start to act to build it. You can be extensible. So what I just described in these situations is this universal parameter graceful extensibility that has to exist in all adaptive systems of any type and at any scale. Why? Because these properties we've talked about are universal and these situations are universally encountered at all scales. And so if you don't have that graceful extensibility extensibility, graceful means from where you are, as things are getting hard, right or different, you've got to be able to smoothly do this. And if you can smoothly do this, you avoid the potential of brittle collapse. What's brittle collapse? You reach saturation.

David Woods, PhD:

I run out of the capacity to respond to growing challenges or changes in the world and when I run out, things break. Now this could be an organization, an organizational collapse, and financial. So the Knight Capital a good example of this is the Knight Capital Financial. It was a runaway automation. So our technology comes in. And well, that technology created all kinds of new things new values, new capabilities, new complexities, et cetera. It's extremely automated, it's extremely autonomous automation under direction of the money managers who are in a very human situation, trying to make massive amounts of money Something highly motivating to people. Right, it's definitely a human story of how automation it gives them an advantage.

David Woods, PhD:

And what did they do? They didn't prepare for the way the automation could misbehave. And so when you have high autonomy, high authority, automation, misbehavior very quickly can lead to bad consequences. So the runaway automation is doing runaway trading right, and all of a sudden the software people, who are not the traders, they're not the quants, they're not the C-suite, they're way down. All of a sudden they have to go and say I have to tell right the top of the organization to authorize shutting off trading. The worst thing, the biggest thing you could do, because we don't know what's wrong and the standard things we did to respond to it didn't work. Normally we have tolerance to a variety of disruptions that occur. In this case, none of them work. We don't know where this came from, we need time and, of course, by the time they got to the people and they could make the decision to shut down, they had lost almost a half a billion dollars and were, for all practical purposes, bankrupt. That's a brittle collapse.

Brian "Ponch" Rivera:

So there's a connection here from resilience engineering into DevOps, development operations, continuous deployment. What we've seen over the last 20 plus years is that sometimes you create software for a context and then 20 years later it may be used for something different. Right, and I think that's where you find brittleness in software these days. It may have worked perfectly well years ago, but the context and how people are using it is changing. Does that make sense?

David Woods, PhD:

It certainly does, and you're absolutely right that the best example of the future, regardless of which technological advance we're focusing on or which other form of advance it could be in communication and connectivity. Your comment again was on and connectivity.

Brian "Ponch" Rivera:

Your comment again was on DevOps, the CD and how software in the past could be perfectly, you know, working fine, and then all of a sudden it doesn't work Right.

David Woods, PhD:

So this is where I got involved with them. I mean, devops is only 13 years old and so when I first they've been trying to get me to come and talk with them and they said they were using some of my stuff, and I kept going well, you understand it, you're using it. This is great. What am I going to say to you? And then I showed up and I looked at them and I said you're not using my stuff. They went of course we are. And I said no, you're the future and it's not as advertised.

David Woods, PhD:

What everybody's touting about more autonomy from whatever family of algorithms. It didn't start with the current machine learning ones and large language models. For the last 20 plus years, algorithms have been the source of. We can get more autonomy from our machines. It turns out what happened is growth, new scales, new complexity, penalties. They changed the model.

David Woods, PhD:

Software is operated. We grew capable software that allowed people to do more things that were important to people, not always in coordination, sometimes in conflict, but as that scale grew, what it means to be a software engineer is you develop and operate. Those two are intimately connected in real time, because everything at its heart is software and that software. So the idea and I've just ran into this this last week that people are still thinking in the AI revolution going on right now that, well, software is just coding and we're going to code faster, or the LLM will code for us, and you're like you have no idea what you're talking about. That's not the way modern software, critical software infrastructure works. It has to be operated. It is already highly autonomous, it is fragile and it does break, despite the fact that we build in fault tolerance. And why does our fault tolerance fail us? Because it keeps changing as the software infrastructure. Friends say, if our company isn't growing, it's going to die in this world, in other words, and if it's growing, that means the infrastructure and the demands on the infrastructure and the challenges and the connections and the interdependencies are all changing.

David Woods, PhD:

An example is when we started doing this, the serious incidents where they acted to prevent a loss of valued services generally happened, mostly within the organization operating that infrastructure. We told people, as we started to see some incidents in our snafu catchers project with the industry, that this wasn't always going to be the case, that you were going to have to see across organizational lines because their software was interacting with your software, given what service you provided, given what service that these other folks provided maybe to you, because everybody, to get more efficient and productive, started utilizing specialists in the software infrastructure world. Dns is an example. It makes everything work better and so everybody uses it, and it's mostly invisible. And so what we started to see, we called the multi-party dilemma, which is you can't tell when you start seeing threats of an outage developing, whether it's from your infrastructure or whether it's from an infrastructure you're interdependent with.

David Woods, PhD:

Is it coming at you from somewhere else? Are you going at somebody else and going to make this all worse? Is that going to feed back on you and make this worse? Is it both of you? Is it a funny interaction? Well, depending on legal and other constraints, you may not be able to see what those other people are doing. You may not be able to get them responsive, because your service contract says you have to put a ticket in and they'll get back to you within 24 or 48 hours. You have to put a ticket in and they'll get back to you within 24 or 48 hours.

Brian "Ponch" Rivera:

Well, crowdstrike took more than 24 or 48 hours to fix. Yeah, so what you're describing there is the system that we create and it's just bigger than software. The whole system is actually driving the behaviors of that system. Right, we're reacting to the context that we created, which could include hey, we need a legal review of this thing before we actually can do anything. Well, in the meantime, we're going to get a failure in the system. We can't. We can't fix that.

David Woods, PhD:

That's right, so different layers, different factors, functions have to now coordinate in real time, whereas before we just think of it as a service contract for run-of-the-mill things, but in a crisis it could lead to a widespread outage that feeds back on the party who isn't interacting quickly. We need responsiveness. Now we need to do that with protecting some other criteria. So we have trade-offs because of finite resources and this is where we reprioritize. How do we know? This is going in a way that means we need to increase our responsiveness. This is what happens in mass casualty events. In emergency medicine, we increase responsiveness and we drop certain criteria that are very important to the normal running, efficient and productive running of a hospital system, and everybody understands that we need to reprioritize. We reconfigure how we do certain things.

David Woods, PhD:

So in one of the mass casualty events we studied pharmacy. When you're running out of supplies a pharmacy because you're inundated with people with similar injuries, pharmacy doesn't say yes, yes, enter it into the computer system and we will get it to you. And today we're probably running about three to four hours to get you some of those supplies Right. No, the answer is no. Pharmacy realizes right away and starts getting them, the supplies, and then backfilling the documentation Also in the. So they are adapting ahead, knowing that constriction is coming, the emergency room is going to run out of critical supplies to treat the injured people coming in. So they change what they're doing, they reconfigure and reprioritize. Meanwhile, guess what somebody one layer up starts doing we don't have enough stuff. If more injured people are coming to our hospital, we don't have enough stuff. If more injured people are coming to our hospital, we don't have enough stuff. This is just stuff we're talking about. To treat that number of patients, we have to find the supplies from somebody else. Well, wait a minute, guess what? There were 500 people burned in this accident, just under 500. Every hospital was getting slammed.

David Woods, PhD:

So where are you going to get the additional supplies? Because all of them are looking for that, those resources, and so this is where you see very quickly the standard plan for handling a surprise event quickly fails. Why? Because backup resources, backup expertise, backup help all get oversubscribed. Either they get oversubscribed and therefore run out, or they are actually affected by the disrupting trigger event that started the whole thing as well as you are.

David Woods, PhD:

This is one of the problems with interdependencies and the scale at which challenge events operate today, because if things happen over bigger areas more intensely, then this ability, this issue of being oversubscribed, happens faster and the issue of what you were counting on to help you in an emergency won't be there because they're undermined by the emergency at the same time won't be there because they're undermined by the emergency at the same time. You know we can. Responsiveness was just demonstrated, or the lack thereof, in a tragic and vivid way in the Guadalupe River flood zone. We don't have to get into some of the other missing preparatory things, but the inability to be responsive as things happen, when you know and it's a great example of anticipation, being prepared to be surprised when you have more sudden, intense rain events over larger areas you're going to have flood events. Right, the standard statistical way to refer to extreme weather events reveals that everybody doesn't understand statistics. Sorry, how many 500-year floods has my son lived through in Houston in 10 years there?

Brian "Ponch" Rivera:

They had one recently.

David Woods, PhD:

Not one, that's right. Not one, all right. 15 years ago there were floods in the Front Range in Colorado, just north of Denver, boulder and north. I was there as a graduate student helping a friend in the summertime in the mountains outside Boulder. Guess what? That 100-year flood had happened 30 years before I was there.

Brian "Ponch" Rivera:

Okay, that's Estes Park. Right, yeah, estes Park. Yeah, so I was a kid when the first flood happened. I was in the 70s. I was there, okay.

David Woods, PhD:

That's.

Brian "Ponch" Rivera:

Estes Park? Right, yeah, estes Park. Yeah, so I was a kid when the first flood happened. I was in the 70s. Yeah, yeah, so I know what you're talking about. Yeah.

David Woods, PhD:

Yeah, I mean I was there. There was no social media. We heard on the radio. We jumped in his Jeep and we started heading for their call for for volunteers and before we could get out of down to the flatlands and head over to the canyons that were flooding, they said they had enough volunteers. This was all emergency radio stuff we were picking up. I mean it was a very different world for hearing about these things.

David Woods, PhD:

Of course we looked at each other later and kind of went it's probably a good idea that you know 20 somethings going. Yeah, we'll go volunteer and help with the rescues and things. We're probably like it's probably a good thing we didn't get there in time. But the issue is we know what we know for sure is that things are changing. What we know for sure things are more interconnected. We know surprise will come at us from surprising directions, even though it's a kind of surprise. Oh yeah, we planned for this. But when it comes in a different way, in a different package, at a different intensity, that doesn't mean at all we're prepared, right, because we think too linearly.

Brian "Ponch" Rivera:

So I'm curious. Folks are asking about methods, and what are I'm going to ask you? How do you create, how do you prepare? I'll call it human agent teaming. You know, how are we going to work with AGI in the future, if that's going to be a thing at all or how do we build the capacity to anticipate, to make sense of the environment? What are the tools and techniques that you're recommending leaders use now, if any?

David Woods, PhD:

So there's a lot. The repertoire is growing all the time. We have two problems with our repertoire, so I'll speak in general, since you talked about people at the top of organizations. So one is they're not as mature as what you're used to, right, because people haven't tried them out, they haven't invested in them. It's like saying you know, if we want to design airframes, there was a day when it was expensive to get aerodynamic studies to say whether potential airframes would. There was a day when it was expensive to get aerodynamic studies to say whether potential airframes would really work and what kinds of instabilities would arise. And then it was wind tunnels and then it was no, we can do a lot of it in the computer and then we can do a little wind tunnel and then we can do a little test pilot flights. We matured the ability to do engineering on complex phenomena and so it got more efficient and we forgot that the basics had to be there. Well, we're much earlier in the maturity development of the techniques to build adaptive capacity. There's a bunch of things we do to try to help people do that, and that's the second constraint is we don't have good ways to do it at scale. The big challenge for us if people would invest in us is help us mature it, but help us scale it up so it can have a bigger effect where your investments can be more leveraged.

David Woods, PhD:

But the recipe for doing this is old. The recipes have been around for a long time. When you look back, I can show examples from you know what's the one I'm about to use in a new chapter is the opening line 1806 from Clausewitz now, of how people are stuck in an old model that was valuable for building safety but is no longer effective today because too much has changed and in fact, the people under stress because of safety being at risk I'm referring to aviation in particular here are retreating into the old model. Oh, if we just fine tune it and just add a little here and there, we'll be back to where we were. And the answer is no. The world is different, the processes are different, the pressures are different. Yes, it's still airplanes with people in the back, but the evidence and the accidents have been telling us and in fact, one of our problems there was they were discounting the evidence of trouble ahead. So you know, the recommendation here is you know you go.

David Woods, PhD:

Don't shortcut systems engineering In a complex world, you need to enhance your systems, engineering techniques, otherwise you end up like Boeing, with 50 billion in losses and counting. You know that was a brittle collapse that has crushed the company's viability. Now, a variety of factors have kept the company going and it will keep going in the future because of its position in the industry and human needs. However, it was an enormous, enormous loss on many, many, many fronts, not just financial and not just lives. So you have to recognize in advance the problem is the C-suite isn't anticipating and isn't set up to see the signals of anticipating, because they're so busy saying we just need to be more productive, we just need to have a higher level financial performance. And so, if you you know, a simple example would be and unfortunately the Senate hearings did not address this If I had been invited, I would have highlighted this the chief engineering officer office failed in Boeing's accidents that killed 346 people because they didn't stand up for engineering. They were just an assistant to the chief financial officer.

Brian "Ponch" Rivera:

So that's a goal. Conflict, that's the deficiency thing, that's, hey, we need to get something out the door, right. So I got to tour the 737 line up in Seattle and I noticed a couple of things. When you go up to the 777, 787 line, it looks like a different company than the 737 line. I mean, 737 line is very efficient, very lean. It is something to be seen, right.

Brian "Ponch" Rivera:

And then you learn about the bigger engines being put on this aircraft. You learn about them changing some landing gear configurations potentially, or why they're going to do it this way so they don't have to go through another evaluation of an aircraft, another platform, and you kind of scratch your head like, well, what could happen? And this is 2015, 14 when I went through there to look through all that. And then, of course, we had the accidents a little bit later on and you make the connection back to a pitot static system that had I forget the name of the platform the software platform that was behind a lot of this. But anyway, the software approach, in my opinion, when I saw this was they were using a pseudoscience approach to building software. They were not following, in my opinion, resilience engineering, a DevOps approach right, they were not doing that and again, this is my perspective from being an outsider at working at Alaska Airlines at the time. So this is what I saw and I warned people. We happened to go up to Boeing and say, hey, you shouldn't be applying the pseudoscience approach. And you're ready for this, dr Woods, and I want to get your view on this you should be applying something that you learn above the wing, and that is the application of team science, crew resource management. How do you trap errors and how do you anticipate threats? Right, that's one of the coolest things.

Brian "Ponch" Rivera:

Now there's a negative side to crew resource management that Decker wrote about, and that is, I guess, weaponizing it against people. Hey, loss of situational awareness, human error, right, those type of things that can be weaponized. So team science is what we recommended software developers use to create psychological safety, to anticipate, to identify weak signals, to do all these things that we learned above the wing right signals, to do all these things that we learned above the wing right. And then, of course, we've talked to Gary Klein about this with his connection to TADMAS, tactical Decision-Making Under Stress.

Brian "Ponch" Rivera:

We've also had Eduardo Salas and Scott Tannenbaum on the show, and my co-authors, nigel Thurlow and then John Turner, who we all looked at Aviation, career Resource Management, the foundation of Team Science, and said, hey, let's bring this over or it's not one for one. I mean, you can't just bring it right out of the cockpit and go, you need to do this but you can use these lessons to actually create the conditions to anticipate Red teaming techniques, things that Gary Klein gave us the premortem, those type of things. And I think that's where a lot of software organizations went wrong as they followed the pseudoscience approach. I'm quoting one of our prior guests, alistair Coburn, who said that these are his words they followed the pseudoscience approach and instead of following I hope you're ready for this Carl Veik's work on high reliability theory and potentially going down the path where you are right now.

David Woods, PhD:

So I don't know if you ever heard this before, but that's what the no, I actually heard this from Boeing when I started in aviation 37 years ago With respect to the cockpit, improving the automation and displays and alarms. It has to quote buy its way onto the airplane. Now, what that illustrates is a fundamental you want action. What can organizations do? So this is driven by fundamentals, like in physics. They don't apply here, it's okay, whatever. No, they apply all the time in this universe. I don't want to get into all those here with you, but from those there are these actions organizations can take.

David Woods, PhD:

And the first one is right, wrote about in the first resilience book and told stories of where this plays out. And this is right. This is the tension between short-term or acute pressures and chronic issues, things that extend over time and play out over time. And I could run you through examples. I have no people in them, it's all physics in power plants. But the point here is you have to know you're always under faster, better, cheaper pressure, acute pressure.

David Woods, PhD:

Now, when we started resilience engineering, that was the policy at NASA in the late 90s, before the three space exploration mishaps occurred in 99. And then in 2000,. In the meetings following those is when I first proposed that we could and needed to develop a resilience engineering, and the issue was you needed to know when to sacrifice productivity for safety, Sacrifice short-term financial short-term productivity for long-term. Boeing's viability in the long-term was harmed by the short-term thinking that had accumulated and persisted for 20 years and eventually right. It's like the old joke about the failed banker is being interviewed by the news person and they say well, how did you go bankrupt? And the ex-banker responds at first slowly, and then all of a sudden, all at once. And so there's cumulative degradation and what we say is you have to be able to build a way to justify longer-term thinking and when to sacrifice the short-term exigencies. I need more now, I need to do more now, I mean.

David Woods, PhD:

Another example that this has nothing to do with money per se is the BP's two accidents in the Gulf, Texas City and Deepwater Horizon. At the time of those two accidents, they were the richest organizations in the history of the human species and yet they were operating various functions within their organizations to reduce costs. Reduce costs, Right. And so you go through the accidents. And while the alarm system didn't work, why didn't somebody fix it?

David Woods, PhD:

Well, operators figure out ways to keep this, keep going. They figured out workarounds. They could keep going because they were under pressure to keep going. So how do you switch the pressure?

David Woods, PhD:

So first thing I say is go read the stories I wrote about in 2006, because they're very vivid, and we've been working with organizations about how do you put out and guide your players at different levels to know and value that they sacrifice the short term for the longer term not always the immediate time horizon and, of course, American business is famous for only having a short time horizon view. If you don't balance the two, you're going to violate the rules. You're going to be more brittle than you realize and you're going to be more surprised when you have a near or partial brittle collapse. And then you're going to attribute it to some component, some person, something, instead of realizing. You designed a system that's more brittle than you realized and yet fails less often because there are some people, ad hoc actors who fill the gaps. There are snafu catchers in your organization, so you don't see the weaknesses that you're building in.

David Woods, PhD:

That's called the law of fluency. It's a law. We can derive the law from first principles. We can derive the law from empirical studies across sectors. Right, Adapting to fill the holes hides the holes from outside views right, Because it goes back to 1956 and the start of control theory and classical automation. Ashby wrote about this in 1956. This is not new folks. We can act on this, Like I said. Maturity we need more mature tools right, Because they'll be more efficient, given the investment you need to make. And we need to build those tools to have larger scale effects, because everything operates at scale. Now, If we can't do things at scale, we can't keep up with what's going on.

David Woods, PhD:

Now the second action for them to do. All right takes a minute to explain After the Columbia space shuttle accident. One of the innovations it's related to another chapter in the original Resilience Engineering book, not because they were inspired by it, but it was all consistent. At the same time, they created a flying reserve. So let's take our military example. What commander goes into action where they may encounter adversaries without a reserve right? Because events, surprising events may happen, things may develop not according to plan, and so you have a reserve. Is the reserve your weakest unit? Is your reserve your least well-equipped units? No, your reserve are strong, effective units, because when you need to put them into action, they had better do the job If they're you know, if a hole is opened in the line and an opponent is ready to rush through, you need to plug the gap, and so our organizations today do not operate with flying technical expertise reserves. Now, these reserves is what NASA created. They created another layer. They called it the NASA Engineering and Safety Group, nesc, and what they did was they sacrificed. They took advantage of their experts, their most experienced people from many different disciplines, backgrounds, and said a part of your job now is to support ad hoc projects that arise because we're having problems places or because new things are occurring and we want to figure out how to best take advantage of them, and so we're going to pool our expertise and we're going to bring it together as tailored to the situation right. So they might need me on one case. Other cases, when they had me on their roster, they didn't need me. Later they had a case and they really did need me. Luckily I was already working on it, so I was for free. But the key here is they look ahead and they learn more from the events that do happen and they integrate those learning lessons across the organization. And it's had a you know this is. It was stood up after the accident in 2003. So it's almost 20 years old and it's still going and it's had a great impact.

David Woods, PhD:

We use an example where they that we almost had a astronaut fatality on a spacewalk and that they didn't take the standard root cause analysis as the end of the story. They came in and saw this as a story of a breakdown in resilience, a breakdown in the processes that would handle this, and they identified a whole range of issues about mission control and space station and space walks. One example is, no one had actually checked the procedures for stopping a spacewalk, stopping it early, because stopping it early means there's something wrong. And two, stopping a spacewalk early is in itself, hazardous multiple kinds of safety issues. So it's how would you do this? Well, they had focused on only on productivity. What, what's all this stuff? How do you get everything done? You need to get done because a spacewalk is a limited resource. No one had thought about that. How do you make a decision to safely stop a spacewalk because something's wrong, despite the movies? Yeah, they imagine it all the time, not well.

Brian "Ponch" Rivera:

You brought up something interesting here about a root cause and I think what I heard there is in this complex environment they identified there is no root cause, that there's probably these causal factors that they identified. It's not just one thing. Right, and I think what we got with James Reason's Swiss cheese model is I think many people misinterpreted it as a static thing that there's always going to be one thing. It's never going to be one thing, it's going to be multiple things, and I think the dynamic models of Swiss cheese is pretty spectacular. But am I reading you correctly on this? Absolutely, absolutely.

David Woods, PhD:

So the name for this now that we have a memory hook for people is the component substitution fallacy. So look, finite resources. You're building things for the future. Things will change in the future in various ways. What that means is components will always have weaknesses, subsystems will always have weaknesses.

David Woods, PhD:

If you watch a development group trying to put together something for a future mission because of a new vehicle with new capabilities and new algorithms on board, you'll see them try to wrestle with trade-offs on a variety of performance dimensions. We can improve this, but then it'll be too heavy and wait a minute. That means the payload will shrink and that won't make it effective for this mission, and they're making all kinds of trade-off decisions. And so when there's a major incident or accident, a major failure out there, when you look down at the system, of course you will see component or subsystem weaknesses. You can't not because they're there, it's back to, the world is messy, it's not almost perfect or getting more perfect over time, and so the problem isn't that they aren't there or that they shouldn't be addressed. The problem isn't that they aren't there or that they shouldn't be addressed.

David Woods, PhD:

The issue is when you stop there, because if you stop there, you're assuming your system is a linear addition of all the components and subsystems. No system is that way today. I won't argue about was that true in the past or not. It's not true today. And so you must also look at this emergent level of what are the broader issues. And the example was nobody had looked at the procedures for a whole bunch of things and it turned out they were fictional documents. How many times have we gone into an organization and looked at procedures for some contingency and found there were procedures and they are mostly fictional about how to actually handle that situation?

Brian "Ponch" Rivera:

Is that work as imagined?

David Woods, PhD:

Follow the procedures and you're like these procedures. They're really going to work here. So these are they?

Brian "Ponch" Rivera:

is this what you would consider work as imagined? The fictional, yeah, okay, yeah.

David Woods, PhD:

Work as imagined is the idea Ultimate. Well, there's two parts to it. One is just simply distant views. Let me go back. Is just simply distant views. Let me go back.

David Woods, PhD:

Work as imagined versus work as done is an English translation we came up with in 2004, kind of serendipitously, of what the French people in human factors have known since the 70s, but there was a translation problem and we finally expressed it properly because they had been working on this that views from a distance, perspectives from a distance, can't see a lot of important things that have to be wrestled with in order to make stuff work at the sharp end. Back to they miss snafu catching, they miss the gaps and they are inevitable. And part of the thing is that you're not bad because you designed a system with gaps. No matter how good you are, no matter how much expertise you recruit, no matter how much resource you gather, there will be gaps. It's the old Yogi Berra quote even if the world were perfect, it wouldn't be. Even if you got it perfectly right, the world will keep changing and it won't be. There'll be gaps. So you have to recognize that you provide for the fact that you first understand the gap filling. That's going on Gap filling, if you look at my paper called Command Adapt Paradox and the Guided Adaptability Approach to Safety, the fact that there's gaps doesn't mean that the adaptations to fill gaps are always wise, always the best, always good.

David Woods, PhD:

The point is somebody has to do something to fill the gap. You need to see what they're doing to say do I change the system? Do I change something else? Do I go and discover what's the best way to really deal with this problem when it arises? Do I need to develop some resources to make, when this happens, to make it smooth to handle, because the way they handle it in an ad hoc way is patchy? Yeah, it kind of works, but I don't want to operate. You know the old word kludge. It's a kludge and it works today. Do I really want to run my system with a bunch of kludges? Well, bp did at Texas City. When you look at that accident, the operators were running that with a whole bunch of kludges. Everybody's like everything's just fine, guys. You're like what do you mean? You're so rich and you can't make sure some of the supporting systems and capabilities actually work.

David Woods, PhD:

So ultimately this gets back to a fundamental about perspective taking, that there is no omniscient perspective. The view from any single point of observation simultaneously reveals and obscures. That's true of all points of observation distant, up, close, so up close reveals some things but obscures others. So in other words, I need a broader perspective to realize this is kind of a patch, or maybe it's an okay patch, but we could do much better if we think about this in general, analyze and develop a much more effective and efficient way to handle this form of stress on the system. Part of it is to recognize there's a stress on the system. The patch might be fine, but now that you know that stress exists, you may realize that might operate over here or over there and we don't have any patch capability over there. We're going to get burned if it happens over there.

David Woods, PhD:

So these all lead to this looking ahead and the idea that I haven't got it all perfect and right now we operate from the top down, as we're almost perfect, and if anything goes wrong, it's not that our plans didn't work right. Our system needs to change. It's those operators, this component. If we just tune up this component, everything will be fine. And the answer is no. The world keeps changing and today the world is changing fast. The turbulence is high, the interdependencies are intense. Surprise is common. I mean it's virtually every week we have a surprise that, if you track various industry sectors, or even the general news that almost every week there is something happening, yeah, nothing's going to change.

Brian "Ponch" Rivera:

It's always going to be continuous here. So I would like to do this. I'm going to introduce you to Mark McGrath. He's joined us about 45 minutes ago. What I'm going to do is Moose is a is a Marine. Marines are fantastic at taking ideas and really synthesizing them. So, Moose, what I'd like to hear from you is just some connections about what you heard in the last 45, 48 minutes from Dr Woods, and then make some connections, make some snowmobiles for us Back to the OODA loop.

Mark McGrath:

Well, thanks, ponchi, I think. Dr Woods, I'm pretty sure that John Boyd would love to have been a part of this conversation, because a lot of the things that you say are affirming of the concepts that he was exploring. I think there's some differences that make sense because he was coming at it from a much different perspective. I mean, like the easy one I think of is just like your. Your stuff is more scientific and morally neutral, whereas in a lot of his work, when it came to leadership, you know, there were some very clear things about implying morals and ethics in certain systems. But the synergy I think is the being really able to thrive under surprise is an expectation of the environment. That that's what's going to happen, um, and and sort of. You know, do you optimize it or around it, or do you maneuver around it? I think that he would probably say it's something that you exploit and maneuver around, but you're essentially saying the same thing I think, and and that's exactly the word we took maneuver.

David Woods, PhD:

Yeah, you outmaneuver complexities. You can't. You know. Yeah, you could, nominally there's some reductions you can do, but generally, as in the software infrastructure case, is to get rid of the complexities. We get rid of the advantage. At the same time, you would take that away, you would not be able to provide as much valued service, you would not be as competitive in the marketplace. A bunch of things would go with it. And so outmaneuvering the complexity is the critical thing, and the basic developments have said hey, this is actually really fundamental, like core science, like gravity. And so how do you do this? Well, the simple way we do it and you'll resonate to this is a form of reciprocity which is based on who is experiencing the greatest risk of reaching saturation, who's about to get over, who's under the most stress.

David Woods, PhD:

Now, in an interconnected system, general stress on everybody, on everything you're doing, doesn't show up equally across every role and every level. It shows up more in some places relative to others. Everybody's affected. If we think of the military analogies, right, if one unit collapses in the face of enemy pressure, the whole line is in big trouble. Somebody saying hey, I'm holding my part of the line. I don't know about you over there, I don't care about you. No, what happens in effective or adaptive organizations is they act to assist and not constrict the unit that is under the most stress. That's fundamental form of reciprocity that's actionable at every scale. I can do it in equations for automation. Right Now I can't do it in equations at all scales, but we do it in training, in small units, in effective militaries where they work together to not constrict but to extend performance under stress, because they know they'll be under stress and that some units will experience the brunt of that and others need to be set up to come to their assistance.

Mark McGrath:

So now you make me think of Buckminster, fuller and the geodesic dome on how the stress is spread load throughout the entire system, that the stress is absorbed by the entire system, not necessarily focused on one point of failure.

David Woods, PhD:

Yeah, and so this is why this idea of reciprocity and how you build it, and that many organizations are run on processes that in practice, not their rhetoric but their actual actions on the ground right, do not build that reciprocity in the face of stress. Instead, it builds up. I'm going to take care of me. I'm going to take care of mine. You can't blame me. That's part of the one of the downsides of blame culture that people have talked about for 50 years is it tends to make everybody just work to roll your role. You know I'm not just a procedure, but I'll work to my role. In fact, that's the worst thing they can do when the system is under stress somewhere. When the system is under stress somewhere, because what happens under stress is you need to spend some of your role Isn't directly the one most stressed, but you need to focus on where you connect to other roles outward, up and down, and spend some of your energy, your capability to build connections, right To strengthen the overall structure under dynamic stress is that like?

Mark McGrath:

is that like like using a real world example that our listeners might be able to relate to because we talk about it a lot? Would that be like and say, in kodak's case, somebody's saying, hey, I did my part. We have a really great graphic design department here at kodak. You know whatever they did, you know? But is that, is that? What? Is that like a fair analogy?

David Woods, PhD:

I think the Kodak case is a little more dramatic for an example of this, given that they but they certainly are an example of an inability to adapt. Often it is. I think in some ways you can say they were slow to anticipate. But I think more in that case study is that they didn't know what to do and they, they, did spend energy. They just didn't spend it very effectively and they did it in a very dispersed kind of way when they needed to re, uh, reframe and rethink, under a period of change, what it meant and how people would connect to their, how photography played a role in people's lives and how to do that, and they were very much stuck in modifying what they already did instead of understanding that the world was changing rapidly, and so did they.

Mark McGrath:

How about, like, maybe a different example, like, say, barings Bank that imploded under the actions of one rogue trader throughout the system? Because I've always thought that over the years that whole ecosystem of Barron's Bank was probably so unhealthy and so incapable of that sort of a surprise catalyst that the entire thing imploded from the actions of one or in a very, very small group, because the overall system wasn't able to, wasn't able to handle that or even allowed for that in the first place.

David Woods, PhD:

Well, I don't know.

David Woods, PhD:

I mean, yeah, I think Enron might be a better example of how some of this stuff plays out. And you know, in some ways let's use a current example that's operating right now and that's the Texas energy market. They're separate from the grid, the national grid, and they under economic theories about regulation and deregulation. I mean, they have regulation, but they have made energy a financial gambling house. In other words, it's not something that is a fundamental necessity for people and we provide it and we make a stable profit in the old utility model. It now is run by the financial people who are buying and trading energy to make profits, and those can be short-term, quarterly profits from various deals as they sign various contracts and from different energy sources, but in the end, what it's done is make the system very brittle. So in 2021, in February, there was a week-long statewide energy crisis due to a winter storm. At least 200 people died as a result of the weather and having no power, at least right up to 600, is the highest estimates. The lowest estimates are in, you know, $2 to $5 billion in costs. Rate payers around the country Minnesota rate payers are certainly Texas rate payers are still today paying for that. Some energy companies made a bundle. Some energy companies lost a lot, everybody, everybody got paid back. The money that was won and the money that was lost all came from rate payers. It's gambling with what people are paying for, a necessity.

David Woods, PhD:

Now this situation is forward-looking because, guess what? One of the big energy users in Texas is Crypto. And how does crypto make money? Every time there's bad weather, the state of Texas pays them to not use energy, not mine cryptocurrency, because they need the energy to support air conditioning in a hot spell in the summer or to keep the grid going in a severe winter, extreme weather situation. So they actually make money from weather. Disasters, right, extreme weather events are profitable for them. Disasters, right, extreme weather events are profitable for them. Now, something's a little. The economists like to use the word perverse incentives here. I mean something's odd here.

David Woods, PhD:

Now let's take AI. How much energy is AI using? Gobs, yep, and now you're going to have private ownership of large scale energy for AI computations. Notice the instability. Multiple jurisdictions with radically different priorities become fundamental to other kinds of operations that are critical to society, and disruptions have costs.

David Woods, PhD:

We can do a simple one. What's everybody who's dependent on energy, who's dependent on computation, financial, the financial world? So we don't have to deal with harm to people. And now, all of a sudden, you say I have an extreme weather season that's punctuated now by extreme, acute events. Energy usage goes down. Remember, if you have a long drought, right, you can't cool the energy plant, they have to run at lower power. France earlier this summer had to cut back energy production from nuclear because it was too hot, water was too warm, couldn't run the planet 100% power anymore. So you can run into these situations that are influencing your ability to generate energy, increase the need in an acute, critical way for energy and now have a huge conflict between where does it go and who does, who pays the cost, who takes the sacrifice of not having energy and who is able to sustain or minimize or maximize their function under degraded conditions.

David Woods, PhD:

Is AI going to get maximum use or only leak a little bit to human safety? Is are we going to be in a trade-off where, well, we can either cool the financial computers or we can turn off, or we can let them heat up and keep the AI going, or we have to turn off the AI. I mean, I'm a little I'm caricaturing it in a little bit oversimplified way, but these kinds of conflicts, so here's the line I use. Let me simplify it.

David Woods, PhD:

Stories of the impact of new technologies either capture or envision the new forms of congestion, cascade and conflict that arise when the apparent benefits get hijacked Because the apparent benefits aren't the real benefits. Other people seeking advantage will figure out what's good for them and they will take the benefits of the new technology in a form that is good for them. Now that will create new forms of conflict or new time periods of conflict. The growth and the hijacking will create it's not that congestion goes away friction and putting military people friction right 1830, we have it written down as doctrine. So congestion will arise in a variety of ways. Interdependencies go up and so the forms and difficulties of handling surprise go up as well. Is that because it's?

Mark McGrath:

too centralized.

David Woods, PhD:

Yes.

Brian "Ponch" Rivera:

Yeah, yeah of handling surprise go up as well? Is that because it's too centralized? Yes, yeah, yeah. So I have a counterfactual, maybe two counterfactuals that we've been following on the podcast. One is going to be clearly the plant-based medicine and the impact of pharmaceutical companies, and we'll shelve that one for now. The other one is a counterfactual in the AI space, and that is if AI follows a natural intelligence approach where it's all about minimizing surprise, minimizing energy.

Brian "Ponch" Rivera:

What you just brought up about conflict still resonates, and that is if the masses of investors, energy companies and AI companies see a future an only future where energy consumption has to go up and they build their infrastructure for that. Consumption has to go up and they build their infrastructure for that. A threat to them would be a lower consumption approach to AI, right? So that's one of the things we're tracking on the show is there is a potentiality out there where platforms that fall, or algorithms or AI that follows a natural approach to how intelligence works, will consume less energy. So I'm just throwing that out there as something we're tracking here on the show and maybe get your thoughts on that.

David Woods, PhD:

Well, I think in this case there's a simple answer, and that's instability. These periods of turbulence and change, especially when we're early in what could be a fluorescence with the new algorithms, in part, introduce enormous instabilities. The problem isn't that somebody sees the future and other people miss it. The issue is nobody sees the future. Everybody who thinks they understand the future is wrong. It will play out in ways that are surprising. We'll re-rationalize it afterwards and say, oh, this person was the key architect or oh, this genius, whatever. But in fact it is all highly unstable. It's unstable in particular in this one because the stuff makes mistakes. It regularly makes mistakes. The mistakes are endemic to the nature of the algorithm.

David Woods, PhD:

For LLM's Gen AI, the way it's being used is to maximize productivity, which means it is doing that where the offsetting costs are not being included in the calculations or in their estimations or in their envisioning. So we've got a thing where it's heavily subsidized. It's got no key revenue generating business case at this stage. It makes lots of mistakes. So for critical functions there's a huge risk that we would never allow in any risk, critical safety, critical world. And then it generates the emergent problem of slop. I think the standard definitions of AI slop are a little too narrow. This is, this is the third ditch in the eighth layer of dante's hell slop, you know, imprisoned in the sea of shit. And uh, that's italian, by the way. That's in the original italian um, the, um, um, the, the. And the reason the slop matters so much is because now you don't know what to trust, and it happened on a widespread basis. I can't tell. I can't tell what's real and what's not real, what's solid, what isn't solid. And let me tell you a simple story related to this, when we were studying how people acted under time pressure in a anomalous situation. Things are not working right and I don't know what information is real. I don't know what information to trust. It has a paralyzing effect.

David Woods, PhD:

We studied this in some energy incidents in the control room, where they knew there was an outage of some of the instrumentation, but they couldn't tell which sensors were erroneous and which sensors were accurate. All they knew was some were wrong and it undermined their ability to be decisive in the face of uncertainty. This increase in uncertainty. Same thing happened in the Air France 447 cockpit Yep. I was thinking about that uncertainty. Same thing happened in the Air France 447 cockpit Yep, I was thinking about that. Captain comes back in and the pilot flying turns and says I don't know what, literally says I don't know what to trust.

David Woods, PhD:

And this slop problem is a widespread thing that undermines the ability to know what's substantive. In studying people successfully deal with uncertainty in a dynamic situation, what we found was that they tended to roll back from. This was AI and other algorithm-assisted decision aids in the 80s and they would roll back to the raw data. We also saw it with Intel analysts when we did studies with them in the early 2000s. When you're highly uncertain about what's going on, what do you do? You roll back to what is the most definitive information that I can count on. And then I rebuild the story up thing telling me this and this thing telling me that, and you know I've got a couple different sources that are contradicting. I've got to go down and say what's something, what's a base that's concrete enough that I can build an interpretation of what's going on. And the slop problem is undermining, in my estimation from the past work, the ability to function in these dynamic, uncertain worlds. If we that slop enters those situations Now we'll see where it goes Because, remember, the bottom line point is instability, and the corollary with that is no one is dealing with that instability.

David Woods, PhD:

They are all too busy claiming ownership of the future, whether that ownership is it's going to ruin humanity, or that ownership is it's going to create an infinite abundance, or it's going to eliminate all white collar jobs, or it's going to do this or that. These are all childish stories of little boys and it's nice that they have an imagination and it's fun when you're a little boy or a little girl to have that imagination. But this is we're dealing with reality now, folks, and instability from a systems engineering point of view is not a good thing. Okay, and we need to start working. We need a big investment in new kinds of systems engineering and safety systems engineering. Someone this last week highlighted that a variety of events that have happened is because the MIL standard on systems engineering is no longer being followed on a widespread basis in aerospace and defense organizations and that maybe we should go back Now. I pulled it back up and went guys, we need a new version of this, because this isn't designed to deal with the complexity of today's systems.

Brian "Ponch" Rivera:

So this is interesting Systems engineering versus complex complexity engineering or resilience engineering there's a difference, right? I mean, the reason you didn't call it complex engineering is why.

David Woods, PhD:

Well, the simple thing was the simple thing. You'll find this hard to believe. Resilience was a very rare, rarely used word. It was not contaminated by other interpretations. Some of the early uses of this were pointing in the right direction. And so a rare word. I mean. When we first started using it, the first reaction everybody went was okay, I guess that's English, what's that word? And you'll see Eric use the Latin definition, not because that's what we meant by resilience, but because this is a word in the English, in all the European English languages, and this is what it classically means. Now let's talk about what it means technically. Now, if we're going to engineer this adaptive capacity so that's why we chose it it's not complexity. Engineering Sounds like you're going to create complexity.

David Woods, PhD:

I thought, we were trying to get rid of complexity. We want to outmaneuver complexity. So that's been one of the other slogans here. How do you outmaneuver complexity? Because that's dynamic. A lot of people are standard, so Punch. The key thing here is that everything needs to get revitalized. It can be right and then become stale. Everything that was once right becomes stale and needs revitalization.

David Woods, PhD:

You referred to crew resource management. When we studied Air France's safety management top to bottom CEO, all the way down, for a year after the 447 accident, first thing we said we didn't wait a year to tell them this. We said all your team training, everybody thinks it's fine, everybody thinks it's running perfectly. You have all these team training programs. You have no teamwork in your organization. They don't work anymore.

David Woods, PhD:

You need to revitalize your team training, your team philosophy, from the bottom up. It doesn't work anymore and it's obvious to an outsider zero disagreement, immediate as we eight of us wandered around their organization. Teamwork has degraded enormously, despite all the teamwork programs that were still working, which were an innovation once upon a time to build teamwork like, and now are stale and ineffective. So we need to, we need to reinvigorate and revitalize systems engineering to outmaneuver complexity and instead of saying, well, it's mostly linear and we can kind of deal with it a little here and a little there, and it'll be OK. Now I will take care of the complexity, because it will tell us what the answer is.

Brian "Ponch" Rivera:

So am I hearing you that teamwork is an imperative in organizations.

David Woods, PhD:

Absolutely. I mean what we focus on is coordination and synchronization. Teamwork can have a narrow reference where we think of a group together, right, so we can think of a unit who works together, trains together the basketball team, the squad, the control room staff. If you notice, in critical software infrastructure the people who come in to handle an incident are connected virtually. They are in very different places for the most part, they are often in very different time zones and so they are working together in the context of handling the incident. The forms of experience and expertise they bring matter a lot, though you don't know in advance whose experience with what will turn out to be critical in figuring out a hard case. So it doesn't meet many of the traditional definitions of a team. They may only occasionally work together as on-call participants, but each time they're on call in an incident they'll be working with a different set of people, some overlapping some, but often new, almost always new people. So we emphasize the coordination and synchronization that has to go on and that that an American culture so obsessed with sports that you know flow. Sports like basketball or soccer are the best, simplest examples. But you know you have simultaneous adaptation going on each time down the court in basketball within the flow of a game, how this plays at the end of the game, at the critical junctures of the game over a playoff series, how the teams are adjusting and changing. A first game blowout often means little because it just means you're stimulating adaptation by the adversary and complacence on the team that blew them out. So who? You know? It's an old story in military. The team that blew them out. So who you know? It's an old story in military. Who adapts the best? It's the Oodaloo. Who adapts the best and fastest wins. You know, I love to use George Marshall.

David Woods, PhD:

Before the US got engaged in the European theater, he knew we would get into the European war. And what did he know? For sure that he didn't know anything about what was the best way to fight in Europe and how to prepare the American units and leadership for that. So what did he do? Well, first off, he got rid of most of the old guys saying they're not going to be able to adapt. The risk of them being set in their ways is too high. Second is he needed people who had contradictory talents. They needed to be highly decisive, but they also needed to be very interested in coordinating. So there was a patent-like general who got fired pretty quick in North Africa because he wouldn't listen to anybody else, he wouldn't work with anybody else, he was decisive, but he didn't coordinate. Friedendahl, lloyd Friedendahl I'd have to double check the name. I remember the story but not the name. Marshall also gave them all a second chance. So it turned out he got back into the fight and had an exemplary record.

David Woods, PhD:

The guy in this example, the key, was people who set up their headquarters at a secure point and stayed there. He fired why? Because you needed to be close to the action to see the surprises, to see what was working, what wasn't working, and so he needed people who were younger and more physically fit to be up there and who would sacrifice some of the command post security. So there was more vulnerability to enemy attack. So there was a real trade-off there. It wasn't somebody just hiding in the back lines, but they needed to be up front to see what was really going on, to adapt fast.

David Woods, PhD:

So the OODA loop and its history, all the way back to Napoleon and inventing it in an ad hoc basis and then the Prussians developing the theory and training to make it the universal way for successful militaries are based on adaptive capacity. It's amazing. Our sports world is based on adaptive capacity, and I have a hard time telling people you should invest in your organization's function to build adaptive capacity. What are you talking about? It's all around us. And then you saw the challenges that are happening regularly in the world that are threatening or breaking large, important chunks of societal functions. Come on, people, this should be an easy sell. Help me mature this.

Brian "Ponch" Rivera:

Yeah, it's hard to sell. And I have to ask you this we're kind of the same line of work. You have to convince leaders that they need to build this capacity and how they do that. We've talked about methods and they have to be grounded in first principles and we agree with you on that. What we find fascinating is we come from a high-performing background in fighter, aviation and military operations.

Brian "Ponch" Rivera:

We're not jumping out of the cockpit and saying we have to close the gap between your high-definition destination and where you are today. That's nonsense. That's not what this is about. Because we can't define the future state. We can't define the future. We know that.

Brian "Ponch" Rivera:

So we abandon things like effects-based operations. They do have a context when you're a complicated domain, using the language of the Kinevin framework. But what we find is, even though we know about team science, we understand the importance of simulation, tactile decision games, we understand red teaming techniques, we understand these things you bring those into an organization and they push back on it because that's not what the other consultants are saying into an organization and they push back on it because that's not what the other consultants are saying. They're saying we need a framework that follows a plan, execute, assess cycle, and that's all we need, and we need to call ourselves Kanban teams, because that's what we get out of the Toyota production system. We don't understand it, but that's what we need, right? So I want to get your thoughts. What are your challenges when you're trying, when you have to convince leaders that adapt to capacity? And I'll just say this it's probably the fundamental thing that organizations need for safety, resilience agility.

David Woods, PhD:

Resilience engineering is all about putting a focus. It doesn't mean other things don't have to be done too, but it's essential to also put a focus on adaptive capacity. So there's many, many points I could make here. One point I want to highlight is we have pulled off. The standard approach is connect to the upper echelon, build relationships and connect to the upper echelon and they will see. We have the evidence, we have things. They will see value. Whether they invest strongly or more weakly in it, they'll see the value. And we've had success.

David Woods, PhD:

I have been connected into periods of change. I had a seat at the table as the patient safety movement developed 96 going forward and the critical start of that. We've had people succeed with organizations and develop resilience-based approaches, organizations and develop resilience-based approaches. We've had NASA succeed with their boring name for a critical function and create this flying reserve of expertise that allowed you to more effectively anticipate and learn to improve performance across every aspect of the organization, not just safety. So we have success stories and the issue is they're not sustainable. Time after time it's not sustainable. We have the turnover in the C-suite, the shift in priority from Congress. Every time we get to the top and have a success. There are factors that work against us and those factors. So we have this problem with sustainability in the face of the folk models. And the folk models are static, structural. They're the opposite of the dynamic coordination synchronization adaptive. They keep believing that they can pin uncertainty to the wall. They can pin a surprise to the wall, tame it, desiccate it, and that their planful function or their introduction of algorithms to take over from people will make it all work smoothly. And the world keeps telling them that's wrong. The world is messy, snafu is normal, snafu catching is essential.

David Woods, PhD:

The second thing is we have a lot of experience with people who have learned and moved forward. We had successes all the way up to the C-suite at times. How did we get those successes? Well, they had to have tangible experience with the consequences of being surprised. They had to be not so up close to it that they were buried in the difficulties and reverberations and they couldn't be too distant and the reason that mattered too distant and you would oversimplify and you would be able to interpret it and not be willing to change anything. The ones in the middle the event highlighted for them the need to reframe, to revise their models of how the world works and that made them curious and open to change. And then all of a sudden we would step in and they'd go. That makes so much sense now. And we've been able to get middle tier people. This happened last week with a conversation with an emergency room doctor or anesthesiologist in England who was part of the pandemic planning response, all the way up to the prime minister's office and he asked me the same question you just said of how, when I look at what you and your colleagues write, it's so obviously correct interpretation and gives us paths forward.

David Woods, PhD:

Why is it such a hard sell? Why do they revert back so much? So that's two. That's an observation about how to reframe.

David Woods, PhD:

The third thing is is this reframing and the OODA loop doesn't really quite get there? It sets up the stage for it but it doesn't quite go all the way. And that is, how do you get people to reframe? And this is important because machines don't reframe. They have a model and they work according to their mechanism and the models that are explicit or implicit in that mechanism. We've never tried to build a machine or algorithms that would reframe. Interestingly, people can reframe but it's very hard because the evidence that says my model of the world is not the world I'm actually living in creates huge dissonance, and dissonance is uncomfortable to painful. And so the easier way to deal with that dissonance is to not listen to what the world says, that it's different than your model, but to reinforce the model, to retreat or retrench in that model, repair the model, ignore the signals from the world, and this is fundamental in what we've seen in safety.

David Woods, PhD:

The signals were there before Columbia. They were not weak. Debris strikes at a different phase of flight. Debris strikes at a different phase of flight. Debris strikes at energy 100 times greater than the maximum they analyze. Debris strikes on orbit for 100 times two orders of magnitude and it was striking different parts of the orbiter structure. And everybody said it's perfectly fine and safe to go on, when the answer was we have no idea what is safe or not safe. We've never looked at this before. Effective engineering would say let's look, let's take a pause and look. But they were under intense, faster, better, cheaper pressure. So they discounted and rationalized away the evidence that they were operating in a different world. And that is critical to reframing. And the issue here is how do we get people to not just reframe but to set up a structure that would help them reframe.

Mark McGrath:

That seems to me like Boyd's whole point, when you take his academic lineage from destruction, creation to conceptual spiral, to essence of winning and losing in OODA Loop sketch, which is the anti-linear OODA Loop, which is the anti linear OODA loop, the whole entire thing is based on reframing. If you don't reframe, you don't, you can't adapt, you can't survive.

Brian "Ponch" Rivera:

Yeah.

Mark McGrath:

Absolutely yeah.

Brian "Ponch" Rivera:

And then when you look at, that.

Mark McGrath:

That's the name of the podcast. It's in there. Yeah, it's a whole.

Brian "Ponch" Rivera:

It's about reframing. So the whole point of, if you were to summarize the OODA loop, it's you update your orientation, your old model, to match the external world or you take action to match, you know, to force your orientation onto that world. So think about what you just said about I'm taking action to ignore these signals. That's an action, right. So you can reframe things by taking action to cut off the flow of information into your system because it's a low energy approach. A higher energy approach is to allow that information in, to update your orientation, and that's why so many people find it hard to do this is in updating that orientation, that internal map of the external world. There's dissonance, there's cognitive dissonance in it.

Mark McGrath:

That's why linear OODA loop is so popular and pervasive, unfortunately, because it doesn't force people to account and create those internal and external internal perception of the external world and there's sliding conceptual spiral which this podcast and our Substack takes the name of that. There's no way out of uncertainty and entropy and everything else. Thus we have to continue on this world of reorientation, which means that we're constantly breaking and reframing our perception and understanding, because if we stop, there's no way we can improve our capacity for free and independent action, which is the ultimate goal of destruction and creation.

David Woods, PhD:

We're in the same place, yeah, no no, I agree.

Mark McGrath:

Just when you said it wasn't open to reframing in the linear sense. Absolutely, I completely agree with you.

David Woods, PhD:

The point for me is for all of us is how do we get, how do we actually help people? We can state it, we can show it, we can document it, we can show you the empirical evidence for it, we can show you the theoretical derivation why it matters. We can do all of this stuff, but we have this very practical thing, which is why I talked about the evidence of what we've seen in the safety world about it is we need to up our game and how we actually get reframing to happen.

Mark McGrath:

We agree a hundred percent. Absolutely. That's why. That's why our company exists, that's why our podcast exists, that's why we talk to you and John Flack and Chet Richards and everybody else, anglo-american culture, business world that wants everything to be nouns, and so there's noun data, noun models and noun math tabulations and these things end up eliminating time, eliminating growth, deterioration.

David Woods, PhD:

All the dynamics start to disappear and it's putting a number on uncertainty, a number on surprise. All of these are now and this is I'm not the only one from a complexity background who's highlighted on this metaphor of nouns versus verbs. Brian Arthur, the complexity economist from Santa Fe, has recently done a piece too. It's why I talk about people want to go. What does resilience mean? I say, well, it's a verb in the future tense. And they go, it's actually not a verb. I go well, nouns can point to verbs. Or I do exercises in the talks. We go. We go nouns versus verbs and I throw up a label and say is this a noun or is this a verb? And we play the noun versus verbs game to get people to start reframing that.

David Woods, PhD:

We're interested in processes and relationships that change over time and you have to make that fundamental in everything everybody does to rebuild this capacity to see the things, use the OODA loop and related ideas. All of this stuff builds out of that. So that's one technique and I can show you some of the exercises we've done with people to just warm them up. But it's interesting because and it's also why people sometimes say I talk in a funny language because I'm using words, often in phrases, Sometimes it's a pair of words, but they're all in reference to processes that play out over time Relationships can be time over time, this relationship changing over time relative to this, and then how does that relationship between the two time dependent things change over time? It's always in reference. I'm trying to get people to always see the relationships and how those are changing over time. It's always in reference. I'm trying to get people to always see the relationships and how those are changing over time. And that's fundamental Remember to many of the algorithms and developments that we've had over the years.

David Woods, PhD:

We know how to build dynamical systems. We can model dynamical systems. We have a base to build on and people are doing it. There's some levels at which the math is being worked out. It's really funny math. It's not AI math, it's different kind of math because it's fundamentally dynamical system. Now the second technique is the one we did right at the beginning of my career because, remember, a lack of reframing, getting stuck in an old frame, was part of the diagnosis for the Three Mile Island accident, and so I literally was assigned two design ways to break frames.

Mark McGrath:

Sounds like a Pepsi syndrome, do you remember? That skit from Saturday.

David Woods, PhD:

Night.

Mark McGrath:

Live.

David Woods, PhD:

Yeah, I do, I do, and this is where we see a tremendous underutilization of the computation plus visualization that the visualizations have become very structural and noun oriented, the animations are limited, despite the power to do animations regularly, and that this all relates to the word you were using Mark maps. Yeah, maps are about capturing relationships in a frame of reference, and there are many frames of reference by which you could create a topology. Yeah, if you have advanced computations, the way to really make them effective is to have them drive visualizations that are topology based, so that you can see relationships and you can navigate. Whatever it's telling you, you know whatever it's highlighting, you can navigate to what actually is most relevant and important to you in context, and the only way to do that is to see in a frame of reference how relationships are changing over time. So, in a dynamic system, can you see deterioration, can you see stabilization, can you see recovery? Can you see overshoot? All of those are dynamic words, they're all verb based and you, you know you talk about how to manage things. You want to see what's deteriorating, what's stabilizing, what? How are we recovering? And that's just one simple example of using verbs. But what goes with that is these visualizations in which you can see these dynamic relationships with that is, these visualizations in which you can see these dynamic relationships. Now, they're not sexy icons, they're not South Park cartoons.

David Woods, PhD:

Sometimes the diagrams that highlight this in a particular world can look a little mundane relative to the design specialists, but they're deep in the sense that you can now see what you couldn't see very well, because you can see how relationships change over time. And what's interesting is calculating deterioration is kind of hard. I'm going to tell you what the deterioration you know nouns tell you. Maps of relationships show you. They show you much more telling. You can only tell you so much. But if I show you right, then you can find within that map of relationships which ones matter to you now. And if I'm wrong in what I tell you, you can find the ones that are the ones that matter now, because I can't always tell you which ones matter now.

David Woods, PhD:

And we actually created displays back then that showed you what was going to happen next. And we've done it again with a visual analytics tool called the what's Next Diagram for automated systems, which is part of the person's job is to understand what the different parts of the automation are doing, and part of that goes back to the three questions from the start of studying aviation automation 1989, which is what are the most common expressions in the flight deck, which is what's it doing? Why is it doing that? What's it going to do next? And how in the world did we get into this mode or configuration and that one we kept saying to people show them what's next. Well, anyway, it turns out doing it for real time cockpit. Why didn't?

David Woods, PhD:

There's another story about why didn't my colleagues and I do that, and that goes back to our conversation about Boeing, because they kept saying if you can't buy your way onto the flight deck, we're not going to, we're not going to listen, we're not going to do anything. And we're like we could design a what's next display. Well, unless the airline says they, they want it, we're not going to spend any time or money or help you with it. Uh, but we've now decided we have to do it. So we created the what's next display.

David Woods, PhD:

We have a paper that describes it, with some accompanying and limited animation, so to see how it would play out on simple cases uh, for example, part of a go around, flying to go around and how it highlights, and we're using it as a analytic evaluation tool for people designing automation to say, let's run through some scenarios and say, can you track what's the automation going to do next? And it can be something that's a plan for event, like a go around, and then you can make the difficulty of the go around change based on adding more to the scenario. But even in a basic go around, all of a sudden you quickly go. This is hard. There's a lot to keep track of. People are pretty good. They can keep and under relatively benign conditions, flying a go around, they can do this really well. When you run through this all the bits that they have to track and the time available and all the things they're doing at the same time, you're going, wow, people can this is amazing, pretty amazing, yeah, yeah, I learned that in the cockpit.

David Woods, PhD:

It's amazing to go around. Yeah, this isn't a hard one. This isn't in New York under bad weather. Yeah, with not enough controllers and the controller's equipment breaking.

Brian "Ponch" Rivera:

Yeah, there's so many other areas we could dive into today, including, you know, the proliferation of drones and airspace and the safety problems with that. There's so much. I do want to point out something when we talk about organizational OODA loops, it is essential that they develop maps, maps of the external world or whatever they're dealing with. And that's what you just confirmed with us. Is that is important. That's the, that's orientation, right. And that's what you just confirmed with us. Is that is important. That's the, that's orientation right. What is the orientation of this collective group of people that have diverse perspectives? Bring them together, build a map, let them argue about the map not about each other, but about the map and then they can solve and anticipate what's next. That's actually how you develop strategy. That's what you want to be able to do, and this connects. You can build maps many ways, by the way. And this connects you can build maps many ways, by the way. So, for those of our listeners out there, they're familiar with Wardley mapping. You know Wardley map connects directly to John Boyd's Observe Orient Decide Act loop, a lot of connections to complexity theory as well, or complexity science. So you nailed that for us.

Brian "Ponch" Rivera:

I appreciate that One of my key takeaways or one of my asks to the safety community and you know I got some time at the Naval Safety Center and got to do some pretty cool things over the last 10 years or seven years since we ran into each other is this the way the OODA loop is presented in safety papers and even in Eric Hallnagel's recent book from like two years ago is a linear depiction of the OODA loop. It's not what the natural science approach, and what we're asking the safety community to do is hey, don't put the linear OODA loop in there. That's just bad, juju. What we're trying to do is get 99% of the population away from that horrible depiction of. I'm going to call it. What do we call it Folk OODA loop? We'll call it the folk model of the OODA loop.

David Woods, PhD:

It's nonsense right, well, we have the same thing with Swiss cheese. Yeah, yeah, yeah. So I deal with this. Somebody does a good point, makes a metaphor or visualization to help you understand it, and then they dilute it or revert, turn it inside out to be the opposite of what was meant.

Brian "Ponch" Rivera:

Yeah, and so I think that's a great connection, is a reason. Swiss cheese model. I learned it years ago, the way that you just described it. Right, you know, but that's not the way I was originally designed. And it's the same thing with Boyd's OODA loop. It was never intended to be a single loop and it's not from fighter aviation. It's actually from the same places that you identified physics. I think you're a fan of Hofstadter. Is that right Of? Yeah.

Mark McGrath:

So there's also a massive you brought up the noun versus verb dilemma. There's also a massive misunderstanding with Boyd between orientation the noun and orientation the verb. That constantly is misunderstood.

Brian "Ponch" Rivera:

Yeah, so having you know John Flack will probably come back on to take a look at his new book. I don't know if you've seen that. We'll have him back on here soon. We've had Todd Conklin on recently. David Slater I think David Slater's connection to the neuroscience aspect is spot on.

Mark McGrath:

Yeah, we could go on for hours with you, Dr Woodson.

Brian "Ponch" Rivera:

Yeah, this is amazing, and we didn't even get into hedge funds yet. Hedge funds need to know your work right now, because you're talking about how do you hedge and how do you create optionality in a instable environment, right, which is what we're talking about.

Mark McGrath:

So, last, question for me, if they're out there listening. I know that we do have some hedge fund clients. You should call us, because this is exactly what we're trying to tell you, and here's living proof again.

Brian "Ponch" Rivera:

Your work connects to so many things. Now, hopefully, our listeners are making that and like wait a minute, this is basically what you're saying, is what we're saying in parallel, but you have the academic background and rigor to go. This is what works, right.

David Woods, PhD:

We're, we're planning. We're a little far afield, it'll be in 26, but hopefully in 26, planning, hopefully more than one Workshop on designing relation know, visualizations, representations, um, to take advantage of advanced computations. That's where you're going to reverse it. Instead of saying the computation is telling you the answer, we're going to say the computation is going to power, a visualization that's going to let you discover the answer. Uh, because the visualization because the computations won't always be right that we have a bunch of knowledge about how to do this in a very practical way, and we want to do it focused on the software world. And the reason we want to do it focused on the software world is the scaling issue, which is I can teach people to do this, we can do this as skilled expert custom work, but we need to build tools and tooling that allow this to happen on a widespread basis and scale. And so the goal here is not to teach people this per se, but rather to work with people who are learning the techniques in order to build scalable tools that allow you to do these on a widespread basis. And the place to do that is since everything depends on software today is do it with software.

David Woods, PhD:

So the op side of software as the primary focus, and so we're hoping to put that together and launch that in 26. And so we're hoping to put that together and launch that in 26. So I'll let you know when we get to a point where we have some, at least tentative plans on where and when those workshops will start. We're debating some things. I want to do it live, at least the first one, debating whether we can do it with people you know virtually. I think there's some people who kind of know the techniques who I could do it virtually with. But I think we need to do the first one live. So we'll let you know.

Brian "Ponch" Rivera:

Well, hey, we've taken up a lot of your time today. Like Moose said, we can go on for hours with you. We'd love to have you back. You know, the network amongst all of us is pretty amazing your connection to Gary Klein, your connection to John Flack, of course, eric Holnagle and Sidney Decker and the safety community. We are so thrilled that you were able to join us today and have this almost two-hour-long conversation. It felt like 10 minutes, it really did. Fascinating.

David Woods, PhD:

Well, I want to thank you guys. Thank you for being out there and fighting the good fight on this front and taking these ideas forward and moving the ball down the field. We're all in this together.

Brian "Ponch" Rivera:

That's right, we're all learning. Thanks so much.

Mark McGrath:

I can't wait. I have to tell you my Columbus Ohio stories.

Podcasts we love

Check out these other fine podcasts recommended by us, not an algorithm.

Shawn Ryan Show Artwork

Shawn Ryan Show

Shawn Ryan
Huberman Lab Artwork

Huberman Lab

Scicomm Media
Acta Non Verba Artwork

Acta Non Verba

Marcus Aurelius Anderson
No Bell Artwork

No Bell

Sam Alaimo and Rob Huberty | ZeroEyes
The Art of Manliness Artwork

The Art of Manliness

The Art of Manliness
MAX Afterburner Artwork

MAX Afterburner

Matthew 'Whiz" Buckley