The Fighter Pilot Who Changed Software Development

For those in the Agile Software Development community who have spent any time at all looking into the Lean underpinnings of Agile, the name John Boyd should be familiar.  Boyd was the originator of the OODA - Observe, Orient, Decide, Act - loop that represents a more detailed refinement of Scrum's Inspect and Adapt cycle.  While OODA was created as a strategy for fighting a war, its underlying goal is to provide agility in any situation and to ideally allow a unit to out-think its opponent.

One of Boyd's 'acolytes', Chet Richards, wrote in his book "Certain to Win: The strategy of John Boyd applied to Business":

It is not more command and control that we are after. Instead, we seek to decrease the amount of command and control that we need. We do this by replacing coercive command and control methods with spontaneous, self-disciplined cooperation based on low-level initiative, a commonly understood intent, mutual trust, and implicit understanding and communications.

As with OODA being a much more detailed strategy than Inspect and Adapt, in one paragraph this explanation details why self-organization is desirable and what is required for it to be effective!

If we dig a bit deeper into Boyd's career, though, there is another very interesting parallel to Agile.  He served briefly in Korea during the last couple of months of that conflict flying F-86 Sabres against Soviet-built MiG-15's. By that point in the war, the American pilots had a 14-1 ratio of kills to losses, and a 10-1 ratio overall during the full 3 years of the conflict.  Boyd was curious as to why this was the case when, on paper at least, the MiG-15 was superior to the F-86.  After the war he revisited this conundrum, and realized that the Sabre pilots enjoyed much better visibility owing to the large bubble canopy, and they could switch from offensive to defensive manoeuvres and back more quickly because the Sabre had fully hydraulic controls.  In other words, the Sabre provided the ability to better observe the situation and the agility to react as required more quickly than the MiG.

Boyd went on to become a student at the Air Force's Fighter Weapons School (FWS), the equivalent to the better known U.S. Navy's Top Gun.  It should be noted that at the time - the early to mid-1950's - the Air Force was essentially a bomber force under the doctrine of massive retaliation to an anticipated attack from the Soviet Union.  There was very little effort and fewer resources allocated to developing fighter tactics, and indeed the belief was that the fighter battles of previous wars were a quaint throwback and had become obsolete.  Boyd, however, worked in near obscurity at the FWS, writing a paper entitled "A Proposed Plan for Ftr. Vs. Ftr. Training".  In it he advised that pilots needed to think differently about how they flew, concentrating not only on their current manoeuvre but also on the effects that manoeuvre would have on their speed and how their enemy would react to the manoeuvre.

E-M Theory
This research began to solidify during the early 1960's into Boyd's Energy-Manoeuvrability Theory, which stated that it wasn't just speed or engine power that gave a pilot the advantage over his adversary, but rather that it was his total energy level.  At this time, Boyd met Tom Christie, who was able to work with Boyd and provide access to the computer time required to develop and validate the equations required to prove the theory.  Indeed, Boyd "stole" the computer time, as it was charged against other projects for which Christie used their authorization codes!

In the end, Boyd and Christie developed tables and graphs that normalized for all aircraft and were based on the simple notion of knowing how quickly a pilot could gain energy when applying full power with the throttle. It was on this basis that they were able to prove that all U.S. fighter aircraft of the mid-60's were inferior to those of the Soviets, with a few exceptions in some limited part of the flight envelope.  This and lack of focus on fighter tactics were being proven by the dismal performance of U.S. fighters in Vietnam despite the technological superiority they supposedly enjoyed.

F-15 Eagle

Based on his experience and calculations, Boyd had been advocating for a small, lightweight, highly manoeuvrable fighter as the next generation to replace the Century Series fighters and the F-4 Phantom.  He was making some progress when fate intervened in the form of the large MiG-25 that appeared at the Domodedevo Air Show in July 1967.  It was reputed to have a top speed of nearly Mach 3, and it was believed that the twin tails made it a highly agile dogfighter.  (A sample was flown in 1976 to Japan by a defector, and the true nature of the MiG-25 as a bomber interceptor with limited range and manoeuvrability was exposed.)

The fear that the Soviets had a larger, better fighter prompted the U.S. Air Force to build the large, complex and very expensive F-15 Eagle.  Boyd railed against the size and proposed Mach 3.0 speed of the F-15, arguing that it was unnecessary and would compromise the agility and range of the aircraft.

Enter the "Mafia"
While the F-15 project proceeded, Boyd was able to convince a small group of generals that a smaller, lightweight fighter should be explored in case the F-15 project failed.  At the time (1967), the future success of the F-15 was anything but certain and both the Air Force and Navy were concerned after the F-111 debacle.  So, Boyd and his group were able to obtain funding to perform a design study on the lightweight fighter.  This group became known as the Fighter Mafia, named by Col. Everest Riccioni, and was an ironic play on the Bomber Mafia of the 1920's led by Gen. Billy Mitchell who faced similar challenges when the Air Corps didn't believe that bombers were important or necessary.

This group split the funds for the design study between two companies - General Dynamic for the YF-16 and Northrop for the YF-17 (the "Y" indicates a prototype aircraft).  Meanwhile, the F-15 project was becoming hugely expensive, and the already expensive F-14 of Top Gun fame was suffering from poor performance.  The government wanted to rein in the procurement process, and the lightweight fighter study seemed to be a good place to start.  As a result, in 1970 they received funding to proceed with the development of the two prototype aircraft and to perform a fly-off evaluation of them in order to make the decision based on real data as opposed to projections.

During 1974 the two aircraft were evaluated, and competed in a much larger competition for NATO orders.  In early 1975 the F-16 was declared the winner, and has since gone on to achieve considerable success with nearly 4,500 being delivered since full production started in 1976.

The YF-17 didn't fade into obscurity, though.  It was redesigned to Navy standards to be able to operate from aircraft carriers and became the F/A-18 Hornet.  It, too, was intended to complement a much larger, more expensive aircraft - the F-14 Tomcat.  As of 2011, the F/A-18 has replaced not only the F-14, but all other strike aircraft on the U.S. carriers.

Ironically, Boyd had argued against any "gold-plating" of the lightweight fighter in the form of powerful radar or computer systems, but those systems have indeed been applied with great success to both aircraft.  The increasing miniaturization and decreasing cost of electronics allowed those systems to be used without the exorbitant costs of previous generations of fighters.

Parallels
So, beyond Boyd's OODA loop, how does this apply to software product development?  Look at the parallels between the push to use small, agile, low-cost fighters in the context of their time.  The doctrine of the Air Force from its creation as a separate service in 1947 until the Vietnam war in the 1960's was that a massive nuclear strike capability was all that stood between the Soviets and an invasion of western Europe.  (Whether that fear was founded or not is beyond the scope of this post.)  This drove the U.S. to focus on aircraft that could deliver the bulky and heavy nuclear weapons of the time, which meant bigger and faster ruled the day.  Similarly, in a defensive role the focus was on stopping the Soviet bombers which mean large interceptors capable of high speeds carrying missiles that could destroy a bomber from a great distance away.  It was assumed that the dogfight had been consigned to the history books, and that technology would rule the skies.

It was the hard lessons of the Vietnam War that proved those assumptions to be mostly incorrect, and indeed the Kennedy administration admitted that the nuclear doctrine served to create more conventional regional conflicts rather than prevent them.

Another aspect is that the military-industrial complex in the U.S. became self-serving with big contracts and big projects resulting in big systems and big aircraft.  These all served to eventually force the need for the lightweight fighter programs that Boyd had advocated.  While the resulting aircraft, the F-16 and F-18, are anything but 'cheap', in the context of the huge F-111, F-14 and F-15 programs of the late 60's they have been unqualified successes.

Now think about the Agile Software Development world:

• In a world of increasingly large and heavy process, one person and later a small group explore and quantify why agility is more effective
• These people and their work are dismissed, most often by those with a vested interest in the status quo of large projects with large processes, often in large and very large companies
• Economic constraints force a second look at their work
• Their work begins to accumulate acceptance as the assumptions behind large process and large projects become exposed as flawed
• Their work becomes fully accepted and begins to flourish
• Others start to add more to their work, despite admonishments to the contrary
• Those additions act to make the original work more robust and even more accepted

Sound familiar?

Agile as we know it in 2011 has its roots in the 1970's.  However, it was the massive build-up of large processes and supporting tools in the 80's and 90's that lead to the "lightweight process mafia" convening at Snowbird in early 2001 in an effort to change how world built software.  That effort exposed the larger, heavier methods as being the massive retaliation targeted at an enemy that would never come, and the dot-com bust would force the post-Vietnam style soul-searching that allowed Boyd's ideas to reach the mainstream.

Today, I hear people talk about OODA constantly, and how it should be used to truly drive agility into the core of a team and organization.  Indeed, the notion of being able to process the loop faster than your competition is a key component of the nascent Lean Startup movement.

We should also be thankful that John Boyd's drive to determine why agility was important, to quantify it in real terms and to implement it in actual practice in the face of the conventional wisdom of the time was shared by those who attended the discussions at Snowbird in 2001, and those who have refined the practice of the Agile Values and Principles since.