Pursuing an integral architecture

In the not so distant past, my software team found itself neck deep in three projects which had a ton of challenges: crazy new features with ambiguous requirements (that never happens!), new team structure, new management, new microprocessors, new tool chain, new technology, and on and on. Did I mention there were three of these projects? Even one should have killed us. Yet somehow we survived three of them.

How we survived was to split the work into three teams: one team to handle the operating system for all the projects, another to handle the middleware (that’s my team) for all the projects, and one to handle the front end graphics for all the projects. This seemed the most efficient way to handle the work, and it actually worked pretty well.

What we did not do well, however, was to define a software architecture. (All software has an architecture, but the extent to which you define it can determine in large part the extent to which your team will succeed.) When we would get a new requirement from the customer, and the three teams would think they knew what to do, but when they met to integrate the software two weeks later, it didn’t work. Team A expected Team B to do something, but Team B would expect Team A to do that very thing!

Because of the volume of work, all teams were in “survival” mode. Everyone had a selfish interest to minimize their workload. But this wasn’t “you do this, so I can leave early” selfishness; it was “I just want to get a few hours of sleep tonight” selfishness. It wasn’t entirely uncommon for me to work a full day (8am to 5pm) and then come home and work another full day (8pm to 5am). Others had the same story. It was “justifiable” selfishness.

Years later, after our software base had stabilized and we had (successfully!) launched these three products and several more, we finally took time to formally define our software architecture. We revisited requirements which we had always fumbled our way through, and carefully defined what each team should do when implementing such requirements. This required a lot of discussion and compromise.

Compromise.

When in survival mode, there was no time for compromise. We all wanted to get our stuff done so we could maybe see our family before they went to sleep. We cared about, and viciously protected, our team’s workload so it wouldn’t get out of control. We didn’t have time to thoughtfully consider other teams’ points of view. We survived by having a narrow focus.

When no longer in survival mode, compromise was no longer a dirty word. We actually stopped seeing it as compromise at all. We started caring about something bigger than each of the three teams. We were One Team. We wanted to make software that made us happy, was well organized and properly modular — not only because it would help us work better together, but it would be higher quality too. The give and take necessary to make this happen no longer threatened our small intellectual silos.

Our One Team grew out of three small teams. The One transcended and included the three.

Three tribes had grown into one bigger, more inclusive tribe.

Integral Theory

While learning about Spiral Dynamics, I stumbled upon the work of a philosopher named Ken Wilber. He has been working for decades on combining several seemingly conflicting schools of thought — models of growing up and waking up, religion, spirituality, social systems, biology, psychology, meditation, and more — into one philosophical framework called Integral Theory. He calls Integral Theory “one suggested architecture of the Kosmos.”

Sound grandiose? Wilber seems to think so. The subtitle of his book The Integral Vision is “A very short introduction to the revolutionary integral approach to life, God, the universe, and everything.” And he has another book called A Brief History Of Everything. And yet another called The Theory Of Everything. You get the idea.

Here’s a more humble description of Integral Theory from a podcast called The Daily Evolver, which is hosted by a follower and friend of Wilber’s:

Integral theory is a school of philosophy that seeks to integrate all of human wisdom into a new, emergent worldview that is able to accommodate the gifts of all previous worldviews, including those which have been historically at odds: science and religion, eastern and western, and pre-modern, modern and post-modern worldviews.

I’m not well-versed in philosophy, but his work grabbed me immediately, because once I grasped it, I realized I had already known it somehow at an intuitive level. And once I saw it, I couldn’t unsee it, and I started seeing everything else through it. Maybe you will feel the same way.

An integral system…. I know this…

Believe it or not, my software architecture story can be explained quite well by Integral Theory. Let’s take a look…

The four quadrants

One of the most important aspects of Integral Theory is the four quadrants. He says that every “occasion” (e.g. organism, situation, or problem) can be viewed from four perspectives, which are represented as quadrants on a graph. Here’s a helpful picture:

Imagine we’re viewing “an engineer working on Jamie’s software team” from these four quadrants. Let’s call her “E” for short.

  • In the “I” quadrant, we see how E feels. Her emotions and internal value systems can be found here. Does she actually care about her work? Why? Something on a spiritual/religious level? Or she’s really enjoying herself? Or she’s a masochist who loves long hours?
  • In the “IT” quadrant, you can observe E from the outside. What does E do? How does she behave? What does she say? What does she produce? What kind of software is she writing — the bare minimum, or something more robust?
  • In the “WE” quadrant, you can see our team (let’s say, the middleware team). Our team values are found in here — things like protection, minimizing our workload, or survival.
  • In the “ITS” quadrant, we can observe our team from the outside. What did we produce? How did we gel as a team? What kind of software did we make? What processes/team structure did we have in place? The software architecture is found in here, because we created it (implicitly) together, and it governs how we write software.

In order to devise a software architecture (“ITS”) that works for all teams, there is no way to do so while staying in the “survival” mindset where all I care about is my own team. In the “I” quadrant, I have to grow first. I have to be willing to hear other points of view. In the “WE” quadrant, my middleware team has to be ready to hear an objection to the boundaries we’ve set up for ourselves. Maybe the graphics team was doing some unnecessary logic that we should have been doing. Maybe we’ve gone the wrong direction for years. Maybe it’s not all about us.

So, in order to create something on the exterior (an architecture) which was more inclusive, we had to do interior work first.

We had to become inclusive on the inside in order to be inclusive on the outside.


See? You knew that. It’s obvious, right? Let’s keep going…

Don’t collapse the quadrants

Different sciences and fields of study often “live” in, or focus on, only one of the quadrants. For example, much of the scientific method of hypothesis and observation lives in the “IT” quadrant. Science focuses on things we can observe, measure, and draw objective conclusions from. However, science may often discount the interior (“I” and “WE”) quadrants. When it does so, it leaves out an aspect of reality.

As an example, let’s take “thoughts.” We’ll take a mundane thought, like “I’m going to Home Depot.” (Not that going to Home Depot isn’t extremely exciting!) What is happening when you have that thought? Certainly on the interior, in the “I” quadrant, you can identify with that thought; you can picture Home Depot in your mind, with the big orange sign and the warehouse floor. But what’s happening on the outside when you have that thought? If a neuroscientist were to probe your brain and measure the electrical signals generated by your neurons firing as you thought about Home Depot, they would find something measurable and observable in this “IT” quadrant.

Which of these points of view are “real?”

The answer is obviously both. But someone entirely focused on one of the quadrants may discount the other three, and collapse them all into one, thinking that view of reality is the only valid one. It’s not. If we integrate them all, a much richer understanding of the world develops.

More aspects of the theory

Integral Theory is also called AQAL (pronounced like “aqua” with an L at the end), which is short for “all quadrants, all levels.” Actually Wilber says AQAL is short for “all quadrants, all levels, all lines, all states, all types.” I suppose AQAL is catchier than AQALALASAT.

We’ve seen the quadrants; let’s look briefly at what those other things are.

Lines

A line is a developmental path. It could be an internal growth path (psychological, spiritual, values, intelligence, worldviews), or an external growth path (organisms, brains, social systems, societies).

Levels

When growing in a developmental line, progress happens through several levels (also called stages). When talking about human development, I like to think of levels in terms of the Spiral Dynamics colors.

States

Levels take time to reach, through evolution and development, but states are always available to us at all levels. They are modes of being, through which we experience reality. Things like waking, dreaming, deep sleep, or the more esoteric gross, subtle, causal, non-dual.

Types

Aside from all the lines and levels and states and things we can “work on,” we also have static types from which we can never escape. For example, your personality type (Myers Briggs, Enneagram, etc.), gender, race, etc. And Integral Theory applies to everyone, no matter their type. (In the picture, I drew types in the “IT” quadrant, but I’m not sure where they really belong. Maybe a combination of “I” and “IT.”)


That’s it! Now, let’s apply it to developing a software architecture.

Modeling team growth with Integral Theory

Here’s how to read this:

  • There are four lines shown, one in each quadrant; there could be infinitely more lines in actuality though.
  • Each line has three levels, indicated by yellow dots. These indicate how you grow in that line.
  • There are three “unifying levels” — or levels of “team consciousness” — shown as concentric gray circles radiating out from the center.

Now, what happened to our engineers while growing through the levels?

LEVEL 1

When our software team was buried under a mountain of work, and unable to cope, we were individually (“I”) in Survival mode. Our team was barely bonded together, in what I call Suffering — loose alliances of teams (“WE”) who could relate to each other because they felt the same pain, but could not relate across team boundaries. Any one engineer probably wrote software (“IT”) which satisfied the bare Minimum requirements, but nothing else — no time for that! Therefore, the software architecture (“ITS”) was completely Undefined — no time for that either!

LEVEL 2

A bunch of engineers in survival mode banded together to form tribes: the three teams I described in my story. They valued Protection (“I”), and teams fought to minimize their workload in a Tribal (“WE”) fashion. In this environment, any one engineer’s software (“IT”) could be described as Defensive, meaning it did what it was supposed to do, and took into account unexpected behavior happening in any “other” team — a practice known as defensive programming, which is a good practice, but when done with an underlying assumption that my team is right and other teams are wrong, it furthers a narrow Team-focused (“ITS”) architecture. We can’t see the big picture, we can only see what our team is doing, and do our best to defend against something stupid that another team might do.

LEVEL 3

Finally, after reaching a stable point at Level 2, we let our guard down and started listening to each others’ point of view, and became (“I”) Cooperative and (“WE”) Compromising. As I said, “compromise” was no longer a dirty word, but part of the culture. At that point, we can finally focus on building an Organizational software architecture (“ITS”) which addresses all teams in the organization, not just my small sub-team. Once doing so, an engineer’s work products (“IT”) become forward-thinking, or Expandable, structured flexibly enough to accommodate not only possible future requirement changes in my team’s specifications, but in other teams’ as well.


This is the beauty of Integral Theory.

By knowing that all lines in all quadrants are related, you encourage growth in one quadrant by addressing all of them together.


Finally, what about states and types?

  • States: In this software engineering example, states are probably like “aha” moments when you can feel more humble, or maybe plugged into the team mission.
  • Types: everyone — whether man or woman, manager or engineer — had to grow through these levels. No one was exempt.

What’s next?

We transcended our three smaller teams and created One Team. What’s next?

If we understand Integral Theory properly, we can say one thing for certain: if we want a LEVEL 4 software architecture that works for a broader, more transcendent team, we have to address not only the technical “ITS” aspects, but all of the quadrants at the same time. We can’t leave out personal feelings (as much as I hate to admit it), team dynamics, and all the rest of that fuzzy “interior” stuff. Nor can we leave out organizational structures, processes, and other “exterior” stuff which may seem unrelated to architecture, but tie back into the “interior” and therefore back again to the “exterior” as well.

It’s all a tangled mess. We are all a tangled mess.

All of the quadrants grow together.


Whew.

That’s the end of my story, and the end of my quick tour of Integral Theory.

Thanks for reading this far! I hope you found this at least somewhat interesting, and if you’re able to apply these concepts to your life/situation/work/whatever, I’d love to hear about it!

Or you may be thinking:

  1. yeah, great story… super obvious
  2. wow, you’re really full of yourself / think you’re so “inclusive”
  3. nobody cares

…to which I would respond…

  1. Yes, it’s obvious. But I missed it for years. Then a light bulb came on.
  2. Yes, I’m full of myself. Every day I gather more evidence of this.
  3. Yes, that’s why I haven’t quit my day job.

Onward anyway!

Next time, I hope to take this one step further, into the realm of human consciousness and spirituality, where it should actually get more useful, not less. (If you can’t wait for that, or want to learn more and prefer firsthand information, you may like this introduction from Ken Wilber.)

Until then, peace!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s