When Murray Gell-Mann was asked how Richard Feynman managed to solve so many hard problems Gell-Mann responded that Feynman had an algorithm:


Write down the problem.
Think real hard.
Write down the solution.


Gell-Mann was trying to explain that Feynman was a different kind of problem solver and there were no insights to be gained from studying his methods. I kinda feel the same way about managing complexity in medium/large software projects. The people that are good are just inherently good at it and somehow manage to layer and stack various abstractions to make the whole thing manageable without introducing any extraneous cruft.

So is the Feynman algorithm the only way to manage accidental complexity or are there actual methods that software engineers can consistently apply to tame accidental complexity?


  When you see a good move, look for a better one.
    —Emanuel Lasker, 27-year world chess champion


In my experience, the biggest driver of accidental complexity is programmers sticking with the first draft, just because it happens to work.  This is something we can learn from our English composition classes.  They build in time to go through several drafts in their assignments, incorporating teacher feedback.  Programming classes, for some reason, don't.

There are books full of concrete and objective ways to recognize, articulate, and fix suboptimal code:  Clean Code, Working Effectively with Legacy Code, and many others.  Many programmers are familiar with these techniques, but don't always take the time to apply them.  They are perfectly capable of reducing accidental complexity, they just haven't made it a habit to try.

Part of the problem is we don't often see the intermediate complexity of other people's code, unless it has gone through peer review at an early stage.  Clean code looks like it was easy to write, when in fact it usually involves several drafts.  You write the best way that comes into your head at first, notice unnecessary complexities it introduces, then "look for a better move" and refactor to remove those complexities.  Then you keep on "looking for a better move" until you are unable to find one.

However, you don't put the code out for review until after all that churn, so externally it looks like it may as well have been a Feynman-like process.  You have a tendency to think you can't do it all one chunk like that, so you don't bother trying, but the truth is the author of that beautifully simple code you just read usually can't write it all in one chunk like that either, or if they can, it's only because they have experience writing similar code many times before, and can now see the pattern without the intermediate stages.  Either way, you can't avoid the drafts.
