Bill Caputo, through repeated conversations we've had, has convinced me of something very surprising. It was something that changed the way I think about the world, and how I do my job.
There is no such thing as software productivity.
As Martin Fowler observed almost a decade ago, productivity in software cannot be usefully measured. The reason why is it just doesn't exist in The Realm of Relevant Things. Put another way, productivity has no applicability as a metric in software. "How much did we create today?" is not a relevant question to ask. Even if it could be measured, productivity in software does not approximate business value in any meaningful way.
This is because software development is not an activity that necessarily produces anything. Here's a thought experiment: Let's say that you have a couple of developers working on the same project, and by accident, both of them pick up the same task on the same day. The first one, Frank, hauls off and writes a 1000 line framework that solves the problem beautifully. The code is well written, well tested, and the deployment and operation of it is well documented. The second developer, Peter, heads off to to the park for the day, where he thinks about the problem while he feeds the pigeons. Around 4:45, Peter wanders back to the office, deletes 100 lines of code, deploys the change...and the problem is fixed.
Which of these two developers was more "productive" today? The answer is: It doesn't matter. What matters the that Peter solved the problem, while simultaneously reducing long term maintenance costs for the team. Frank also solved the problem, but he increased maintenance costs by producing code, and so (all other things being equal) his solution is inferior. To call Peter more "productive" is to torture the metaphor beyond any possible point of utility.
I would argue that what good software developers do is remove problems. The opposite, in fact, of production. The creation of technological artifacts such as code, documentation, data, etc...are all necessary evils to achieve the goal of removing problems. That's why, sometimes, the most effective solution to a problem is a 5 minute conversation.
The strongest evidence for this model is that when you look at software development this way, a large number of thorny, observable-but-unquantifiable problems suddenly become easy to understand. For example, why do developers (usually) seem to be less effective when you isolate them from stakeholders and/or users. Wouldn't keeping them from getting distracted make them more productive? The answer is yes, it does often make them more productive...but it also makes them less effective. Since what they're doing is solving problems for stakeholders, isolating them from those stakeholders makes it harder for them to identify and solve problems. Having access to the people whose problem is being solved is more effective than the alternative, even if it means not having hands-on-keyboards for 8 full hours a day.
Which leads us to another example: Why do maintenance costs usually (but not always) dwarf other costs in building software? Why can't we ever seem to get it "right" the first time? One explanation is that software is a fixed solution to a potentially changing problem. When the problem changes (or when our understanding of it changes), it creates a gap between the problem and the solution. It gets expensive to constantly fill these gaps over time as the problems shift, even though the software doesn't change unless we change it. This also explains why video games traditionally have low maintenance costs compared to other software projects. It could be that the problem being solved (getting people to buy and play the game) is mostly dependent on human psychology, and less likely to change.
What about the mythical 10x "productivity" difference between good and bad developers? Everyone claims to have seen it, and yet nobody seems to be able to measure it directly. This theory explains that too. Problem solving is a much easier thing to "lever" (in the financial sense) than productivity, which makes order-of-magnitude differences easier to achieve. Problem solving requires only information and insight. Either you have it or you don't. There are no raw materials, no throughput limits. It's not that the bad developers are typing slowly. It's not that they would do better if they just "tried harder". They just don't have the insight or the information to solve the problem as effectively. Perhaps the reason you can't measure the difference in productivity between the good and bad developer is because there's nothing to measure.
There may be other phenomena that this theory explains. You can probably figure them out if you try, anyway. Here's what I've been doing lately...try this and see if it works for you. Every time you catch yourself talking about being "productive", ask if you're approaching the problem in the right way. Remember that if we could solve the problem without making anything at all, anything that we actually produce is wasteful.