Methodologies considered risable
A recent blog on The Agile Fallacy got me thinking. I've been programming for 45 years and still am, although slowly. I have seen the wheel go round many times, and I have a somewhat different perspective to that expressed in the blog. I won't mention any specific technologies because that will only yank various folk's chains. In the 1960's most big development projects delivered late and lousy. It became a standing joke that programmers used all of the project time and budget to deliver the first 90% of the agreed deliverable, and an equal amount of time and money to deliver the other 90%, assuming they were allowed to carry on that long. Peter Jackson famously remarked that movies don't actually get "released", they escape into the wild, and the same can be said for most large application development efforts. Whatever isn't done right at the time of release is done later, and called maintenance. The programmer's ultimate refuge is to claim "the users' requirements changed", even if the requirements were to implement double-entry bookkeeping which has been around in its present form for about 800 years. I rambled on about this issue in an earlier blog entry.
Fred Brooks, manager of IBM's OS/360 development project, wrote The Mythical Man Month to describe many of the pitfalls found in the 1960's. A lot of DP managers (that's what we used to call them) got fired as a result of non-delivery, and in self-defense they developed "methodology", elaborate systems of checks and balances, reports and reviews, that slowed the whole development process down to a point where there were far fewer ugly surprises. You can tell that "methodology" was a con game from the start by its name – "methodology" should mean the study of methods, not the practice thereof. It should have been called just "methods", but that was far to simple a word for the grand collusion then underway. With the introduction of "methodology" productivity plummeted, but the DP managers got to keep their jobs, and that's what counted most (to them).
The underlying problem with predicting how long development will take is that the productivity of individual programmers varies hugely. A research project at Dartmouth College back in the 1960's showed that the slowest student programmers were over 100 times slower than the fastest. The only reason that the study showed "only" a factor of 100 is that the researchers got tired of waiting for the last few laggards, closed the lab and went home. Sadly, you can't tell which programmers are fast and which are slow just by looking that them, so it's hard to pick a winning team. Ornate methodologies overlay so many admin tasks on all programmers that the difference in productivity between the best and the worst is masked.
While DP managers were happy with "methodologies", the really competent programmers were totally frustrated with them because they knew that they were being hobbled. As they say, "how can you fly like an eagle when you are surrounded by turkeys?" Some of the more cunning ones solved the problem by developing revolutionary new "methodologies" that allowed them to deliver running code dozens of times more quickly than the old, high-bulk "methodologies". On inspection, these new "methodologies" succeeded by jettisoning most of the padding that had been packed into the old ones, thus allowing the good programmers to get the job done fast. In order to allow them to use these new techniques, the practitioners had to dress them up with fancy new names, buzzwords, smoke, and mirrors.
Things went well for a while with the competent programmers delivering code at speed, but after a while the rest of the industry demanded the right to catch up. Snake oil salesmen were quick to "analyse and to codify" the new fast-track methodologies, and to come up with books and courses that would allow any pedestrian programmer to fly like an eagle. Or so they said. Institutions all over the world tried the new panacea, and found out the hard way that they didn't work for them. The new paradigms became discredited, but the fast programmers now had the bit between their teeth, and they realized that the way forward was to coin a new name for low-bulk programming every time the previous name got discredited. To date we have seen a number of iterations of this particular wheel, with the smart programmers soaring like eagles while the rest of the pack follow behind, crashing through the undergrowth and gobbling wildly as they struggle to become airborne.
So what should we do about this problem? The reality is, we can't afford to do without the really competent programmers, and if the new methodology game is the price that we have to pay in order to allow them to get the important stuff done fast, so be it. Just be aware that your mileage may vary.
Fred Brooks, manager of IBM's OS/360 development project, wrote The Mythical Man Month to describe many of the pitfalls found in the 1960's. A lot of DP managers (that's what we used to call them) got fired as a result of non-delivery, and in self-defense they developed "methodology", elaborate systems of checks and balances, reports and reviews, that slowed the whole development process down to a point where there were far fewer ugly surprises. You can tell that "methodology" was a con game from the start by its name – "methodology" should mean the study of methods, not the practice thereof. It should have been called just "methods", but that was far to simple a word for the grand collusion then underway. With the introduction of "methodology" productivity plummeted, but the DP managers got to keep their jobs, and that's what counted most (to them).
The underlying problem with predicting how long development will take is that the productivity of individual programmers varies hugely. A research project at Dartmouth College back in the 1960's showed that the slowest student programmers were over 100 times slower than the fastest. The only reason that the study showed "only" a factor of 100 is that the researchers got tired of waiting for the last few laggards, closed the lab and went home. Sadly, you can't tell which programmers are fast and which are slow just by looking that them, so it's hard to pick a winning team. Ornate methodologies overlay so many admin tasks on all programmers that the difference in productivity between the best and the worst is masked.
While DP managers were happy with "methodologies", the really competent programmers were totally frustrated with them because they knew that they were being hobbled. As they say, "how can you fly like an eagle when you are surrounded by turkeys?" Some of the more cunning ones solved the problem by developing revolutionary new "methodologies" that allowed them to deliver running code dozens of times more quickly than the old, high-bulk "methodologies". On inspection, these new "methodologies" succeeded by jettisoning most of the padding that had been packed into the old ones, thus allowing the good programmers to get the job done fast. In order to allow them to use these new techniques, the practitioners had to dress them up with fancy new names, buzzwords, smoke, and mirrors.
Things went well for a while with the competent programmers delivering code at speed, but after a while the rest of the industry demanded the right to catch up. Snake oil salesmen were quick to "analyse and to codify" the new fast-track methodologies, and to come up with books and courses that would allow any pedestrian programmer to fly like an eagle. Or so they said. Institutions all over the world tried the new panacea, and found out the hard way that they didn't work for them. The new paradigms became discredited, but the fast programmers now had the bit between their teeth, and they realized that the way forward was to coin a new name for low-bulk programming every time the previous name got discredited. To date we have seen a number of iterations of this particular wheel, with the smart programmers soaring like eagles while the rest of the pack follow behind, crashing through the undergrowth and gobbling wildly as they struggle to become airborne.
So what should we do about this problem? The reality is, we can't afford to do without the really competent programmers, and if the new methodology game is the price that we have to pay in order to allow them to get the important stuff done fast, so be it. Just be aware that your mileage may vary.
posted by Trevor Turton at 10:39 am
0 comments
