Glenn Vanderburg

: Blog

« 9 to 16 of 40 articles »
Bridges and Software
Thu, 08 Feb 2007 (18:42) #
(I sent this to the NFJS speakers’ mailing list last week, and Ben Galbraith suggested I repost it here.)

Every now and then I hear someone compare software development to bridge building. (Bridge building, of course, is just a placeholder here for "real engineering," which in the speaker’s mind is much cleaner and more manageable than the current messy state of software development.) Sometimes it’s "software development isn’t like building bridges," while on other occasions it’s "software development should be more like building bridges." In either case, though, the implication is clear: bridge building is predictable, rote, unexciting, very manageable work, and software development is not. The only difference is whether the speaker likes software development the way it is, or wishes it could be different.

I think both positions are misinformed. And no, I’m not about to pull out my magic prescription for how to solve all the software industry’s problems by making it more like bridge building. In my experience, software developers tend to have an idealized, unrealistic view of what "real engineering" is like. Sure, some kinds of bridges are so well understood by now that there’s very little risk involved; freeway overpasses and the like are churned out regularly and routinely (in much the same way that simple CRUD applications, whether web- or desktop-based, are usually safe bets for even inexperienced development teams). But from what I’ve learned, bridge building in general is a lot more like modern software development than most people realize. And I think the software industry can learn some lessons from the history of bridge building.

Take, for example, the bridges of Swiss engineer Robert Maillart. His best known bridge, Salginatobel, was just featured in a really nice piece about some of the best man-made structures.

Maillart was seeking new designs that would take advantage of the properties of a new material: reinforced concrete. It had been in use for some time, and builders had figured out how to work with it, but Maillart realized that reinforced concrete had unique properties that would permit the use of new forms, resulting in significant savings (due to reduced material costs).

The formal methods used by civil engineers at the time weren’t up to the challenge of analyzing these structures (known today as "hollow box arches" and "deck-stiffened arches"). Maillart verified the designs empirically, by building models, rolling barrels full of concrete over them, etc. etc. The civil engineering establishment of the day vilified him as a charlatan who was endangering lives and cheating his customers by building bridges that would fall down. But he got customers anyway, because his designs were much, much cheaper to build. (The fact that they were strikingly beautiful didn’t hurt.)

Another engineer of the time was Leon Moisseiff, a strong proponent of formal methods and the developer of "deflection theory," at the time the state of the art in mathematical analysis of suspension bridges. Moisseiff designed a bridge intended to be a showpiece for the power of deflection theory. It was the Tacoma Narrows bridge. After its famous collapse, other bridges that had been designed with Moisseiff’s assistance (such as the Golden Gate) were retrofitted with stiffening trusses. It turned out that deflection theory was deeply flawed in a way that nobody had yet realized.

One of Maillart’s bridges did fall down … after being buried under an avalanche. One was demolished because more capacity was required. The rest are still in use, and the forms he pioneered are now standard design taught to civil engineers. The math eventually caught up with Maillart’s methods. As the story I linked to above notes, Maillart is an inspiration to the current superstar of bridge design, Santiago Calatrava.

I think there are some important lessons here for the software profession. The lesson is definitely not that "real engineering" is a mechanistic, purely construction-oriented process, which is the lesson that is usually assumed when software is compared to bridges.

Note: I have at best an interested layman’s knowledge of the history of bridge engineering. Sources include Henry Petroski’s wonderful Engineers of Dreams: Great Bridge Builders and the Spanning of America for information about Moisseiff, and David P. Billington’s article "The Revolutionary Bridges of Robert Maillart" (from the July 2000 edition of Scientific American). For what I believe to be the best description of the true relationship between software development and other engineering disciplines, I encourage you to read "What is Software Design?", Jack Reeves’ brilliant essay.

Sun, 18 Jun 2006 (16:09) #
I was thrilled to learn today that my friend Eitan Suez has released his JMatter application development framework as open-source software.

I’ve been a huge fan of the idea of Naked Objects since I first saw Richard Pawson talk about the idea at OOPSLA in 2000. (He called the idea "expressive systems" then, but only the name has changed.) I introduced the idea to Dave Thomas at OOPSLA the following year, and he began spreading the word through a series of talks at NFJS symposiums.

Unfortunately, programmers who became interested in Naked Objects as an application-development strategy frequently turned away from it again after becoming frustrated with the poor quality and design of the default naked objects application framework. Eitan, however, took a better approach: he decided to write a better framework. And he wrote it in the context of a real application he was developing for his employer, which is always the best way to drive framework design: validating ideas in the crucible of real-world constraints. The resulting framework is JMatter, and it’s a great tool. If you’ve ever wanted to explore Naked Objects as a way of building cool, powerful business applications quickly — or if you’ve already tried Naked Objects but decided it wasn’t ready yet — you owe it to yourself to give it a try.

(Disclaimer: my enthusiasm for JMatter has nothing to do with the fact that my visage is prominently featured in one of Eitan’s sample application screenshots. Quite to the contrary, in fact. :-)

OOPSLA Essay: Extreme Programming Annealed
Fri, 03 Feb 2006 (04:02) #
(Brian Marick blogged a couple days ago about the OOPSLA 2006 essays track, and I remembered that I should’ve blogged about last year’s event a few months ago.)

Last year was the inaugural OOPSLA essays track. I was thrilled to see it, because I enjoy the kind of papers that OOPSLA used to be famous for. The essays track (as well as the "Onward!" track that has been featured at OOPSLA the past few years) is intended to provide a forum for research that’s a little less "hard" and empirical than the typical modern OOPSLA fare — but no less important.

When I saw the announcement of the essays track last year, I was thrilled, partly because I had done some unpublished work that seemed perfect for the new forum. I wrote up the essay, and was thrilled to have the opportunity to present it at the conference. Brian, the track chair, lined up Kent Beck to respond to my essay during the presentation, which made for a fun discussion.

As it happens, my essay, A Simple Model of Agile Software Processes, -or- Extreme Programming Annealed, was the only essay accepted last year, out of several submissions. I wasn’t privy to the details of the review process, of course, so I don’t know why the other submissions weren’t accepted.

But the track was very well received, and I certainly hope it continues and grows.

I certainly wouldn’t hold my essay up as an exemplar of an OOPSLA essay, but I do hope it might suggest the kind of form that the essays committee is looking for. I hope to see more (and better) examples in this year’s essays track.

Pragmatic Studio: Ajax
Mon, 16 Jan 2006 (17:40) #
On Mondays I typically scan my calendar for the next few weeks to see what’s coming up, and this morning I was reminded that the first Pragmatic Studio for Ajax is coming up in just over three weeks. I’m thrilled that I have the opportunity to be there. I’m one of the "Ajax experts" scheduled to speak at The Ajax Experience in May, but never mind about that — I’ve never failed to learn new things whenever I’ve heard Stu Halloway and Justin Gehtland speak. Besides … I’ve noticed that cool people like Rael Dornfest are planning to be there, and the photos from the first Rails studio show what a fun time these training events are. (I’m seriously disappointed I haven’t been able to make it to a Rails studio.)

Ajax is a funny beast, and you can expect to meet Java, Rails, .NET, and PHP developers there — and that’s just part of the fun, to me. There’s still time to sign up and get good airfare!

The Right Team for Rails
Mon, 31 Oct 2005 (15:51) #
A few weekends ago, in Chicago, Ted Neward asked me an interesting question. I had a kind of lame, generic response at the time; it took me a few hours to really think things through and work out what I really think about the issue. I’m finally getting around to blogging about it.

The occasion was a No Fluff, Just Stuff speakers’ panel. As is usual these days, there was a question about Rails: "What do you think it’ll take for Rails to hit the mainstream," if I recall correctly. I answered that Rails is already on a path that will lead to very widespread use, but that it won’t necessarily be smooth. I predicted that the next year would see a highly publicized "Rails failure," caused not by Rails itself but by a company that tried to do Rails with the wrong team.

That was when Ted, who was moderating the panel, asked his question. "What is the right team, Glenn? Does it have to have Dave Thomas on it?" Of course, Ted was deliberately overstating the question to be provocative; that’s part of a moderator’s job. But there’s a serious question there. If you want to experience the benefits that are being claimed for Rails, what kind of team should you have?

If I had to explain Rails’ secret sauce in one short sentence it would be this one: Rails makes it easy to do the right thing. "The right thing" there means the kind of basic software design and development practices that are widely known, but not so widely practiced because the typical team isn’t very disciplined. Things like

  • Writing clean, expressive code
  • Avoiding duplication
  • Keeping responsibility well defined (in layers, components, classes, and methods)
  • Testing thoroughly
  • Aggressively employing automation for repetitive tasks

Others might see different things behind Rails’ success, but for me, making it easy to do the right thing is crucial. Things that take a lot of discipline in other languages and frameworks become much easier in Rails. In some cases they’re so easy that they’re actually the path of least resistance. In others they still take discipline, but the drudgery has been reduced dramatically.

The key to building a Rails team is to choose a team that’s prepared to exploit that characteristic.

A good Rails team will consist of programmers who know what they should be doing, and usually have the discipline to do it. They have a good, solid grounding in software design principles and development practices, and a good track record of sticking to them on real projects. Such a team will fall right into the Rails way of doing things, seeing an environment where many of the barriers and obstacles fall away and they can do good work with less effort. This kind of team will see big productivity increases once they get the hang of Rails.

If you feel like gambling, you might not do too badly with some less skilled teams: teams that understand what the right thing is, but aren’t quite disciplined enough to do it consistently. That kind of team may find that, with Rails making the job easy for them, they can be a little more disciplined and start doing much better work. (Just making the right thing easier wouldn’t be enough, but Rails also exploits the No Broken Windows effect: the skeleton application Rails gives you to start out with is in such good shape that it feels wrong to mess it up.) Again, it’s not a sure bet, and you’ll be much better off with a team that’s already strong.

The wrong Rails team is one that doesn’t understand those principles and practices. The fact that Rails makes things easy won’t be enough. In my experience, such teams expend amazing effort and ingenuity to do the wrong thing.

(On the other end of the spectrum, there’s a fourth kind of team that is the ideal Rails team. The truly great Rails team will understand not only what Rails gives them but how it does it, and the team won’t merely benefit from Rails but will build on it, carrying Rails’ design philosophies and techniques into the domain of the application being built. Expect these teams to create their own Ruby domain-specific constructs, extending Rails to meet their own needs and amplifying their own productivity through the course of the project.)

So (you might ask) how are Rails teams any different from any other project teams? The answer is that they’re not. An experienced, skilled team will, in all likelihood, find ways to be successful. An inexperienced one can find many ways to fail. Somewhere in the middle are the teams where each project is a gamble: they might fail, or succeed modestly, or even succeed spectacularly, but it’s very hard to predict the outcome. No tool, language, or framework — not even Rails, which I think is the best thing going — will change that. When you pick up a powerful tool, it doesn’t eliminate your weaknesses; rather, it amplifies whatever characteristics you bring to it, strengths and weaknesses alike.

Rails gives a team some great tools for success. But the team has to understand those tools and be prepared to exploit them.

Same as it ever was.

I think Ted does get Rails
Thu, 18 Aug 2005 (15:31) #
I have to admit that my first reaction to Ted Neward's blog about Rails was the same as Dion's and Justin's: "Ted’s missing the point." But his followup has mostly changed my mind. Ted’s still skeptical, which is a good thing, but what he thinks he sees in Rails, and what he’s looking for, is (to my mind) the right thing, and my belief is that he’ll find it there.

Ted writes: "Look, guys, at the end of the day, if Rails is about Ruby and the things that a scripting language can do that a compiled, statically-typed language can’t, then Rails definitely has a place in the world and I’ll take the time to learn it." My position is that that’s precisely what Rails is about. Sure, some of the lessons that Rails teaches can be carried across to statically-typed languages just fine, but some of the most important ones simply won’t translate. Rails is a testament to the power of dynamic languages in general, and of Ruby in particular.

Here’s one example: it’s a mistake Ted did make in his first blog, and it’s a mistake others make as well. "I mean, I see a bunch of intelligent code-generation," Ted wrote, and if you’re coming from a static language background it’s easy to look at Rails’ scaffolding support and default views and think that’s what you’re seeing. I thought so, too. If that’s what the scaffolds are, they aren’t too impressive — partly because that’s easy to do, but mostly because generated code like that eventually becomes a burden. Most of us have seen code generation tools before, and they’re great for getting started, but when you move beyond what the generated code will do for you you’re in a world of hurt, often having to throw away what’s built into the framework and supply your own versions of everything.

But Rails isn’t doing "intelligent code-generation" — unless what you mean by that is that it’s doing the least possible code generation. Crack open those scaffolds and look inside. They’re almost empty! Usually they’re just trivial class definitions, with no methods or fields. All of the behavior comes at runtime, through inherited reflective logic. Rails is generating structure, not behavior, and that turns out to be a huge win. Ruby makes that possible, partly through its rich support for runtime reflection, but mostly because its dynamic, open nature makes it a breeze to fill in those empty spaces incrementally, overriding and extending little by little the default behavior that’s built into the framework.

And that’s just one example of how a dynamic language makes Rails possible, and why some of Rails’ goodness is easy for we static-language types to miss at first glance.

What’s Rails about?

  • It’s about Ruby and the things that a scripting language can do that a compiled, statically-typed language can’t.
  • It’s about confirming some of the earliest thinking about frameworks: that they should be extracted from well-designed applications, rather than being designed on their own.
  • It’s about demonstrating the fundamental importance of the DRY principle for software design. (Bruce Eckel calls it "the most fundamental concept in computing.")
  • Oh … and it’s about bringing the pendulum back away from the layers-upon-layers default approach in Java projects.
Build Languages
Thu, 11 Aug 2005 (01:31) #
When Mike Clark was writing Pragmatic Project Automation, he and I conspired about a few things. A biggie was build scripts — in particular, the expressiveness and power thereof. We persuaded James Davidson to publicly admit that the choice of an XML format was … well, not a mistake exactly, but ultimately a limiting strategy. And we wanted to demonstrate the usefulness of having a real programming language at your disposal in a build script. A lot of this came from our interest in Ruby and our use of Rake, but in the context of a Java-focused book Mike decided to demonstrate by using Groovy to script Ant. That was cool, but it just touched the surface. Since it wasn’t really the main topic of the book, Mike wisely kept it short.

Today, Martin Fowler blogged an excellent article about the strengths of using Rake. Martin gives it the full treatment, explaining Rake, how it’s similar to both Make and Ant, and how it goes beyond both of those older tools. He demonstrates in several ways how powerful it is to have all of Ruby available in build scripts.

My favorite line: "After all, until we tried it I thought XML would be a good syntax for build files."

Read it now.

James Bach on best practices
Sun, 10 Jul 2005 (12:32) #

In my maturity model of the industry, promoting a maturity model is mere level 2 behavior. By level 3, we outgrow it.

(Bach's article speaks directly of software testing practices, but it's all applicable to other development practices as well.)