news.samba.org

Jeremy Allison Column Archives

The Low Point — a View from the Valley — Column 17

Reliability Rules

One of the most interesting parts of my job is reviewing previously proprietary software on behalf of my employer to decide if it's worth it to “Open Source” the code, and if so under what license it should be released. This isn't as easy as you might think. On the one hand, my Sheffield “Independent Socialist Republic of South Yorkshire” roots (sadly now long since deceased under “New” Labour, as far as I can tell from the distant view of California's Silicon Valley) push me into recommending the GPL for everything. The GNU General Public License forces all redistribution to include the full source code, thus satisfying my “share and share alike” socialist principles.

On the other, to do a decent job for my employer (evil capitalists that all corporations are :-), I sometimes have to recognize that the GPL isn't always the correct license to use. When the desire is that certain software technology become ubiquitous for example it's much better to recommend a BSD (Berkley Software Distribution, one of the original UNIX authors) style of license, which essentially allows the software to be used as you wish except for claiming that you wrote it.

Unfortunately sometimes the right answer is not to Open Source some code at all. There's no use “throwing source code over the corporate wall” as it's sometimes described, without an ecosystem of motivated programmers ready and interested in working with, maintaining, and extending it. The original release of Netscape's “Mozilla” browser code almost fell into that category: which would have been a catastrophe for the freedom and diversity of computing platforms on the Web. That source code release was widely derided as a failure for the first few years, and only the hard work of the dedicated group of programmers at the Mozilla foundation have preserved the ability to see any Web pages without first having to install a version of Windows in order to use Internet Explorer. The currently successful Firefox browser had a lonely and difficult childhood. Source code out on its own is a fragile thing, and like fine silk it rots with disuse. Having to say “no” to a well meaning and keen group of engineers who want to release some source code that has no chance of making it out there as a separate project is the hardest thing I ever have to do.

It's a nasty job but someone has to do it, and at least it's something I have experience in, having done something similar at many previous companies. It also turns out it's a common task in many different software companies wrestling with the Open Source/Free Software phenomenon, as I have found out over the years talking with people doing the same job elsewhere. Engineers love to gossip, especially about other engineers and projects, which is how I learned something new about Open Source development recently. I'm not naming names in order to protect the parties I'm describing and I'm obscuring the details enough that hopefully no one will recognize themselves in the story I'm describing: the point I'm trying to make will hopefully still survive.

A group of engineers wanted to release a large and incredibly useful piece of source code comprising a library. This wasn't source code without a history, a version of this library had been widely used within the company in question for many years, yet strangely, despite its usefulness, it had fallen into disuse and was no longer integrated into the projects that once had eagerly used it. In fact the other projects had forked off a version of the code and were maintaining it themselves. The latest version of the library was not being used in the company, and so the engineers wished to release it to the world in order that it not be lost in some dusty archive tape vault, never to be seen again.

Let me be honest; this is the worst reason in the world to release source code and had I been on that companies review board I would have instantly recommended not releasing it. The reason is simple. If no one is using a library, just making it Open Source/Free Software isn't going to instantly generate a new set of eager programmers willing to embrace it. It's going to languish in obscurity and be left unused as another lesson for corporations in how not to engage the Open Source/Free Software community. But then according to my informant something unexpected happened.

The internal projects who had been using earlier versions of the library and who had rejected the later version stepped forward and said “of course if this library is going to be Open Source things would be different”, and committed to use the Open Source version in their projects going forward. When I inquired what made the difference I was told they had felt that if it was Open Source then it was more reliable and they could assume it would be maintained for longer. This was unprecedented in my experience, and when I heard about it I started to wonder what it was that made then think that, and how it had happened ?

The answer I came up with was based on my own experience developing proprietary software in the past. I have created software similar to this library, only to have it languish unused in the corporate vault of abandoned projects. Sometimes this was through no fault of my own, I had been moved to a different project, the company priorities had changed and the software just decayed in place, as does all unmaintained software. The engineers developing the internal projects must also have had such experiences, they're common to anyone working in proprietary software. In short, they didn't trust their own corporate engineering process to maintain the libraries they depended on. But they did trust the Open Source/Free Software development process to do so.

Why is this ? Over the years I've worked with both proprietary and Open Source/Free Software I have seen the difference and it's mainly a difference in quality and accountability. No, this doesn't mean all Open Source software is superior to proprietary software, but on average I find that Open Source software is higher quality, and thus safer to commit to be used in other projects. The recent comparison of bugs per-line-of-code between proprietary and Open Source code done by the (ironically) proprietary software company Coverity have shown hard evidence for this, with Open Source code being proven measurably superior. This argument is nothing new, and misses the Freedom part of Free Software, something that Richard Stallman and the Free Software Foundation often point out. The accountability comes with knowing the actual names of the engineers who wrote and tested the code, and being able to communicate with them directly, as peers. It's the difference between the claim that “washing power brand X washes clothes better” with the implied “just trust us” and a peer-reviewed scientific study of detergents, at a more basic level the difference between science and “alchemy”.

Open Source/Free Software is more reliable, in every use of the term, and it seems that more than just the people who develop it are starting to understand that.