Presentations, tutorials, etc. by Samba Team members are listed in our:
"It was the best of protocols, it was the worst of protocols, it was the age of Linux, it was the age of Windows..." sorry, sometimes the tag line for the column gets me carried away. In developing Samba we're creating a bridge between two of the most popular standards currently deployed in the computing world, and I thought it might be instructive to examine the difference between how they came to be, and why that's important in deciding what standards to support in government and computing in general.
The first standard to examine is POSIX. Named (like so many things in the software world) by Richard Stallman, it stands for "Portable Operating System Interface" - X (yes OK, so acronyms aren't Richards strong point); meaning a portable definition of a UNIX like operating system API. Few people have actually seen an official POSIX standard document, because the damn thing costs money to buy from the IEEE. Back before the Web became really popular I bought one from the sorely missed Computer Literacy bookshop here in Silicon Valley, just to take a look at what the real thing looked like. It wasn't cheap (a few hundred dollars as I recall). Amusingly enough I don't think Linus Torvalds ever read one or referred to it when originally creating Linux.
Reading the paper standard however is very interesting. It reads like a legal document; every clause is numbered so it can be referred to in other parts of the text. It's detailed. Really detailed. The reason for such detail is it was designed during the great age of the splintering of the UNIX operating system families, and if someone wanted to write a compatible operating system they needed a specification that told them everything about the way that system has to behave at the API level. It tells you what structure members must be defined (for us C programmers that's rather important). It tells you what error messages are permitted to be returned for what errors. It even says what API's you can depend on being present and which might be optional.
Having started with a base of documentation that detailed the standards community managed to keep it up to date when such new things as POSIX threads were added, and they kept the same level of detail in the text. It doesn't cover absolutely everything you might want to know about writing programs on a UNIX-like OS, and it has certain elements that make programmers want to scream due to the stupidity of the design (POSIX file locking comes to mind...) but it's enough to write something like Samba, which is quite a large and complex application program. It's also enough to enable us to make Samba portable across Linux, Solaris, HPUX, AIX, the "BSD-of-the-month" club, and almost every UNIX out there in existence.
The second standard is the de-facto standard of the Windows Win32 API. This originally also wasn't free, but was much cheaper to buy than POSIX. Like most things in Windows, on the surface it looks great ! It covers much more than POSIX tries to bite off, and so offers what appear to be portable interfaces for graphics, sound, pen computing or whatever is the next flavor of the month API Microsoft wants to embed into Windows to kill a competitor, such as HTTP protocol API's. Microsoft claim they document everything, and they update it regularly and used this fact to great effect in the USA anti-trust case to claim that Windows has open interfaces.
Then you start to program with it. If you're used to the POSIX specifications you almost immediately notice something is different. The details are missing. It's fuzzy on the details. You notice it the first time you call an API at runtime and it returns an error that's not listed anywhere in the API documentation. "That's funny....?" you think. The deeper you get into it, the worse it gets. Take file locking for instance (a boring subject for non-programmers I know, but bear with me). You make a mistake and take a lock on a region you already got and it silently succeeds (the lock has been stacked, like a plate in a self-service restaurant plate pile). The people who wrote the Win32 documentation aren't giving the level of detail needed to re-create the API by someone else, as the POSIX docs were designed to do, but they're only giving rough hints to help programmers using it. I once gave a presentation on how Win32 file locking actually works at the yearly CIFS protocol conference (back when Microsoft bothered to attend), and the Microsoft representative sheepishly asked if they could use it internally to document the locking behavior for their own programmers.
What happens over the years is that experienced Windows programmers gain this sort of folk-knowledge about the API's, how they really work versus what the documentation says. I often hang out on Usenet Windows discussion groups (yeah, I'm too old for IRC, what can I do; "you kids, get off my lawn !") and it's very interesting to watch the attitudes of the experienced Windows programmers. They usually hate telling novices how stuff works, it's almost like learning it was a badge of honor, and they don't want to make it too easy for the neophytes. The exude an air of "They must suffer as I did".
The lack of detail is one of the reasons that the Wine project finds it difficult to create a working implementation of the Win32 API on Linux. How do you know when it's done ? Remember that Linus with some help was able to create a decent POSIX implementation within a few years. The poor Wine developers have been laboring at this for twelve years and it's still not finished. There's always one more wrinkle, one more undocumented behavior that some critical application depends on.
It's not entirely Microsoft's fault. They haven't documented their API because, well they haven't needed to. POSIX was documented to this detail due to need; the need of the developers creating implementations. Micrososft know that whatever they make the API do in the next service pack, that's still the standard. "Where ever you go, there you are", so to speak.
So what is important is when governments and large corporations are selecting products based on standards, they pay attention to Open Royalty-Free Public standards. No Microsoft Word ".DOC" format standard (which suffers from the same problem as Win32). No de-facto vendor standards, no matter how convenient. They need to select standards that are at the same level as POSIX. Standard to the level that other implementations can be created from the documentation. It's simple to tell when a standard meets that criteria because other implementations of it exist.
There is one sign of progress. Microsoft have now released their "Services for UNIX" product for free, which includes a NT-kernel subsystem that implements the full POSIX standard in the Windows kernel. So if you want to write a completely portable application, the one standard you can count on to be there on Apple MacOS X, Linux and Windows is POSIX. If you'll excuse me, I'm going to look at porting Samba to Windows......