In the week of February 27 to March 2, 2012, a few Samba developers accepted an invitation by Microsoft and attended an SMB2.2 testing opportunity at Microsofts Enterprise Engineering Center in Redmond. Jeremy Allison, Steve French, Volker Lendecke, Chris Hertel, Christian Ambach, Matthieu Patou and I found our way to Redmond with Stefan Metzmacher participating to some extent via IRC and mumble. For me, the event was a big success, and I am happy that I finally made up my mind to go there. This is my personal report.
Background
Microsoft will ship a new version 2.2 of the SMB protocol with Windows 8. Along with this, a whole new scale out clustering mode is added. The target of these new features is clearly server workload instead of client workload, the two most prominent applications being Virtualization (Hyper-V) and SQL. These two applications that were originally typical applications that ran from SAN-Storage, can now run directly from SMB2.2, and they can go even further when an RDMA adapter is installed, thanks to the new RDMA support in SMB2.2 called SMB Direct. Other intersting features in SMB2.2 are multi-channel sessions and persistent file handles that can survive server failures without data loss.
These new features were first presented at the Storage Developer Conference in September 2011.
There are preview versions of the documents for these new features available from the msdn library, but they are not complete yet and partly still subject to change. Since February 29, the beta version of Windows 8 can be tested. While the client is freely available, the server variant is only available via MSDN subscriptions. Before this date, only a preview from September 2011 was available that did not yet support many of the announced features.
The Test Setup
Microsoft had established two test environments for us with a domain and Windows 8 clients equipped with some test suites. One network contained a Windows 8 cluster server installation, and in the other network was intended for us to integrate our own server implementations to run Microsoft’s test suite against.
After we had trouble accessing our test network the first day, it worked nicely from the second day on and gave us the opportunity to run tests against Windows 8 beta. It was especially useful to run tests against a fully installed Windows 8 cluster.
Signing
Since the beta release, Windows 8 sports a new signing algorithm, aes-cmac, that had not been available in earlier Window 8 previews. In a joint effort with Jeremy and Metze, we were able to fix the last bugs in the code that Metze had written in the last days before the event (by just looking at the docs). So we now have a working signing code against Windows 8, (client and server side), which we did not have before.
Persistent Handles – Test Suites
My focus for the event was on durable (SMB2.0) and persistent (SMB2.2) file handles and clustering features. I gained a better understanding of persistent file handles and I was able to extend our testsuites and add precision to them. I also spotted some bugs in the documentation and a bug in Windows 8 durable-handle-vs-oplock behaviour (a regression from Windows 2008R2). I am still working on extending our testsuites with respect to durable and persistent handles.
It was extremely useful to have several of the core engineers from Microsoft available during the test lab for discussions about product behaviour and the documentation. It helped me to improve my understanding of the new clustering concepts and to deepen my knowledge of durable and persistent file handles.
Persistent Handles – Server Hacking
On the server side, Christian set up a Samba-CTDB cluster in the partner environment and we installed the code from the durable handle work-in-progress-branch that Metze and I are working on, and we started hacking some 2.2 features into it. In the end, we were able to talk SMB 2.2 with the client and offered persistent handles – in a somewhat faked up manner, not being able to give the full set of guarantees attached to persistent handles.
The nice visible success was that we were able to do a transparent node failover of the client copying a dvd image to the samba cluster. We ran “ctdb disable” on the connected node and the client switched over to the other node and kept copying. In the end, the dvd image was complete and its md5sum correct!
You can see this as a record-my-desktop video here.
Note: The new copy progress bar is a really nifty feature in Windows 8. 
As a background, it is important to know that in the Windows 8 clustering, it is the client that implements most of the logic for failover scenarios. The client notices the server outage and reconnects to the new node consciously. It then does a replay of the last write that failed. So the additions on the server side for this first success were resonably small after having some support (still work-in-progress support) for durable handles.
Summary
Summing up, it was a very successful and productive event. I would also like to emphasize that the Microsoft folks have really been very cooperative, and they are also happily accepting comments on documentation and product behaviour.
The outcome for me is that we are on the right track with our work on durable handles, and it is good to see that some of it is already working. There still is a lot of work needed to get the things right, and we have for instance not even seriously touched RDMA yet, but we have a much clearer picture now. The good bit that has been confirmed now, is that the logic of the new Windows clustering is largely in the client. And we were able to demonstrate that we have a chance to take advantage of it in our CTDB clusters without having to throw the established CTDB-clustering away and implement something completely different.
