Tuesday, June 24, 2008

WAN-accelerator for NFS

I've been working a bit with WAN-acceleration for CTDB. Actually with two different approaches
for two different purposes.


WAN-accelerator #1 (general purpose)

The first approach was to add new "capabilities" to the CTDB daemon so that you could have a cluster of CTDB nodes where some nodes were located at a very remote site, across a high-latency WAN-link. This was tricky to solve since eventhough you have nodes that participate synchronously in the cluster you do not want the high WAN-link latency to affect performance on the nodes in the main datacentre.

Initial tests seems to indicate that it works quite well. Surprisingly well.

But this is not really a WAN-accelerator. A classic WAN-accelerator is more a device that performs a man-in-the-middle attack on the CIFS/NFS protocols and performs some (sometimes unsafe) caching.

In the CTDB approach above there is no man-in-the-middle attack, nor is it really a WAN-accelerator.
It is conceptually more like one single multihomed CIFS server where one on the NICs (the remote site) happens to be a few hundred ms away. Thus we dont have to play any tricks, nor do any questionable caching, we are still a single cifs server, with fully and 100% correct cifs semantics, its just that this cifs server is spread out across multiple sites.

I.e. clients on the remote site talk to the genuine real cifs server. Not an man-in-the-middle imposter that may or may not provide correct semantics.

WAN accelerator #2 (nfs)

A different solution was based on FUSE and providing very aggressive caching of data and metadata for NFS. This one also seems to perform really well but is obviously less cool than "a single multihomed cifs server spanning multiple sites".