A recent thread on the zfs-fuse mailing list has announced that the long-awaited Lustre project to make a native ZFS module for Linux has made good progress. This was announced as being the future for Lustre probably a year ago, but I haven't heard anything about it until this post on the list.

Things are still pretty early it sounds like -- zfs-fuse is likely to be the best choice for probably the next 6 months at least, but it is a significant step toward getting ZFS native under Linux.

The ZFS license is still CDDL, which means that it won't be included in the kernel.org kernel, instead it'll be an out-of-kernel module like DRBD (until recently) or Xen, etc...

This comes at the same time as the ZFS-FUSE 0.6.9 release, which includes deduplication and many other great features. In my testing of 0.6.9b3, it's been working really great. I've been hammering on it with both "zfsstress" and also running it on a test backup server, and it's been running very solidly.

The deduplication has been working well, though you really do need a lot of memory in the ARC cache if you want it to perform well. For this system with 8x2Tb drives, I figure I'll need to put at least 8, and possibly 16GB in the ARC cache. I currently have 8GB RAM, and a 2GB ARC, which is about as much as I can do in an 8GB system. The host will take up to 32GB RAM though, so I have room to grow. My plan is to upgrade it to 8GB and push the ARC up to 8GB, then see how it works. I blew out the original 800MB ARC with deduplication at around 900GB stored in the pool.

It looks like with compression plus deduplication I'm getting a 1.9:1 space savings. Not sure how this compares to the deduplication+compression in BackupPC, but I'm expecting it to do much better simply because I can do block-level changes (large files that just have small appends/updates to them, like databases or log-files).

Anyway, that's the ZFS news for today. :-)
(Post Reply)