Your Linux Data Center Experts

Isn't it a shame that rsync has to spend all this time walking your file-system to see what has changed on the source side? Even for modest servers, this can take minutes or hours to just figure out what has changed. Too bad the file-system can't help us out… Oh, wait! btrfs to the rescue! Read on for examples of how to see what has changed.

My previous entry is about a program that makes snapshots every 5 minutes or every hour. There's a feature of btrfs that will tell you what has changed, given a “generation number”. Unfortunately there doesn't seem to be a “what is the current generation” call, the “find-new” call will tell you the generation ID as the last line:

$ sudo btrfs subvolume find-new / 99999999
transid marker was 2100

Here I gave it a bogus generation ID which I knew was much later than what I have. You could also put in “0” for the generation ID and just do a “tail -1” to get the last line, but that takes a while to run as it generates all the diffs between the file-system creation and now…

So what was the set of changes to my file-system most recently?

$ sudo btrfs subvolume find-new / 99999999
transid marker was 2108
$ sudo btrfs subvolume find-new / 2107
inode 185148 file offset 978944 len 4096 disk start 12439048192 offset 0
      gen 2108 flags NONE var/log/audit/audit.log
inode 187390 file offset 0 len 232 disk start 0 offset 0
      gen 2107 flags INLINE etc/openvpn/openvpn-status.log
inode 216339 file offset 159744 len 4096 disk start 12455149568 offset 0
      gen 2108 flags NONE home/jafo/.zsh_history
inode 1047615 file offset 426917888 len 983040 disk start 12454166528 offset 0
      gen 2107 flags NONE home/jafo/.xsession-errors
inode 1047615 file offset 427900928 len 983040 disk start 12455157760 offset 0
      gen 2108 flags NONE home/jafo/.xsession-errors
inode 1058654 file offset 0 len 4096 disk start 11968630784 offset 0
      gen 2107 flags NONE home/jafo/.vim-tmp/tjedentry.1288661473.068206.swp
inode 1058654 file offset 8192 len 4096 disk start 11968643072 offset 0
      gen 2108 flags NONE home/jafo/.vim-tmp/tjedentry.1288661473.068206.swp
inode 1059030 file offset 0 len 32 disk start 0 offset 0
      gen 2107 flags INLINE var/run/cups/certs/0
inode 1059050 file offset 0 len 16384 disk start 11881242624 offset 0
      gen 2108 flags NONE home/jafo/.viminfo
inode 1059082 file offset 0 len 28672 disk start 11881336832 offset 0
      gen 2108 flags NONE home/jafo/.purple/blist.xml
inode 1059085 file offset 0 len 73728 disk start 11884965888 offset 0
      gen 2108 flags NONE var/lib/pgsql/data/pg_stat_tmp/pgstat.stat
transid marker was 2108
$ 

So I had some logs change, my command-line history, I was editing this blog post… Hmm, ~/.xsession-errors… What's that all about. Oh, it's over 400MB and growing fast… I'd better look into that…

With the “btrsnap” keeping snapshots every 5 minutes, I can use those snapshots to see what the latest generation ID is, and what files have changed between them:

$ sudo btrfs subvolume find-new /.snapshots/5m-20101101-190501 9999
transid marker was 2027
$ sudo btrfs subvolume find-new /.snapshots/5m-20101101-191001 9999
transid marker was 2039
$ sudo btrfs subvolume find-new /.snapshots/5m-20101101-191001 \
      2027 | awk '{ print $17 }' | sort | uniq

etc/openvpn/openvpn-status.log
[...]
var/db/sudo/jafo/12
var/lib/pgsql/data/pg_stat_tmp/pgstat.stat
var/log/audit/audit.log
var/log/cron
var/log/messages
$ 
comments powered by Disqus

Join our other satisfied clients. Contact us today.