External (and Live) snapshots with libvirt

Previously, I posted about snapshots here , which briefly discussed different types of snapshots. In this post, let’s explore how external snapshots work. Just to quickly rehash, external snapshots are a type of snapshots where, there’s a base image(which is the original disk image), and then its difference/delta (aka, the snapshot image) is stored in a new QCOW2 file. Once the snapshot is taken, the original disk image will be in a ‘read-only’ state, which can be used as backing file for other guests.

It’s worth mentioning here that:

• The original disk image can be either in RAW format or QCOW2 format. When a snapshot is taken, ‘the difference’ will be stored in a different QCOW2 file
• The virtual machine has to be running, live. Also with Live snapshots, no guest downtime is experienced when a snapshot is taken.
• At this moment, external(Live) snapshots work for ‘disk-only’ snapshots(and not VM state). Work for both disk and VM state(and also, reverting to external disk snapshot state) is in-progress upstream(slated for libvirt-0.10.2).

Before we go ahead, here’s some version info, I’m testing on Fedora-17(host), and the guest(named ‘testvm’) is running Fedora-18(Test Compose):

$ rpm -q libvirt qemu-kvm ; uname -r
libvirt-0.10.1-3.fc17.x86_64
qemu-kvm-1.2-0.2.20120806git3e430569.fc17.x86_64
3.5.2-3.fc17.x86_64
$ 

External disk-snapshots(live) using QCOW2 as original image:
Let’s see an illustration of external(live) disk-only snapshots. First, let’s ensure the guest is running:

$ virsh list
 Id    Name                           State
----------------------------------------------------
 3     testvm                          running


$ 

Then, list all the block devices associated with the guest:

$ virsh domblklist testvm --details
Type       Device     Target     Source
------------------------------------------------
file       disk       vda        /export/vmimgs/testvm.qcow2

$ 

Next, let’s create a snapshot(disk-only) of the guest this way, while the guest is running:

$ virsh snapshot-create-as testvm snap1-testvm "snap1 description" \
  --diskspec vda,file=/export/vmimgs/snap1-testvm.qcow2 \
  --disk-only --atomic

Some details of the flags used:
– Passing a ‘–diskspec’ parameter adds the ‘disk’ elements to the Snapshot XML file
– ‘–disk-only’ parameter, takes the snapshot of only the disk
– ‘–atomic’ just ensures either the snapshot is run completely or fails w/o making any changes

Let’s check the information about the just taken snapshot by running qemu-img:

$ qemu-img info /export/vmimgs/snap1-testvm.qcow2 
image: /export/vmimgs/snap1-testvm.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 2.5M
cluster_size: 65536
backing file: /export/vmimgs/testvm.qcow2
$ 

Apart from the above, I created 2 more snapshots(just the same syntax as above) for illustration purpose. Now, the snapshot-tree looks like this:

$ virsh snapshot-list testvm --tree

snap1-testvm
  |
  +- snap2-testvm
      |
      +- snap3-testvm
        

$ 

For the above example image file chain[ base<-snap1<-snap2<-snap3 ], it has to be read as – snap3 has snap2 as its backing file, snap2 has snap1 as its backing file, and snap1 has the base image as its backing file. We can see the backing file info from qemu-img:

#--------------------------------------------#
$ qemu-img info /export/vmimgs/snap3-testvm.qcow2
image: /export/vmimgs/snap3-testvm.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 129M
cluster_size: 65536
backing file: /export/vmimgs/snap2-testvm.qcow2
#--------------------------------------------#
$ qemu-img info /export/vmimgs/snap2-testvm.qcow2
image: /export/vmimgs/snap2-testvm.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 3.6M
cluster_size: 65536
backing file: /export/vmimgs/snap1-testvm.qcow2
#--------------------------------------------#
$ qemu-img info /export/vmimgs/snap1-testvm.qcow2
image: /export/vmimgs/snap1-testvm.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 2.5M
cluster_size: 65536
backing file: /export/vmimgs/testvm.qcow2
$
#--------------------------------------------#

Now, if we do not need snap2 any more, and want to pull all the data from snap1 into snap3, making snap1 as snap3’s backing file, we can do a virsh blockpull operation as below:

#--------------------------------------------#
$ virsh blockpull --domain testvm \
  --path /export/vmimgs/snap3-testvm.qcow2 \
  --base /export/vmimgs/snap1-testvm.qcow2 \
  --wait --verbose
Block Pull: [100 %]
Pull complete
#--------------------------------------------#

Where, –path = path to the snapshot file, and –base = path to a backing file from which the data to be pulled. So from above example, it’s evident that we’re pulling the data from snap1 into snap3, and thus flattening the backing file chain resulting in snap1 as snap3’s backing file, which can be noticed by running qemu-img again.
Thing to note here,

$ qemu-img info /export/vmimgs/snap3-testvm.qcow2
image: /export/vmimgs/snap3-testvm.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 145M
cluster_size: 65536
backing file: /export/vmimgs/snap1-testvm.qcow2
$ 

A couple of things to note here, after discussion with Eric Blake(thank you):

• If we do a listing of the snapshot tree again(now that ‘snap2-testvm.qcow2’ backing file is no more in use),

$ virsh snapshot-list testvm --tree
snap1-testvm
  |
  +- snap2-testvm
      |
      +- snap3-testvm
$

one might wonder, why is snap3 still pointing to snap2? Thing to note here is, the above is the snapshot chain, which is independent from each virtual disk’s backing file chain. So, the ‘virsh snapshot-list’ is still listing the information accurately at the time of snapshot creation(and not what we’ve done after creating the snapshot). So, from the above snapshot tree, if we were to revert to snap1 or snap2 (when revert-to-disk-snapshots is available), it’d still be possible to do that, meaning:

It’s possible to go from this state:
base <- snap123 (data from snap1, snap2 pulled into snap3)

we can still revert to:

base<-snap1 (thus undoing the changes in snap2 & snap3)

External disk-snapshots(live) using RAW as original image:
With external disk-snapshots, the backing file can be RAW as well (unlike with ‘internal snapshots’ which only work with QCOW2 files, where the snapshots and delta are all stored in a single QCOW2 file)

A quick illustration below. The commands are self-explanatory. It can be noted the change(from RAW to QCOW2) in the block disk associated with the guest, before & after taking the disk-snapshot (when virsh domblklist command was executed)

#-------------------------------------------------#
$ virsh list | grep f17btrfs2
 7     f17btrfs2                      running
$
#-------------------------------------------------#
$ qemu-img info /export/vmimgs/f17btrfs2.img
image: /export/vmimgs/f17btrfs2.img
file format: raw
virtual size: 20G (21474836480 bytes)
disk size: 1.5G
$ 
#-------------------------------------------------#
$ virsh domblklist f17btrfs2 --details
Type       Device     Target     Source
------------------------------------------------
file       disk       hda        /export/vmimgs/f17btrfs2.img

$ 
#-------------------------------------------------#
$ virsh snapshot-create-as f17btrfs2 snap1-f17btrfs2 \
  "snap1-f17btrfs2-description" \
  --diskspec hda,file=/export/vmimgs/snap1-f17btrfs2.qcow2 \
  --disk-only --atomic
Domain snapshot snap1-f17btrfs2 created
$ 
#-------------------------------------------------#
$ qemu-img info /export/vmimgs/snap1-f17btrfs2.qcow2
image: /export/vmimgs/snap1-f17btrfs2.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 196K
cluster_size: 65536
backing file: /export/vmimgs/f17btrfs2.img
$ 
#-------------------------------------------------#
$ virsh domblklist f17btrfs2 --details
Type       Device     Target     Source
------------------------------------------------
file       disk       hda        /export/vmimgs/snap1-f17btrfs2.qcow2
$ 
#-------------------------------------------------#

Also note: All snapshot XML files, where libvirt tracks the metadata of snapshots are are located under /var/lib/libvirt/qemu/snapshots/$guestname (and the original libvirt xml file is located under /etc/libvirt/qemu/$guestname.xml)