Snapshotting with libvirt for qcow2 images

Libvirt 0.9.6 was recently out. Take a look at 0.9.5 changelog for truckload of features/bugfixes/cleanups(specifically snapshot related) from the libvirt team.

So, I grabbed the F14 srpm from Libvirt ftp, and made a quick Fedora koji scratch build of libvirt-0.9.6 for Fedora 15 and gave the snapshot features a whirl. Here it goes:

(Also noted below is some very useful discussion I had(on #virt, OFTC) with Eric Blake (Upstream/Red Hat’s Libvirt dev, very friendly guy.) on snapshots. It was way informative not to capture it.)

Context on types of snapshots
At the moment, snapshotting in KVM/QEMU/Libvirt land is supported primarily for QCOW2 disk images. I briefly discussed about Qcow2 previously here.

There are several different types of snapshots possible. Some idea on that:

Internal snapshot: A type of snapshot, where a single QCOW2 file will hold both the ‘saved state’ and the ‘delta’ since that saved point. ‘Internal snapshots’ are very handy because it’s only a single file where all the snapshot info. is captured, and easy to copy/move around the machines.

External snapshot: Here, the ‘original qcow2 file’ will be in a ‘read-only’ saved state, and the new qcow2 file(which will be generated once snapshot is created) will be the delta for the changes. So, all the changes will now be written to this delta file. ‘External Snapshots’ are useful for performing backups. Also, external snapshot creates a qcow2 file with the original file as its backing image, and the backing file can be /read/ in parallel with the running qemu.

VM State: This will save the guest/domain state to a file. So, if you take a snapshot including VM state, we can then shut off that guest and use the freed up memory for other purposes on the host or for other guests. Internally this calls qemu monitor’s ‘savevm’ command. Note that this only takes care of VM state(and not disk snapshot). To try this out:

 
#------------------------------------------------
# Memory before saving the guest f15vm3
$ free -m
             total       used       free     shared    buffers     cached
Mem:         10024       5722       4301          0        164       4445
-/+ buffers/cache:       1112       8911
Swap:            0          0          0
#------------------------------------------------
$ virsh list
 Id Name                 State
----------------------------------
  5 f15guest             running
  6 f15vm3               running
#------------------------------------------------
# Save the guest f15vm3 to a file 'foof15vm3'
$ virsh save f15vm3 foof15vm3
Domain f15vm3 saved to foof15vm3
#------------------------------------------------
# Now, f15vm3 is gracefully saved/shutdown.
$ virsh list
 Id Name                 State
----------------------------------
  5 f15guest             running
#------------------------------------------------
# Notice the RAM being freed
$ free -m
             total       used       free     shared    buffers     cached
Mem:         10024       5418       4605          0        164       4493
-/+ buffers/cache:        760       9263
Swap:            0          0          0
#------------------------------------------------
# Let's restore the guest back from the file 'foof15vm3'
$ virsh restore foof15vm3
Domain restored from foof15vm3
#------------------------------------------------
# List the status. f15vm3 is up and running.
$ virsh list
 Id Name                 State
----------------------------------
  5 f15guest             running
  7 f15vm3               running
#------------------------------------------------

For brevity, let’s try out internal disk snapshots where all the snapshot info. (like disk and VM state info) are stored in a single qcow2 file.
Virsh(libvirt shell interface to manage guests) has some neat options for snapshot supports. So, I’ve got an F15 guest (Qcow2 disk image).

Internal Disk Snapshots when the guest is online/running

For illustration purpose, let’s use a Fedora-15 guest called ‘f15guest’ .

$ virsh list
 Id Name                 State
----------------------------------
  4 f15guest             running

$ 

For clarity, ensure there are no prior snapshot instances around.

$ virsh snapshot-list f15guest
 Name                 Creation Time             State
------------------------------------------------------------

$ 

Before creating a snapshot, we need to create a snapshot xml file with 2 simple elements (name and description) if you need sensible name for the snapshot. Note that only these two fields are user settable. Rest of the info. will be filled by Libvirt.

$  cat /var/tmp/snap1-f15guest.xml
<domainsnapshot>
    <name>snap1-f15pki </name>
    <description>F15 system with dogtag pki packages </description>
</domainsnapshot>

$ 

Eric Blake noted that, the domainsnapshot xml file is optional now for ‘snapshot-create’ if you don’t need a description for the snapshot. And if it’s okay with libvirt generating the snapshot name for us. (More on this, refer below)

Now, I’m taking a snapshot while the ‘guest’ is running live. Here, Eric noted that, especially when running/live, the more RAM the guest has, and the more active the guest is modifying that RAM, the longer the it will take to create a snapshot. This was a guest was mostly an idle guest.

$ virsh snapshot-create f15guest /var/tmp/snap1-f15guest.xml
Domain snapshot snap1-f15pki  created from '/var/tmp/snap1-f15guest.xml'
$ 

While the snapshot-creation is in progress on the live guest, the state of the guest will be ‘paused’.

$ virsh list
 Id Name                 State
----------------------------------
  4 f15guest             paused

$ 

Once, the snapshot is created, list the snapshots of f15guest

$ virsh snapshot-list f15guest
 Name                 Creation Time             State
------------------------------------------------------------
 snap1-f15pki         2011-10-04 19:04:00 +0530 running

$

Internal snapshot while the guest is offline

For fun, I created another snapshot, but after shutting down the guest. Now, the snapshot creation is just instantaneous.

$ virsh list
 Id Name                 State
----------------------------------

$ virsh snapshot-create f15guest
Domain snapshot 1317757628 created

List the snapshots of ‘f15guest’ using virsh.

$ virsh snapshot-list f15guest
 Name                 Creation Time             State
------------------------------------------------------------
 1317757628           2011-10-05 01:17:08 +0530 shutoff
 snap1-f15pki         2011-10-04 19:04:00 +0530 running

To see some information about the VM size, snapshot info:

$ qemu-img info /export/vmimgs/f15guest.qcow2
image: /export/vmimgs/f15guest.qcow2
file format: qcow2
virtual size: 8.0G (8589934592 bytes)
disk size: 3.2G
cluster_size: 65536
Snapshot list:
ID        TAG                 VM SIZE                DATE       VM CLOCK
1         snap1-f15pki           1.7G 2011-10-04 19:04:00   32:06:34.974
2         1317757628                0 2011-10-05 01:17:08   00:00:00.000
$ 

To revert to a particular snapshot, virsh snapshot-revert domain snapshotname

Also, discussed with Eric, in what cases does virsh invoke Qemu’s ‘savevm‘ and ‘qemu-img snapshot -c‘ commands while creating different types of snapshots discussed earlier above. Here is the outline:

  • it uses ‘qemu-img snapshot -c‘ if the domain is offline and –disk-only was not specified
  • it uses qemu’s ‘savevm‘ if the domain is online and –disk-only was not specified
  • it uses qemu’s ‘snapshot_blkdev‘ if the domain is online and –disk-only is specified

(Note: –disk-only is an option to capture only ‘disk state’ but not VM state. This option is available w/ virsh ‘snapshot-create’ or ‘snapshot-create-as’ commands.)

Thanks Eric for the detail.

Advertisements

18 Comments

Filed under Uncategorized

18 responses to “Snapshotting with libvirt for qcow2 images

  1. NLD

    Have you tried to do external snapshot while the guest is either online or offline? If you, please le me know.

  2. Olivier DELEMAR

    OK, you can create snapshots. But did you tried to create/modify files after a snapshot, revert to a given snapshot, go back and forth, etc.? Making snapshot hasn’t more interest than making backup: only restoration is of interest.

  3. SR

    Hi ,

    thanks for the great info, especially for newbies like me who rely on internet for gaining knowledge.

    Note :
    for UBuntu 12.04 with libvirt version 0.9.8 – the creation of snap.xml is required/imperative. failing to do so will give out an I/O error, and virsh snapshot-list will not return details about any snapshots.

    However when you start “qemu-system-x86_64 -hda path/to/hd.qcow2 -m 1024? and then from within the monitor (clicking within the qemu and then pressing CTRL+ALT+2 ) you will notice that a snapshot has been created successfully.

    Command for viewing : info snapshots

    Rgds
    SR

  4. Fábio Nogueira

    Hi Kashyap! Congrats by excellent post…
    I have one scenario: One virtual machine (.img file) and I want working with this machine (One lab tests)… How I can comeback in this virtual machine using snapshot? Example: I create this machine, users, password, partitions, etc… and make one snapshoot (I don’t know wath is the best method) and after many tests, modifications, I roolback to snapshot clean … Do you understand me? ;)
    One “abraço” from Brazil…

    • If I understand you correctly: You have an original image which has some content and you want to take a snapshot of it. Once snapshot is taken, you start to make a some modifications and end you don’t up liking them and want to rollback to the snapshot. For this, as I noted in the post, there are several different ways. If you’ve never used snapshots, to begin with, you can try ‘internal snapshots’ (where your original and its delta is stored in a single QCOW2 file). NOTE: This only works for QCOW2 images.

      To create an ‘internal’ snapshot (note: this can be live or offline; but offline is more robust)

      $ virsh snapshot-create-as f20vm1 snap1 "Clean F20"

      If you want to revert to the above snapshot:

      $ virsh snapshot-revert f20vm1 snap1
  5. rudolph shih

    Hi Kashyap,

    You mentioned that:
    — snip —
    External snapshot: Here, the ‘original qcow2 file’ will be in a ‘read-only’ saved state, and the new qcow2 file(which will be generated once snapshot is created) will be the delta for the changes. So, all the changes will now be written to this delta file. ‘External Snapshots’ are useful for performing backups.
    — snip —

    If I have a backing chain (all in qcow2 format, and “lazy_refcounts”=on) like below:
    base.qcow2 <- sn1.qcow2
    And then I create an external snapshot called sn2.qcow2:
    base.qcow2 <- sn1.qcow2 <- sn2.qcow2
    Does it mean that all cached data of sn1.qcow2 will be flushed once the external snapshot sn2.qcow2 is created? And sn1.qcow2 will be read-only so that I can copy it as a backup right away?

    Thanks

    • Yes, when you create ‘sn2’, no more writes will happen on sn1, so you can immediately take a backup of it. Alternatively, you can even take a live backup of your disk image using ‘blockcopy’. An old example here — https://kashyapc.fedorapeople.org/virt/lc-2012/live-backup-with-blockcopy.txt

      • rudolph shih

        I used “blockcopy” to perform incremental backup (pseudo code list below), and create a external snapshot once the backup is synced. However, this caused the running VM complain I/O error and remount disk as read-only.
        —- pseudo code —-
        ….
        blockcopy $DOM $BLKDEV /path/to/backup –shallow –wait
        suspend $DOM
        blockjob $DOM $BLKDEV –abort
        snapshot-create-as $DOM … -diskspec diskspec –disk-only –atomic
        resume $DOM
        ….
        —- pseudo code —-

        So I was thinking if a backing file can be read-only (all cached data be flushed when cache mode = write back???) once an external snapshot created, then I just backup the backing file instead of using blockcopy.
        Although it’s not a live backup.

  6. Payes Anand

    Hi,
    You mentioned about saving the VM State to a file using virsh save. But the VM shuts down after that. Is there a command which could save the VM state to a file without shuting it down.

    Regards,
    Payes

    • ‘virsh save’ saves the state including RAM and the guest will no longer be running – that’s by *design*. The below live external snapshot (disk and memory) command will work:

      $ virsh snapshot-create-as --domain vm1 snap1 \ 
        --diskspec vda,file=/export/vmimages/disk-snap.qcow2,snapshot=external \ 
        --memspec file=/export/vmimages/mem-snap.qcow2,snapshot=external \ 
        --atomic
      
  7. rudolph shih

    Hi Kashyapc,

    I got a problem to take a internal live snapshot when running guest has 2 qcow2 images.

    I have a guest “win7″ with 2 qcow2 images. And I take a live internal snapshot when guest is running:

    [/share/SATA/VMs/test] # TM=$(date +”%s”)
    [/share/SATA/VMs/test] # virsh snapshot-create-as win7 $TM “$TM,$TM” –atomic

    And the 2 qcow2 image became:

    [/share/SATA/VMs/test] # qemu-img info w7.img
    …. ….
    Snapshot list:
    ID TAG VM SIZE DATE VM CLOCK
    1 1422011556 551M 2015-01-23 11:12:36 00:01:33.268
    …. ….
    [/share/SATA/VMs/test] # qemu-img info disk2.img
    …. ….
    Snapshot list:
    ID TAG VM SIZE DATE VM CLOCK
    1 1422011556 0 2015-01-23 11:12:36 00:01:33.268

    However, when I delete the snapshot 1422011556, and then try to take another internal snapshot. the libvirt report an error :

    [/share/SATA/VMs/test] # virsh snapshot-delete win71422011556
    Domain snapshot 1422011556 deleted
    [/share/SATA/VMs/test] # TM=$(date +”%s”)
    [/share/SATA/VMs/test] # virsh snapshot-create-as win7 $TM “$TM,$TM” –atomic
    create an internal snapshot 1422012000 on test
    error: operation failed: Failed to take snapshot: Error while creating snapshot on ‘drive-ide0-0-1’

    And the 2 qcow2 image became:

    [/share/SATA/VMs/test] # qemu-img info w7.img
    …. ….
    Snapshot list:
    ID TAG VM SIZE DATE VM CLOCK
    1 1422012000 925M 2015-01-23 11:20:00 00:08:51.134
    …. ….
    [/share/SATA/VMs/test] # qemu-img info disk2.img
    …. ….
    Snapshot list:
    ID TAG VM SIZE DATE VM CLOCK
    1 1422011556 0 2015-01-23 11:12:36 00:01:33.268

    It looks like that the previous snapshot 1422011556 of disk2.img is not removed by libvirt, so that the new snapshot 1422012000 cannot be created on disk2.img.

    Is this normal? or how can I delete all snapshot data when the snapshot span on 2 qcow2 images by libvirt?

    Thanks

    • If you have enough backups, you can probably try to remove ‘1422011556’ snapshot using qemu-img tool like this:

      $ qemu-img snapshot -d 1422011556 disk2.img
      

      (NOTE: In regular circumstances, I wouldn’t recommend directly using `qemu-img` tool behind libvirt’s back, because libvirt still will have the old metadat – though this can be cleaned up.)

  8. gunnar

    it would really be helpful if anybody took the time for ‘external’ or ‘internal’ when using the term snapshot

  9. gunnar

    ok, after complaining an actual question: I am looking for a good procedure to backup a qemu/kvm VM. You recommend external snapshots for that purpose. I assume you will just stack up on top of a base base < sn1 < sn2 < sn3 <…
    Actually the base for each new layer of external snapshots is its predecessor, and not the base (qemu-img create -f qcow -b sn1.qcow2 sn2.qcow2) right? Isn't it a big risk that just one of those will fail and make your whole stack unusable? Let's say you want to take 2 backups a day … well I guess you see what I am getting at here.
    Also I was wondering … can you commit for example sn2 to the base and still use run sn3? I probably should just try that out

  10. gunnar

    if you use external snapshots to make backups of those I assume you end up with one (committed) image as your production system (because you commit snapshots once in a while assumingly) and a very long sn chain on top of the original base image as your backups. The last snapshot of that backed up snapshot chain will (if running) be identical to the current production system.

    So, if anything breaks in your production system I guess you’d have to recover from let’s say sn100 (1 backing file + 99 snapshots on top [or even longer chains]). Would such still work? I mean do they run?

    I guess the whole (maybe not the only) point of external disk snapshots for backups is to keep the fractions small enough to make taking backups manageable

    P.S. not yet 100% clear whether `qemu-img` or virsh `snapshot-create` was the preferable tool of choice

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s