Libvirt 0.9.6 was recently out. Take a look at 0.9.5 changelog for truckload of features/bugfixes/cleanups(specifically snapshot related) from the libvirt team.
So, I grabbed the F14 srpm from Libvirt ftp, and made a quick Fedora koji scratch build of libvirt-0.9.6 for Fedora 15 and gave the snapshot features a whirl. Here it goes:
(Also noted below is some very useful discussion I had(on #virt, OFTC) with Eric Blake (Upstream/Red Hat’s Libvirt dev, very friendly guy.) on snapshots. It was way informative not to capture it.)
Context on types of snapshots
At the moment, snapshotting in KVM/QEMU/Libvirt land is supported primarily for QCOW2 disk images. I briefly discussed about Qcow2 previously here.
There are several different types of snapshots possible. Some idea on that:
Internal snapshot: A type of snapshot, where a single QCOW2 file will hold both the ‘saved state’ and the ‘delta’ since that saved point. ‘Internal snapshots’ are very handy because it’s only a single file where all the snapshot info. is captured, and easy to copy/move around the machines.
External snapshot: Here, the ‘original qcow2 file’ will be in a ‘read-only’ saved state, and the new qcow2 file(which will be generated once snapshot is created) will be the delta for the changes. So, all the changes will now be written to this delta file. ‘External Snapshots’ are useful for performing backups. Also, external snapshot creates a qcow2 file with the original file as its backing image, and the backing file can be /read/ in parallel with the running qemu.
VM State: This will save the guest/domain state to a file. So, if you take a snapshot including VM state, we can then shut off that guest and use the freed up memory for other purposes on the host or for other guests. Internally this calls qemu monitor’s ‘savevm’ command. Note that this only takes care of VM state(and not disk snapshot). To try this out:
#------------------------------------------------ # Memory before saving the guest f15vm3 $ free -m total used free shared buffers cached Mem: 10024 5722 4301 0 164 4445 -/+ buffers/cache: 1112 8911 Swap: 0 0 0 #------------------------------------------------ $ virsh list Id Name State ---------------------------------- 5 f15guest running 6 f15vm3 running #------------------------------------------------ # Save the guest f15vm3 to a file 'foof15vm3' $ virsh save f15vm3 foof15vm3 Domain f15vm3 saved to foof15vm3 #------------------------------------------------ # Now, f15vm3 is gracefully saved/shutdown. $ virsh list Id Name State ---------------------------------- 5 f15guest running #------------------------------------------------ # Notice the RAM being freed $ free -m total used free shared buffers cached Mem: 10024 5418 4605 0 164 4493 -/+ buffers/cache: 760 9263 Swap: 0 0 0 #------------------------------------------------ # Let's restore the guest back from the file 'foof15vm3' $ virsh restore foof15vm3 Domain restored from foof15vm3 #------------------------------------------------ # List the status. f15vm3 is up and running. $ virsh list Id Name State ---------------------------------- 5 f15guest running 7 f15vm3 running #------------------------------------------------
For brevity, let’s try out internal disk snapshots where all the snapshot info. (like disk and VM state info) are stored in a single qcow2 file.
Virsh(libvirt shell interface to manage guests) has some neat options for snapshot supports. So, I’ve got an F15 guest (Qcow2 disk image).
Internal Disk Snapshots when the guest is online/running
For illustration purpose, let’s use a Fedora-15 guest called ‘f15guest’ .
$ virsh list Id Name State ---------------------------------- 4 f15guest running $
For clarity, ensure there are no prior snapshot instances around.
$ virsh snapshot-list f15guest Name Creation Time State ------------------------------------------------------------ $
Before creating a snapshot, we need to create a snapshot xml file with 2 simple elements (name and description) if you need sensible name for the snapshot. Note that only these two fields are user settable. Rest of the info. will be filled by Libvirt.
$ cat /var/tmp/snap1-f15guest.xml <domainsnapshot> <name>snap1-f15pki </name> <description>F15 system with dogtag pki packages </description> </domainsnapshot> $
Eric Blake noted that, the domainsnapshot xml file is optional now for ‘snapshot-create’ if you don’t need a description for the snapshot. And if it’s okay with libvirt generating the snapshot name for us. (More on this, refer below)
Now, I’m taking a snapshot while the ‘guest’ is running live. Here, Eric noted that, especially when running/live, the more RAM the guest has, and the more active the guest is modifying that RAM, the longer the it will take to create a snapshot. This was a guest was mostly an idle guest.
$ virsh snapshot-create f15guest /var/tmp/snap1-f15guest.xml Domain snapshot snap1-f15pki created from '/var/tmp/snap1-f15guest.xml' $
While the snapshot-creation is in progress on the live guest, the state of the guest will be ‘paused’.
$ virsh list Id Name State ---------------------------------- 4 f15guest paused $
Once, the snapshot is created, list the snapshots of f15guest
$ virsh snapshot-list f15guest Name Creation Time State ------------------------------------------------------------ snap1-f15pki 2011-10-04 19:04:00 +0530 running $
Internal snapshot while the guest is offline
For fun, I created another snapshot, but after shutting down the guest. Now, the snapshot creation is just instantaneous.
$ virsh list Id Name State ---------------------------------- $ virsh snapshot-create f15guest Domain snapshot 1317757628 created
List the snapshots of ‘f15guest’ using virsh.
$ virsh snapshot-list f15guest Name Creation Time State ------------------------------------------------------------ 1317757628 2011-10-05 01:17:08 +0530 shutoff snap1-f15pki 2011-10-04 19:04:00 +0530 running
To see some information about the VM size, snapshot info:
$ qemu-img info /export/vmimgs/f15guest.qcow2 image: /export/vmimgs/f15guest.qcow2 file format: qcow2 virtual size: 8.0G (8589934592 bytes) disk size: 3.2G cluster_size: 65536 Snapshot list: ID TAG VM SIZE DATE VM CLOCK 1 snap1-f15pki 1.7G 2011-10-04 19:04:00 32:06:34.974 2 1317757628 0 2011-10-05 01:17:08 00:00:00.000 $
To revert to a particular snapshot, virsh snapshot-revert domain snapshotname
Also, discussed with Eric, in what cases does virsh invoke Qemu’s ‘savevm‘ and ‘qemu-img snapshot -c‘ commands while creating different types of snapshots discussed earlier above. Here is the outline:
- it uses ‘qemu-img snapshot -c‘ if the domain is offline and –disk-only was not specified
- it uses qemu’s ‘savevm‘ if the domain is online and –disk-only was not specified
- it uses qemu’s ‘snapshot_blkdev‘ if the domain is online and –disk-only is specified
(Note: –disk-only is an option to capture only ‘disk state’ but not VM state. This option is available w/ virsh ‘snapshot-create’ or ‘snapshot-create-as’ commands.)
Thanks Eric for the detail.
Have you tried to do external snapshot while the guest is either online or offline? If you, please le me know.
Not yet. Sorry, I couldn’t get time to do the external snapshotting blog post. I’ll post that soon.
OK, you can create snapshots. But did you tried to create/modify files after a snapshot, revert to a given snapshot, go back and forth, etc.? Making snapshot hasn’t more interest than making backup: only restoration is of interest.
Making snapshots is also of interest(particularly, external,disk-only) because — you can take a ‘backup’ of original image while the machine is running live.
Reverting was briefly dealt in — https://kashyapc.wordpress.com/2011/10/04/snapshotting-with-libvirt-for-qcow2-images/
But for external, disk-only snapshots, where several qcow2 files are to be handled(because, each time you take a disk-only external snapshot, a new snapshot is created), ‘snapshot revert’ is not quite ready. That said, there’s a way to ‘merge’ existing snapshot files using ‘blockcommit’ or ‘blockpull’ (for which patches are in latest qemu/libvirt git)
If you’re curious, you can see some of my notes on blockcommit tests here, to see how they work — http://kashyapc.fedorapeople.org/virt/blockcommit-tests/
Hi ,
thanks for the great info, especially for newbies like me who rely on internet for gaining knowledge.
Note :
for UBuntu 12.04 with libvirt version 0.9.8 – the creation of snap.xml is required/imperative. failing to do so will give out an I/O error, and virsh snapshot-list will not return details about any snapshots.
However when you start “qemu-system-x86_64 -hda path/to/hd.qcow2 -m 1024? and then from within the monitor (clicking within the qemu and then pressing CTRL+ALT+2 ) you will notice that a snapshot has been created successfully.
Command for viewing : info snapshots
Rgds
SR
Hi Kashyap! Congrats by excellent post…
I have one scenario: One virtual machine (.img file) and I want working with this machine (One lab tests)… How I can comeback in this virtual machine using snapshot? Example: I create this machine, users, password, partitions, etc… and make one snapshoot (I don’t know wath is the best method) and after many tests, modifications, I roolback to snapshot clean … Do you understand me? ;)
One “abraço” from Brazil…
If I understand you correctly: You have an original image which has some content and you want to take a snapshot of it. Once snapshot is taken, you start to make a some modifications and end you don’t up liking them and want to rollback to the snapshot. For this, as I noted in the post, there are several different ways. If you’ve never used snapshots, to begin with, you can try ‘internal snapshots’ (where your original and its delta is stored in a single QCOW2 file). NOTE: This only works for QCOW2 images.
To create an ‘internal’ snapshot (note: this can be live or offline; but offline is more robust)
If you want to revert to the above snapshot:
Hi Kashyap,
You mentioned that:
— snip —
External snapshot: Here, the ‘original qcow2 file’ will be in a ‘read-only’ saved state, and the new qcow2 file(which will be generated once snapshot is created) will be the delta for the changes. So, all the changes will now be written to this delta file. ‘External Snapshots’ are useful for performing backups.
— snip —
If I have a backing chain (all in qcow2 format, and “lazy_refcounts”=on) like below:
base.qcow2 <- sn1.qcow2
And then I create an external snapshot called sn2.qcow2:
base.qcow2 <- sn1.qcow2 <- sn2.qcow2
Does it mean that all cached data of sn1.qcow2 will be flushed once the external snapshot sn2.qcow2 is created? And sn1.qcow2 will be read-only so that I can copy it as a backup right away?
Thanks
Yes, when you create ‘sn2’, no more writes will happen on sn1, so you can immediately take a backup of it. Alternatively, you can even take a live backup of your disk image using ‘blockcopy’. An old example here — https://kashyapc.fedorapeople.org/virt/lc-2012/live-backup-with-blockcopy.txt
I used “blockcopy” to perform incremental backup (pseudo code list below), and create a external snapshot once the backup is synced. However, this caused the running VM complain I/O error and remount disk as read-only.
—- pseudo code —-
….
blockcopy $DOM $BLKDEV /path/to/backup –shallow –wait
suspend $DOM
blockjob $DOM $BLKDEV –abort
snapshot-create-as $DOM … -diskspec diskspec –disk-only –atomic
resume $DOM
….
—- pseudo code —-
So I was thinking if a backing file can be read-only (all cached data be flushed when cache mode = write back???) once an external snapshot created, then I just backup the backing file instead of using blockcopy.
Although it’s not a live backup.
Hi,
You mentioned about saving the VM State to a file using virsh save. But the VM shuts down after that. Is there a command which could save the VM state to a file without shuting it down.
Regards,
Payes
‘virsh save’ saves the state including RAM and the guest will no longer be running – that’s by *design*. The below live external snapshot (disk and memory) command will work:
Hi Kashyapc,
I got a problem to take a internal live snapshot when running guest has 2 qcow2 images.
I have a guest “win7″ with 2 qcow2 images. And I take a live internal snapshot when guest is running:
[/share/SATA/VMs/test] # TM=$(date +”%s”)
[/share/SATA/VMs/test] # virsh snapshot-create-as win7 $TM “$TM,$TM” –atomic
And the 2 qcow2 image became:
[/share/SATA/VMs/test] # qemu-img info w7.img
…. ….
Snapshot list:
ID TAG VM SIZE DATE VM CLOCK
1 1422011556 551M 2015-01-23 11:12:36 00:01:33.268
…. ….
[/share/SATA/VMs/test] # qemu-img info disk2.img
…. ….
Snapshot list:
ID TAG VM SIZE DATE VM CLOCK
1 1422011556 0 2015-01-23 11:12:36 00:01:33.268
However, when I delete the snapshot 1422011556, and then try to take another internal snapshot. the libvirt report an error :
[/share/SATA/VMs/test] # virsh snapshot-delete win71422011556
Domain snapshot 1422011556 deleted
[/share/SATA/VMs/test] # TM=$(date +”%s”)
[/share/SATA/VMs/test] # virsh snapshot-create-as win7 $TM “$TM,$TM” –atomic
create an internal snapshot 1422012000 on test
error: operation failed: Failed to take snapshot: Error while creating snapshot on ‘drive-ide0-0-1’
And the 2 qcow2 image became:
[/share/SATA/VMs/test] # qemu-img info w7.img
…. ….
Snapshot list:
ID TAG VM SIZE DATE VM CLOCK
1 1422012000 925M 2015-01-23 11:20:00 00:08:51.134
…. ….
[/share/SATA/VMs/test] # qemu-img info disk2.img
…. ….
Snapshot list:
ID TAG VM SIZE DATE VM CLOCK
1 1422011556 0 2015-01-23 11:12:36 00:01:33.268
It looks like that the previous snapshot 1422011556 of disk2.img is not removed by libvirt, so that the new snapshot 1422012000 cannot be created on disk2.img.
Is this normal? or how can I delete all snapshot data when the snapshot span on 2 qcow2 images by libvirt?
Thanks
If you have enough backups, you can probably try to remove ‘1422011556’ snapshot using
qemu-img
tool like this:(NOTE: In regular circumstances, I wouldn’t recommend directly using `qemu-img` tool behind libvirt’s back, because libvirt still will have the old metadat – though this can be cleaned up.)
it would really be helpful if anybody took the time for ‘external’ or ‘internal’ when using the term snapshot
ok, after complaining an actual question: I am looking for a good procedure to backup a qemu/kvm VM. You recommend external snapshots for that purpose. I assume you will just stack up on top of a base base < sn1 < sn2 < sn3 <…
Actually the base for each new layer of external snapshots is its predecessor, and not the base (qemu-img create -f qcow -b sn1.qcow2 sn2.qcow2) right? Isn't it a big risk that just one of those will fail and make your whole stack unusable? Let's say you want to take 2 backups a day … well I guess you see what I am getting at here.
Also I was wondering … can you commit for example sn2 to the base and still use run sn3? I probably should just try that out
I see what you’re getting at. By just creating a qcow2 overly, obviously you won’t have a “fully contained” backup. I recently wrote this document which outlines taking a live disk backup using libvirt’s
blockcommit
http://wiki.libvirt.org/page/Live-disk-backup-with-active-blockcommit. Maybe you can experiment with this and see if that works for you?Note: That needs at-least QEMU 2.1 and libvirt-1.2.9.
if you use external snapshots to make backups of those I assume you end up with one (committed) image as your production system (because you commit snapshots once in a while assumingly) and a very long sn chain on top of the original base image as your backups. The last snapshot of that backed up snapshot chain will (if running) be identical to the current production system.
So, if anything breaks in your production system I guess you’d have to recover from let’s say sn100 (1 backing file + 99 snapshots on top [or even longer chains]). Would such still work? I mean do they run?
I guess the whole (maybe not the only) point of external disk snapshots for backups is to keep the fractions small enough to make taking backups manageable
P.S. not yet 100% clear whether `qemu-img` or virsh `snapshot-create` was the preferable tool of choice