Resize a fedora 19 guest with libguestfs tools

The inimitable Rich Jones writes some incredibly useful software. I can’t count how many times they helped me debugging disk images, or gave great insights into different aspects of linux virtualization. One of those instances again — I had to resize my OpenStack Fedora guest as its root file system merely had 5.4 GB to start with. So, I wanted to add atleast 15 GB more. After a bit of trial & error, here’s how I got it working. I’m using Fedora-19 in this case, but any other distro which supports libguestfs should be just fine.

Firstly, let’s check disk space inside the guest:

$ df -hT
    Filesystem              Type      Size  Used Avail Use% Mounted on
    /dev/mapper/fedora-root ext4      5.4G  5.4G     0 100% /
    devtmpfs                devtmpfs  4.7G     0  4.7G   0% /dev
    tmpfs                   tmpfs     4.7G     0  4.7G   0% /dev/shm
    tmpfs                   tmpfs     4.7G  392K  4.7G   1% /run
    tmpfs                   tmpfs     4.7G     0  4.7G   0% /sys/fs/cgroup
    tmpfs                   tmpfs     4.7G  472K  4.7G   1% /tmp
    /dev/vda1               ext4      477M   87M  365M  20% /boot
    /dev/loop0              ext4      4.6G   10M  4.4G   1% /srv/node/device1
    /dev/loop1              ext4      4.6G   10M  4.4G   1% /srv/node/device2
    /dev/loop2              ext4      4.6G   10M  4.4G   1% /srv/node/device3
    /dev/loop3              ext4      4.6G   10M  4.4G   1% /srv/node/device4
    /dev/vdb                ext4       17G   44M   16G   1% /mnt/newdisk

Print the libvirt XML to get the source of the disk

$ virsh dumpxml f19-test | grep -i source
      <source file='/var/lib/libvirt/images/f19-test.qcow2'/>

Above, I’m using a qcow2 disk image, I converted it to raw:

$ qemu-img convert -f qcow2 -O raw \
  /var/lib/libvirt/images/f19-test.qcow2 \
  /var/lib/libvirt/images/f19-test.raw

List the filesystems, partitions, block devices inside the raw disk image:

$ virt-filesystems --long --all -h -a \ 
  /var/lib/libvirt/images/f19-test.raw
Name              Type        VFS   Label  MBR  Size  Parent
/dev/sda1         filesystem  ext4  -      -    500M  -
/dev/fedora/root  filesystem  ext4  -      -    5.6G  -
/dev/fedora/swap  filesystem  swap  -      -    3.9G  -
/dev/fedora/root  lv          -     -      -    5.6G  /dev/fedora
/dev/fedora/swap  lv          -     -      -    3.9G  /dev/fedora
/dev/fedora       vg          -     -      -    9.5G  /dev/sda2
/dev/sda2         pv          -     -      -    9.5G  -
/dev/sda1         partition   -     -      83   500M  /dev/sda
/dev/sda2         partition   -     -      8e   9.5G  /dev/sda
/dev/sda          device      -     -      -    10G   -

Now, extend the file size of the raw disk image, using truncate:

# Create a new file based on original
$ truncate -r f19-test.raw f19-test.raw.new
# Adjust the new file size to 15G
$ truncate -s +15G f19-test.raw.new

List the file system partition info to find out the block device name:

$ virt-filesystems --partitions \
  --long -h -a f19-test.raw
Name       Type       MBR  Size  Parent
/dev/sda1  partition  83   500M  /dev/sda
/dev/sda2  partition  8e   9.5G  /dev/sda

Now, resize the new disk image using virt-resize. Note that, the --lv-expand option expands the root file system (thx Rich!):

$ virt-resize --expand /dev/sda2 --lv-expand \
  /dev/fedora/root f19-test.raw f19-test.raw.new
    Examining f19-test.raw ...
    **********
    
    Summary of changes:
    
    /dev/sda1: This partition will be left alone.
    
    /dev/sda2: This partition will be resized from 9.5G to 24.5G.  The LVM 
        PV on /dev/sda2 will be expanded using the 'pvresize' method.
    
    /dev/fedora/root: This logical volume will be expanded to maximum size. 
         The filesystem ext4 on /dev/fedora/root will be expanded using the 
        'resize2fs' method.
    
    **********
    Setting up initial partition table on f19-test.raw.new ...
    Copying /dev/sda1 ...
     100% ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧ 00:00
    Copying /dev/sda2 ...
     100% ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧ 00:00
    Expanding /dev/sda2 using the 'pvresize' method ...
    Expanding /dev/fedora/root using the 'resize2fs' method ...
    
    Resize operation completed with no errors.  Before deleting the old 
    disk, carefully check that the resized disk boots and works correctly.

Now, the size of the both new guests

$ ls -lash f19-test.raw f19-test.raw.new
2.7G -rw-r--r--. 1 qemu qemu 10G Apr 10 11:12 f19-test.raw
11G -rw-r--r--. 1 root root 25G Apr 10 12:13 f19-test.raw.new

Replace the old one w/ new one (you might want to take backup of the old one here, just in case):

$ mv f19-test.raw.new f19-test.raw

Also, update the libvirt XML file of the guest to reflect the raw disk image:

# Update source file path
$ virsh edit f19-test 
# grep the xml file to ensure.
$ grep source /etc/libvirt/qemu/f19-test.xml 
	
	

List file systems inside the newly created guest:

$ virt-filesystems --partitions --long -h -a f19-test.raw
Name       Type       MBR  Size  Parent
/dev/sda1  partition  83   500M  /dev/sda
/dev/sda2  partition  8e   25G   /dev/sda

Start the guest & ensure if everything looks sane:

$ virsh start f19-test --console

Advertisement

A few common operations while testing OpenStack on systemd based distros

While testing OpenStack on Fedora, I had to do some common operations to restart, list services. I started using systemctl — which is used to control & introspect the Systemd services. Noting them here, for reference. These should ideally be scripted, typing such long commands would be cumbersome.

List all enabled systemd unit files for OpenStack services:

$ systemctl list-unit-files --type=service | \
  grep -i openstack | grep -i enabled \
  | awk '{print $1}'

Status of OpenStack services active, inactive:

$ for i in `systemctl list-unit-files --type=service | \
  grep -i openstack |  grep -i enabled | \
  awk '{print $1}'`; do systemctl status $i ; done

Status of all OpenStack services, active:

$ for i in `systemctl list-unit-files --type=service | \
  grep -i openstack | grep -i enabled | awk '{print $1}'`; \
  do systemctl status $i ; done | grep Active -B1

Status of OpenStack services, inactive:

$ for i in `systemctl list-unit-files --type=service | \
  grep -i openstack | grep -i enabled | awk '{print $1}'`; \
  do systemctl status $i ; done | egrep -i 'Active|dead' -B1

Restart existing services. In systemd’s parlance, all enabled OpenStack unit files:

$ for i in `systemctl list-unit-files --type=service | \
  grep -i openstack | grep -i enabled | awk '{print $1}'`; \
  do systemctl restart $i ; done

Update: Updated the title to reflect its applicable for systemd based distros

Finding serial console log of a nova instance

The other day I was debugging why a volume booted nova instance doesn’t get dhcp leases. For that I need to find networking information of the instance during boot. Fortunately, nova attempts to log the boot details to a serial console log. To find where it’s located..

First, let’s list the instances which we care about:

$ nova list | grep f17
| 220d6e57-f7ad-4256-a861-7855ffc6788f | f17-builder       | ACTIVE  | novanetwork=192.168.32.4 |
| f1169841-5e67-45d9-85c4-0d44bd3bec90 | f17_volume_backed | ACTIVE | novanetwork=192.168.32.6 |

Then, grep that instance UUID in /etc/libvirt/qemu/* to find what’s the libvirt id of a specific nova instance:

$ grep f1169841-5e67-45d9-85c4-0d44bd3bec90 \
  /etc/libvirt/qemu/* 
/etc/libvirt/qemu/instance-00000007.xml:  f1169841-5e67-45d9-85c4-0d44bd3bec90
/etc/libvirt/qemu/instance-00000007.xml:      f1169841-5e67-45d9-85c4-0d44bd3bec90

$ virsh list | grep instance-00000007
 10    instance-00000007              running

From above, the libvirt instance is called instance-00000007

To find path to the serial console log of the instance, grep its libvirt xml information for the relevant fragment:

$ virsh dumpxml  instance-00000007 | grep -i console
      <source path='/var/lib/nova/instances/instance-00000007/console.log'/>
    <console type='file'>
      <source path='/var/lib/nova/instances/instance-00000007/console.log'/>
    </console>

The console log of the cirros image runs a bunch of useful debug commands after booting an instance — which gave me an insight that it wasn’t getting dhcp leases. Listing the commands it ran from the console.log for reference:

$ /etc/rc.d/init.d/sshd start
$ ifconfig -a
$ route -n
$ cat /etc/resolv.conf
# pinging nameservers
$ uname -a
$ dmesg | tail
$ tail -n 25 /var/log/messages

Also, for reference, relevant xml snippet of the console information:

$ virsh dumpxml instance-00000003 | \
  egrep -i 'serial type' -A9
    <serial type='file'>
      <source path='/var/lib/nova/instances/instance-00000003/console.log'/>
      <target port='0'/>
      <alias name='serial0'/>
    </serial>
    <serial type='pty'>
      <source path='/dev/pts/1'/>
      <target port='1'/>
      <alias name='serial1'/>
    </serial>
    <console type='file'>
      <source path='/var/lib/nova/instances/instance-00000003/console.log'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>