Intel’s new transistor technology — 3D Tri-gate (Ivy Bridge)

It’ll be interesting to see what level of impact does this innovative transistor technology called 3D Tri-gate will have on Virtualization(KVM based). Apparently, the upcoming processors will be nearly 40% more faster than the current generation processors and more interestingly, consumes 50% less power.

Technical details are here, from Intel’s Silicon Architecture section. They also note some fun facts to visualize how small is 22nm.

Also, a hardware based DRNG(Digital Random Number Generation) technology which can produce quality random numbers at a very high rate is in store(which will assist in making Cryptographic operations more stronger). Technical details are here

Pretty staggering.

Advertisement

Share an interactive ssh session b/n two users with ‘tmux’

I recently started using tmux, a terminal multiplexer much similar to GNU screen(which I still use on a lot on RHEL machines). Personally, I find tmux more elegant and less cryptic while configuring. Specially, sharing interactive(or read-only) ssh sessions w/ remote folks for debugging is very trivial.

Get it:

  $ yum install tmux -y   

Create a tmux session(let’s say on testbox1) by specifying path to a socket(its name can be anything)to be used:

 $ tmux -S /var/tmp/share-ses 
# Where -S is the absolute path to the server socket.

Now, to share the above created session w/ a remote user, ask the person to ssh into testbox1, and then try the below for sharing a session interactively or read-only mode:

• Read/Write session, so that both users can interactively edit on the shell:

 $ tmux -S /var/tmp/share-ses attach 

• For ‘Read-Only’ session(for remote user) :

$ tmux -S /var/tmp/share-ses attach -r 
# Where r is 'read-only' mode for the remote user,
# so he/she can just 'watch' the session.

To detach from the session: $ tmux detach (or use the binding — Ctl+b d when in ‘read-only’ mode)

Unattended guest install with a local kickstart

Up until recently, I was doing all my unattended installs to provision linux guests with a kickstart hosted over http. Now, with this neat flag ‘–initrd-inject‘ in virt-install , guests can be created using a local kickstart file. This is available in python-virtinst-0.500.4-1 and above)

Here is the invocation, it assumes network bridging is configured. Refer this to quickly configure bridging (if you haven’t already).

This creates a minimal guest w/ these specs: 20G disk image(RAW format); 2GB ram ; 2 Virtual CPUs.

#!/bin/bash
set -x

# Note: Replace this with your local Fedora tree if you have one.
tree= http://download.fedoraproject.org/pub/fedora/linux/releases/15/Everything/x86_64/os

virt-install --connect=qemu:///system \
    --network=bridge:br0 \
    --initrd-inject=/export/fed-minimal.ks \
    --extra-args="ks=file:/fed-minimal.ks \
      console=tty0 console=ttyS0,115200" \
    --name=f15testbox \
    --disk /var/lib/libvirt/images/f15testbox.img,size=20 \
    --ram 2048 \
    --vcpus=2 \
    --check-cpu \
    --accelerate \
    --hvm \
    --location=$tree \
    --nographics 

Also a serial console(ttyS0) is configured so that all the booting/package-install process(complete install should take around around 4-5 mins) could be seen from the shell.

Later on, for remote management using libvirt, you can connect to guests using:

  $ virsh console f15testbox 

And the contents of the kickstart file ‘fed-minimal.ks’. This is the smallest possible install:

# Minimal Kickstart file
install
text
reboot
lang en_US.UTF-8
keyboard us
network --bootproto dhcp
#Choose a saner password here.
rootpw testpasswd
firewall --enabled --ssh
selinux --enforcing
timezone --utc America/New_York
#firstboot --disable
bootloader --location=mbr --append="console=tty0 console=ttyS0,115200 rd_NO_PLYMOUTH"
zerombr
clearpart --all --initlabel
autopart

#Just core packages
%packages
@core
%end

I placed all the above in a quick script

A note on bridging, network device names:
With the recent Fedora 15 feature Consistent Network Device Naming , network device names from here on would be renamed from ethX to something like emX . Refer the Fedora test day for more information and a script to determine if your system is impacted due to the Biosdevname change. (I had to do this for my DELL optiplex Xeon test box.)

Experiment with ‘Native Linux KVM Tool’

Over the weekend I was tinkering around with the recently announced version 2 of ‘Native Linux KVM tool’. My aim was to boot a minimal Linux (RAW) disk image and a QCOW2 based image. Read on for details:

A little bit of context
‘Native Linux KVM tool’ was first announced by Pekka Enberg on lkml/kvm-upstream lists. From the announce email:

“Right now it can boot a Linux image and provide you output via a serial
console, over the host terminal, i.e. you can use it to boot a guest
Linux image in a terminal or over ssh and log into the guest without
much guest or host side setup work needed.”

Essentially, the initial goal of the tool appears to be a featherlight-weight userspace alternative to QEMU, which can boot Linux guests. And this Native tool lives inside the kernel tree under /tools(this means, if/when this tool is merged in the mainline kernel tree, a linux distro will by default get minimal userspace tool to boot linux guests). However, QEMU does plenty more than booting a linux guest.(Quick googling will provide all the info.)

Getting to the matter
On Saturday, I started off by pulling Penberg’s kernel git tree[1]. And proceeded to configure it with some build time options required for ‘Native Linux KVM tool’ by following the Version 2 announcement. And I enabled each of the necessary options in the kernel configuration. Before I proceed, Sasha Levin(Native KVM tool dev.) suggested that I compile the guest kernel without any modules. Reason being, if I built it with modules, all of them needs to be explicitly loaded into the disk image before booting. If we have all of them built in, we just use the ‘bzImage’. So, I did a quick `sed -i ‘s/=m/=y/’ .config` to *include* everything and compiled the kernel with ‘make -j5’ (so that it builds with 5 parallel threads).

Build complete, bzImage generated. Without any delay I went ahead and launched the hypervisor to boot the minimal disk image:

  [root@moon kvm]#./kvm run -d /export/test-images/linux-0.2.img 
 # kvm run -k ../../arch/x86/boot/bzImage -m 448 -c 4 --name guest-9408
.
.
.
[   63.903734] Copyright (c) 2009 - 2010 Intel Corporation.
[   63.903753] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version
3.3.8-k2
[   63.903755] ixgbe: Copyright (c) 1999-2011 Intel Corporation.
[   63.903774] ixgbevf: Intel(R) 10 Gigabit PCI Exp[root@moon kvm]# 

No dice! As it can be noticed, it just exits abruptly w/o providing any useful error(And later I was pointed out that it didn’t go through the regular exit path either). So I provided my kernel .config and bzImage for debugging.

Meanwhile, Sasha provided me his kernel .config file to get a working ‘kvm’ tool. Rinse and repeat – re-compilation and booting the disk image.

Woot! this time it boots into the minimal linux disk image.

 [root@moon kvm]# ./kvm run -d /export/test-images/linux-0.2.img
 # kvm run -k ../../arch/x86/boot/bzImage -m 448 -c 4 --name guest-9408
.
.
.
[    1.386747] Bluetooth: Virtual HCI driver ver 1.3
[    1.387838] Bluetooth: HCI UART driver ver 2.2
[    1.388615] Bluetooth: HCI H4 protocol initialized
[    1.389481] Bluetooth: HCI BCSP protocol initialized
[    1.390366] Bluetooth: HCILL protocol initialized
sh-2.05b# mkdir /export 
sh-2.05b# pwd 
/export
sh-2.05b# echo test > foo.txt
sh-2.05b# cat foo.txt 
test
sh-2.05b# ls /
bin   dev  export  lost+found  proc  sbin  usr
boot  etc  lib	   mnt	       root  tmp   var
sh-2.05b# df -hT
Filesystem    Type    Size  Used Avail Use% Mounted on
rootfs      rootfs     20M   17M  1.7M  92% /
/dev/root     ext4     20M   17M  1.7M  92% /
devtmpfs  devtmpfs    150M     0  150M   0% /dev
sh-2.05b# 

Next morning, on Sunday, I tried to do a couple of regular virtual guest operations like pause, resume, list, stop. I first chose to ‘pause’ the guest. Poof! I’m thrown out of my ssh session to the *host* , where I’m doing all of this. I can’ re-ssh any more, the SSH session was killed. All I see is a connection refused. Confused, I checked with Penberg, he replied, it’s a nasty bug(which sends the pause signal to the wrong process), but fixed and pointed me to the git commit. Ok, that confirmed I didn’t screw up anything. I re-pulled and re-compiled kernel tree on my Lenovo X200 laptop . (Later in the day, I went and fixed the host test machine(this reminded me to tie this machine to a remote power management console).

With the up2date git tree (and post recompilation), I was able to do all the regular virtual guest operations fine.

Results
Test env.
OS: Fedora -15
Processor: 4 Intel(R) Xeon(R) CPU E5405 @ 2.00GHz
RAM: 8GB

I tested w/ both minimal RAW and a debian squeeze (de-compressed) QCOW2 disk images.

With minimal RAW image: I was able to run; pause; resume; list; stop the minimal RAW linux disk image. Also my (limited)test results, guest networking is a little fragile(understandably) at the moment.
Posted my results here.

With QCOW2 image: Currently a QCOW2 image boots in ‘read-only’ mode. To try it, below is the working syntax.

# ./kvm run -p "root=/dev/vda1" -d /export/test-images/debian_squeeze_i386_standard_decompressed.qcow2

For more verbose details, errors I encountered, conversion of compressed to de-compressed images, I posted my notes and results here.

NOTE: I’m yet to boot any of the Fedora/RHEL RAW disk images which I use in my daily work. Also, I haven’t tried out the ‘virtio ballon’ feature. Will write more on these when I get to it.

For a quick shot
If you don’t want to compile your own kernel and do all the stuff, to give a quick shot, all that needs to be done is:

Download the working Kernel x86 boot executable ‘bzImage’(I built with Sasha’s config); compiled ‘kvm’ tool binary ; and a minimal linux disk image(linux-0.2.img)
– ‘bzImage’ and ‘kvm’ tool binary are located on my fedora people page
– minimal linux disk image — # wget http://wiki.qemu.org/download/linux-0.2.img.bz2 && bunzip2 linux-0.2.img.bz2

And, run the hypervisor to boot into the minimal disk image:

 
# ./kvm run -k ~/tinker/native-linux-kvm/linux-kvm/arch/x86/boot/bzImage  -d /export/testimages-nlt/linux-0.2.img
 

I jotted down a README here.
If you’re interested to compile your own kernel from scratch w/ latest git, I uploaded a working kernel .config (from Sasha) on my fedora people page.

Thanks a lot to Sasha Levin and Pekka Enberg. They were very helpful answering all my questions and also walked me through some of the issues I was facing.

References

1. git://github.com/penberg/linux-kvm.git
2. http://kashyapc.fedorapeople.org/native-linux-kvm-tool/working-native-linux-kvm-tool/sashal-linux-config

Update:
With this commit 6533f7913743742fdd690eee0930fb7ba1bcbb1f, Pekka introduced an ‘init’ target . So if there are any errors( like /usr/bin/ld: cannot find -lc) while compiling the kvm binary on Fedora, ensure to have glibc-static package installed to get them resolved.

Using OZ tool to create virtual guests (with minimal input)

OZ(part of the Aeolus project) is a neat tool by Chris Lalancette, to create JEOS(just enough operating system)  guests with minimal input. Where minimal input include things like: name of the guest ; OS details like – version, arch, url for the OS tree(or ISO), root-passwd. All of these attributes can be defined in a simple XML type file called TDL(template definition language). Read on for details.

Under the hood, OZ uses all the existing Virtualization infrastructure components like kvm/qemu, libvirt, libguestfs.

Ok,  OZ workflow goes like this:

1/ Firstly, ensure to have all the Fedora Virtualization packages.(if not already present)

$ yum groupinstall Virtualization 

2/ Enable packet forwarding on the host machine. (ensure this).

$ echo 1 > /proc/sys/net/ipv4/ip_forward 

3/ Prepare a simple TDL(template definition language)
(Don’t forget to replace the ‘url’ with a working Fedora install tree. Or ISO could also be used as install type. Refer to the ‘References’ section at the bottom)

 $ cat fedora_15_x86_64.tdl
<template>
  <name>zeus-f15</name>
  <os>
    <name>Fedora</name>
    <version>15</version>
    <arch>x86_64</arch>
    <install type='url'>
      <url>http://foo.bar.com/fedora/linux/releases/15/Fedora/x86_64/os/</url>
    </install>
    <rootpw>testpwd</rootpw>
  </os>
  <description>Fedora 15</description>
</template>

4/ Now, create the guest using the oz-install tool.(takes roughly 8-9 mins.)

$ /usr/bin/oz-install -d 4 fedora_15_x86_64.tdl 2>&1 | tee f15.out 

End of successful ‘oz-install’ output should look like this:

.
.
.

INFO:oz.Guest.FedoraGuest:Cleaning up after install
DEBUG:oz.Guest.FedoraGuest:Removing modified ISO
Libvirt XML was written to zeus-f15Aug_08_2011-16:16:08

Note: zeus-f15Aug_08_2011-16:16:08 is the xml definition file for the guest just created.
A successful run of oz-install looks like this on the stdout — http://kashyapc.fedorapeople.org/virt/oz/stdout-oz-install.txt

5/ Define the guest with the above XML.

$ virsh define zeus-f15Aug_08_2011-16:16:08
Domain zeus-f15 defined from zeus-f15Aug_08_2011-16:16:08

$ virsh start zeus-f5

$ virsh list --all
 Id Name                 State
----------------------------------
  1 zeus-f15             running

I’ve composed most of this into a quick shell script here – http://kashyapc.fedorapeople.org/virt/oz/oz-jeos.bash

References:

1. oz project page — http://aeolusproject.org/oz.html.
2. More customizations can be done. All the supported attributes for TDL file can be found in the OZ git repo — https://github.com/clalancette/oz/blob/master/docs/tdl.rng

Dogtag Certificate System 9.0 release…

Late post  by a couple of weeks, but still…  Dogtag Certificate System  team announced the release of 9.0 release (for Fedora -15)

Release Notes can be found here.

To try out on your Fedora-15 system:

1/ Install the pre-requisites (389-ds ldap instance) :

 # yum install 389-ds-base -y

2/ Configure the 389-ds instance by running( accept all the default/typical options.) :

 # /usr/sbin/setup-ds.pl

3/ Install the dogtag certificate system CA(certificate Authority) package.

# yum install pki-ca pki-silent -y

(NOTE:  To get all Dogtag CS packages, use — # yum install dogtag-pki -y )

4/ Create a Certificate Authority(CA) instance using ‘pkicreate’ tool. I placed the default CA instance create script in a simple file noted below:

# wget http://kashyapc.fedorapeople.org/dogtag-pki/instance-create-scripts/ca.bash
 # chmod +x ca.bash
 #./ca.bash
 (CA instance should be successfully created.)

5/ Now, let’s configure the previously created CA instance. There are 2 ways to configure an instance:  One – via the browser ; the other, using ‘pkisilent’ tool to silently configure the CA.
For illustration purposes, let’s go the  ‘pkisilent’ route. I composed a sanitized  pkisilent configuration script for CA here — http://kashyapc.fedorapeople.org/dogtag-pki/pkisilent-scripts/ca-silent.bash.
Also, before you try the ‘ca-silent.bash’  – replace the passwords in the script accordingly.

# wget http://kashyapc.fedorapeople.org/dogtag-pki/pkisilent-scripts/ca-silent.bash 
# chmod +x ca.bash 
# ./ca-silent.bash 
# service pki-cad restart
# service pki-cad status
 pki-ca (pid 1418) is running...                            [  OK  ]
 Unsecure Port       = http://lovelock1.foo.bar.com:9180/ca/ee/ca
 Secure Agent Port   = https://lovelock1.foo.bar.com:9443/ca/agent/ca
 Secure EE Port      = https://lovelock1.foo.bar.com:9444/ca/ee/ca
 Secure Admin Port   = https://lovelock1.foo.bar.com:9445/ca/services
 EE Client Auth Port = https://lovelock1.foo.bar.com:9446/ca/eeca/ca
 PKI Console Port    = pkiconsole https://lovelock1.foo.bar.com:9445/ca
 Tomcat Port         = 9701 (for shutdown)
PKI Instance Name:   pki-ca
PKI Subsystem Type:  Root CA (Security Domain)
Registered PKI Security Domain Information:
 ==========================================================================
 Name:  silentdom
 URL:   https://lovelock1.foo.bar.com:9445
 ==========================================================================

6/ Now, Let’s try to: [a] Export the CA agent certificate into a p12 file from the temporary NSS db. [b] Install the CA Agent’s certificate into NSS database of the browser(from where CA Agent pages can be invoked):

#####################################
 # certutil -L -d .
Certificate Nickname                                         Trust Attributes
 SSL,S/MIME,JAR/XPI
testnick                                                     P,,
 Certificate Authorityca-t1 - silentdom                       c,c,
 ca-agent                                                     u,u,u
 #
 ####################################
 # /usr/bin/pk12util -o ca-agent.p12 -n "ca-agent" -d /var/tmp/testdb/
 Enter Password or Pin for "NSS Certificate DB":
 Enter password for PKCS12 file:
 Re-enter password:
 pk12util: PKCS12 EXPORT SUCCESSFUL
 #####################################
 # /usr/bin/pk12util -i ca-agent.p12 -d /home/kashyap/.mozilla/firefox/irphredr.default/ 
 #####################################