Welcome to the new location of Alien's Wiki, sharing a single dokuwiki install with the SlackDocs Wiki.

Welcome to Eric Hameleers (Alien BOB)'s Wiki pages.

If you want to support my work, please consider a small donation:

This is an old revision of the document!


Hardware virtualization with QEMU

If you want to experience the kick of looking at a Windows boot screen in a desktop window while you're working in Slackware, read this article on the advancements in open source solutions for hardware virtualization - aka running a computer inside a computer.

Virtualization in Linux

One of the often cited reasons people have for not completely switching to Linux, is the fact that for some Windows applications, there is no real Linux alternative. Solutions for running DOS/Windows applications in Linux have been available for quite some time though: there is the Wine Project that allows you to run individual Windows applications by translating the Windows library calls to Linux library calls. In DOSemu and DOSBox you can run old DOS based programs. DOSemu and DOSBox provide a more or less complete virtual computer environment to the DOS Operating System. For the Windows OS there are a few virtualization solutions, too.
The most prominent is the commercial VMWare which is also the fastest performer. Open Source alternatives like Bochs and Plex86 (formerly called FreeMWare) have tried, but failed to result in a workable environment.

QEMU

The new kid on the block is QEMU which is an actively developed project with a fairly large base of enthousiast followers and contributors. QEMU is able to provide a virtualized computer hardware environment on which you can install and run Windows, Linux, and lots of other Operating Systems. QEMU calls this System Emulation. QEMU can also operate in User Emulation mode, where it is able to run single Linux processes - thereby translating calls for a specific architecture into another architecture. Most notably, User Emulation is used on non-intel platforms to run a version of the Wine Emulator, so that individual Windows applications can actually be used on foreign platforms.

Because QEMU source code is licensed under the GPL, a third party has used it to develop an enhanced version they sell under the name Win4Lin Pro. Modifications to the GPL-ed source code are contributed back to qemu, so both parties profit.

QEMU virtualizes a complete hardware environment, including CPU (i386, x86_64, ppc, sparc, mips, arm platforms) and peripheral devices like VGA card, network interfaces, IDE, sound and USB components, and more. This enables you to to run an unmodified operating system in QEMU. Since QEMU is a user process, crashes of the Guest Operating System do not affect the host at all.
Supported operating systems include for instance Linux and BSD distributions, and the more recent releases of Microsoft Windows such as Windows 2000 and XP (and 98 and NT4 too, if you do without the acceleration support from kqemu). On the QEMU website, you can download one or more of several disk images of fully operational virtual computers. More disk images (always virtual computers installed using freely distributable Operating Systems such as FreeDOS, Linux and BSD) are available from the Free OS Zoo website.

The strengths of QEMU are that it's evolving quite fast, it is uncomplicated to setup and use, and there is no requirement to patch your kernel in order to get it running. The fact that this is free software does not relieve you of the requirement to own valid licenses of any software - like MS Windows - you want to install in QEMU.

Unlike Xen, QEMU allows you to run closed-source Operating Systems like MS Windows as a guest in a virtual machine without the requirement to modify the code. Xen offers near-native speed for the OS-es that run as guests in a Xen virtual machine, but requires you to modify the code of the guest OS in order to get these speeds. This leaves you with Open Source OS-es within Xen for the moment.

Emulated hardware and supported Guest OS-es

The QEMU Virtual Machine emulates a set of hardware components that is independent of the real hardware on which it is running. Since Slackware runs on x86 architecture, I will limit myself to a list of emulated hardware available to Slack (other emulated architectures may have other hardware peripherals available to the Guest OS).

  • IDE-Controller supporting up to 4 drives (the drives are disk images on the host computer
  • IDE CDROM device (in the form of a CD ISO image, or a real CDROM device)
  • Floppy disk controller supporting up to 2 drives (floppy disk images)
  • Graphics card (either a Cirrus Logic GD5446 PCI, or VGA-VESA)
  • PS/2 Mouse
  • Ethernet network card (Realtek RTL8139 PCI or NE2000 PCI)
  • A serial port (COM 1)
  • A parallel port (LPT 1)
  • Soundcard (Soundblaster 16 and/or ES1370)
  • A USB-UHCI host controller (the Intel SB82371)

The list of Operating Systems that run inside QEMU is quite long. Here is an unofficial list of supported Guest OS-es. I have run various Linuxes (for x68 and x86_64) and Windows 98/2000/XP inside QEMU.

Performance

Recent developments in the kqemu accelerator module bring performance of QEMU on par with the commercial VMware. However, VMware offers quite a few extra features compared to QEMU. Now that you can get certain VMware software versions for free, VMware has again become a compelling alternative (although Open Source lovers will still prefer QEMU naturally!).

You should realize that running an operating system directly on the hardware is always faster than running the same operating system inside a virtual machine, because of the processor overhead that the virtualization of all hardware dictates. Even so, the guest operating system inside the QEMU virtual machine feels suprisingly responsive if you have a decent CPU and lots of RAM.

Obtaining QEMU software

Qemu is not part of Slackware, but it is easily built from source, and ready-made packages are available from my SlackBuild repository. If you want to build yourself you can use my SlackBuild script found at the above URL, or go to the QEMU website and compile the source code yourself.
The qemu virtualizer has two parts: the qemu program(s) which is under the GPL license, and a separate kernel module called kqemu,which is able to accelerate a lot of the virtualized hardware to a level where the virtual machine (the guest) is running at nearly the perceived speed of the host machine. The kqemu software has a proprietary license, but nevertheless, you're free to use it for non-commercial as well as commercial purposes. You may distribute kqemu only with the author's explicit permission.

Running QEMU and kqemu

Running QEMU is as easy as just… running qemu <many parameters>. QEMU all by itself results in quite sluggish emulation which makes the experience working with the Guest OS not an exhilarating one. The accelerator module kqemu improves QEMU's emulation speed a lot. If you installed my kqemu package, several files in /etc/ will be altered, so that kqemu's acceleration will be available to you after each boot of the computer. If you do not want to reboot, but want to use QEMU with kqemu right away, then you should run these commands once (as root) before you start QEMU:

mount /dev/shm
modprobe kqemu

You should prepare your Slackware system for the kqemu module so that it can operate as efficiently as possibble. Your computer will automatically have been setup in case you installed my Slackware packages from the URL I provided in the previous section. If you compiled everything yourself, please follow these instructions:

  • The acceleration that kqemu offers works best if kqemu can map the Virtual Machine's memory pages to your computer's physical RAM. If kqemu finds a mounted RAM filesystem at /dev/shm it will create a hidden file there with the dimensions of the VM's memory size. If it does not find a RAM filesystem, the VM's memory will be mapped to a disk file in your /tmp directory, and of course this will not nearly be as fast as working entirely in RAM.

So, add this line to /etc/fstab

none        /dev/shm        tmpfs   defaults        0 0

By default, adding this line in /etc/fstab will allow a maximum of 50% of your physical RAM to be used up by the RAM filesystem. If you want your QEMU to emulate a computer with an X MB amount of memory, then this amount of memory “X” must be available to qemu in /dev/shm, or else qemu will refuse to start. This means for instance, that if your computer has 1GB of RAM, and the above line is present in /etc/fstab, qemu will be able to emulate a computer with a maximum of 500MB RAM (actually, a little less than 50% is effectively available to the VM, so try to give QEMU a couple of percents less than 50% of your physical RAM).
If you need more RAM for QEMU, you must change the defaults like this:

none        /dev/shm        tmpfs   size=750M         0 0

which makes 750MB of your RAM available for use by (k)qemu.

Reserving part of your RAM to be used by /dev/shm and a tmpfs, does not mean that you permanently loose this amount of RAM for other applications! Only the memory in /dev/shm that is actually used by an application will become unavailable to other applications. Having a line like above in your /etc/fstab does in no way affect your machine and it's performance if no program uses it. So would be OK to always have such a line.
  • The installation of kqemu creates a device node /dev/kqemu that will by default be writeable by anyone. If you don't like that, you can limit access to the device node by letting it be owned by a Linux group (for instance, create a group called qemu) restrict access to the device node to just that group and add yourself to that group.

If you're doing all of this yourself, you should add these lines to /etc/rc.d/rc.local to ensure that the device /dev/kqemu is available and accessible:

## For 2.6 kernels, as long as udev does not re-create the ##
## /dev/kqemu device node automatically, you can keep      ##
## the lines below:                                        ##
##                                                         ##
# Create the KQEMU device
if ! [ -e /dev/kqemu ]; then
  mknod /dev/kqemu c 250 0
  chmod 666 /dev/kqemu
fi

My kqemu package also installs a udev rule file as /etc/udev/rules.d/50-kqemu-rule in case you run QEMU on a 2.6 kernel. If you decided to restrict access to /dev/kqemu, then you should modify this file.

  • To make the kqemu module load on boot, you should add the following line to the file /etc/rc.d/rc.modules:
    /sbin/modprobe kqemu

This concludes the alterations needed for a performance boost of your Virtual Machines inside QEMU. As I said earlier, running qemu is really simple - it is a single binary with a lot of optional commandline parameters that customize the virtual machine it will setup for use. QEMU will use the kqemu accelerator if it finds the kernel module loaded in memory (and if it's built with support for kqemu). The newer versions of QEMU and kqemu (at the time of writing, it's only available in the CVS repository of QEMU, but it will be released as QEMU 0.8.1) provide an additional layer of acceleration called kernel-kqemu. Here, the Guest kernel processes will also be accelerated as opposed to the “regular” function of kqemu to only accelerate the Guest's user processes. You do need to supply an explicit parameter to the qemu commandline:

qemu -kernel-kqemu <other parameters>

If you don't want kqemu functionality at all, for instance because some programs and OS-es will not work reliably or not at all, you can tell qemu on the commandline as well:

qemu -no-kqemu <other parameters>
QEMU uses SDL for it's output (video as well as sound), and you need to run QEMU in an X Window session. There is an option to the qemu binary called -nographic that you can use to circumvent the X session requirement, but you still need the X and SDL libraries installed on your system. Typically, you would use -nographic for a virtual machine you've already installed previously, and that allows remote access through SSH, VNC or XDMP, and where you no longer need the virtual machine's local console.

Installing an Operating System in QEMU

Remember that QEMU builds a virtual computer around the Operating System you're going to use inside of it. So, the hard drives are virtualized as well. Before starting QEMU, we will create a big file that QEMU can then use as the virtual hard disk:

dd if=/dev/zero of=winxp.img bs=1k count=0 seek=4000000

This will create a so-called sparse file with a size of about 4GB. A sparse file means that it can ultimately grow to 4GB of occupied disk blocks, but the file will only allocate enogh blocks to store the actual data. So, if you run du on that directory, it will report 4GB used, but actually, the file will currently use no space at all! Now, a real file using up 4GB of disk space before you even wrote to it, can be created using this command (it's up to you to decide what you want to use, either command is OK):

dd if=/dev/zero of=winxp.img bs=1k count=4000000

I think that 4GB is enough to hold Windows XP and several large applications, but if you think you need more, just change the 4000000 into a bigger value.

Now, to install Windows, you need a Windows installation CD (and a valid license key of course). You have two options; if you have a “real” CDROM medium, you can let QEMU use the physical drive. If all you have is an ISO image of the installation CD, then QEMU can present this ISO image as a CDROM drive to the virtual machine. We will use an ISO file in this excercise, and assume you keep it stored as ./ISOS/winxp_pro_us.iso.

This is the big moment… we're going to fire up QEMU and tell it to use the 4GB empty file we have just created as the virtual computer's first hard drive, and use an ISO image to represent as a CD-ROM drive in the virtual machime. QEMU should also boot from that CD. Run

qemu -localtime -m 256 -hda winxp.img -cdrom ./ISOS/winxp_pro_us.iso -boot d

Alternatively, as an example, the commandline would have looked like this if we had chosen to use a real CD in the CDROM drive /dev/cdrom: qemu -localtime -m 256 -hda winxp.img -cdrom /dev/cdrom -boot d.
The '-boot d' parameter instructs QEMU to let the virtual machine boot from the (virtual) CD-ROM drive we've specified using -cdrom. The '-hda winxp.img' of course means that the first IDE harddisk is to be represented by our image file (QEMU can use a hdb, hdc and hdd as well, in case you need more hard disks inside QEMU).

When you run the above command, you'll see a new window appear, and you'll actually see the Windows XP installer boot (assuming you didn't mess up during one of the previous steps)! To use the mouse in QEMU, you need to click in the window to give it focus. QEMU grabs the mouse, and to give it back to your other applications you press Ctrl+Alt (it even says so in QEMU's window bar). The installation of XP is a process that I'm not going to repeat, I am assuming you'll know how to proceed. The installation of Windows 2000/XP in QEMU still has a couple of annoyances. These are: excruciatingly slow progress from time to time, especially during the stage where Windows tries to detect the hardware in your system. QEMU even knows a separate commandline parameter -win2k-hack that you should add exclusively for the duration of a Windows 2000 installation if you experience disk full errors during the installation!
Even though the slow pace of the Windows installation can make you wonder if you did the right thing installing QEMU, these annoyances will be gone once Windows installation has finished and you reboot into the OS for the first time.

When you're done with installing from CD, do not forget to at least remove the -boot d commandline parameter, so that the virtual computer will start from it's hard drive.

Networking your virtual machine

By default, QEMU uses a feature which is called user-mode network. QEMU will run an internal DHCP server that can assign an IP address to the virtual computer in case that is configured to use DHCP. The network range and gateway are hard-coded into QEMU but it allows your virtual machine to call out to your host machine and beyond, if you setup your host. I will probably document user-mode networking sometime later, but for now I suggest you read my article on VDE where I show how you can use VDE (virtual distributed ethernet) in combination with dnsmasq to get a much enhanced network experience.

Advanced topics

Using copy-on-write files

QEMU uses a disk image file containing your installed Operating System. We created that image file in a previous section as a raw image, meaning the file has no special format. QEMU knows of several disk image formats, and one of them I want to introduce to you: the QCOW or ““qemy copy-on-write” format. The QCOW file can be used in combination with a base image file. Suppose you're happy with the installed OS in your winxp.img disk image file. You want to try out a new application, or you want to do some potentially disastrous tests in your virtual machine. You can then create a QCOW file that is based on the winxp.img file. Create the QCOW (qemu copy-on-write) file like this:

qemu-img create -b winxp.img -f qcow winxp.qcow

When you tell QEMU to use the QCOW file instead of the RAW image file (something like qemu -hda winxp.qcow), QEMU will do all file writes to the QCOW image, thus treating the original RAW file as a read-only file. The effect is, that if you stop QEMU after you did some work, the original image file is unaltered and all changes are recorded into the QCOW files. If you delete the QCOW subsequently and re-start QEMU now using the original winxp.img file, there is no trace left of the session you ran using the QCOW file.
So, this is ideal for testing stuff out - in fact, this is how I test and build my new Slackware packages. When you're happy with the result, you can commit the changes to the base image file and get rid (if you want) of the now emptied QCOW file:

qemu-img commit winxp.qcow

Read the qemu-img man page if you want to know more about the supported image file formats (some support compression and encryption!).

The qemu monitor

Besides the graphical screen, QEMU has a monitor screen. You can toggle the QEMU output from graphics to monitor mode (after giving the window focus by clicking in it with the mouse) by pressing Ctrl+Alt+2 and back to graphics mode by pressing Ctrl+Alt+1. The QEMU monitor is used for several purposes:

  • Remove or insert removable medias images (such as CD-ROM or floppies)
  • Freeze/unfreeze the Virtual Machine (VM) and save or restore its state from a disk file.
  • Inspect the virtual machine state without an external debugger.
  • Send key presses to OS running inside the virtual machine.
Changing a CD/floppy in the virtual machine

The most interesting feature of the monitor from an end user perspective is the ability to change removable media images. Two scenarios:

  1. Suppose you want to install Slackware in QEMU. You start QEMU with something like
    qemu -localtime -m 256 -hda slackware.img -cdrom ./ISOS/slackware-10.2-install-d1.iso -boot d

    and after Slackware finishes with CD1 it will ask for CD2. How does one “change the CD” in QEMU? This is simple; switch to the monitor by pressing Ctrl+Alt+2 and run these commands (the help monitor command is always at hand if you should forget what to do)

    eject -f cdrom
    change cdrom ./ISOS/slackware-10.2-install-d2.iso

    i.e. supply the full path to the ISO image on the Host filesystem.

  2. You are running QEMU and decide you need a CDROM to install a piece of software on the currently running OS. Unfortunately, you did not start qemu with the parameter -cdrom /dev/cdrom or currently have another CDROM medium in the drive. Neither do you want to stop your QEMU session to change the CD in the drive. In this case too, you switch to the QEMU monitor. If there already was a CDROM in the drive, you first run
    eject -f cdrom

    Next, you change/insert the physical medium for the CD you want to use in the running QEMU. Finally, make the new CDROM known to the guest OS in QEMU:

    change cdrom /dev/cdrom

    This is enough to make the new drive visible in the guest OS.

Switch back out of the monitor to the Guest using Ctrl+Alt+1.

Send key presses

In some cases, the host OS, or the host hardware, won't let you use certain keyboard combinations. Two examples:

  1. You own an Apple computer. The PowerBook for instance does not have the Del key - and Windows really needs this key for to compose Ctrl+Alt+Del combination in order to login to the computer. The solution is to switch to the QEMU monitor using Ctrl+Alt+2 keypresses and then run the literal command
    sendkey ctrl-alt-delete
  1. You're running X on the host as wellas on the guest and in the guest, want to switch to a console using Ctrl+Alt+F6. The host will intercept this key combination even if the QEMU window has focus. The solution here is to switch to the QEMU monitor using Ctrl+Alt+2 keypresses and then run the literal command
    sendkey ctrl-alt-f6

Mounting filesystems inside the virtual disk image

On the host, you can mount filesystems created in partitions that are present in QEMU's virtual disk image - and access the files contained in there - on certain conditions:

  1. QEMU most not be running when you mount the virtual disk partitions!
  2. The disk image must have a RAW format. Other formats, such as QCOW for instance, can not be accessed from outside QEMU.

A QEMU disk image file will contain one or more partitions. We need to find out at what location in the file these partitions start, so that we can tell the mount -o loop command at which file offset to look. The fdisk -ul command will show us exactly that type of information. The '-u' parameter will output sector count. For example:

$ /sbin/fdisk -lu slack102.img
       You must set cylinders.
       You can do this from the extra functions menu.

       Disk slack102.img: 0 MB, 0 bytes
       255 heads, 63 sectors/track, 0 cylinders, total 0 sectors
       Units = sectors of 1 * 512 = 512 bytes

              Device Boot      Start         End      Blocks   Id  System
       slack102.img1   *          63      208844      104391   83  Linux
       slack102.img2          208845      401624       96390   82  Linux swap
       slack102.img3          401625     4192964     1895670   83  Linux

You can safely ignore the warnings about the “set cylinders”, they do not matter here.
The output shows a disk image consisting (apart from a swap partition) of two Linux partitions, which probably contain the / and /home filesystems. The first partition starts at sector 63 while the other data partition (partition #3) starts at sector 401625. These are the offsets we have to multiply with 512 (the sector byte size) to get the offset number we will feed to the mount command. Proceed with mounting the two partitions (I create the mount points too for completeness sake):

mkdir -p /mnt/qemu/part1 /mnt/qemu/part2
mount -o loop,offset=$(( 63 * 512 )) -t auto slack102.img /mnt/qemu/part1
mount -o loop,offset=$(( 401625 * 512 )) -t auto slack102.img /mnt/qemu/part2

I assume here, that setting the filesystem type to “auto” will work for you. If not, and you know what type the filesystems are (for instance ext3), then you can change the ”-t auto“ to ”-t ext3“ in these two mount commands.
Once mounted, you can copy files from and to the disk image.

If you do want to be able to access your data stored in a QEMU partition, and still use QCOW for the advantages that file format offers, I suggest you create two disk image files, and use QCOW on the disk image where you will install your Guest Operating System. Create the second disk image as a RAW file, and use this to for a filesystem where you will keep your data. That way, at least the data remains accessible in the Host.

Accessing the host parallel port

If you need to print to a printer that is attached to the host computer's parallel port, you will have to pass an additional parameter on the qemu command line, like this:

qemu -parallel /dev/parport0 <other qemu parameters>

You might not have this device node /dev/parport0 available on your host, but QEMU requires the parallel port device name to start with the string parport. If your parallel port is called lp0 you can easily create a symbolic link called “parport0” to that device by running

cd /dev
ln -s lp0 parport0

Using an USB device in QEMU

FIXME

The VNC patch and other user contributions

FIXME

A QEMU start script

For ease of use, we will create a script that will start QEMU with a lot of default options. The script will write the output into a .log and .err file and then change into a background process, so that you can just start the script in a terminal and get your prompt back, or add it to your Window Manager menu. Call the script for instance start_winxp.sh and make it executable.

#!/bin/sh
#
# Start Windows XP Pro in QEMU
 
PARAMS=$*
 
# Qemu can use SDL sound instead of the default OSS
export QEMU_AUDIO_DRV=sdl
 
# Whereas SDL can play through alsa:
export SDL_AUDIODRIVER=alsa
 
# Change this to the directory where _you_ keep your QEMU images:
IMAGEDIR=/home/alien/QEMU
 
# Change this to the directory where _you_ keep your installation CDROM's ISO images:
ISODIR=/home/alien/ISOS
 
# Now, change directory to your image directory
cd $IMAGEDIR
 
# If you want the WinXP CD available, use a '-cdrom' parameter:
qemu -net user,vlan=0 -net nic,vlan=0 -m 256 -localtime -soundhw all -hda winxp.img -cdrom ${ISODIR}/winxp_pro_us.iso  1>winxp.log 2>winxp.err ${PARAMS} &

A script that uses VDE for a better networking support, would look like this:

#!/bin/sh
#
# Start Windows XP Pro in QEMU using VDE for better network support
 
PARAMS=$*
 
# Qemu can use SDL sound instead of the default OSS
export QEMU_AUDIO_DRV=sdl
 
# Whereas SDL can play through alsa:
export SDL_AUDIODRIVER=alsa
 
# Change this to the directory where _you_ keep your QEMU images:
IMAGEDIR=/home/alien/QEMU
 
# Change this to the directory where _you_ keep your installation CDROM's ISO images:
ISODIR=/home/alien/ISOS
 
# Now, change directory to your image directory
cd $IMAGEDIR
 
# If you want to boot from the WinXP CD add a '-boot d' parameter to the commandline;
#   if you don't need the CDROM present in the VM, leave '-cdrom ${ISODIR}/winxp_pro_us.iso' out:
# I made the MAC address up - make sure it is unique on your (virtual) network.
 
# This command returns to the command prompt immediately,
#   and QEMU's error output is redirected to files.
vdeqemu -net vde,vlan=0 -net nic,vlan=0 -m 256 -localtime -soundhw all -hda winxp.img -cdrom ${ISODIR}/winxp_pro_us.iso  1>winxp.log 2>winxp.err ${PARAMS} &

Further info and pointers

 Hardware virtualization with QEMU ()
SlackDocs