System emulation using QEMU

[This was last edited 2016-10-07. Please send feedback about this page! See page footer for contact information.]

The qemu project is really cool, and their goal is to support lots of computers, not just PCs.

The PC emulation works well and doesn't suffer from major incompatibility problems from qemu release to qemu release. It is really simple to get started. Create 4GiB disk image:

dd if=/dev/zero of=disk.img bs=1048576 count=4096
Install:
qemu-system-i386 -hda disk.img -cdrom your-favourite-os-install.iso -boot d
Boot the installed system:
qemu-system-i386 -hda disk.img

To instead emulate a 64-bit PC, use qemu-system-x86_64. See also below. Note that the install suggestions below will result in faster systems than these basic examples.

The downside of qemu is lack of documentation, and in particular usage examples. Also, if you try some random GNU/Linux or BSD release for a non-PC with some random qemu release, it will very likely not work as easily as above. Here I try to show one working variant of each base OS. The install examples below suggest outdated guest systems in some cases. I don't suggest that you use outdated systems, but these are the last with which I have tried and succeeded. Hopefully these examples will get you started!

If you upgrade a system, do not forget to first bring the system down and make a copy of the old disk image. It might be hard to go back once you've made your disk image unbootable. (The habit of copying disk images is a good one when working with any emulation, and is in fact a key feature of such environments!)

The qemu project relies on code inspection, but they perform limited regression testing. My experience is that regressions are not uncommon with qemu. If you are going to use qemu for something else than fun, you need to keep several builds and choose the one that works reliably for a particular system.

My motive for this entire exercise is software testing; qemu allows me to test things for systems to which I have no other access. I just need ssh access, and that's exactly what these examples do.

FAQ

Q: Do these examples really work? Have they been tested?

A: I have tested exactly what I suggest below in each case. They do work.

Q: What about networking? What's this "tap" thing?

A: If your host is set up to allow network bridging (think of that a simulated Ethernet switch) and if you either configure an IP address manually during guest OS installation, or have a dhcp server, then network should just work. If you don't want to configure the host to allow tap to work, check qemu's documentation about "user" networking and perhaps the hostfwd feature; many people find that easier to get going.

Q: The amount of memory (as specified by -m) varies between these examples, why?

A: There isn't too much science behind that. I tend to give slightly more memory to a 64-bit guest than a 32-bit guest.

Q: The qemu version varies between these examples. Any good reasons for that?

A: Some past versions of qemu have had regressions affecting particular targets. Therefore, I recommend qemu versions that I have made sure works. By all means, try other versions, but my recommendation is to start with exactly my setup and then go from there.

Q: Disk and network are specified in varying ways. Why not be more consistent?

A: I believe the variation is mostly well-motivated by limitations in the emulated OS and/or qemu. It is possible that things could be made more consistent, but I have tried to minimize unmotivated variation.

Q: Qemu offers several disk formats. In these examples, "raw" format seem to be either understood or explicitly enforced with the format=raw specifier. Reasons?

A: I actually use either lvm partitions or sparse plain files for my guest disks (the latter created with dd's seek operator). These imply minimal host CPU overhead. If you want to use qcow2 or some other setup, that should work fine (but make sure to remove (or adapt) the format=raw specifier).

Q: I want to emulate machine M with CPU C, a combination which I believe qemu supports. How should I do that?

A: The goal of my page here is to allow for testing on various CPU architectures. I realise that not all sub-architectures are covered (e.g., armv4, sparc millennium, mips r6). Also, the machine type is largely ignored. I don't currently have plans to try to make it much more complete, as that would be hard to maintain in the long run. So, I'm afraid I can only wish you luck with machine M emulation.

Q: There is something called qemu user-level emulation, skirting this thing about OS installation. Why don't you give examples of that instead?

A: I intend to do that. In fact, I have a shell script which installs all Debian versions which I can get to work. The main problem here is that the binfmt GNU/Linux feature is poorly designed and that the qemu pattern files which come with most (perhaps all) GNU/Linux dists are pretty buggy.

Current matrix

x86-32 x86-64 mips32 mips64 sparc32 sparc64 ppc32 ppc64 arm32 arm64 s390x alpha
FreeBSD yes1 yes4 ? n/a n/a (yes)10 hangs yes12 ? ? n/a
NetBSD yes2 yes5 ? n/a yes9 yes10 no n/a ? n/a n/a
OpenBSD ? ? ? ? yes9 yes10 hangs n/a ? n/a n/a
GNU/Linux yes3 yes6 yes7 yes8 n/a yes10 yes11 yes13 yes14,15 yes16 (yes)17 yes18,19
GNU/Hurd yesTBD n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a

Install details

These are rudimentary qemu guest install instructions. They assume the reader knows how to install each OS on real hardware.

You probably want to enable networking both during the install and boot process. You can either use -net tap like below, or -net -user,hostfwd=tcp::PORT-:22 for some adequate PORT number. (What "tap" means and how to do that is beyond the scope of this page, the qemu side of it is described in the qemu man page, the rest is quite system specific. It can be evil to get it right, since advanced networking is not too well designed on all systems.)

I would recommend that you initially do exactly like in these examples, since the system is fragile and even seemingly innocent deviations might lead to failure. When you have something that works, you can always go from there.

If you start several qemu guests at the same time, you might get networking problems since they by default use the same MAC address. The solution is to pass -net nic,macaddr=XX:XX:XX:XX:XX:XX, for some suitable (e.g., random) values of the X'es, different for each running qemu. Each example below uses a different MAC address.

These examples' index numbers correspond to the table above.

  1. X86-32 FreeBSD
    Download: compressed iso image
    Unpack: xz -d FreeBSD-11.0-RELEASE-i386-disc1.iso.xz
    common_args="-drive file=disk.img,if=virtio,format=raw,index=0 \
      -net nic,macaddr=52:54:00:fa:ce:01,model=virtio -net tap"
    
    Install:
    qemu-system-i386 $common_args \
      -cdrom FreeBSD-11.0-RELEASE-i386-disc1.iso \
      -boot d
    
    Boot:
    qemu-system-i386 $common_args
    
     
  2. X86-32 NetBSD
    Download: iso image
    common_args="-drive file=disk.img,if=virtio,format=raw,index=0 \
      -net nic,macaddr=52:54:00:fa:ce:02,model=virtio -net tap"
    
    Install:
    qemu-system-i386 $common_args \
      -cdrom NetBSD-6.1.5-i386.iso \
      -boot d
    
    Boot:
    qemu-system-i386 $common_args
    
     
  3. X86-32 Debian
    Download: iso image
    common_args="-drive file=disk.img,if=virtio,format=raw,index=0 \
      -net nic,macaddr=52:54:00:fa:ce:03,model=virtio -net tap"
    
    Install:
    qemu-system-i386 $common_args \
      -cdrom debian-7.11.0-i386-CD-1.iso \
      -boot d
    
    Boot:
    qemu-system-i386 $common_args
    
     
  4. X86-64 FreeBSD
    Download: compressed iso image
    Unpack: xz -d FreeBSD-11.0-RELEASE-amd64-disc1.iso.xz
    common_args="-drive file=disk.img,if=virtio,format=raw,index=0 \
      -net nic,macaddr=52:54:00:fa:ce:04,model=virtio -net tap"
    
    Install:
    qemu-system-x86_64 $common_args \
      -cdrom FreeBSD-11.0-RELEASE-amd64-disc1.iso \
      -boot d
    
    Boot:
    qemu-system-x86_64 $common_args
    
     
  5. X86-64 NetBSD
    Download: iso image
    common_args="-drive file=disk.img,if=virtio,format=raw,index=0 \
      -net nic,macaddr=52:54:00:fa:ce:05,model=virtio -net tap"
    
    Install:
    qemu-system-x86_64 $common_args \
      -cdrom NetBSD-6.1.5-amd64.iso \
      -boot d
    
    Boot:
    qemu-system-x86_64 $common_args
    
     
  6. X86-64 Debian
    Download: iso image
    common_args="-drive file=disk.img,if=virtio,format=raw,index=0 \
      -net nic,macaddr=52:54:00:fa:ce:06,model=virtio -net tap"
    
    Install:
    qemu-system-x86_64 $common_args \
      -cdrom debian-7.11.0-amd64-CD-1.iso \
      -boot d
    
    Boot:
    qemu-system-x86_64 $common_args
    
     
  7. MIPS32 Debian
    Download: kernel and initrd
    common_args="-M malta \
      -drive file=disk.img,if=virtio,format=raw,index=0 \
      -net nic,macaddr=52:54:00:fa:ce:07,model=virtio -net tap \
      -nographic"
    
    Install:
    qemu-system-mips $common_args \
      -kernel vmlinux-3.16.0-4-4kc-malta \
      -initrd initrd.gz \
      -append "console=ttyS0"
    
    Follow the instructions in the section Copying out Linux kernel and initrd below.
    Boot (console-less):
    qemu-system-mips $common_args \
      -kernel vmlinux-3.16.0-4-4kc-malta \
      -initrd initrd.img-3.16.0-4-4kc-malta \
      -append "root=/dev/vda1 console=ttyS0"
    
     
  8. MIPS64 Debian
    Use: qemu 2.7.0 (other versions might work, older than 2.2.0 known to not work.)
    Download: kernel and initrd
    common_args="-M malta \
      -drive file=disk.img,if=virtio,format=raw,index=0 \
      -net nic,macaddr=52:54:00:fa:ce:08,model=virtio -net tap \
      -nographic"
    
    Install:
    qemu-system-mips64 $common_args \
      -kernel vmlinux-3.16.0-4-4kc-malta \
      -initrd initrd.gz \
      -append "console=ttyS0"
    
    This is now a 32-bit system (and indeed the install command should remind of the mips-32 command above). But the Debian installer should have put the 64-bit vmlinux-3.16.0-4-5kc-malta and initrd.img-3.16.0-4-5kc-malta in /boot. If not, install them using apt-get. However, this kernel and initrd will not get used; you need to copy them out into the host system and boot using them. See the section Copying out Linux kernel and initrd below.
    Boot (console-less):
    qemu-system-mips64 $common_args \
      -cpu 5KEc -m 256 \
      -kernel vmlinux-3.16.0-4-5kc-malta \
      -initrd initrd.img-3.16.0-4-5kc-malta \
      -append "root=/dev/vda1 console=ttyS0"
    
    Notes:
  9. SPARC32 NetBSD
    Use: qemu 2.7.0 (older recent versions should work)
    Download: iso image
    common_args="-m 256 \
      -drive file=disk.img,format=raw,index=0 \
      -net nic,macaddr=52:54:00:fa:ce:09 -net tap \
      -nographic"
    
    Install:
    qemu-system-sparc $common_args \
      -cdrom NetBSD-7.0.1-sparc.iso \
      -boot d
    
    Boot:
    qemu-system-sparc $common_args
    
    Notes:
  10. SPARC64 NetBSD
    Use: qemu 2.7.0.
    Download: iso image
    common_args="-m 512 \
      -drive file=disk.img,format=raw,index=0 \
      -net nic,macaddr=52:54:00:fa:ce:10 -net tap \
      -nographic"
    
    Install:
    qemu-system-sparc64 $common_args \
      -cdrom NetBSD-7.0.1-sparc64.iso \
      -boot d
    
    Boot:
    qemu-system-sparc64 $common_args
    
    Notes:
  11. PPC32 Debian
    Use: qemu 2.7.0
    Download: iso image
    common_args="-drive file=disk.img,if=virtio,format=raw,index=0 \
      -net nic,macaddr=52:54:00:fa:ce:11,model=virtio -net tap \
      -display vnc=:11"
    
    Install:
    qemu-system-ppc $common_args \
      -cdrom debian-8.6.0-powerpc-CD-1.iso \
      -boot d
    
    Perform a default install using vncviewer. Note that the boot might hang on input with a screen with minimal contrasts; just hit ENTER to continue to a readable install dialogue.
    Follow the instructions in the section Copying out Linux kernel and initrd below.
    Boot:
    qemu-system-ppc $common_args \
      -kernel vmlinux-3.2.0-4-powerpc \
      -initrd initrd.img-3.2.0-4-powerpc \
      -append "root=/dev/vda3"
    
    You may need to modify the root= argument depending on where your / partition was put. It should be the one given here if you used the default layout.
  12. PPC64 FreeBSD
    Use: qemu 2.7.0
    Download: compressed iso image
    Unpack: xz -d FreeBSD-11.0-RELEASE-powerpc-powerpc64-disc1.iso.xz
    common_args="-M pseries -cpu POWER8 -m 512 \
      -drive file=disk.img,if=scsi,format=raw,index=0 \
      -net nic,macaddr=52:54:00:fa:ce:12 -net tap \
      -nographic -vga none"
    
    Install:
    qemu-system-ppc64 $common_args \
      -cdrom FreeBSD-11.0-RELEASE-powerpc-powerpc64-disc1.iso \
      -boot d
    
    Boot:
    qemu-system-ppc64 $common_args \
      -serial null -monitor null
    

  13. PPC64 Debian
    Use: qemu 2.7.0
    Download: kernel and initrd.
    common_args="-M pseries -cpu POWER8 -m 512 \
      -drive file=disk.img,if=scsi,format=raw,index=0 \
      -net nic,macaddr=52:54:00:fa:ce:13 -net tap \
      -display vnc=:13
    
    Install:
    qemu-system-ppc64 $common_args \
      -kernel vmlinux \
      -initrd initrd.gz
    
    Perform a default install using vncviewer.
    Boot:
    qemu-system-ppc64 $common_args
    
    Notes:
  14. ARM32 Debian ABI:armhf
    Use: qemu 2.6.2 (2.7.0 causes hangs)
    Download: kernel and initrd.
    common_args="-M virt -cpu cortex-a15 -m 256 \
      -drive file=disk.img,if=none,format=raw,id=hd0 \
      -device virtio-blk-device,drive=hd0 \
      -netdev type=tap,id=net0 \
      -device virtio-net-device,netdev=net0,mac=52:54:00:fa:ce:14 \
      -nographic"
    
    Install:
    qemu-system-arm $common_args \
      -kernel vmlinuz \
      -initrd initrd.gz \
      -append "console=ttyAMA0 --"
    
    Follow the instructions in the section Copying out Linux kernel and initrd below.
    Boot:
    qemu-system-arm $common_args \
      -kernel vmlinuz \
      -initrd initrd.img \
      -append "root=/dev/vda2 rw console=ttyAMA0 --"
    

  15. ARM32 Debian ABI:armel
    Use: qemu 2.6.2 (2.7.0 causes hangs)
    Download: kernel and initrd
    common_args="-M versatilepb -m 256 \
      -drive file=disk.img,format=raw,index=0 \
      -net nic,macaddr=52:54:00:fa:ce:15 -net tap \
      -display vnc=:14"
    
    Install using vncviewer over the network:
    qemu-system-arm $common_args \
      -kernel vmlinuz-3.16.0-4-versatile \
      -initrd initrd.gz
    
    Follow the instructions in the section Copying out Linux kernel and initrd below.
    Boot:
    qemu-system-arm $common_args \
      -kernel vmlinuz \
      -initrd initrd.img \
      -append "root=/dev/sda1"
    

  16. ARM64 Debian
    Use: qemu 2.7.0
    Download: kernel and initrd.
    common_args="-M virt -cpu cortex-a57 -m 256 \
      -drive file=disk.img,if=none,format=raw,id=hd0 \
      -device virtio-blk-device,drive=hd0 \
      -netdev type=tap,id=net0 \
      -device virtio-net-device,netdev=net0,mac=52:54:00:fa:ce:16" \
      -nographic"
    
    Install:
    qemu-system-aarch64 $common_args \
      -kernel linux \
      -initrd initrd.gz \
      -append "console=ttyAMA0 --"
    
    Follow the instructions in the section Copying out Linux kernel and initrd below.
    Boot:
    qemu-system-aarch64 $common_args \
      -kernel vmlinuz-3.16.0-4-arm64 \
      -initrd initrd.img-3.16.0-4-arm64 \
      -append "root=/dev/vda1 rw console=ttyAMA0 --"
    
    Notes:
  17. S/390 Debian PRELIMINARY PRELIMINARY PRELIMINARY
    Use: qemu 2.7.0
    Download: kernel and initrd.
    common_args="-M s390-ccw-virtio -m 512 \
      -drive file=disk.img,if=none,format=raw,id=hd0 \
      -device virtio-blk-ccw,drive=hd0,id=virtio-disk0 \
      -netdev type=tap,id=net0 \
      -device virtio-net-ccw,netdev=net0,mac=52:54:00:fa:ce:17,devno=fe.0.0001 \
      -nographic"
    
    Install:
    qemu-system-s390x $common_args \
      -kernel kernel.debian \
      -initrd initrd.debian
    
    Boot:
    qemu-system-s390x $common_args
    
    This will likely fail after a timeout. The problem is that the root filesystem is not given correctly. A workaround is to from the (initramfs) prompter make /dev/disk/by-path/ccw-virtio0-part1 a symlink to ../../vda1. The exit and see things boot!
  18. Alpha Debian (obsolete release!)
    Use: qemu 2.6.2 (newer probably works too).
    Download: iso, kernel and initrd.
    Uncompress vmlinuz to vmlinux.
    common_args="-m 256 \
      -drive file=disk.img,media=disk,format=raw,index=0 \
      -net nic,macaddr=52:54:00:fa:ce:18 -net tap \
      -kernel vmlinux \
      -nographic"
    
    Install:
    qemu-system-alpha $common_args \
      -initrd initrd.gz \
      -drive file=debian-5010-alpha-netinst.iso,if=ide,media=cdrom \
      -append "console=ttyS0"
    
    Follow the instructions in the section Copying out Linux kernel and initrd below.
    Boot:
    qemu-system-s390x $common_args \
      -initrd initrd.img \
      -append "root=/dev/hda3 console=ttyS0"
    
  19. Alpha Gentoo
    Use: qemu 2.6.2 (newer probably works too).
    Download: install-alpha-minimal-[DATE1].iso from current-iso and stage3-alpha-[DATE2].tar.bz2 from current-stage3 for suitable values of [DATE1] and [DATE2]. Grab the latest if there is a choice.

    Qemu's Alpha firmware (SRM) is not ready and is probably not yet useful for anything. Therefore, we need to boot a kernel directly.

    I've tried hard to make the kernel use the installation ISO as root image (with and without Gentoo's initrd image gentoo.igz), but the cdrom isn't found and a kernel panic ensues. I therefore use the trick of copying the ISO's contents into a temporary disk image ("inst-disk.img").

    Note that these commands assume a GNU/Linux host (mkfs.ext3 command and loop mount syntax are non-portable). The end result should run under other OSes, though.

    I recommend that you cut-and-paste the commands here and follow the progress of each command. The "set -e" is there should you be tempted to use it as a script; do not issue it to an interactive shell or you will find yourself logged out before long!

    ISO=install-alpha-minimal-[DATE1].iso
    STAGE3=stage3-alpha-[DATE2].tar.bz2
    TARG=try1		# Subdir where to put results
    DISK_SIZE=16		# Disk size in GiB, should be >= 4
    
    set -e			# Don't paste this command to an interactive shell!
    
    mkdir $TARG
    cd $TARG
    
    mkdir iso iso_sub targ
    
    mount $ISO iso
    
    # Copy out a useful kernel from ISO
    zcat iso/boot/gentoo >kernel
    touch -r iso/boot/gentoo kernel
    
    # Prepare temporary install image "inst-disk.img".
    mount iso/image.squashfs iso_sub
    dd if=/dev/zero of=inst-disk.img bs=$((1024*1024)) seek=$((2*1024)) count=0
    mkfs.ext3 inst-disk.img
    mount inst-disk.img targ
    cp -a iso_sub/* targ
    
    # Edit inst-disk.img to not scramble the root password.
    sed --in-place="" 's/echo/#echo/' targ/etc/init.d/pwgen
    
    # Edit inst-disk.img password files to make initial root password empty.
    sed --in-place="" 's/^root:[^:]*:/root::/' targ/etc/passwd
    sed --in-place="" 's/^root:[^:]*:/root::/' targ/etc/shadow
    
    # Edit inst-disk.img config to allow root logins (but do not start ssh yet!).
    sed --in-place="" 's/.*PermitRootLogin.*/PermitRootLogin yes/' targ/etc/ssh/sshd_config
    
    umount targ
    umount iso_sub
    umount iso
    
    # Create main disk image.
    dd if=/dev/zero of=disk.img bs=$((1024*1024)) seek=$(($DISK_SIZE*1024)) count=0
    
    # Clean up
    rmdir iso iso_sub targ
    
    That finishes the disk preparations. Now it is time to boot up the system and continue installation.
    common_args="-m 512 \
      -drive file=disk.img,media=disk,format=raw,index=0 \
      -net nic,macaddr=52:54:00:fa:ce:19,model=e1000 -net tap \
      -kernel kernel \
      -display vnc=:19 \
      -serial null -monitor null"
    
    Install:
    qemu-system-alpha $common_args \
      -drive file=inst-disk.img,media=disk,format=raw,index=1 \
      -append "root=/dev/sdb"
    
    Perform an almost plain Gentoo installation using vnc. Be warned that this is a difficult installation even for being Gentoo, since things are highly inconsistent; commands do not work as suggested or documented, partition numbers and names vary between wrong and dead wrong, etc, etc. Things to consider: Boot:
    qemu-system-alpha $common_args \
      -append "root=/dev/sda"
    
    Notes:
    As an alternative, download the pre-installed experimental image disk.img.xz. This unpacks into a 10 GiB sparse file.
    sha256: 16898e6a85695bab0a3ffafd846a47f232877e1b5bef669d89154aa1d147762e img.xz
    sha256: 70e43ee95b6e40c1666661d4c9144323a311e73cbd4e5e241bb38dc54f754484 img
    The image can be booted by this sh script which also needs this kernel. Note that this is a better kernel than in /boot of the image. CAUTION: The root password is "x". Security experts might consider that to be a tad too short. Therefore, I highly recommend that you comment out the -net line from the boot script, make an initial boot and connect using vncviewer :19, change the root password to something sensible, halt the system, and the re-instate the -net line before re-booting the system. (If there is access from the Internet to the vnc port, the suggested precaution isn't going to do much good.)

Copying out Linux kernel and initrd

When a boot loader cannot be installed, as in many cases above, one needs to supply a kernel and some initial files via an "initrd". In the case of GNU/Linux, the needed files should be in /boot at the end of installation.

There are several alternative ways one may copy these files to the host file system.

  1. Just before the end of the installation.

    In Debian (and presumably several of its derivatives), just as the Finishing the installation/Installation complete step is performed, choose Go back and then scroll down the menu to Execute a shell. In that shell do:

    mount -t proc proc /target/proc
    mount --rbind /sys /target/sys
    mount --rbind /dev /target/dev
    chroot /target bash
    /etc/init.d/ssh start

    Then, from the host system, do

    ssh -p [arguments] [host] "tar -c -f - --exclude=lost+found /boot" | tar xf -

    where [arguments] should be any required port number and [host] is the IP address or name of the target host, or perhaps "localhost" if port forwarding was used. Note that root logins might not work unless you edit /etc/ssh/sshd_config to allow root login.

    Return to the installation screen, and exit the shell, and choose Finishing the installation.

  2. Let the installation finish, then mount the disk image read-only from the host. This will usually work, but it might be hard if the host system cannot grok the disk label. Typically,

    { echo p; echo q; } | fdisk disk.img

    will work. You should see a list such as:

    Device    Boot   Start     End Sectors  Size Id Type
    disk.img1 *       2048  333823  331776  162M 83 Linux
    disk.img2       333824 3893247 3559424  1.7G 83 Linux
    disk.img3      3895294 4192255  296962  145M  5 Extended
    disk.img5      3895296 4192255  296960  145M 82 Linux swap / Solaris

    You can now mount the partitions,

    mount -o loop,ro,offset=333823 disk.img /mnt

    where the offset argument comes from the relevant partition. Now just copy the files from /mnt.

  3. Use qemu-nbd. I haven't tried that myself.

Now, point the -kernel and -initrd arguments to the files in just copied-out directory boot. Exact arguments appear in each example above.


Please send corrections and any additional instructions to tg at gmplib dot org