CSE 522S: Studio 15


I/O devices are generally the most difficult hardware to virtualize in a high-performance fashion. Traditional approaches use device emulation, whereby the hypervisor exposes a set of hardware interfaces – device memory regions and I/O ports, mainly – that mimic the appearance of real hardware devices. This allows the guest to load its unmodified drivers to manage them as if they were real hardware. While this makes it possible to run unmodified guest operating systems, this approach is problematic because it requires a lot of hypervisor code to emulate devices, buses, chipsets, etc., and also because this emulation leads to a large number of traps in response to operations on the emulated hardware (such as reads/writes on emulated I/O ports). To address these issues, many modern hypervisors implement paravirtual devices, including virtio devices in KVM/Linux systems. Though paravirtual devices require changes to guest device drivers, they lead to simpler device interfaces and higher performance in virtual machines.

In this studio, you will:

  1. Use QEMU/KVM to boot emulated and virtio-enabled hardware in VMs on the Raspberry Pi
  2. Measure the performance of two different workloads using virtio and emulated devices
  3. Compare these measurements with the same workloads running natively on the Raspberry Pi

Please complete the required exercises below. We encourage you to please work in groups of 2 or 3 people on each studio (and the groups are allowed to change from studio to studio) though if you would prefer to complete any studio by yourself that is allowed.

As you work through these exercises, please record your answers, and when finished upload them along with the relevant source code to the appropriate spot on Canvas.

Make sure that the name of each person who worked on these exercises is listed in the first answer, and make sure you number each of your responses so it is easy to match your responses with each exercise.

Required Exercises

  1. As the answer to the first exercise, list the names of the people who worked together on this studio.

  2. As you did for the Virtual Machine Performance with QEMU and KVM studio, boot an Ubuntu VM using QEMU/KVM on your Raspberry Pi. We are going to need more memory in the VM, so run with 512MB instead of 256MB as follows:

    sudo qemu-system-aarch64 -M virt -cpu host -m 512M -smp 2 -nographic -bios QEMU_EFI.fd -cdrom seed-kvm-bionic-01.iso -drive if=none,file=bionic-image-01.img,id=hd0 -device virtio-blk-device,drive=hd0 -device virtio-net-device,netdev=vmnic -netdev user,id=vmnic -accel kvm

    Note the two "-device" options on the QEMU command line. These tell QEMU to expose two virtio devices to the guest: a block device (virtio-blk-device) and a network device (virtio-net-device).

    Once the guest has booted, log into it (recall from the previous studio that the username is ubuntu and the password is cse522studio unless you set a different password). Then, from the terminal of the virtual machine, run the following command:

    lsmod | grep virtio

    This will determine which virtio device drivers have been loaded into the guest kernel. As the answer to this exercise, show the output from this command.

  3. Next, you'll need to disable the automatic update feature in Ubuntu Server, to keep it from preventing you from installing new software packages. This can be done easily with the following commands:

    sudo apt-get update
    sudo systemctl mask apt-daily.service apt-daily-upgrade.service
    sudo systemctl disable apt-daily.service apt-daily.timer apt-daily-upgrade apt-daily-upgrade.timer

    Download two new applications in the VM that we will use to evaluate performance: fio, which we'll use for disk I/O performance analysis, and iperf, which we'll use for network performance analysis:

    sudo apt install fio iperf

    (If you receive an error message indicating the apt can't proceed because a file is locked, you may need to reboot the VM to apply the changes you made to the automatic update settings in the guest OS.)

    We'll first analyze guest file I/O performance with fio. For more information, see the online man page or issue the command:

    man 1 fio

    As is standard for I/O performance analysis, we'll evaluate two different parameters: workloads that are read vs write heavy, and workloads that access data sequentially vs randomly. First, we'll evaluate sequential read performance with the following command:

    sudo echo 3 | sudo tee /proc/sys/vm/drop_caches && fio --thread --name=read --rw=read --size=256M --filename=file.dat && rm file.dat

    Note that we are first clearing the guest kernel's page cache by writing 3 to the drop_caches sysfs file, and then using the fio program. When this program completes, it will write a line to standard output that start with: READ: bw=..... Record the bandwidth measurement, and then also run the sequential write, and then random read/write benchmarks:

    sudo echo 3 | sudo tee /proc/sys/vm/drop_caches && fio --thread --name=write --rw=write --size=256M --filename=file.dat && rm file.dat
    sudo echo 3 | sudo tee /proc/sys/vm/drop_caches && fio --thread --name=randread --rw=randread --size=256M --filename=file.dat && rm file.dat
    sudo echo 3 | sudo tee /proc/sys/vm/drop_caches && fio --thread --name=randwrite --rw=randwrite --size=256M --filename=file.dat && rm file.dat

    As the answer to this exercise, show the bandwidth measurements for these 4 workloads, and explain how and why they are similar or different.

    Once you are done, shut down the virtual machine:

    sudo shutdown -h now

  4. Now, we will download and run the same workloads on the host to determine how much performance overhead exists.

    Repeat the previous exercise, but from a terminal on the host system, rather than within the virtual machine. You should not have to disable the automatic updates in the host, as you likely already did this in the previous studio.

    As the answer to this exercise, show the bandwidth data for all 4 workloads, and determine how much performance overhead exists for each in comparison to the guest environment. If some workloads exhibit significantly more virtualized overhead than others, provide a possible explanation for why that might happen, using your knowledge of device I/O behavior and how the I/O is virtualized.

  5. We will now investigate virtualized networking performance by transferring data between the host and the guest over a virtualized network. First, determine the public IP address of the host by using the ifconfig command and looking for the IPV4 address associated with either your Ethernet or WiFi device, depending on which network you are using for Internet access.

    You will use iperf to perform network throughput tests between the host and the guest. First, For more information, see the online man page or issue the command:

    man 1 iperf

    Now, start up an iperf server on the host with the following command:

    iperf -s -p 8080

    In a separate terminal, again boot the VM using the command from above. In the VM, run an iperf client with:

    iperf -c $SERVER_IP -p 8080

    replacing $SERVER_IP with the host's IP address you found above. This command will run a TCP networking test for a few seconds, and then report the networking bandwidth it sustained. Run this test 3 times, and as the answer to this exercise record the median bandwidth from the 3 experiments.

  6. Teardown the VM (with sudo shutdown -h now). We will now boot the VM with an emulated networking device, which will appear to the guest as an e1000 Intel Gigabit Ethernet device rather than a virtio device. To do this, modify your QEMU command line, removing:

    -device virtio-net-device,netdev=vmnic

    and replacing it with

    -device e1000,netdev=vmnic

    Boot and log back into the guest, and again run the following command to determine which virtio drivers are now loaded into the guest kernel.

    lsmod | grep virtio

    Then, run the same iperf experiment 3 times.

    As the answer to this exercise, report the output of the lsmod command, again report the median of the 3 iperf experiments, and determine the performance loss compared with the virtio network device. Based on on your knowledge of virtio, explain (briefly) why you think the performance is different in the 2 environments.

  7. Things to Turn In:

    Page updated Thursday, April 7, 2022, by Marion Sudvarg and Chris Gill.