"A crash reduces
your expensive computer
to a simple stone."
—Anonymous, GNU error message haiku page
Welcome to the first meeting of the Advanced Operating Systems course! Today we're going to configure and compile a Linux kernel from source code. Even if you've done this kind of thing before, this studio will introduce the source code and resources that this class depends on, along with giving you additional experience working with the Raspberry Pi platform.
In this studio and the next studio, and throughout the semester, you will use a two-stage cross-compilation approach:
Having the code and the build process on other machines has some key advantages besides
probably building faster than on your Pi. First,
if you get locked out of your Pi, or at some point a (modified) kernel crashes (see the
error message haiku above :-) most of the work you have done is safe in another
location which allows you to start over efficiently by re-imaging your Pi to
the default setup, fixing whatever bugs may have caused the crash, recompiling the kernel,
installing it, and proceeding from there. Second, the CEC servers (such as
shell.cec.wustl.edu) give you a place to create backup copies of any code
before you modify it - or even to run version control software such as svn
or git so that if a change you make goes awry you can quickly recover to
a previous (working) version of your code.
Please complete the required exercises. We encourage you to please work in groups of 2 or 3 people on each studio (and the groups are allowed to change from studio to studio) though if you would prefer to complete any studio by yourself that is allowed.
As you work through these exercises, please record your answers, and when you finish them each and every person who worked on them should please log into Canvas, select this course in this semester, and then upload a file containing them, or put them into a text entry box, and submit them for this studio assignment (there should be a separate submission from each person who worked together on them).
Make sure that the name of each person who worked on these exercises is listed in the first answer, and make sure you number each of your responses so it is easy to match your responses with each exercise.
To save time, and to avoid potential freezing of your Raspberry Pi during
long-running local compiles, we will
be cross-compiling the kernel using a university-managed server
(shell.cec.wustl.edu) and dedicated Linux machines.
First, ssh into shell.cec.wustl.edu using your WUSTL Key id and password.
On a Mac or Linux computer, you can just run ssh directly from within a terminal window.
On many versions of Windows, you can run ssh directly from the command prompt.
If the command is not recognized, you will need to install and use an SSH client,
such as Putty or WinSCP.
Be careful! When you connect with ssh, you must specify your username, e.g.
ssh username@shell.cec.wustl.edu
or you risk getting locked out of the server.
If you connect with the wrong username or password 5 times in 30 minutes,
your IP address will be blacklisted for 4 weeks.
To help prevent lockouts due to incorrect password attempts, you may want to log in using an SSH key instead of a password, as detailed on this page.
Because kernel compiles are
computationally intensive, you should not run them on the shell.cec.wustl.edu server, and instead you should run:
qlogin
which will let you log into a dedicated Linux host using your WUSTL Key id and password.
You must remember the hostname of the machine you are now running on. To determine the hostname, execute the
hostname command. The result should look something like:
linuxlab{id}.engr.wustl.edu
You must remember this hostname. It will store your linux kernel as well as other files you implement as part of this and future studios and labs.
As the answer to this exercise, write down the hostname of the linuxlab machine you're using, as well as the CPU model name and number of (logical) CPUs. Note: the hostname can and likely will be different for all students in your group
Determine the hostname of the machine you are now running on by issuing the hostname command.
Additionally, execute the lscpu command
to get information about the CPU(s) on the server.
As the answer to this exercise, write down the hostname of the linuxlab machine you're using,
as well as the CPU model name and number of (logical) CPUs.
Now, exit out of the machine you were issued via qlogin by typing exit. You are now running
on the shell.cec.wustl.edu machine. Now, use ssh to log back into the linuxlab machine you wrote down by issuing the ssh command:
ssh linuxlab{id}.engr.wustl.edu
replacing {id} with the specific ID found in your hostname.
Now, you are just about ready to start downloading and compiling the Linux kernel. However, in order to ensure that your compilation does not interfere with others using the machine, you must first create a private directory on the local networked filesystem that is isolated from the traffic created by other users.
To do so, issue the following commands:
cd /project/scratch01/compile
mkdir "your username"
chmod 700 "your username"
cd "your username"
replacing "your username" with your actual username.
These commands created a directory that you should use to store your linux
kernel. Whenever you log into the machine via ssh, you must first change your
directory to this location by typing:
cd /project/scratch01/compile/"your username"
As the answer to this exercise,
please issue the command cd /project/scratch01/compile
to navigate to the directory containing the one you just created.
Then issue the command ls -ld "your username"
and give the complete output from that command.
Additionally, please write what the above chmod
command accomplished, including what the octal value 700 denotes.
(Hint: look at the manual page loaded from the command man 1 chmod.)
Note that this directory only exists on the linuxlab machine that you are currently using. You must always ssh to this specific machine, as you did at the beggining of this step.
Now it's time to download the Linux kernel source code. For a general-purpose project, you would go to kernel.org and download the Linux source from there. However, since we're using the Raspberry Pi we'll be starting with a version that is designed for our platform. The Raspberry Pi project maintains it's own distribution on GitHub, at https://github.com/raspberrypi. We'll be modifying the kernel based on the building guide that you can find here.
From your specific /project/scratch01/compile/user-name/ directory, create a folder called
linux_source in which to keep all of your source code and build files organized.
Use the cd command to move into your new folder and issue the following
commands, each of which may take a while to finish: the second of which may take over 20 minutes to complete:
wget https://github.com/raspberrypi/linux/archive/raspberrypi-kernel_1.20210527-1.tar.gz
tar -xzf raspberrypi-kernel_1.20210527-1.tar.gz
This has the effect of downloading a specific version of the
Raspberry Pi Linux distribution, which is the version that
this course was developed with. Newer versions exist, which we won't
use for this class but you can pursue on your own time if you'd like. Once
the files finish unpacking you'll have a new directory, which we
suggest renaming as something simpler with the mv
command. We renamed ours linux, which is what the
rest of these instructions will assume you've done as well.
To save space, please delete the .tar.gz file once it is unpacked.
Move into your new linux directory, and issue the
command make kernelversion, which will tell you which Linux
kernel version you've just checked out.
As the answer to this exercise, give the output
from that command.
Additionally, open the Makefile in a text editor (e.g. emacs, vim, or nano) and look at the first several lines that define constants related to the kernel version. Write the name of this version of the kernel (as defined by the NAME constant).
If you are using the Raspberry Pi 4 or 4B,
you will need to modify the arm-pmu entry in the device tree for the Pi 4's BCM2711 board,
which will allow the Linux kernel to load the hw perfevents driver that will be used in a later studio.
To do so, open the file arch/arm/boot/dts/bcm2711.dtsi,
find the entry for the arm-pmu,
then set the "compatible" line as follows:
compatible = "arm,cortex-a72-pmu", "arm,cortex-a15-pmu", "arm,armv8-pmuv3";
module add arm-rpi (this adds the cross-compiler to your PATH variable)
module add gcc-8.3.0 (this adds an updated C compiler to your PATH variable)
Note: You can add the two commands above as individual lines at the end of the file ~/.bashrc,
and then in order to ensure the command is executed when you log in via SSH, please add code like the following to your ~/.bash_profile file (if it's not already there):
if [ -f ~/.bashrc ]; then
. ~/.bashrc;
fi
Now, issue the following command if you are using a Raspberry Pi 3B+:
make -j8 ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2709_defconfig
If you are using the Raspberry Pi 4 or 4B, instead use the following command:
make -j8 ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2711_defconfig
To reiterate, if you were building a general purpose kernel, you wouldn't use the previous commands, which set up a default configuration for the Raspberry Pi.
Next, we want to set custom configuration options. Issue the command:
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig
After a moment, you'll get a kernel configuration menu. As you can see, there are a lot of options. For now, we're just going to do three things.
First, you'll add your own unique identifier to the kernel you build.
Navigate to "General setup", then select "Local version."
The local version string you specify here will be appended to the output of the
uname command. If you applied the default Raspberry Pi
configuration correctly, this currently should be set to:
"-v7" if you are using a Raspberry Pi 3B+, or to
"-v7l" if you are using a Raspberry Pi 4 or 4B.
Go ahead
and append your own unique identifier to that string, though recall that uname
already gives you the kernel version number, the day and time the kernel was
compiled, as well as other info. Warning: do not include any spaces in the
local version string - this will break the build script when you run
sudo make modules_install in the future.
Next, we will change the kernel's preemption model. While still in "General setup," select "Preemption Model," and then select the "Preemptible Kernel (Low-Latency Desktop)" option.
Now, enable the ARM Performance Monitor Unit driver. This will enable you to use the hardware counters provided by the Raspberry Pi. While still in "General setup," select "Kernel Performance Events and Counters." Make sure "Kernel performance events and counters" is enabled. Then, return to "General setup" and ensure that "Profiling support" is enabled. Exit "General setup" to return to the main configuration menu.
Next you'll add your own unique identifier to the kernel you build.
Exit "Kernel Features" and navigate to "Local version" under "General
setup". The local version
string you specify here will be appended to the output of the
uname command. If you applied the default Raspberry Pi
configuration correctly, this currently should be set to "-v7". Go ahead
and append your own unique identifier to that string, though recall that uname
already gives you the kernel version number, the day and time the kernel was
compiled, as well as other info.Warning: do not include any spaces in the
local version string - this will break the build script when you run
sudo make modules_install in the future.
Choose an interesting sounding option and use the "H" key to bring up a short description. As the answer to this exercise, give the option's name, a short summary, and the option's symbol. Additionally, please describe why you think we have chosen the "Preemptible Kernel (Low-Latency Desktop)" preemption model. (Hint: look at the corresponding configuration option in the kernel code here.)
Once you're done, exit the configurator and be sure to answer "Yes" when asked to save your changes.
To track how long the compilation step takes, issue the following command, which compiles the kernel, while writing start and end times to a file:
date>>time.txt; make -j8 ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs; date>>time.txt
The compile may take a while (possibly an hour or more, depending on the machine you're compiling on). Once it's done, issue the following command:
mkdir ../modules
This will create a directory that the cross-compiler will use to store the kernel modules that it creates. You will later transfer these files to your Raspberry Pi. Then issue the command:
make -j8 ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf-
INSTALL_MOD_PATH=../modules modules_install
As the answer to this exercise please indicate the time it took to compile the kernel (by running cat time.txt and comparing the start and end times).
Additionally, please describe (in a sentence or two) why we have to use the cross-compiler to compile the kernel on the linuxlab server.
Note the time again, and as the answer to this exercise please indicate approximately how long it took to complete it.
Congratulations! You've just compiled an operating system from source!
For this studio, please turn in the following:
Page updated Friday November 19, 2021, by Marion Sudvarg and Chris Gill.