E81 CSE 522S: Advanced Operating Systems

Spring 2022

Acknowledgement: portions of this course and web site are adapted from materials
that were originally developed and taught at Washington University by David Ferry and by Brian Kocoloski.

Instructors Chris Gill (office hours by appointment)
Marion Sudvarg (office hours by appointment)
Course Web Site
Course Piazza Page
Course Meetings Tuesdays and Thursdays 2:30 - 3:50 PM in Urbauer 216.
First Exam
Thu, Feb 24th, 2:30 - 3:50 PM in Jubel 121
Second Exam
Tue, Apr 19th, 2:30 - 3:50 PM in Lopata 103
Prerequisites CSE 422S (Operating Systems) and C/C++ programming experience are firm prerequisites for this course.

  1. Course Description
  2. Prerequisites
  3. Discussions/Studios
  4. Course Project
  5. Class FAQ
  6. Computers, Textbooks, and Other Resources
  7. Grading
  8. Academic Support
  9. Title IX, Diversity, and Inclusion
  10. Academic Integrity

Course Description

This course intends to give hands-on experience working with system software based on the Linux operating system kernel, to address issues well beyond the scope of a basic operating systems course. Profiling, quantifying, modifying, and optimizing program and operating system behaviors as different policies and mechanisms interact are key themes of this course, as is understanding how those policies and mechanisms govern those behaviors. Particular attention will be paid to containers and virtualization, and how they leverage and extend particular features and services provided by Linux.

The objectives of this course are for each student to:

This is a hands-on graduate-level course dealing with complex systems software, which is being substantially revised again this semester (including the addition of entirely new material on containers and virtualization). Hiccups are to be expected, and finding new and exciting ways to break course exercises is encouraged, as the instructors continue to fine-tune the content and scope of each class meeting.


Discussions and Studios

This course is conducted in a studio format, both to emphasize the practical and hands-on nature of "kernel hacking" and to engage students in applied active learning exercises. The lectures are designed to facilitate in-class discussion of kernel features and their design and use, while the aim of the studio exercises is (1) to offer direct experience working with (and within) the kernel and (2) to familiarize students with key tools and techniques for kernel development, profiling, analysis, and optimization. The studios and course project will be completed as small group exercises, and are submitted for credit. All studios assigned prior to the first exam are due the night before it, and all studios assigned after the first exam are due the night before the second exam (please see the day-by-day syllabus for the precise deadlines).

Most class periods are accompanied by suggested readings. The Linux kernel is a particularly compelling subject of study, in that many of the discussions (and disagreements) among the original developers have been saved verbatim in repositories such as the the Linux Kernel Mailing List (LKML) or documented by firsthand witnesses at sites such as The course textbooks can be used as technical references for kernel mechanisms, and supplemental readings in them can be used to understand the particular design choices made in the course of kernel development.

The course schedule is as follows. Note that this schedule may change over the course of the semester. If changes occur, students will be notified in class and in Canvas and given enough advance notice so that readings and other preparation may be accommodated.

# Date Topic Assigned Readings/Resources Studios/Projects
1 Tue, Jan 18th Course Introduction

Academic Integrity
Cross-compiling the Linux kernel

Setting up the Raspberry Pi
2 Thu, Jan 20th Hardware Performance Monitoring and Kernel Modules Hardware Counters and Loadable Kernel Modules
3 Tue, Jan 25th Filesystem Monitoring and Events Observing File System Events
4 Thu, Jan 27th Process Observability and Isolation Process Forensics
5 Tue, Feb 1st Isolation with Namespaces Isolation with Namespaces

Course Project Assigned
6 Thu, Feb 3rd User Namespaces and Capabilities Capabilities and User Namespaces
7 Tue, Feb 8th Subsystem Observability and Control Groups

Course Project Overview and Ideas
Observing Memory Events
8 Thu, Feb 10th Timing Forensics and CPU cgroups CPU Control and Timing Events

Project proposals due Fri, Feb 11th at 11:59PM
9 Tue, Feb 15th Timers and Interrupts No Studio Assigned – Catch-Up Day
10 Thu, Feb 17th I/O Cgroups Controlling I/O Behavior

Project proposal feedback returned by Fri, Feb 18th at 11:59PM
11 Tue, Feb 22nd First Exam Review in Urbauer 216   All studios assigned so far are due Wed, Feb 23rd at 11:59pm
12 Thu, Feb 24th
First Exam, 2:30 - 3:50 PM in Jubel 121
13 Tue, Mar 1st Introduction to Docker Introduction to Docker Containers and Images

Revised project proposals due Tue, Mar 1st at 11:59pm
14 Thu, Mar 3rd Docker Container Management (optional) DKR pages 70-83, 85-110, 115-130, 134-138, & 141-142 Container Control with Docker
15 Tue, Mar 8th Network Namespaces No Studio Assigned – Catch-Up Day
16 Thu, Mar 10th
Docker Communication and Networking Docker Web Applications
Tue Mar 15th and Thu Mar 17th: Spring Break - no classes
17 Tue, Mar 22nd CPU Virtualization No Studio Assigned – Catch-Up Day
18 Thu, Mar 24th Memory Virtualization Virtual Machine Performance with QEMU and KVM
19 Tue, Mar 29th I/O Virtualization virtio: Towards a De-Facto Standard For Virtual I/O Devices No Studio Assigned – Catch-Up Day
20 Thu, Mar 31st Project Milestone Demos in Urbauer 216

Project milestone code and reports are due Friday, Apr 1st at 11:59pm
21 Tue, Apr 5th Device I/O No Studio Assigned – Catch-Up Day
22 Thu, Apr 7th Device Memory Interaction
  • LKD pages 231-245
  • LDD chapter 15: pages 412-424 & 440-448
  • Virtio
    23 Tue, Apr 12th Device Ecosystem No Readings Assigned No Studio Assigned – Catch-Up Day
    24 Thu, Apr 14th Second Exam Review in Urbauer 216   All studios assigned after the First Exam are due Mon, Apr 18th at 11:59pm
    25 Tue, Apr 19th
    Second Exam, 2:30 - 3:50 PM in Lopata 103
    26 Thu, Apr 21st In-class project time
    27 Tue, Apr 26th Project Presentations I in Urbauer 216
    28 Thu, Apr 28th Project Presentations II in Urbauer 216

    All project code and reports are due Monday, May 2nd at 11:59pm

    Course Project

    The course project will give each team of two or three students an opportunity to propose and complete their own significant extension of, or modification to, system software based on the Linux kernel. This may be an entirely new capability or an alteration to some existing functionality. Each team will evaluate the effectiveness of its modification by developing their own test cases that can successfully demonstrate the new behavior. Students are encouraged to look to their own labs, interests, and/or research projects for inspiration and ideas for the project.

    In either case, students will justify their proposal in a written document, in order to ensure that the scope of the project is achieveable. The proposal document will:

    1. Propose a modification of, or extension to, Linux kernel behavior
    2. Clearly explain the purpose of the modification or extension
    3. Identify any kernel files that will require modification or extension
    4. Identify any kernel data structures that will need to be changed or extended
    5. Identify any kernel concepts and control paths that will need to be changed or extended
    6. Propose a set of test cases that demonstrate and test the modification or extension
    7. Identify any kernel modules or user space programs that will be produced in order to implement, or to validate and test, the modification or extension
    8. Include a multi-week planned schedule for the project including: initial design and implementation; initial evaluation; detailed design, implementation, and evaluation; project presentation; and project writeup.

    The project will be evaluated in part based on a live presentation at the end of the semester, as well as code and documentation which will be submitted and graded both at an interim point in the semester and at the end of the semester. The presentation and writeup should explain the modification or extension to the other students in the course; describe any alterations or extensions to kernel files, data structures, or concepts; present timing graphs or other data to verify the behavior of the modified system; and may include a short demo (if possible within the allotted time).

    Class FAQ

    Current and past instructors, TAs, and students of this course have contributed various tips, tricks, and solutions to problems that they have encountered. Please let your instructor know if you have something you'd like to add here!

    Class FAQ

       What if your Pi is freezing up?

       Leaving your Pi at home

    Generating SSH Keys

    Using Visual Studio Code Remotely

    Computers, Textbooks, and Other Resources

    To be able to complete the studio assignments this semester, you will need to purchase a CanaKit Raspberry Pi 3 B+ kit (or, if not available, a CanaKit Raspberry Pi 4 B kit), which is the platform on which we have tested out our solutions to those assignments, and on which we will be grading them.

    Kits may be purchased from any of the following links. For purchases directly from the CanaKit website, any kit with an "Add to Cart" button that doesn't show a pre-order date means it's in stock.

    The course has been designed with the intention that students bring these devices to class, plug them in, and do some hacking there. Monitors, keyboards, and mice will all be provided for this purpose – you will need to provide the Raspberry Pi, Micro SD card, and power cord (all of which should be included in the kits linked above).

    If you use a Raspberry Pi 4, you will need to additionally purchase a Micro-HDMI to HDMI adapter. You do not need this if you are using a Raspberry Pi 3 this semester. We recommend the following adapter:

    If you so desire, you may set up your Raspberry Pi in your home or office and configure it for remote access. However, the instructors cannot support you in this, and we are modifying the kernel so you may be rather lost if you reboot and can't ssh back into your machine. Also, be forewarned that some exercises will require an X11 window, so simple ssh access will not be enough to do everything remotely.

    The in-class studios and assigned readings offer a reasonable coverage of Linux. However, the studios and assigned readings will not yet make you an expert on the Linux kernel. When it comes to learning the details of a code base as deep and complex as Linux, there is no substitute for reading source code. Some of the lectures and studios will give pointers to relevant source code files, and for overall understanding it is expected that students will spend some time absorbing the content there as well.

    To be clear, you are not expected to understand or memorize every line of the Linux, KVM, or Docker source. Exams will not have questions derived from the source (though basic concepts and pseudo-code are fair game). However, the Linux, KVM, and Docker implementations each constitute huge sets of deeply interdependent source code files. The only way you will ever understand them in-depth is by looking through them many times.

    There are two required course textbooks for this course:

    These are both excellent, compact, and inexpensive texts that give the reader a basic understanding of Linux kernel and library design and usage, written in depth by an expert Linux veteran.

    Readings in two optional additional course textbook also will be suggested:

    Please note that the Linux kernel source code is frequently (and sometimes rapidly) updated, so that while the general information in these textbooks is correct, many details of the source code itself, the names of source code files, and where those files exist, are somewhat likely to change (and have done so since the required text books above and the additional references below were written).

    There are a number of other references that you may find useful for this course and more broadly for your study and use of the Linux kernel.


    There are three activities for which you will receive credit in this course: studios, exams, and the course project. Studios are daily guided assignments primarily designed to familiarize students with course concepts, development tools, and the kernel source code (i.e. knowledge and comprehension tasks). The course project requires that students propose and complete a novel kernel modification, and students will be graded on a project proposal; an initial milestone that includes a presentation, code, and writeup describing progress so far; and on their final project submission. Two in-semester exams will evaluate your technical understanding of course concepts.

    Your grade will be determined as follows:

    Activity Grade Percentage
    First Exam 15%
    Second Exam 15%
    Studios 20%
    Course Project Proposal 10%
    Course Project Milestone 15%
    Course Project Final Submission 25%

    Academic Support, Health and Wellness, and Disabilities

    Especially with the challenges many students may face in an on-line learning environment, we encourage everyone to take a look at, and potentially make use of, the many academic support resources available at Washington University in St. Louis, including: The Learning Center, and The Writing Center, and Student Technology Services.

    Washington University recognizes that students serving in the U.S. Armed Forces and their family members may encounter situations where military service forces them to withdraw from a course of study, sometimes with little notice. Students may contact the Office of Military and Veteran Services at (314) 935-2609 or and their academic dean for guidance and assistance.

    The Habif Health and Wellness Center, WashU Cares, Mental Health Services, and RSVP are important university resources for issues pertaining to physical and mental health.

    Students with disabilities or suspected disabilities are strongly encouraged both to bring any additional considerations to the attention of the instructor and to make full use of Washington University's Disability Resources, potentially including accommodations for studios, projects, and/or exams.

    Title IX, Diversity, and Inclusion

    Washington University is firmly committed to addressing and preventing sexual misconduct on our campuses: please see the Washington University statement on new Title IX rules. If a student discusses or discloses an instance of sexual assault, sex discrimination, sexual harassment, dating violence, domestic violence or stalking, or if a faculty member otherwise observes or becomes aware of such an allegation, the faculty member will keep the information as private as possible, but as a faculty member of Washington University, they are required to report it immediately to their Department Chair or Dean or directly to Ms. Jessica Kennedy, the University's Title IX Director, at (314) 935-3118 or Additionally, incidents or complaints can be reported to the Office of Student Conduct and Community Standards or by contacting WUPD at (314) 935-5555 or contacting your local law enforcement agency. To explore options for medical care, protections, or reporting, free and confidential support resources and professional counseling services are available through the Relationship and Sexual Violence Prevention (RSVP) Center in Seigle Hall, Suite 435, which can be reached at or at (314) 935-3445. For after-hours emergency response services, call 314-935-6666 or 314-935-5555 and ask to speak with an RSVP Counselor on call.

    Washington University's Center for Diversity and Inclusion supports and advocates for undergraduate, graduate, and professional school students from underrepresented and/or marginalized populations, collaborates with campus and community partners, and promotes dialogue and social change to cultivate and foster a supportive campus climate for students of all backgrounds, cultures, and identities. The University has a process through which students, faculty, staff, and community members who have experienced or witnessed incidents of bias, prejudice, or discrimination against a student can report their experiences to the University's Bias Report and Support System team. For procedures and information on reporting an instance of bias, please visit the Bias Report and Support System page on the Center for Diversity and Inclusion web site. In order to affirm each person's gender identity and lived experiences, it is important that we each ask and check in with others about pronouns. This simple effort can make a profound difference in a person's experience of safety, respect, and support. Please see the University's Pronouns Information page and the Office of the Registrar's Preferred Name page for additional information and resources.

    Academic Integrity

    Each studio and project assigned in this course is expected to be completed in teams of up to three people (and not more than that) – we want to encourage collaborative work though you are also allowed to work by yourself if you prefer. During the regularly scheduled class times that the instructors are attending (and only during those times), all students are allowed to share their screens to show code for purposes of getting help with debugging, illustrating solution approaches that are being discussed, etc., and communication about a particular studio or project within your team of (3 or fewer) people who are working together on it is allowed at any time. Communication between different teams is not allowed outside of the scheduled class times that the instructors are attending unless the instructors give permission in advance. Student teams may change from assignment to assignment, but the sharing of code between teams without prior permission of the instructors is strictly prohibited, and you must acknowledge and document in detail all contributions that anyone has made to the work.

    To encourage you to read the Linux man pages and the LPI, LKD, LSP, and DKR text books carefully, you are allowed to use snippets of code from those sources in your studios, projects, and exams without asking permission, as long as (in comments before and after it) you clearly identify the start and end of each such code snippet and attribute the source from which it came (including page and/or line numbers where those are available). For code from any other sources you must first ask for and receive permission from your instructors (and should identify and attribute it similarly if so).

    Exams must be completed individually without assistance from any other person, and without reference to external materials except as is specifically allowed by the instructor (documentation of what is allowed will be described in the review lecture, provided in the corresponding review slides, and written on the font page of the exam).

    Cheating costs everyone something. Someone who cheats misses out on the intended opportunity to improve through the assigned work, and like anyone helping them cheat is at risk of diminished reputation as well as specific sanctions (see below). Cheating also degrades the value of the degree earned by those who complete their work with integrity.

    Academic integrity is a serious matter in this course, and the instructor will decide whether any situation of concern in this course warrants referral to the appropriate formal academic integrity adjudication process at the school or university level. If in any doubt, please ask first. Anyone found to have cheated or to have helped someone else cheat will face penalties that may include receiving a negative score equal in magnitude to the value of the assignment in question (e.g., cheating on an assignment worth 10% of the course grade would result in -10% being assigned for a total loss of 20% from the course grade, which is twice as large a loss as simply not turning in the assignment).

    Academic integrity is itself worth studying and thinking about as a key component of your education. Please read, familiarize yourself with, and reflect on the Engineering School's and Washington University's undergraduate and graduate policies on academic integrity.