CSE 422S: Studio 17

Linux Pipes and FIFOs

Regular pipes are the method used to "pipe" the output of one program into the input of another; they are created in memory via a system call and do not exist on any filesystem. Named pipes act like regular pipes but are accessed via a file, called a FIFO special file. Unrelated processes can access this file and communicate.

—Robert Love, Linux System Programming, 2nd Edition, Chapter 1, pp. 14-15.

Pipes and FIFOs provide efficient ways to do file-like input and output between processes on the same host machine. Their biggest benefit is the ability to treat the endpoints as formatted input and output streams, just like you would read and write from files.

In this studio, you will:

  1. Create and pass data through pipes with the pipe() system call
  2. Create and pass data through FIFOs with the mkfifo() function
  3. Implement a rudimentary active object pattern with processes and FIFOs
  4. Use the getline() function to read a line of data at a time from a file into a dynamically allocated buffer, and then send it over a FIFO

As a refresher on Linux processes, re-read (skim) LPI pp. 515-522 and 563-571 and LKD pp. 29-33 and 38-40, and please run the following commands on the Linux command line (on your Raspberry Pi or on one of the Linux Lab machines) for more documentation on the fork and exec library functions:
man 2 fork
man 3 exec

You may want to also skim the slides and studio exercises to refresh your understanding of parent and child processes.

Please complete the required exercises below. We encourage you to please work in groups of 2 or 3 people on each studio (and the groups are allowed to change from studio to studio) though if you would prefer to complete any studio by yourself that is allowed.

As you work through these exercises, please record your answers, and when finished upload them along with the relevant source code to the appropriate spot on Canvas. If you work in a group with other people, only one of you should please upload the answers (and any other files requested) for the studio, and if you need to resubmit (e.g., if a studio were to be marked incomplete when we grade it) the same person who submitted the studio originally should please do that.

Make sure that the name of each person who worked on these exercises is listed in the first answer, and make sure you number each of your responses so it is easy to match your responses with each exercise.

Required Exercises

  1. As the answer to the first exercise, list the names of the people who worked together on this studio.

  2. First we will use the pipe() system call to create a program with communicating child and parent processes. Once created, pipes cannot be shared between unrelated processes. Thus, the standard way to use this system call is first to call pipe(), which will create separate pipe endpoints for reading and writing, and then to call fork(), which will create a child process. Thus, the child will inherit the pipe endpoints from the parent, and both processes now will have access to the pipe.

    On your Raspberry Pi, create a C program that:

    1. Calls pipe() to create a pipe with read and write file descriptors (see man 2 pipe for details)
    2. Calls fork() to create a child process
    3. After the fork, the child process should close its copy of the read file descriptor with close(), and the parent process should close its copy of the write file descriptor similarly
    4. After the fork, the child process should write several test messages to the pipe
    5. After the fork, the parent process should read several messages from the pipe and print them to the standard output stream (stdout)

    Pipe I/O, like file I/O in C, is comparatively low level. There are two approaches you can use to read from and write to a pipe. You could use the integer file descriptors returned by pipe() directly with the read() and write() system calls, which are documented at man 2 read and man 2 write. However you might find it easier to instantiate a FILE* stream variable, which allows you to work with your pipe like it was any other file object (i.e. you write to the pipe with fprintf() and read from the pipe with input operations such as fgets() or fscanf(). You can open a file stream from a file descriptor with the fdopen() function. You can Read more about the difference between file descriptors and file streams here. Of course, all of the above functions have informative man pages as well.

    Hint: Blocking functions that read the pipe will block indefinitely until they are able to return successfully or the write-end of the pipe is closed by the writer. The writer should write all of it's data and then call close() on its file descriptor when it is done. This allows you to write code such as while( fgets() != NULL ) to do open-ended reads, on the pipe, which will eventually fail and return once the write-end is closed.

    Build and run your program on your Raspberry Pi. As the answer to this exercise, copy and paste the terminal output that is produced by running your program.

  3. Now we are going to implement a program that provides a rudimentary active object. An active object is an executable context that performs services for other contexts upon request. In this case our active object will be a process that listens to one end of a FIFO, does some trivial computation when data is recieved, and prints the result of that computation to standard output. Any number of other processes can open the FIFO's read side and submit data, and the order that requests are scheduled depends on a race for the FIFO (recall that writes and reads are atomic for less than PIPE_BUF sized data).

    To begin, create a new file for your program code. This program should:

    1. Create a new FIFO with the function mkfifo(), giving it both read and write permissions (S_IRUSR | S_IWUSR per the table on page 295 of the LPI textbook). This is documented at man 3 mkfifo. Be careful not to confuse this function with the program called mkfifo that is documented at man 1 mkfifo (which is also the man page you get if you say just man mkfifo). Note: if the FIFO already exists, the call to mkfifo() will fail and errno (declared in errno.h, which you can find in the /usr/include/ directory on the machine on which you are compiling in case you would like to browse through it and the other files it includes, to see the symbols and functions it declares) will be set to EEXIST - in that case your program should simply continue to the next step.
    2. Open the FIFO for reading (as though it were a file) using fopen() (run the command man 3 fopen to see return values, parameters, etc. for that call).
    3. Enter a while loop that continually attempts to read from the FIFO until EOF is read. Whenever something is read, print it to the standard output.

    Build and run your program: it will create a FIFO that will appear in the filesystem. In a separate terminal, write some data into the FIFO and validate that it is read by your program correctly. For example, if your FIFO is named my_ao_fifo, the command echo "my_message" > my_ao_fifo will insert the string "my_message" into the FIFO. Test your program with larger amounts of input as well, for example: cat my_file.c > my_ao_fifo.

    As the answer to this exercise, describe how you tested your program and why you think it is working correctly.

  4. Now, we will make two modifications to your active object program. After opening a read stream to the FIFO, also open a write stream (i.e. fopen( fifo_name, "w" ). Your active object will not ever write to this stream, but the fact that it is held open will prevent your program from automatically quitting.

    Next, modify the active object to read from the FIFO with the function fscanf(). This allows you to perform formatted input. Your program should read integer inputs from the pipe, double the input, and then print out the original and the new values. Your input stream should not have any non-integer characters and you do not need to handle this case: test your program by passing integers into your FIFO.

    Finally for this exercise, we want to write a set of programs that concurrently use the FIFO to request that the active object perform work. Write two programs that open the write end of the FIFO with fopen(fifo_name, "w") and insert integer values with fprintf(). One of these programs should only insert even numbers, and the other should only insert odd numbers. Run these programs long enough to verify that they can both write concurrently to the FIFO without problem. Examine the output of your active object to verify correctness.

    As the answer to this exercise, please explain briefly (and show output fragments as evidence) why you think your program is working correctly based on that output.

  5. Create a new FIFO writer program that again shares the same FIFO with the active object reader program as in the previous exercise. Have your new FIFO writer program take the name of a standard file (i.e., one on disk) as a command line argument, open the file, and then repeatedly: use the getline library function to move a line of text out of the file into a dynamically allocated memory buffer, and then output the entire line into the FIFO, until the end of the file is reached. When the end of the file is reached, your new FIFO writer program should free the dynamically allocated memory buffer.

    Please see the getline man page for different ways to have getline manage buffer allocation, resizing, etc. as needed in case the length of a line from the file (or FIFO).

    Test your new FIFO writer program with the active object reader program from the previous exercise, using files with different line lengths. As the answer to this exercise, please explain briefly (and show output fragments as evidence) why you think your program is working correctly based on that output.

Things to turn in: