This page describes the format of the test cases posted for CSE 241 labs and provides hints on how to interpret the output of the auto-grader when it fails your code.
NB: the posted test cases for each lab are identical to the pre-turn-in cases tested by the auto-grader. However, we reserve the right to add additional post-turn-in test cases, which will not be disclosed to students ahead of the due date, to evaluate and grade your code. Moreover, passing all the pre-turn-in test cases in the auto-grader does not guarantee that your code is correct. We may read your code after turn-in in order to determine its correctness.
The test cases used by the auto-grader for each lab will be posted along with the lab, so that you can use them locally to debug your code. Test cases include two parts: a test case file that describes the inputs and expected output for each test case, and a directory (typically named after the lab) that provides input files needed for some test cases.
The test case file contains one or more test cases, each beginning with a comment line (starting with a #) and ending with two slashes "//". The first line after the comment is the input specification, which gives the list of arguments passed to the program. Arguments that start with @DIR/ refer to files; after deleting this prefix, the remaining path points to the location of the file in the test case archive.
The remaining lines of each test case give the expected output for the test case. This may not include every line printed by your program; in some cases, only the parts essential for checking correctness are given.
For each lab, the auto-grader uses a lab-specific script that compiles your program, runs it on one or more test cases, parses the output, and compares it to the contents of the test case file.
If your code fails to compile, the auto-grader will typically report a compilation error. (We have occasionally seen other errors, such as failure to find main(), as the result of a compile failure, but these occurrences should be quite rare.) If it compiles but produces wrong output for some test case(s), the offending output lines will be shown, along with some information about the correct output.
A type of output used for most (but not all) labs in the case of an error is the "diff", or difference between your result and the correct result. A diff contains lines prefixed with "+" or "-". The former indicates that you printed a line that is not present in the correct output, while the latter indicates that you did not print a line that is present in the correct output. Lines with neither prefix are present in both your output and the correct output.
For example, if the correct code should print
b
c
d
e
k
c
e
-b
+k
c
-d
e