Written by Dr. James S Plank, Gradescripts are an amazing way to check your current grade on an assignment. There are (usually) 100 test cases which it will run your program through, comparing it to the solution executable. For each test case, you must match the output exactly. This means that missing a newline or adding an extra space will flag it incorrect!

NOTE: This is a guide exclusively for the UTK Computer Science program, as it requires SSH access to UTK's Hydra/Tesla computers.

SSH in to Hydra or Tesla. The script path is determined by what class you are in, and what lab you are on.

The general directory where the gradescripts are in can be generated based on this (Assuming Dr. Plank doesn't change the directory names any time soon):

/home/plank/csXXX/Labs/LabX/

In each directory, there is a gradeall and a gradescript shell script file, as well as a Gradescript-Examples directory, which contains the test cases.

NOTE: Sometimes, the directory names are lowercase. So you'll have to let auto-completion help you out.

Example: If I was in CS140 and was on Lab 4...

/home/plank/cs140/Labs/Lab4/

Example: If I was in CS360 and was on Lab 8...

/home/plank/cs360/labs/lab8/

As noted before, cs360's labs are in "labs", not "Labs".

"Okay cool Clara, but how do I run these?"

Copy gradeall and gradescript in to your own lab directory (with your executables). Then you can run them in one of two ways:

Test my programs with ALL test cases

Just run this:

./gradeall

That was easy...

Test my programs with a specific test case

Lets say you got test case 24 incorrect... well you can test that one individually via:

./gradescript 24

So let's say you messed up on a test case. That doesn't really say much. There are two things you can do:

Method 1: Compare your output with the solution output

When you run gradescript, it generates files in the following format:

tmp-???-test-stdout.txt
tmp-???-test-stderr.txt
tmp-???-correct-stdout.txt
tmp-???-correct-stderr.txt

The ??? is replaced with the number of the test case you just ran.
You can compare your output to the solution's output in 2 ways (there are more but let's keep it simple).

Diff

diff -y file1 file2

NOTE: I'd recommend pairing this with less since your program's output may be larger than your terminal. Preview (Note the third line is different):


Vimdiff

vimdiff file1 file2

Vimdiff highlights the lines and characters that are different, so you'll instantly know what is different between your output and the solution executable's output.

Preview (Again, third line is different):


You can tell if X lines are the same via vimdiff if it says something like "190 lines: 255" and is gray like in the picture above. Vimdiff is basically telling you "These lines are the same and it is pointless for me to print them".

To Quit Vimdiff, use :qa (Quit All)

Method 2: See how the gradescript runs your program

This method varies per lab. The test cases can vary:

1. The file is read in to the program via stdin
2. The file is read as command line arguments for the application
3. The file is used in command line arguments for the program to read in via ifstream, FILE*, etc.

Go in to the Gradescript-Examples directory (found via the path above) and then use cat or vi to view the test case files directly. It should be obvious, when you get there, what files are used in test cases. If your program uses command line arguments, the contents of the file may be command line arguments.

For instance, CS140's Bigwhite executable for Lab 2 accepts input like this:

./bigwhite width height

The lab has a Gradescript-Examples directory that looks like this:


Opening 021.txt via vim gives us this:

14 14

It's obvious, based on the input method above (The lab accepts 2 command line arguments) that this file specifies command line arguments. Therefore, we can run the test ourselves without the gradescript:

./bigwhite 14 14

"Clara, what about those pgm files prior to 021.txt?"

Yes, the lab I used as an example has 5 executable files. Bigwhite accepts command line arguments. But the one before it (called pgminfo) reads a PGM file in from stdin.

You know what this means? Those first 20 files are PGM files read directly in to your executable via stdin. You can look at the files manually and see how it's structured. Some files purposefully contain errors that you can probably catch via looking at it manually.

Gradescripts are also used by TAs to grade assignments. Though it doesn't completely determine your grade, you'll know how you will generally do on the lab before even submitting it. You may still get points taken off for:

1. Lack of good commenting
2. Bad coding practices
3. Inefficient code (too slow)
4. Hard-coding solutions based on the gradescript (generally results in a 0 for a lab)

I'd recommend going here for guides on writing and commenting clean code: http://web.eecs.utk.edu/~bvz/cs140/labs.html#readable