Homework 13 (Project 4) Feedback
Paul Krzyzanowski
December 4, 2020
Grades
Grades for homework 13 (project 4) are on a 10-point scale.
Roughly speaking, grades fall in the following buckets:
|:—|:—-|
| Grade | Description |
| 10 | Superb work, no glitches: A+ |
| 9 | Perfectly solid but perhaps a single test case that didn’t work properly or incorrect command invocation produced a cryptic message: A ||
| 8 | Really good work - maybe a couple of glitches: B++ |
| 6–7 | Generally working assignment but quite a few conditions that aren’t handled: B..B+|
| 4–5 | Something’s seriously wrong. pow-create may be generating invalid data or pow-check may not be working properly: C+ |
| 2–3 | Far too many things wrong for this to even be considered a functional piece of work: C |
| 1 | You got credit for submission but the program either didn’t prodcue the right data or was just too buggy: D |
You can get an approximate feel for your grade on a 4-point GPA scale (A=4) via the formula (⅓p + 1), where p is your non-zero score.
Deductions
You lose points for various errors and I don’t bother with fractional points.
Errors also include deviations from the specified output formats. These cause my tests to fail and required effort on my part to patch your program to generate the right output (if possible).
Command names
You lost a point if you submitted programs with names such as pow-create`.py`, `powcreate`, orpow-create.sh. All these required me to do extra work to run them against my scripts.
Command usage
Possibly the most important aspect of programming with security in mind is to validate all inputs. I did not check how you alert the user to things such as unreadable files but did try a few variations of improper command usage. If you didn’t handle these gracefully and caused a runtime environment to dump some junk, you lost a point.
PoW generation
I tested your pow-create output by running your output headers against my test program. If your headers had the wrong names, I fixed your code so you’d have a chance to not lose all credi. Unfortunately, this cost you two points.
I tested pow-create header generation on four sample files (you can download them).
timeout 600s ./`pow-create` 5 $testfiles/magnacarta.txt >results/out-00.txt
timeout 600s ./`pow-create` 15 $testfiles/magnacarta.txt >results/out-01.txt
timeout 600s ./`pow-create` 15 $testfiles/colossus.txt >results/out-02.txt
timeout 600s ./`pow-create` 20 $testfiles/eggs.txt >results/out-03.txt
timeout 600s ./`pow-create` 20 $testfiles/spools.jpg >results/out-04.txt
The results were checked by running my version of pow-check using the same input file and validating your headers.
You lost up to 4 points for bad PoW generation. You lost up to 2 points for mistakes in other headers. The most common mistake was
counting the number of leading zeros incorrectly: it should be value actually present in your hash value, not the value requested.
If a 20-bit proof-of-work generation took over a minute, you lost a point. I used a Raspberry Pi 4 for running the programs, which is 3.2x slower than the iLab systems. The vast majority of submissions did not exhibit atrocious performance, although a few did reach my 10-minute timeout.
PoW checking
I ran your validation against data that your pow-create program generated (which I expect you did).
To test whether your pow-check program works, I ran it against a set of good headers and a set
of headers that had a single component whose value was modified to be incorrect.
You lost points if you didn’t identify the bad data or if your runtime environment generated some crap.
You may lose a maximum of 4 points for a bad pow-check program.
Credit for effort
I truly hope that most of you were able to finish the assignment quickly. The majority of students earned grades of 8 or higher. These were submissions that ran properly (for the most part) and did not require any significant effort on my part to get to run. If this assignment took you many hours, take this as a sign that you need to brush up on your programming.
Some assignments simply didn’t work. The programs would either crash or not generate valid proof-of work values or not be able to validate valid headers. Beyond the 1 point, you get no credit for effort. If it took you many hours to do this assignment and you received a poor grade, you really need to spend more time getting comfortable with programming and debugging.
Mistakes
If you believe I made a mistake in your grading, please email me before Monday and identify the precise item that you believe was misgraded.
Most submissions ran without problems. In some cases, I had to create pow-create and pow-check scripts or modify your code so it would run with my tests.
It’s possible that I ran your program incorrectly and you lost a lot of points because of that. If so, please let me know.
Do not send me screen shots. Any messages with screenshots will be deleted immediately.
- Copy and paste the command you ran and the output you get.
- Identify why the output is correct but was misgraded.
Before contacting me:
Run
pow-createorpow-checkon all the sample files. For example, there were a couple of submissions that failed on the filespools.jpg. As far as I could guess, for some unfathomable reason, the program insisted on validating that the input data was text-only.Validate your
pow-createoutput manually withopensslexactly as described in the assignment writeup.
Please do not contact me if you don’t honestly believe I made a mistake but are simply hoping to get more points.