<instructions>Your goal is to analyze the following comment and estimate how highly it will be upvoted by the Hacker News community.</instructions><comment><author>bobbiechen</author><text>When I was a TA at CMU, we used Gradescope &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.gradescope.com&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.gradescope.com&amp;#x2F;&lt;/a&gt; for this. Every exam would be scanned and divided into problems (based on a predefined template - fixed page space for answers).&lt;p&gt;Then, each problem was assigned to a TA. Either there&amp;#x27;s a predefined rubric, or you create it as you go (-1 point for mistake X, half credit for mistake Y, etc.). There&amp;#x27;s a pretty slick interface where you just read the answer, and use keyboard shortcuts to apply the relevant deductions.&lt;p&gt;It still has the issue that every time you change the rubric, you&amp;#x27;d need to go back and re-do previously-graded instances of that problem. But it was way faster and (equally important) less tiring.</text><parent_chain><item><author>cvwright</author><text>My strategy was to, like you said, grade problem by problem.  Then for each problem, first find all those who got full marks.  Then group the others into piles based on what mistakes they made.&lt;p&gt;This ensures that everyone who made the same mistake(s) gets the same grade.  It also tends to shuffle the order of the exams after every problem.&lt;p&gt;Obviously you don’t need this strategy for simple multiple choice questions, and it’s probably also not a great fit for long-form essays.  But it worked great for technical short answer problems in CS and security.</text></item><item><author>lqet</author><text>I work in academia. When we grade exams, the order of the exams on the stack is the order in which they were collected in the room (people can sit wherever they like). For grading, we are usually 5 people in a single room, and everyone grades a specific exercise for consistency. The exams are getting shuffled &lt;i&gt;heavily&lt;/i&gt;, with everyone just grabbing stacks, looking for exams where &amp;quot;their&amp;quot; exercise was not yet graded, and taking them out. So basically, the order in which we grade exams can be considered random.&lt;p&gt;However, I also grade weekly exercise sheets during the semester, and these are committed into a repository, where each student has a folder that... begins with the first letter of their first name. &lt;i&gt;Everyone&lt;/i&gt; I have ever worked with acknowledges that you have to shuffle the order in which you grade these submissions each week, for fairness. Several effects come into play: (1) your are usually less tired at the beginning, (2) your mood gets better during the last 2 sheets because you know you are done soon, (3, and crucially) at the beginning, you have not yet seen all the common errors &amp;#x2F; developed a &amp;quot;feeling&amp;quot; for them, and you might thus miss them in early submissions, but spot them immediately in later submissions.&lt;p&gt;Another alphabetic effect: In elementary school, my name was on top of the list of students in my class. I remember that I often had to do some special job simply because I was the first name on this list (for example, carry a group ticket when we visited some museum, keep track of something, be the first at something where nobody wanted to be the first, with everyone watching, be the first to be graded in PE, again with everyone watching, etc.). As a fairly shy kid, this already annoyed me in first grade.</text></item></parent_chain></comment><story><title>U-M finds students with alphabetically lower-ranked names receive lower grades</title><url>https://record.umich.edu/articles/study-alphabetical-order-of-surnames-may-affect-grading/</url></story>
<instructions>Your goal is to analyze the following comment and estimate how highly it will be upvoted by the Hacker News community.</instructions><comment><author>jcla1</author><text>This sounds like an organisational nightmare to be honest. You&amp;#x27;d be going through the pile of exams multiple times (at least twice) and what do you do if there are multiple mistakes that are common in a single exam question?&lt;p&gt;Also: if you&amp;#x27;re sorting into &amp;quot;mistakes piles&amp;quot; for single exercises, how can you parallelise marking of separate and independent questions?</text><parent_chain><item><author>cvwright</author><text>My strategy was to, like you said, grade problem by problem.  Then for each problem, first find all those who got full marks.  Then group the others into piles based on what mistakes they made.&lt;p&gt;This ensures that everyone who made the same mistake(s) gets the same grade.  It also tends to shuffle the order of the exams after every problem.&lt;p&gt;Obviously you don’t need this strategy for simple multiple choice questions, and it’s probably also not a great fit for long-form essays.  But it worked great for technical short answer problems in CS and security.</text></item><item><author>lqet</author><text>I work in academia. When we grade exams, the order of the exams on the stack is the order in which they were collected in the room (people can sit wherever they like). For grading, we are usually 5 people in a single room, and everyone grades a specific exercise for consistency. The exams are getting shuffled &lt;i&gt;heavily&lt;/i&gt;, with everyone just grabbing stacks, looking for exams where &amp;quot;their&amp;quot; exercise was not yet graded, and taking them out. So basically, the order in which we grade exams can be considered random.&lt;p&gt;However, I also grade weekly exercise sheets during the semester, and these are committed into a repository, where each student has a folder that... begins with the first letter of their first name. &lt;i&gt;Everyone&lt;/i&gt; I have ever worked with acknowledges that you have to shuffle the order in which you grade these submissions each week, for fairness. Several effects come into play: (1) your are usually less tired at the beginning, (2) your mood gets better during the last 2 sheets because you know you are done soon, (3, and crucially) at the beginning, you have not yet seen all the common errors &amp;#x2F; developed a &amp;quot;feeling&amp;quot; for them, and you might thus miss them in early submissions, but spot them immediately in later submissions.&lt;p&gt;Another alphabetic effect: In elementary school, my name was on top of the list of students in my class. I remember that I often had to do some special job simply because I was the first name on this list (for example, carry a group ticket when we visited some museum, keep track of something, be the first at something where nobody wanted to be the first, with everyone watching, be the first to be graded in PE, again with everyone watching, etc.). As a fairly shy kid, this already annoyed me in first grade.</text></item></parent_chain></comment><story><title>U-M finds students with alphabetically lower-ranked names receive lower grades</title><url>https://record.umich.edu/articles/study-alphabetical-order-of-surnames-may-affect-grading/</url></story>