Research summary
Reactions to Grades
We study how students respond to randomly more lenient or harsh grades.
Main takeaway
The evidence suggests that a reduction in perceived grading standards harms students.
Students who receive a favorable grader-induced grade become more confident about the next short writing assignment, yet perform worse on it.
Experimental Design
The study uses 8,498 short writing assignments submitted by 3,397 students in two large introductory economics courses. Each assignment was graded on a 100-point scale using the same ten-item rubric.
Each SWA asked students to analyze an economic problem, answer a multi-part prompt in essay form, and attach a handwritten graph. The assignment counted for 10 percent of the course grade, with 70 points based on the correctness of the content and 30 points based on writing quality.
View the graph from a sample assignmentBecause students did not choose their graders, differences in grader leniency generate grade luck within the same course and assignment.
How much graders differ
Each dot is a grader-semester combination, measured relative to other graders on the same assignment.
Estimation and empirical strategy
Isolating grader-induced grades
We compare students within the same course, assignment, and semester and use the randomly assigned grader indicators as instruments for the grade received. This isolates the portion of a grade generated by which grader happened to evaluate the submission.
Why UJIVE?
The analysis has 48 grader-semester instruments. With many instruments, conventional two-stage least squares can be biased toward the ordinary correlation between grades and outcomes. The unbiased jackknife instrumental variables estimator, or UJIVE, removes each observation's own contribution when constructing its instrumented grade and is designed to reduce this many-instrument bias.
Checking the design
After the semester, two additional graders independently rescored every submission without seeing the original grade. These blind regrades let us verify that the component of grades generated by the original grader is unrelated to the underlying quality of the work. Interpreting the UJIVE estimates also requires random grader assignment, an exclusion restriction, and average monotonicity. We examine these assumptions in the paper.
Results
Students treat grader-induced variation as an informative signal, but the response is confined to similar assignments.
Students who receive higher grader-induced grades contest them less, perceive the grading as more lenient, and become more overconfident about the next assignment. These higher grades partly undo themselves: approximately a quarter of the gain is offset by lower performance on the next assignment. Exam confidence and performance are unchanged.
Explore individual outcomes
Explore the nonlinear estimates
Our main estimates assume that one point up has the same-sized effect as one point down. We also estimate the full dose-response because students may react more strongly to increasingly harsh grades and ignore increasingly lenient grades, or vice versa. The nonparametric estimates closely reproduce the linear estimates for the outcomes we observe.
Explore heterogeneity across students
The average response is not driven by a specific group of students. Choose a student characteristic, then select an outcome for more detail.
Some subgroup results are suggestive, but the equality tests and multiple-testing adjustments caution against placing too much weight on any individual pattern.
Interpreting the Results
Receiving a higher grade increases overconfidence about performance on the next short writing assignment, suggesting that students treat the grade as an informative signal. The question is what they learn from it.
The pattern is most consistent with students updating their beliefs about the grading standards applied to the writing assignments.
Students do not appear to interpret grader-induced variation as random grade luck. The absence of corresponding effects on exam confidence, exam-related effort, or exam performance suggests that they do not broadly update their beliefs about their overall course performance.
Why it matters
If students mistake idiosyncratic variation in grades for evidence that the standards applied to similar assignments are low, correcting that perception may improve subsequent performance. The findings point to a potential role for clearer information about those standards. Another possibility is to separate formative feedback from the numerical score, for example by providing comments before releasing the score.
Limits to the interpretation
We do not directly observe students' beliefs about grading standards, and the design cannot fully distinguish among reassurance, reduced perceived need to improve, or other responses to favorable feedback. The setting is a large introductory economics course with short writing assignments, numerical rubric scores, and no written comments from graders. Student responses may differ in other settings.
Frequently asked questions
How can you separate the effect of the grade from the quality of the essay?
Assignments were anonymized and assigned to graders at random within each course and assignment. Two independent blind regrades provide a separate measure of the underlying quality of the work.
Were graders using different standards or rubrics?
No. All graders received the same training and grading guidelines and used the same ten-item rubric. They nevertheless differed systematically in leniency.
Could students simply feel better about their own ability?
That is possible in principle. However, the response is specific to the writing assignments. Confidence rises for the next writing assignment, while exam confidence and performance remain unchanged. This task-specific pattern is more consistent with students updating beliefs about grading standards.
Do students work less after receiving a favorable grade?
The effort estimates are generally negative, but they are small and statistically insignificant. We cannot determine whether the decline in performance reflects reduced effort, reassurance, or some other response to favorable feedback.
Does this mean higher grades are always harmful?
No. The analysis identifies responses to the range of grades induced by grader leniency in this setting. Responses to larger or more salient grade differences, detailed written comments, or higher-stakes assessments could be different.
Was the study pre-registered?
Yes. The data collection was registered with the AEA RCT Registry (AEARCTR-0010898).