An Item Response Theory for peer assessment

This paper proposes an Item Response Theory for peer assessment. The proposed model is a modified Graded Item Response model which incorporates the assessor's assessment criterion parameters. Its advantages are as follows: 1. The learners' abilities are evaluated on the same scale even if the assessors have different assessment criteria. 2. The learners' abilities are evaluated by taking into consideration each assessor's reliability, which produces reliable evaluation results. 3. Parameters from missing data can be easily estimated. 4. From 1-3, the parameter estimation accuracy is improved. 5. Characteristics of the tasks and assessors can be analyzed from the estimated parameters. Moreover, the two indices, "strictness of the assessment criterion" and "consistency of the assessor," are proposed based on the model. The proposed model was applied to actual data, which demonstrated its effectiveness.