Collaboration in writing assessment: insights into novice and experienced iranian EFL raters’ critical thinking and criteria

Abstract

There is a growing body of literature on effective techniques through which critical thinking (CT) can be fostered in an educational setting. In a way that agrees with teaching as an interactive process, considerable debate has taken place over enhancing teachers’ CT abilities in recent years. Concerning the positive effects of collaboration on CT, the present study examines whether the frequency of CT components among novice and experienced Iranian EFL raters is different while assessing a passage individually and collaboratively (in pairs). For this end, a mixed-method approach was adopted. Writing assessments made by two groups of novice and experienced raters were coded in search of statements classified as one of the five components of CT (Stapleton, 2001). The data obtained included recorded think-aloud protocols and written comments made by raters while assessing 3 passages written by IELTS examinees. Then, novice and experienced raters were compared based on the total frequency of CT components used while assessing individually and collaboratively. Afterwards, their criteria for scoring a passage (without using a rubric)were attempted to be established. Findings revealed that collaboration in writing assessment improves raters’ CT skills irrespective of their experience level, while CT indices were significantly higher for experienced collaborative pairs. Although, novice and experienced raters’ CT skills were not significantly different while assessing individually.With regard to the criteria applied by novice raters, qualitative analysis revealedcentral concerns about the text surface level. Actually, they were believed to be in the process of forming their personal strategies and establishing their own criteria as raters. While experienced raters went one step ahead of syntax into semantics.

There is a growing body of literature on effective techniques through which critical thinking (CT) can be fostered in an educational setting. In a way that agrees with teaching as an interactive process, considerable debate has taken place over enhancing teachers’ CT abilities in recent years. Concerning the positive effects of collaboration on CT, the present study examines whether the frequency of CT components among novice and experienced Iranian EFL raters is different while assessing a passage individually and collaboratively (in pairs). For this end, a mixed-method approach was adopted. Writing assessments made by two groups of novice and experienced raters were coded in search of statements classified as one of the five components of CT (Stapleton, 2001). The data obtained included recorded think-aloud protocols and written comments made by raters while assessing 3 passages written by IELTS examinees. Then, novice and experienced raters were compared based on the total frequency of CT components used while assessing individually and collaboratively. Afterwards, their criteria for scoring a passage (without using a rubric)were attempted to be established. Findings revealed that collaboration in writing assessment improves raters’ CT skills irrespective of their experience level, while CT indices were significantly higher for experienced collaborative pairs. Although, novice and experienced raters’ CT skills were not significantly different while assessing individually.With regard to the criteria applied by novice raters, qualitative analysis revealedcentral concerns about the text surface level. Actually, they were believed to be in the process of forming their personal strategies and establishing their own criteria as raters. While experienced raters went one step ahead of syntax into semantics.