Assessing Writing in MOOCs: Automated Essay Scoring and Calibrated Peer Review

Balfour, Stephen P.
June 2013
Research & Practice in Assessment;Summer2013, Vol. 8 Issue 1, p40
Academic Journal
Two of the largest Massive Open Online Course (MOOC) organizations have chosen different methods for the way they will score and provide feedback on essays students submit. EdX, MIT and Harvard's non-profit MOOC federation, recently announced that they will use a machine-based Automated Essay Scoring (AES) application to assess written work in their MOOCs. Coursera, a Stanford startup for MOOCs, has been skeptical of AES applications and therefore has held that it will use some form of human-based "calibrated peer review" to score and provide feedback on student writing. This essay reviews the relevant literature on AES and UCLA's Calibrated Peer Reviewâ„¢ (CPR) product at a high level, outlines the capabilities and limitations of both AES and CPR, and provides a table and framework for comparing these forms of assessment of student writing in MOOCs. Stephen Balfour is an instructional associate professor of psychology and the Director of Information Technology for the College of Liberal Arts at Texas A&M University.


Related Articles

  • Looking at MOOCs Rapid Growth Through the Lens of Video-Based Learning Research. Giannakos, Michail N.; Jaccheri, Letizia; Krogstie, John // International Journal of Emerging Technologies in Learning;2014, Vol. 9 Issue 1, p35 

    Video learning is becoming an increasingly important part of contemporary education. In the decade there has been an increase of many and diverse forms of research efforts on video learning. This paper focuses on the video learning research of the last years based on 166 peer-reviewed published...

  • Self and Peer Assessment in Massive Open Online Courses. Admiraal, Wilfried; Huisman, Bart; Saab, Nadira; Smit, Ben; Theeuwes, Brigitte; van De Ven, Maarten; van Der Rijst, Roeland; van Driel, Jan; Verstelle, Marja // Proceedings of the European Conference on e-Learning;2014, p1 

    Open online distance learning in higher education has quickly gained popularity, expanded, and evolved, with Massive Open Online Courses (MOOCs) as the most recent development. New web technologies allow for scalable ways to deliver video lecture content, implement social forums and track...

  • Remarks on MOOCS and Mini-MOOCS. Spector, J. // Educational Technology Research & Development;Jun2014, Vol. 62 Issue 3, p385 

    In an earlier issue of ETR&D, the editors provided an hierarchical framework of components to support learning and instruction. That hierarchy included information objects, knowledge objects, learning objects, instructional objects, courses, programs and ongoing efforts, with each subsequent...

  • The influence of MOOCs to enhance graphic design education. McNamara, Patrick // Art, Design & Communication in Higher Education;2015, Vol. 14 Issue 1, p57 

    There is increased pressure on higher education (HE) graphic design lecturers to provide a more engaged platform for the evolving digitally engrossed student. This article explores Massive Open Online Courses (MOOCs) as an educational experiment to help envision changes in HE design teaching....

  • FROM THE EDITOR. TRITELLI, DAVID // Liberal Education;Fall2013, Vol. 99 Issue 4, p4 

    An introduction is presented in which the editor discusses reports within the issue on topics including massive open online courses (MOOC), difference between liberal education and online discussions and broadcast lectures, and the transformative education offered by MOOC.

  • MASSIVE OPEN ONLINE COURSES: AN OVERVIEW AND RESEARCH AGENDA. Murphy, Jamie; Hofacker, Charles; Brogan, Patricia; Kim, Paul // AMA Summer Educators' Conference Proceedings;2013, Vol. 24, p260 

    An abstract of the article "Massive Open Online Courses: An Overview and Research Agenda" by Jamie Murphy, Charles Hofacker, Patricia Brogan, and Paul Kim is presented.

  • Will Free Online Courses Ever Replace a College Education? Winterhalter, Benjamin // Distance Education Report;9/1/2014, Vol. 18 Issue 17, p2 

    The article offers information on the difference between free online courses and college education and mentions the possibility of replacement of college education by online education.

  • Special Issue: Research into Massive Open Online Courses. Siemens, George // International Review of Research in Open & Distance Learning;Nov2014, Vol. 15 Issue 5, preceding p2 

    An introduction to the journal is presented in which the editor discusses the special issue on massive open online courses (MOOCs) and its impact on education.

  • MOOCs. Milman, Natalie B. // Distance Learning;2012, Vol. 9 Issue 4, p91 

    The article presents questions and answers related to Massive Open Online Courses (MOOC) including what these are, what organizations are offering MOOCs, and the what type of content is best taught in a MOOC.


Read the Article


Sorry, but this item is not currently available from your library.

Try another library?
Sign out of this library

Other Topics