Should a Computer Grade Your Essays?
Updated: Jul 31
Technology has evolved drastically in the last 15 years, especially after the occurrence of the Internet. Since the Internet enabled people to connect with the `world`, the advancement and alteration gained momentum. Therefore, many people assert that the Internet was one of the breaking points in the history of mankind. The following break point would be the Artificial Intelligence (AI), which is frequently mentioned not only in the science fiction movies anymore. According to John McCarthy, who coined the term in 1955, defines it as “the science and engineering of making intelligent machines”.
Without a doubt, Artificial Intelligence would transform people’s lives in many ways. With the help of enhanced intelligent techniques and knowledge management systems, the main aim of the AI is to leverage decision making. As Kenneth C-Jane P. Laudon stated in their book titled ‘Essentials of MIS’, “AI technology consists of computer-based systems (both hardware and software) that attempt to emulate human behavior and thought patterns.” In other words, computers will make complex decisions by discovering patterns and behaviors in excessive amounts of data and generate solutions that are too complex for human beings.
One of the major areas that AI would have a great impact is education. According to the article titled ‘What is AIED and why does Education need it?’ by Joshua Underwood and Rosemary Luckin, “one of AIED’s (Artificial Intelligence in Education) scientific goals is to make computationally precise and explicit forms of educational, psychological and social knowledge which are often left implicit”. Although there are many notable technological successes and yet these technologies are not fully exploited for educational purposes, it has already started to affect the way that people learn and interact. For instance, in April 2013, EdX, a Harvard and MIT joint venture which they claim to be the only leading MOOC provider that is both nonprofit and open source, launched an essay scoring program. It enables students to revise, resubmit, and improve their grade as many times they want by using artificial intelligence technology.
However, although automated grading systems for multiple-choice and true-false tests are now widespread, the use of artificial intelligence technology to grade essay answers has not yet received widespread endorsement by educators and has many critics. As a matter of fact, a computer cannot measure accuracy, reasoning, adequacy of evidence, good sense, ethical stance, convincing argument, clarity, and veracity in essays with the current technology.
Les Perelman, the former director of writing at MIT, conducted studies on the Electronic Essay Rater (e-Rater) developed by the Education Testing Service (ETS). The e-Rater uses syntactic variety, discourse structure and content analysis and is based on natural language processing technology. Perelman analyzed the algorithms and unsurprisingly found out that AES programs cannot distinguish fact from fiction. After he examined e-Rater by answering the essay questions semantically wrong, he received the top score of 6. Since computer programs are not able to understand meaning, essay length is a proxy for writing fluency, conjunctive adverbs such as “however” and “moreover” use for complex thinking, and extensive words for vocabulary aptitude.
On the other hand, the free and open source EdX program seems to be more sophisticated than some other vendors such as e-Rater in the marketplace since it develops grading standards and to adapt to grading preferences. First, instructors grade 100 essays and these are input to the program. In the light of these guidelines, the tool creates its own grading metrics and follow the grading method preferred by the instructor. As Mark Shermis, University of Akron College of Education dean, stated that it is infeasible to give a comprehensive feedback, which is one of the most essential parts for students to learn the material covered in class, to every student when the classes are crowded.
Although there are some imperfections of AI currently, I believe that scientists who work on the field will remedy the deficiencies in the near future. As Stephen P. Balfour stated in his article titled ‘Assessing Writing in MOOCs: Automated Essay Scoring and Calibrated Peer Review’, “Machine evaluation is distinguishable from human evaluation because it is more consistent than human evaluation”. Even now, there are lots of benefits of automated essay grading software programs. These programs are not replacing human graders yet; it is supplementing them. It has been using in schools as a classroom tool which allows students to revise their essays before they turn them in to an instructor to be graded. According to an article in New York Times titled as ‘Essay-Grading Software Offers Professors a Break’, “There is a huge value in learning with instant feedback,” Dr. Agarwal said. “Students are telling us they learn much better with instant feedback.”
Since technology is constantly developing, these grading systems might take the place of graders in following years. In fact, these systems increase the productivity because human graders are not as quick as the grading software. The money spending on human graders would be less and the grades would be more standardized as well. Moreover, the other element that increases the productivity is the massively open online courses (MOOCs) that provided by some institutes. One of these institutes is EdX, its course offerings are free and students earn a certificate after they complete the program. There are hundreds of courses available for students from Introduction to Programming to Psychology. Although some authors asserted that MOOCs represented a decline in educational quality, its benefits are non-negligible. Thousands of students are able to take the courses that they want to and access whenever they need to.
To conclude, artificial intelligence will be keep changing the way people live enormously with the help of the exponentially growing technological advancements. Education is one of the major fields that AI would change in several ways. First, with the help of the automated grading software programs, students are able to get instant feedbacks that make them learn faster. Although these automated grading systems have limited capacity today, they will eventually evolve enough to take away the jobs of most of the graders. In addition to that, free online courses are widely available to students from all around the world which provides flexibility. It’s even possible that these online courses might take the places of the most schools one day.
Assessing Writing in MOOCs: Automated Essay Scoring and Calibrated Peer Review by Stephen P. Balfour, Ph.D. Available at: http://www.rpajournal.com/dev/wp-content/uploads/2013/05/SF4.pdf
A report for the UK’s TLRP Technology Enhanced Learning – Artificial Intelligence in Education Theme. May 2011 by Joshua Underwood and Rosemary Luckin, The London Knowledge Lab. Available at: http://tel.ioe.ac.uk/wp-content/uploads/2011/06/telaied_whyaied.pdf
Essentials of MIS, Eleventh Edition by Kenneth C. Laudon and Jane P. Laudon