Grammatical accuracy & length across computer-assisted and pencil– and-paper assessments: Variation in advanced-level EFL writing

The sweeping trend of communication technologies increasingly makes computers and computer-based operations an indispensable part of our everyday lives. Education as a institution deeply rooted within the context of the broader community can, therefore, hardly afford to do without the digital era. Assessment, as a subcategory of education, is expected to be under the influence of the same trend. The present study, focusing on the EFL situation in Iran compared two equal-sized advanced learners’ each containing 20) writing accuracy and length. One group employed the traditional paper-and-pencil PPT) mode while the other used computer-assisted language testing CALT). Results indicated a significantly more accurate writing for the CALT group, while a longer writing production for the PPT group. Implications of the study are discussed.


Introduction
Writing as a means of communication occupies a central position in the professional and personal spheres of human life. Ina addition to being a vehicle of communication for formal and official correspondence, it is so closely associated with the academic skills. In the latter vein, it requires not only the mastery of language skills e.g. the right style, genre, ec), but also a knowledge of content. In being so, it is supposed to be quite a time-consuming activity Dastjerdi & Samian, 2011 Through making use of language's depth and originality [in writing], the personal growth process is recorded by individuals. This enriches their lives and vitalizes the employment of language. Composition can also be regarded as a combined representation of all four language skills consisted of writing, reading, speaking, and listening) and can be an appropriate means of reflecting the overall language competence and proficiency. But, due to the complex nature of the composition's content, it is a particularly difficult subject for students.
It can also be argued that all four skills are involved interactively in composition writing, and for this very reason can be a good source of judgment as to the learners' overall language competence and proficiency. According to Obemeata 1995), English as a foreign language functions as a system of communication which is superimposed upon an already existing medium of communication, that is the first language. Also he argues that writing and reading are demanding and take more effort for instruction than listening and speaking in that the latter develops naturally like a growing child.
It can be argued that writing is the most complicated skill of all that necessitates complicated cognitive activity and a simultaneous control of all variables see Kolade, 2012). Here are some of the factor that are presumably effective and helpful in writing improvement:


Awareness of the students' needs Tribble 1996),  Purposefulness of the writing and alignment with the students' interest Bryan, 2004)  Quality enhancement of the thinking process on the students' part Barrass,2005)  The possibility of drafting and self-revision Doughty and Pica, 1986)  Emphasizing the process of writing which encourages autonomy Vanessa, 2004)

Computer-assisted writing in ESL
Computers were used in the classrooms as early as the 1980s when the writing teachers in American schools and universities applied word-processing to facilitate students' writing outputs. Kulik 2003) indicates improvements in the performance of students using a word processor. Despite the fact that findings do not lend a high support for impacts in the majority of word processing researches, the impacts were still large enough to be viewed as educationally meaningful ibid). As Pennington 2003) declares, writing in a computer context is more advantageous for students than writing by penand-paper due since the former is more practical and convenient and quite flexible. According to Kupelian 2001), an effective educational resource, the electronic tool has not only changed the composing process, but also strengthens participation in writing activity. He goes on to lay emphasis on the non-threatening atmosphere provided by the computers in which writers get encouraged to be productive in terms of language. Another reason is that the Web provides a field for writers to present their work to a real and larger audience that spreads beyond classroom and school frames Karchmer, 2001). One of the e-mediums which can be used in language learning and teaching writing is weblog or blog. According to Bella 2005) a weblog, also called a blog, is an easily created and updateable website that allows people to publish to the Internet instantly. Soares 2008) asserts that the blogs because of their public and interactive aspects can be applied for educational purposes such as students' interests and class content. When students realize that they put their work on the weblog for readers in the real world, they are motivated to write Leibowitz, 1999). Ward 2004) sees blogs as useful and helpful for language learning and teaching. When students are writing large reports, essays, or research papers, ask them to submit their rough drafts onto their blogs. Then comment on the papers constructive criticisms. This is not to say that personal, face-to-face assistance is not needed. The advantage of using a blog is that all comments and developments are archived for study and reference" p. 154).
Halic, Lee, Paulus and Spense 2010), in this concern, have adeptly noted that: The popularity of blogs among young people has made the map pealing to educators seeking to integrate computer-mediated communication CMC) tools at the university level. These tools are seen as having the potential for enhancing student engagement and providing an environment for collaboration and creation of knowledge p. 1).
In the same vein, Campbell 2003) discusses the potentiality and practicability of integrating weblogs into the educational context, particularly in the field of language teaching.
A study was conducted by Zare-ee, Shekarey and FathiVajargah 2009) among the Iranian undergraduate and graduate students in Iran on the use of technology in education. In their study, students' views on the application of blogs in teaching-learning processes and their actual use of blogs were investigated. The participants' familiarity with blogs which were written in English and Persian was studied and their views about the usefulness of blogs as an instructional tool in higher education were declared. The findings of the study showed that blogs were actually used less than it is believed they should be. It was one of the studies done in Iran related to the use of the blog in education at university level. The study did not investigate the students' writing performance and was aimed to view the students' perceptions about blog in teaching-learning process. Also, the research methodology of that study was qualitative survey) and not experimental.

Pen-and-Paper versus Technology-Based Writing
Lam and Pennington 1995) conducted a study in which they compared two groups of secondary students who used penand paper and computer for writing. Their writings were assessed by raters based on ESL Composition Profile 1981) and the results presented that that overall, the writing of students in the Computer group was better than that of the one in a Pen group and they were hugely significant differences in every aspect of their writing except organization and content with the superior exhibition of performance belonging to the Computer group.
A few studies indicate the positive effect of using weblogs to enhance students' writing development.  reported that students enjoyed the experience of writing in a blog with signs of improvement due to a high level of motivation. Though they are not intended for language learning and education purposes, blogs seem to have a great potential for EFL pedagogy Kavaliauskienė & Mažeikienė, 2006). However, most of the researchers in this field believe that the full potential of writing class blogs still needs to be learned and explored and also a lot of work needs to be done in order to effectively use blogs in the writing class. Aside from other pedagogical merits, writing improvement and enhancing motivation, there are other effects on learners such as fostering autonomy, peer and group work and feedback provision.
Similar findings were reported in studies by Russell 1999) and Russell and Plati 2001), who showed that students who were accustomed to writing using a computer performed better on the open-ended tests when they wrote using a computer than when they wrote by hand. Pommerich 2004) investigated the item-level mode effects of English, reading, and science reasoning tests in grades 11 and 12 and found that the mode effect in general was small. Two comparability studies on the online versions of the NAEP math and writing tests showed that the paper group significantly outperformed the computer group in the eighth-grade NAEP mathematics test, but no mode effect was found for the eighth-grade NAEP essay test Sandene, Horkay, Bennett, Allen, Braswell, Kaplan, 2005). The NAEP studies also found that students' familiarity with computers was related to their performance. Particularly, hands-on measures of keyboarding skill were found to be a significant predictor of students' performances on the NAEP online writing test Sandene, et al., 2005).
Experimenting the effect of different modes i.e. conventional and computer-based) on children's performance, Barnes 2010) could prove a significant improvement for children; however, a lot seemed to depend on the content and age. With this background in mind, the present study seeks to answer the following research questions: R.Q. 1. Does advanced EFL learners' writing accuracy vary significantly across pencil-and-paper and computer-assisted testing?
R.Q. 2. Does advanced EFL learners' writing length vary significantly across pencil-and-paper and computerassisted testing?

METHOD Participants
Participants in the study were two equal-sized groups each consisting of 20 all male students. Both groups were in fact classes in an institute in Shabestar, a township in Eastern Azerbaijan Province in Northwest of Iran. Their ages ranged between 22 and 32,. They had already passed conversational courses for at least 10 semesters and were currently doing their advanced level spoken English courses. As participants, they were given Oxford Placement Test based on which they were ranked as advanced level students. Their first language was Azerbaijani as they belonged to the local context of the township.

Instrumentation
The instruments of the study were as follows:  Oxford Placement Test Packages: The Oxford Placement Tests provide a reliable and efficient means of placing students at the correct places.
The new edition consists of: The test is divided into two main sections Listening Test and Grammar Test), each of 100 items. The first section is primarily a test of reading and listening skills, in which the learner's performance is dependent on knowledge of the sound and writing system of English and on the ability to make use of this knowledge at a task-speed well within the competence of a native speaker of English. The second section is a test of grammatical structure and involves a carefully selected range of those structures consistently found in course books and examinations at elementary, intermediate and advanced levels.
The two sections of the test are, however, designed to be used together to produce a total score out of 200. It is this aggregate score from the two sections that should ideally be used to establish a rank order for placement or other purposes.
A written key to the test appears on the inside cover of the

Procedures
There were 315 EFL learners studying English in padideh Language Center, Shabestar. The language proficiency level of 100 learners had been determined through an Oxford Placement Test by the authorities of the institute, in advance.
In the main study, 20 participants out of 100 intermediate EFL learners were selected non-randomly.
At the beginning of the study, the participants were engaged in traditional writing, which is the participant wrote essays about the topic what can be an ideal marriage in your country) on the paper and with pen or pencil during an hour. While, after two weeks the participants were engaged in modern writing. They typed their essays on the computer by keyboard about the same topic and during an hour, too.
In the PPT performance, papers were scored by a rater according to the grammar errors on the papers. Also the length of the essays was considered the number of complete sentences in an essay was counted).
In the computerized system, essays were scored automatically by software that was connected on to the Internet according to the grammar errors.

DATA ANALYSIS & RESULTS
In order to analyze the data, the researcher used both descriptive as well as inferencial statistics. A series of SPSS were performed on the data whether there were any significant differences in performance across the two modes. To answer the research questions the researcher conducted a series of paired samples t-tests to observe how each group performed independently and comparably. And also the researcher used rank order analysis for the comparison of the performance of two modes, CALT and PPT. The details of the statistical analyses, the results, and the outcomes of the study are presented and discussed in the following lines.
The descriptive data of the results for placement test OPT), grammatical accuracy, and length are presented in Tables 1  to 3. As can be seen, Table 1 shows that the mean of students on OPT Oxford Placement Test) is high enough for the researchers to place them at the advance level of proficiency.  * PPT = Pencil-and-paper test; ** CALT = Computer-assisted language test As can be seen in Table 3the mean length in PPT exceeds that of CALT which is quite diametrically opposed to the difference regarding grammatical accuracy across the two modes.
As for the first research question, "Does advanced EFL learners' writing accuracy vary significantly across pencil-and-paper and computer-assisted testing?", the results were analyzed employing paired-samples t-test. The inferential statistics appears below in Table 4. Regarding the grammatical accuracy, the results clearly illustrated that the mean difference 13.5) was simply significant at p < 0.05. Therefore the mean of the scores across the two sets of tests were numerically distanced enough beyond random variation.

Figure 1. Means of grammatical accuracy in PPT and CALT methods
Having made sure that the difference is significant at p < 0.05, a look back at the mean can clearly illustrate that the mean of the CALT group is higher than that of the PPT group. Therefore, the CALT group outperformed the PPT group regarding grammatical accuracy.
For the second research question, namely "Does advanced EFL learners' writing accuracy vary significantly across penciland-paper and computer-assisted testing?" the results are presented in Table 5. The calculated t-value, 7.10, was found to be significant with p < 0.05 thus, rejecting the null hypothesis Table 4.10). The research indicated that there was a significant difference between subjects' length performance on paper-and pencil test and computer-assisted test.

Figure 2. Means of writing length in PPT and CALT methods
Now that the mean difference between the two groups i.e. PPT & CALT) was statistically significant, comparing the means in terms of their numerical value clearly indicates that the PPT group outperformed CALT group in length.

DISCUSSION & CONCLUSION
As far as the first research question is concerned, the difference in terms of grammatical accuracy across Paper-and-Pencil Test PPT) and Computer-assisted Language Testing CALT) turned out to be significant. The participants who had taken the writing test through CALT had produced significantly more accurate writings than their PPT counterparts. This finding is in line with Barnes  Concerning the second research question PPT group outperformed CALT which means that using the traditional paper and pencil mode prompted the students to write more in terms of length. This finding is in line with the earlier findings such Kern and Warshauer 2000), but is contradicted by Barnes  This study bears implications for language assessment where CALT can offer great potentials for gauging learner performance. Educational systems at large that are facing a sweeping trend of electronic and digital developments can benefit from CALT and its vast array of data. Also, material and syllabus designers, language teaching practitioners, and many other local pedagogical concerns with language learning and teaching.