The effects of a test taker-centered computer-based writing test on Thai first year university students' English writing performance

Abstract

The objectives of the present study were to investigate the effects of facilitative features and required multiple drafts on the writing performance and processes of EFL students in a test taker-centered computer-based writing test (T-CBWT); the computer writing behaviors of the test takers; and the attitudes of the test takers towards the T-CBWT. Subjects were 144 Thai first-year undergraduates of the Faculty of Commerce and Accountancy year 2006. They were purposively sampled and randomly assigned to four equal test groups. Subjects were also classified into three writing proficiency levels using computer-based writing pretest (CBWT) scores as a basis. Research instruments included (1) the T-CBWT, the posttest test with facilitative features allowed and required multiple drafts) (2) analytical rating scales measuring three major aspects of content, organization and language use (3) retrospective questionnaires and (4) stimulated retrospective interview questions. Quantitative data were analyzed via two-way analysis of variance, Kruskal-Wallis H and Mann-Whitney [mu] statistics. Qualitative data were analyzed though textual analysis of test takers’ written drafts, and content analysis of verbal reports and attitudes towards the T-CBWT. Findings from the quantitative analyses in this study seem to provide evidence that facilitative features assist test takers across writing proficiency levels in the improvement of mechanics (spelling and punctuation) but not the overall quality of the essay. In addition, imposed multiple drafts seem to have some influence on the content (topic development & supporting ideas and clarity & explicitness scores) of test takers in the advanced writing proficiency level who were in the ‘with drafts’ test condition. Furthermore, textual analyses reveal that some test takers of the intermediate and low-intermediate writing proficiency levels in the ‘with drafts’ test condition heavily performed content-related changes at the sentence level that were instrumental in improving the overall quality of their essays. These findings together suggest that required multiple drafts and facilitative features might ultimately be effective in assisting test takers in performing their best on a writing test. Findings also indicate that a majority of subjects across writing proficiency levels wrote in a recursive style, confirming established models of writing processes. Finally, test takers in general had a positive outlook towards the T-CBWT and have shown interest in opting to take the T-CBWT again. With further development and research especially in relation to washback, the T-CBWT serves as a potentially practical approach to assess writing and may positively impact the EFL learning and teaching of writing