The effects of test authenticity and test delivery mediums on the computer-based reading comprehension ability and the test takers' attitudes towards the test

Abstract

This study proposed, designed and created Reading Comprehension Tests for assessing first year students’ reading proficiency at King Mongkut’s Institute of Technology North Bangkok. The objectives of this study were 1) to investigate the effects of test authenticity and test delivery mediums on test takers’ reading proficiency and 2) to explore the attitudes of the test takers towards test authenticity and test delivery mediums. The Target Language Use domain was considered to develop authentic and inauthentic reading tests administered via paper-based and computer-based (computer-adaptive online) tests. Students’ attitudes towards these different versions of the tests were also investigated. The main research instruments included the Authentic English Reading Comprehension Conventional Paper-and-Pencil Test (ACON), the Inauthentic English Reading Comprehension Conventional Paper-and-Pencil Test (ICON), the Authentic English Reading Comprehension Computer-Adaptive Online Test (ACOM), and the Inauthentic English Reading Comprehension Computer-Adaptive Online Test (ACOM). A retrospective interview was conducted to investigate students’ attitudes towards those tests. The data obtained were analyzed by two-way analysis of variance (2*2 ANOVA) to identify the main effects and the interaction effects. Partial Eta square reported by SPSS program was used to measure the effect sizes of test authenticity and test delivery mediums. Content analysis and chi square (X²) tests were applied to find out whether there was any significant difference in the proportions of subjects’ attitudes towards test authenticity and test delivery mediums in each group. Findings reveal that (1) there was significant difference between the mean score obtained from the test considered authentic (ACOM and ICOM) and those obtained from the test considered inauthentic (ICOM and ICON) at the 0.05 level (p< .05, F = 49.164). Partial Eta Squared value (0.142) shows the medium effect size of the test authenticity on test takers’ reading ability score (2) there was no significant difference between the mean score obtained from the test delivered by means of computer (ACOM and ICOM) and that obtained from the test delivered by means of paper (ACON and ICON) at the 0.05 level (p> .05, F = 2.556). The effect size value of test delivery mediums is 0.009. This means that the effect size of the mediums on test takers’ reading ability scores was small. (3) The interaction effect between Mediums and Authenticity value obtained from the 2*2 ANOVA was significant at the level of 0.05 (p .05, F = 14.474). Regarding the test takers’ attitudes towards the test authenticity and test delivery mediums, it can be concluded that test takers tend to have positive attitudes towards the tests they took. However, there were some issues on which they reported negative attitudes.