The impact of an exit English test on Hong Kong undergraduates: A study investigating the effects of test status on students' test preparation behaviours

The impact of an exit English test on Hong Kong undergraduates: A study investigating the effects of test status on students' test preparation behaviours

• Stoneman, Bernadette Wun Han

2006

• L Educational Measurement & Evaluation
  • LC Academic Achievement Tests
• G Curriculum, Teaching & Learning: Specific Disciplines
  • GB English Language

Educational Evaluation Higher Education

• Hong Kong

• 1997.7 onwards

• Post-Secondary Education

The notion that testing exerts influence on teaching and learning is accepted by many. In Hong Kong, some educationalists indeed see tests and examinations as one way of instilling positive changes in learners' learning. In 1993, amidst growing concern that Hong Kong students' standard of English was declining, the Hong Kong Polytechnic University (PolyU) proposed to the then University and Polytechnic Grants Committee that a Graduating Students' Language Proficiency Assessment (GSLPA)be devised and introduced to encourage university students to improve their English proficiency. In 1999, the GSLPA-English was mandated by the PolyU as a university exit English test for its graduates. There was no accompanying formal teaching/learning curriculum, as the test was intended to assess students' level of English proficiency rather than their achievement in some taught English courses. Whilst it was reasonable to expect that some students in this university would engage in some form of learning activities to help them learn/train for their exit English test, it was not clear exactly how these students - of their own accord, without a specified learning syllabus and receiving no input from teachers in the regular English classroom - would approach test preparation. Hitherto, there has been little research on how washback operates for learners rather than teachers. This study was principally motivated by a desire to better understand learners as participants in the washback cycle. It investigated how a sample of PolyU graduating students perceived the GSLPA-English as a university exit English test, and whether and how they prepared for it. Initially, the study found that this test was perceived by many respondents as lacking in status and stakes, and that not many took steps to prepare for it. The GSLPA-English was superseded by the lELTS-CEPAS in 2002, when the University Grants Committee, in a further attempt to raise students' awareness of the importance of English proficiency, chose to adopt the International English Language Testing System (IELTS) over the GSLPA-English and other English tests as the test of choice for Hong Kong's territory-wide common English proficiency assessment scheme (CEPAS) for university graduates. Insofar as the stated functions of a university exit English test are concerned, i.e. as a tool for assessing students' English proficiency and as a source of motivation to encourage students to improve their English, the GSLPA-English and the lELTS-CEPAS had identical functions. However, as regards test status, these two tests differed significantly from each other in that the former was a locally developed test adopted by a single institution only, whereas the latter was an internationally recognised test implemented territory-wide in Hong Kong and with overt government sanction. This study took advantage of a timely opportunity to test a washback hypothesis, put forth by Alderson and Hamp-Lyons (1996: 296), which states that the status/stakes of a test will affect the amount and type of washback The research investigated whether, given the differences in test status described above, perceptions of the exit English test and test preparation behaviours would be similar, or different, between students who sat the GSLPA-English and those who sat the lELTS-CEPAS. The ultimate purpose was to determine the role of test status in mediating washback on learners. The study investigated and compared the nature and extent of the test preparation activities reported by two samples of PolyU students. The first sample sat the GSLPA-English in its last year of implementation in April 2002, while the second sat the lELTS-CEPAS during its first year in 2002/2003. The data collection method included questionnaire surveys for collecting quantitative data, and semi-structured interviews for obtaining in-depth qualitative data. A final supplementary study was conducted in February/March 2004 to shed further light on some observations and find support or otherwise for claims made in earlier phases of the research.
As far as the occurrence of washback was concerned, the study found a significantly higher percentage of lELTS-CEPAS respondents took up some form of test preparation - 74.9% as opposed to 18.8% for the GSLPA-English sample. The status of the lELTS-CEPAS was found to have positively predisposed students to take up test preparation, as results of statistical analysis showed that respondents who were aware of the test's international and official status were more likely to opt to engage in test preparation. Regarding the type of washback, this study found little difference in the nature of the test preparation activities reported by respondents in the two samples. In general, students from both groups chose activities mainly intended for test orientation and/or test-specific training/coaching. The interview data revealed that rather than being influenced by the status of the test that they were taking, students' choices of the types of test preparation activities were in fact mainly influenced by their past learning and test taking experiences. The majority of respondents in this study were accustomed to using test-specific materia Is and practising with sample papers to train for English exams. When faced with a university exit English test, many resorted to the same test preparation strategies which seemed to have worked for them in the past. As for the amount of washback, this was positively affected by test status. The lELTS-CEPAS respondents spent on average approximately four times the amount of time in test preparation than did the GSLPA-English respondents. However, irrespective of the fourfold increase, 81.8% of the lELTS-CEPAS respondents said that the effort they put into preparing for this test was less than what they had put into other public exams that they had taken, and 46.3% started to prepare for the lELTS-CEPAS only one week or less before the test date. When asked to explain this behaviour, respondents typically evoked test stakes, rather than test status, as the deciding factor. They were primarily concerned with whether or not the test carried high stakes for them personally rather than with its status in the community at large. In the case of the lELTS-CEPAS, the voluntary nature of the test and the fact that students could opt not to have the test results recorded on their transcript rendered the test low-stakes, despite having the status of being a highly recommended, government approved and globally recognised test The implication of this finding is that test status and test stakes are test qualities that do not necessarily go hand in hand, and that their consequences on washback may well be different. In this respect, this study has highlighted a need to distinguish between test status and test stakes. In general, the study' s conclusions attest to current views about the complexity of washback. In particular, they suggest that as a strategy used to promote desirable changes in learners and their learning, tests may, or may not, bring about the predicted results. This unpredictability is partly the results of our inability to anticipate learners' reactions accurately - caused mainly by our lack of understanding about learners' beliefs and expectations in testing situations, and partly because we have yet to fully understand how washback operates in various social-economic-political situations. Although this study has made one small step to further our understanding in these directions, further research is clearly needed.

PhD

Hong Kong Polytechnic University

Hong Kong

• English

Dissertation Theses

https://bibliography.lib.eduhk.hk/en/bibs/08752934

2010-12-16