( 1999) define face validity as “the degree to which a test appears to measure the knowledge or abilities it claims to measure, as judged by an untrained observer” (p.
More specifically, the study examines to what extent test-taker perception and test developer intention agree and discusses the importance of investigating test-taker perception or face validity when new tests are developed and incorporated. We investigate test-taker perception of the skills being measured by high-stakes examinations administered in two EFL countries, Japan and South Korea. If test-takers fail to correctly recognize which skills the test developers intend to measure, they are not likely to focus on and spend time on the skills that the test developers wish them to pay attention. In this context, the gap between test-takers’ perception of the skills being measured and test developers’ intention in developing items is arguably one of the factors that undermines intended positive washback on learning. As can be seen, language tests are often used as de facto language education policies (Menken, 2008) and disciplinary tools (Shohamy, 2001), and test developers measure abilities and skills that they wish test-takers to focus on in their learning activities. In Korea, similarly, a key purpose of developing a new university entrance examination in Korea-the National English Ability Test (NEAT)-was claimed to be “to activate the teaching of speaking and writing of English at schools, which has not actually been conducted despite its compelling necessity” (Lee, 2012, p. For example, developers of Test of English for Academic English (TEAP)-a high-stakes test jointly developed by Sophia University and EIKEN recently-explicitly stated their expectation that the newly introduced test will draw attention to productive and cognitive skills and that it would lead to better English education and learning in Japan ( Eiken Foundation of Japan, n.d.). In many cases, the introduction or revision of examinations has been done to promote positive washback on teaching and learning. Since high-stakes examinations are extremely influential to the students, having an immense impact on what is to be taught and learned (Cheng, 2008), test developers often develop tests in order to bring about the intended impact on language learning or washback. In some monolingual EFL countries such as Japan and Korea, high-stakes examinations are the primary opportunities to use English and a powerful instrumental motivation to learn English. In order to obtain the positive and intended washback effect on English learning, the present study suggests that the Japanese and Korean test committees need to (a) widely inform test-takers of the ability measured by each test item and (b) incorporate performance testing that measures test-takers’ productive skills more directly. Conclusionsįace validity could be integrated for test development with the ultimate aim of promoting positive washback on students, which should be one of the intentions of test developers. Furthermore, many participants perceived items designed to indirectly measure writing ability as those tapping into reading ability.
ResultsĪlthough the overall agreement rate was moderately high, items aiming to measure an ability to read between the lines were perceived to be measuring an ability to understand the content objectively. University students in Japan and Korea (N = 179) were given past entrance examinations administered in the respective countries and asked to read test items and record what ability they thought each item was measuring.
This study aims to investigate test-taker perception of the ability being measured by items (i.e., face validity) in high-stakes tests and examines the extent to which test-taker perception and test developer intention agree. However, if students fail to recognize the abilities that the test developers intend to measure, they are less likely to learn what the test developers wish them to learn. High-stakes tests have an immense washback effect on what students learn and affect the content of student learning.
0 kommentar(er)
