University of Macau

Title: Challenging the Standards and Argument Approaches and Offering 

          an Ethical Basis for Assessment Evaluation
 

Abstract

Despite the publication and influence of the Standards for Psychological and Educational Studies (US-based APA, AERA, and NCME, 1954 to 2014), the impact on language assessment has been minimal. The main reasons could be numerous: (1) the Standards are seen as an idealized set of practices with no connection to the practical world of assessment agencies; (2) the Standards are seen as best practices developed by theoreticians and directed to researchers; (3) the Standards do not have principled reasoning behind them; they are lists of things to do; (4) the Standards did not trickle down to training and education programs in assessment. 

An alternative approach, the Argument-based approach (Kane, 2012; Bachman and Palmer, 2014) to evaluation of assessments has challenged the Standards approach in the last decade. But its appeal mainly rests on using the Toulmin approach (with claims, warrants, backing, rebuttal etc.) that is used to map the argument. This approach has not received wide acceptance in terms of publications or acceptance among assessment agencies for the following reasons: (1) the approach provides a framework and graphical model with plenty of new and somewhat unclear terminology; (2) the acceptance or rejection of arguments are not clearly articulated; and (3) the approach does not relate to the assessment professionals and practitioners as it works as a top-down approach and involves only researchers.  

In Kunnan (2017), I proposed a new approach that uses an ethical basis with practical assessment-related scenarios. This approach requires the use of ethical thinking by professionals and practitioners in evaluating dilemma-type scenarios with the support of ethical principles. Such an approach can be seen as a ground-up approach that can involve all practitioners not just researchers. In addition, this approach can be used in education and training programs as the scenario-based approach uses dilemmas that commonly occur in everyday assessment settings. In my talk, I will discuss this approach with examples and dilemmas from assessment settings. 
 

​

Bio

Antony John Kunnan is a language assessment specialist. His research interests are fairness of tests and testing practice, assessment literacy, research methods and statistics, ethics and standards, and language assessment policy. After completing his Ph.D. from UCLA in 1991, he was awarded a post-doctoral fellowship at the University of Michigan, Ann Arbor for a year. From 1992 until 2013, he was assistant, associate and full professor at California State University, Los Angeles. In 2006, he received a Fulbright scholarship to Tunghai University, Taiwan where he was a visiting professor and scholar. He also was professor (and now Honorary Professor) at the University of Hong Kong and at Nanyang Technological University, Singapore. He joined the University of Macau as professor in 2016.