Seminar 2024-02-02

Psychometric Consideration for Multilingual Learners in Large-Scale Assessments in Education

Jung Yeon (Ellie) Park

Assistant Professor of Quantitative Research Methods

College of Education and Human Development

George Mason University

Date: Friday, February 2, 2024

Time: 11:00 A.M. – 12:00 P.M. Eastern Time

Location: Nguyen Engineering Building, Room 1109

Large-scale standardized tests, such as the Programme for International Student Assessment (PISA), aim to measure students’ academic ability equitably but can pose challenges for students from diverse linguistic and cultural backgrounds. In this study, we use item response theory, a psychometric modeling method, to investigate potential test bias based on language status in PISA 2018, focusing on multilingual students and their peers in the U.S. To address potential bias, we employ IRT, a framework that models student ability based on their response patterns to items with varied difficulty levels. Specifically, we focus on detecting differential item functioning, where items may favor a specific group while the two groups are matched on abilities. We compare multilingual (ML) and mainstream students using the root mean squared deviation statistic, identifying problematic items based on item characteristic curves. Additionally, we approach the DIF-related problem and fairness issues based on the students’ other test-taking behaviors, utilizing differential response times.

Dr. Jung Yeon (Ellie) Park is an Assistant Professor of Quantitative Research Methods at GMU (Spring 2020 - Current). She received her PhD in Measurement, Evaluation, and Statistics from Columbia University (2015) and M.S. in Statistics from The George Washington University (2009). Prior to joining Mason, Dr. Park was a postdoctoral researcher at ITEC - KU Leuven's Smart Education Program for projects to advance adaptive e-learning assessment and learning analytics methods to support technology-enhanced learning (2017-2019); and a postdoctoral associate for projects identifying risk and protective factors for suicides among sexual minority youth at New York University's Institute of Human Development and Social Change (2016). She served as a program committee for European Conference on Artificial Intelligence (ECAI; 2020) and a proposal review panelist at NSF (2021, 2022).    

David Kepplinger

Nicholas Rios