MA Xiaomei, DU Wenbo

（Xi’an Jiaotong University）

　　Abstract：

　　As a core component of higher-order thinking ability, inferencing in EFL reading refers to the process of constructing new semantic information in a specific context by integrating textual explicit information, textual clues, or readers’ background knowledge. Traditional English proficiency tests, however, only provide students with a summative score, lacking a detailed diagnosis of students' strengths and weaknesses in inferencing. To make up this deficiency, cognitive diagnostic assessment (CDA) appears on the scene. CDA decomposes the macro-level ability of a specific domain into various knowledge, skills, and strategies denoted as cognitive attributes. These attributes are embedded in test items in the form of 1/0represented in a Q-matrix. With entries of Q-matrix and student' response data, cognitive diagnostic models(CDMs)will estimate students’ mastery status upon the defined attributes and generate a multi-dimensional diagnostic report subsequently.

　　Under the framework of CDA, this study incorporated China’s Standards of English Language Ability(CSE)to develop a cognitive diagnostic model for EFL reading inferential ability, aiming at offering a useful diagnostic tool to remedy the void. To this end, this study was conducted in two phases: phase I, constructing CSE-based reading inferential subskills; phase II, cognitive diagnostic modeling and generating a diagnostic report.

　　In phase I, six CSE-based reading inferential skills were defined based on the literature review, including referential, lexical, causative, temporal, premise-conclusion, and thematic inferences, with their difficulty level ranging from CSE 3 to CSE 6. The defined inferential skills were then validated by seven experts’ judgments and 16 students’ instant test-taking records. All experts reached an agreement upon the definition of inferential skills after a two-round survey. Meanwhile, 16 students’ test-taking records proved that all six inferential skills were adopted during the test-taking process with only different frequencies, thus validating the effectiveness of the defined inferential skills. Additionally, two language knowledge skills, i.e. “understanding sentence literal meaning” and “understanding discourse literal meaning” were extracted from students' test-taking process data and were added to the diagnostic process.

　　In phase II, 1, 083 valid test responses to an online diagnostic reading test for inferential ability were estimated by five different cognitive diagnostic models (CDMs), i. e. G-DINA, ACDM, RRUM, DINA, and DINO. Model-data fit results showed that the G-DINA model demonstrated the best fit on both absolute fit statistics and relative fit statistics. Diagnostic reliability results further proved that the G-DINA model showed the highest reliability coefficient both on the test level and attribute level among the five CDMs, indicating a very high classification accuracy. Hence, the diagnostic power of G-DINA is reliable and valid enough to be used in real applications in this study. Subsequently, the group-level and individual-level diagnostic results obtained from the G-DINA model were synthesized to generate personalized diagnostic reports with Python for each student. The individualized diagnostic report vividly depicts students’ strengths and weaknesses in each inferential reading skill, which offers instructive suggestions for their remedial learning.

　　The research findings provide advice and reference for the revision and application of CSE in further research as follows. First, the level and content of inferential descriptors in CSE reading scales should be modified based on inferencing theories to increase the precision of each inferential descriptor and avoid construct-irrelevant factors. Second, different types of inferential descriptors are advised to be supplemented in different levels and across different genres to make up the unbalanced distribution in the current CSE version. It will be beneficial for teachers to apply CSE self-assessment scales to evaluate students' learning progress in class. Finally, when applying CSE to construct reading items, analytical tools for assessing textual difficulty, such as Coh-Metrix, are suggested as guidance to distinguish complex, less complex, and simple language in CSE.

　　Key words：

　　China’s Standards of English Language Ability; cognitive diagnostic model; reading inferential ability; diagnostic report

　　Source:

　　Journal of China Examinations

　　No. 12, 2022