Modeling Linguistic Structure in Language Awareness Assessment

Insights into Item Design and Student Response Behavior within the Austrian Competence Assessment iKMPLUS

Abstract

This paper examines items in the “language awareness” domain of the Austrian competence assessment iKM^PLUS. It focuses on item design and student response behavior. Three items that were excluded during the validation process are analysed using data from one primary-level pilot study (n = 623) and one secondary-level pilot study (n = 822). Two items target orthography-related knowledge of written structure and strategies (capitalisation cues and punctuation), the third item focuses on the distinction between verbs and nouns/nominalised verbs. Items were developed to improve the assessment of procedural and metacognitive knowledge rather than declarative knowledge. Analyses highlight design challenges and response patterns. Results demonstrate the difficulty of operationalising strategic metacognitive competence in test items. Student responses indicate limited awareness of (morpho-)syntactic determination. Findings contribute to discussions on translating didactically informed constructs into standardised test items in the field of language awareness.