Development and validation of multimedia-based learning product instruments: SEM and Rasch model approaches

Abstrak

Despite the rapid expansion of multimedia-based learning in education, the availability of psychometrically validated evaluation instruments remains limited, underscoring the urgent need for reliable, standardized measurement tools. This study aims to develop and validate a multimedia-based learning product evaluation instrument that measures three main constructs: feasibility, practicality, and attractiveness. The study used an instrument development design with an integrated psychometric approach that combines classical measurement theory (Classical Test Theory), Structural Equation Modeling (SEM), and the Rasch Model. Content validity was assessed using expert judgment, as measured by Aiken's V coefficient and the Content Validity Ratio (CVR). Construct validity was tested through Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA). Structural analysis was conducted to examine the relationships between constructs, and multi-group analysis (teachers and students) was conducted to test model invariance. Data were collected from 257 respondents, comprising 153 students and 104 teachers. The results showed that the instrument had high content validity (mean Aiken's V = 0.89), very high reliability (α = 0.93), and a good measurement model fit (CFI = 0.96; RMSEA = 0.054). Rasch analysis showed item reliability of 0.96, and no serious misfit was found. Multi-group analysis showed the model was invariant between teachers and students. Thus, this instrument meets modern psychometric standards and is suitable for use in evaluating multimedia-based learning products.