Measurement equivalence: A non-technical primer on categorical multi-group confirmatory factor analysis in school psychology☆
Section snippets
Role of measurement
In psychology and education, researchers and practitioners regularly attempt to assess constructs that are believed to exist but cannot be observed directly (e.g., intelligence, mathematics achievement, depression, happiness). Because these constructs cannot be directly observed, the degree to which respondents possess an unobservable trait is typically inferred based on their responses to items (often responses to items on scales with Likert-type formats) that are believed to reflect the
What techniques are available to examine ME/I with ordinal data?
Rating scales with Likert-type formats are commonly used to assess the outcomes of social, emotional, and behavioral interventions in school settings, and these rating scales usually produce ordinal data. Ordinal variables consist of three or more mutually exclusive levels, which are presumed to be rank-ordered (Kline, 2010). Though commonly treated as interval variables within psychological and educational research, scales with Likert-type item formats typically lack the necessary
Necessary sample size
Prior to beginning analyses, researchers should ensure that they have enough participants to accurately evaluate ME/I. MG-CFA is based on CFA which is a large sample technique that generally requires a minimum of 200 participants (200 per group for MG-CFA). That said, a variety of factors should be considered in decisions about sample size including reliability of indicators, scaling, and strength of factor loadings. For more information about minimum sample sizes needed for CFA, see MacCallum,
Meaningful non-invariance: what next?
In this study, ME/I across race was supported for SABRS scores. However, in practice, it is common to encounter situations where invariance does not hold. For example, the SABRS (described previously) has been shown to meet criteria for configural, metric, and scalar/threshold invariance across gender (Kilgus & Pendergast, 2016). However, suppose that the scale developers added the following item, Gossips about classmates, to the social behavior factor. Gossiping is a form of relational
References (128)
Structural equation modelling: Adjudging model fit
Personality and Individual Differences
(2007)On tests and indices for evaluating structural models
Personality and Individual Differences
(2007)- et al.
Testing factorial invariance across groups: A reconceptualization and proposed new method
Journal of Management
(1999) - et al.
Comparison of methods for factor extraction for cognitive test-like data: Which overfactor, which underfactor?
Intelligence
(2016) - et al.
Psychometric defensibility of the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS) Teacher Rating Scale and multiple gating procedure within elementary and middle school samples
Journal of School Psychology
(2016) The golden rule is that there are no golden rules: A commentary on Paul Barrett's recommendations for reporting model fit in structural equation modelling
Personality and Individual Differences
(2007)- et al.
Detecting a gender-related DIF using logistic regression and transformed item difficulty
U.S.-China Education Review
(2011) - et al.
Standards for educational and psychological testing
(2014) - et al.
Multiple-group factor analysis alignment
Structural Equation Modeling: A Multidisciplinary Journal
(2014) - et al.
Robust chi square difference testing with mean and variance adjusted test statistics
On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA
Structural Equation Modeling
Assessment of depression
Significance tests and goodness of fit in the analysis of covariance structures
Psychological Bulletin
Structural equations with latent variables
When does measurement invariance matter?
Medical Care
Measurement invariance versus selection invariance: Is fair selection possible?
Psychological Methods
Conducting measurement invariance tests with ordinal data: A guide for social work researchers
Journal of the Society for Social Work and Research
Confirmatory factor analysis for applied research
Confirmatory factor analysis
Alternative ways of assessing model fit
Principles and practice of scaled difference chi-square testing
Structural Equation Modeling: A Multidisciplinary Journal
Structural equation modeling with Mplus: Basic concepts, applications and programming
Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance
Psychological Bulletin
Sensitivity of goodness of fit indexes to lack of measurement invariance
Structural Equation Modeling
What happens if we compare chopsticks with forks? The impact of making inappropriate comparisons in cross-cultural research
Journal of Personality and Social Psychology
Teacher's corner: Testing measurement invariance of second-order factor models
Structural Equation Modeling
Evaluating goodness-of-fit indexes for testing measurement invariance
Structural Equation Modeling
Change in learning strategies during and after the transition to higher education: The impact of statistical choices on growth trend estimates
The insidious effects of failing to include design-driven correlated residuals in latent-variable covariance structure analysis
Psychological Methods
How to compare means of latent variables across countries and waves: Testing for invariance measurement. An application using eastern European societies
Sociológia
Development and validation of the Student Internalizing Behavior Screener: Examination of reliability, validity, and classification accuracy
Assessment for Effective Intervention
Testing the invariance of values in the Benelux countries with the European Social Survey: Accounting for ordinality
Social Darwinism, scientific racism, and the metaphysics of race
The Journal of Negro Education
Testing for factorial invariance in the context of construct validation
Measurement and Evaluation in Counseling and Development
Academic enablers and student achievement: Implications for assessment and intervention services in the schools
Psychology in the Schools
Testing measurement invariance in the target rotated multigroup exploratory factor model
Structural Equation Modeling
DIF detection and description: Mantel-Haenszel and standardization
School-based screening: A population-based approach to inform and monitor children's mental health needs
School Mental Health
Equivalence of psychological measurement in heterogeneous populations
Journal of Applied Psychology
The student risk screening scale (SRSS)
Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement
Quality of Life Research
Evaluating the use of exploratory factor analysis in psychological research
Psychological Methods
Scientific racism: The cloak of objectivity
Journal of Social Issues
Exploratory factor analysis: A users' guide
International Journal of Selection and Assessment
An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data
Psychological Methods
Old and new ideas for data screening and assumption testing for exploratory and confirmatory factor analysis
Frontiers in Psychology
Response shifts in mental health interventions: An illustration of longitudinal measurement invariance
Psychological Assessment
Factor analysis with ordinal indicators: A Monte Carlo study comparing DWLS and ULS estimation
Structural Equation Modeling
Implementation science and school psychology
School Psychology Quarterly
Service delivery for response to intervention: Core components and directions for future research
School Psychology Review
Cited by (151)
Examining ethnic identity, school climate, and academic futility among minoritized students
2024, Journal of School PsychologyPsychometric evaluation of the United States-adapted perceived perioperative competence scale-revised: A national survey
2024, International Journal of Nursing Studies AdvancesThird variables in longitudinal research: Application of longitudinal mediation and moderation in school psychology
2024, Journal of School PsychologyMeasuring social-emotional development in schoolchildren: A national-level analysis of ECLS-B cohort data
2024, Journal of School PsychologyMeasurement Invariance of the Social, Academic, and Emotional Behavior Risk Screener - Teacher Rating Scale
2024, Journal of Psychoeducational Assessment
- ☆
Data needed to reproduce analyses in this article are available as supplemental materials (online only) in the Appendix of this article.