OpenAIRE is about to release its new face with lots of new content and services.
During September, you may notice downtime in services, while some functionalities (e.g. user registration, login, validation, claiming) will be temporarily disabled.
We apologize for the inconvenience, please stay tuned!
For further information please contact helpdesk[at]

fbtwitterlinkedinvimeoflicker grey 14rssslideshare1
Jak, S.; Jorgensen, T.D. (2017)
Publisher: Frontiers Media S.A.
Journal: Frontiers in Psychology
Languages: English
Types: Article
Subjects: Conceptual Analysis, Psychology, measurement invariance, cross-level invariance, BF1-990, multilevel structural equation modeling, multilevel confirmatory factor analysis, multilevel reliability
Data often have a nested, multilevel structure, for example when data are collected from children in classrooms. This kind of data complicate the evaluation of reliability and measurement invariance, because several properties can be evaluated at both the individual level and the cluster level, as well as across levels. For example, cross-level invariance implies equal factor loadings across levels, which is needed to give latent variables at the two levels a similar interpretation. Reliability at a specific level refers to the ratio of true score variance over total variance at that level. This paper aims to shine light on the relation between reliability, cross-level invariance, and strong factorial invariance across clusters in multilevel data. Specifically, we will illustrate how strong factorial invariance across clusters implies cross-level invariance and perfect reliability at the between level in multilevel factor models.
  • The results below are discovered through our pilot algorithms. Let us know how we are doing!

    • Bakker, A. B., Sanz-Vergel, A. I., Rodríguez-Mu-oz, A., and Oerlemans, W. G. (2015). The state version of the recovery experience questionnaire: A multilevel confirmatory factor analysis. Eur. J. Work Org. Psychol. 24, 350-359. doi: 10.1080/1359432X.2014.903242
    • Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychol. Bull. 107, 238-246. doi: 10.1037/0033-2909.107.2.238
    • Bollen, K. A. (1989). Structural Equations with Latent Variables. Hoboken, NJ: Wiley.
    • Bottoni, G. (2016). A multilevel measurement model of social cohesion. Soc. Indic. Res. doi: 10.1007/s11205-016-1470-7. [Epub ahead of print].
    • Browne, M. W., and Cudeck, R. (1992). Alternative ways of assessing model fit. Soc. Methods Res. 21, 230-258. doi: 10.1177/0049124192021002005
    • Byrne, B. M., Shavelson, R. J., and Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: the issue of partial measurement invariance. Psychol. Bull. 105, 456-466. doi: 10.1037/0033-2909.105.3.456
    • Davidov, E., Dülmer, H., Cieciuch, J., Kuntz, A., Seddig, D., and Schmidt, P. (2016). Explaining measurement nonequivalence using multilevel structural equation modeling the case of attitudes toward citizenship rights. Soc. Methods Res. doi: 10.1177/0049124116672678. [Epub ahead of print].
    • ESS Round 6: European Social Survey (2014). ESS-6 2012 Documentation Report. 2.1 Edn. Bergen: European Social Survey Data Archive, Norwegian Social Science Data Services.
    • Geldhof, J. G., Preacher, K. J., and Zyphur, M. J. (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychol. Methods 19, 72-91. doi: 10.1037/a0032138
    • Hanges, P. J., and Dickson, M. W. (2006). Agitation over aggregation: clarifying the development of and the nature of the GLOBE scales. Leadersh. Q. 17, 522-536. doi: 10.1016/j.leaqua.2006.06.004
    • Hox, J. J. (2010). Multilevel Analysis: Techniques and Applications, 2nd Edn. New York, NY: Routledge.
    • Hu, L., and Bentler, P. M. (1999). Cutoff criteria for fit indices in covariance structure analysis: conventional versus new alternatives. Struct. Equat. Model. 6, 1-55. doi: 10.1080/10705519909540118
    • Huppert, F. A., Marks, N., Clark, A., Siegrist, J., Stutzer, A., Vittersø, J., et al. (2009). Measuring well-being across europe: description of the ESS well-being module and preliminary findings. Soc. Indicat. Res. 91, 301-315. doi: 10.1007/s11205-00 8-9346-0
    • Jak, S. (2017). Testing and explaining differences in common and residual factors across many countries. J. Cross Cult. Psychol. 48, 75-92. doi: 10.1177/0022022116674599
    • Jak, S., Oort, F. J., and Dolan, C. V. (2013). A test for cluster bias: Detecting violations of measurement invariance across clusters in multilevel data. Struct. Equat. Model. 20, 265-282. doi: 10.1080/10705511.2013. 769392
    • Jak, S., Oort, F. J., and Dolan, C. V. (2014). Measurement bias in multilevel data. Struct. Equat. Model. 21, 31-39. doi: 10.1080/10705511.2014.856694
    • Kim, E. S., Cao, C., Wang, Y., and Nguyen, D. T. (2017). Measurement invariance testing with many groups: a comparison of five approaches. Struct. Equat. Model. 24, 524-544. doi: 10.1080/10705511.2017.1304822
    • Lord, F. M., and Novick, M. R. (1968). Statistical Theories of Mental Test Scores. Reading, MA: Addison-Welsley Publishing Company.
    • Lubke, G. H., Dolan, C. V., Kelderman, H., and Mellenbergh, G. J. (2003). On the relationship between sources of within-and between-group differences and measurement invariance in the common factor model. Intelligence 31, 543-566. doi: 10.1016/S0160-2896(03)00051-5
    • McDonald, R. P. (1999). Test Theory: A Unified Treatment. Mahwah, NJ: Erlbaum.
    • Mehta, P. D., and Neale, M. C. (2005). People are variables too: multilevel structural equations modeling. Psychol. Methods 10, 259-284. doi: 10.1037/1082-989X.10.3.259
    • Mellenbergh, G. J. (1989). Item bias and item response theory. Int. J. Educ. Stat. 13, 127-143. doi: 10.1016/0883-0355(89)90002-5
    • Meredith, W. (1993). Measurement invariance, factor analysis, and factorial invariance. Psychometrika 58, 525-543. doi: 10.1007/BF02294825
    • Meredith, W., and Teresi, J. A. (2006). An essay on measurement and factorial invariance. Med. Care 44, S69-S77. doi: 10.1097/01.mlr.0000245438.73837.89
    • Millsap, R. E., and Everson, H. T. (1993). Methodology review: statistical approaches for assessing measurement bias. Appl. Psychol. Meas. 17, 297-334. doi: 10.1177/014662169301700401
    • Muthén, B. (1990). Mean and Covariance Structure Analysis of Hierarchical Data (UCLA Statistics Series No. 62). Los Angeles, CA: University of California, Los Angeles.
    • Muthén, B., and Asparouhov, T. (2017). Recent methods for the study of measurement invariance with many groups: alignment and random effects. Soc. Methods Res. doi: 10.1177/0049124117701488
    • Muthén, L. K., and Muthén, B. O. (1998-2015). Mplus User's Guide, 7th Edn. Los Angeles, CA: Muthén and Muthén.
    • Rabe-Hesketh, S., Skrondal, A., and Pickles, A. (2004). Generalized multilevel structural equation modeling. Psychometrika 69, 167-190. doi: 10.1007/BF02295939
    • Raftery, A. E. (1986). Choosing models for cross-classification. Am. Sociol. Rev. 51, 145-146. doi: 10.2307/2095483
    • Raftery, A. E. (1995). Bayesian model selection in social research. Sociol. Methodol. 25, 111-163. doi: 10.2307/271063
    • Raykov, T. (1997). Estimation of composite reliability for congeneric measures. Appl. Psychol. Meas. 21, 173-184. doi: 10.1177/01466216970212006
    • Raykov, T., and Shrout, P. E. (2002). Reliability of scales with general structure: point and interval estimation using a structural equation modeling approach. Struct. Equat. Model. 9, 195-212. doi: 10.1207/S15328007SEM0902_3
    • Sörbom, D. (1974). A general method for studying differences in factor means and factor structures between groups. Br. J. Math. Stat. Psychol. 27, 229-239. doi: 10.1111/j.2044-8317.1974.tb00543.x
    • Spilt, J. L., Koomen, H. M., and Jak, S. (2012). Are boys better off with male and girls with female teachers? A multilevel investigation of measurement invariance and gender match in teacher-student relationship quality. J. School Psychol. 50, 363-378. doi: 10.1016/j.jsp.2011.12.002
    • Stapleton, L. M., Yang, J. S., and Hancock, G. R. (2016). Construct meaning in multilevel settings. J. Educ. Behav. Statist. 41, 481-520. doi: 10.3102/1076998616646200
    • Steiger, J. H., and Lind, J. C. (1980). “Statistically based tests for the number of common factors,” in Paper Presented at the Annual Meeting of the Psychometric Society, Vol. 758 (Iowa City, IA).
    • Tay, L., Woo, S. E., and Vermunt, J. K. (2014). A conceptual and methodological framework for psychometric isomorphism validation of multilevel construct measures. Org. Res. Methods 17, 77-106. doi: 10.1177/1094428113517008
    • van de Vijver, F. J. R., and Poortinga, Y. H. (2002). Structural equivalence in multilevel research. J. Cross Cult. Psychol. 33, 141-156. doi: 10.1177/0022022102033002002
    • Wei, W., Lu, H., Zhao, H., Chen, C., Dong, Q., and Zhou, X. (2012). Gender differences in children's arithmetic performance are accounted for by gender differences in language abilities. Psychol. Sci. 23, 320-330. doi: 10.1177/0956797611427168
    • Werts, C. E., Linn, R. L., and Jöreskog, K. G. (1974). Intraclass reliability estimates: testing structural assumptions. Educ. Psychol. Meas. 34, 25-33. doi: 10.1177/001316447403400104
    • Whitton, S. M., and Fletcher, R. B. (2014). The group environment questionnaire: a multilevel confirmatory factor analysis. Small Group Res. 45, 68-88. doi: 10.1177/1046496413511121
    • Widaman, K. F., and Reise, S. P. (1997). “Exploring the measurement invariance of psychological instruments: applications in the substance use domain,” in The Science of Prevention: Methodological Advances from Alcohol and Substance Abuse Research, eds K. J. Bryant, M. Windle, and S. G. West (Washington, DC: American Psychological Association), 281-324.
    • Yuan, K. H., and Bentler, P. M. (2000). Three likelihood-based methods for mean and covariance structure analysis with nonnormal missing data. Sociol. Methodol. 30, 165-200. doi: 10.1111/0081-1750.00078
    • Zee, M., Koomen, H. M., Jellesma, F. C., Geerlings, J., and de Jong, P. F. (2016). Inter-and intra-individual differences in teachers' self-efficacy: a multilevel factor exploration. J. Sch. Psychol. 55, 39-56. doi: 10.1016/j.jsp.2015.12.003
  • No related research data.
  • No similar publications.

Share - Bookmark

Cite this article

Cookies make it easier for us to provide you with our services. With the usage of our services you permit us to use cookies.
More information Ok