Alternative Approaches to Validate a Critical Thinking Measurement Instrument for University Students

Background/purpose. The aim of this study is to present classical and robust alternatives for the validation of an instrument that assesses critical thinking (CP). The Chilean adaptation of the Halpern Critical Thinking Assessment Using Everyday Situations (HCTAES) test was analyzed using three factorial models and a short-version model, with two estimation methods: maximum likelihood (ML) and weighted least squares (WLSMV). Internal consistency was examined with point and interval estimation for different reliability coefficients.

Materials/methods. The sample consisted of 214 students from a Colombian university. The instrument used was the Chilean adaptation of the Halpern Critical Thinking Assessment Using Everyday Situations (HCTAES), which assesses five dimensions of critical thinking through open-ended and closed-ended items. For the psychometric analysis, three-factor models and an abbreviated version were considered, using Maximum Likelihood Estimation (MLE) and Weighted Least Squares (WLS) as estimation methods. Internal consistency was evaluated using reliability coefficients with confidence intervals, while construct validity was examined using Confirmatory Factor Analysis (CFA), complemented by convergent and discriminant validity indicators.

Results. The results indicate that the complete test and the open-ended question factor exhibit satisfactory internal consistency (ω=0.78 and ω=0.79, respectively). The confirmatory factor analysis using the robust WLSMV estimator yielded better fit across the absolute, incremental, and parsimony fit indices, corroborating Halpern's theoretical model.

Conclusion. It is concluded that the configural and open-format models retain adequate psychometric properties, i.e., good fit and stable factor configurations across the evaluated structures. The study contributes methodological elements to the line of validation and evidence of psychometric properties for PC, or serves as a reference for other analogous constructs.

Keywords: Critical thinking, validity metrics, robust alternative, HCTAES-Colombia

References

Acosta Jiménez, J. C., & Torres Rojas, I. S. (2025). Validation of an instrument using modern psychometric techniques for the measurement of quantitative reasoning. Caracas Medical Gazette, 133.

Anggraeni, D. M., Prahani, B. K., Suprapto, N., Shofiyah, N., & Jatmiko, B. (2023). Systematic review of problem based learning research in fostering critical thinking skills. Thinking Skills and Creativity, 49, 101334. https://doi.org/10.1016/j.tsc.2023.101334

Ato, M., López-García, J. J., & Benavente, A. (2013). A classification system for research designs in psychology. Anales de Psicología, 29(3), 1038–1059. https://doi.org/10.6018/analesps.29.3.178511

Basto, M., & Pereira, J. M. (2012). An SPSS R-menu for ordinal factor analysis. Journal of Statistical Software, 46(4), 1–29. https://doi.org/10.18637/jss.v046.i04

Bezanilla, M. J., Domínguez, H. G., & Ruiz, M. P. (2021). Importance of teaching critical thinking in higher education and existing difficulties according to teachers’ views. REMIE: Multidisciplinary Journal of Educational Research, 11(1), 20–48. https://doi.org/10.4452/remie.2021.02

Butler, H. A., Dwyer, C. P., Hogan, M. J., Franco, A., Rivas, S. F., Saiz, C., & Almeida, L. S. (2012). The Halpern Critical Thinking Assessment and real-world outcomes: Cross-national applications. Thinking Skills and Creativity, 7(2), 112–121. https://doi.org/10.1016/j.tsc.2012.04.001

Butler, H. A. (2024). Predicting everyday critical thinking: A review of critical thinking assessments. Journal of Intelligence, 12(2), 16.

Cain, M. K., Zhang, Z., & Yuan, K.-H. (2017). Univariate and multivariate skewness and kurtosis for measuring nonnormality: Prevalence, influence, and estimation. Behavior Research Methods, 49(5), 1716–1735. https://doi.org/10.3758/s13428-016-0814-1

Campo, L., Galindo-Domínguez, H., Bezanilla, M. J., Fernández-Nogueira, D., & Poblete, M. (2023). Methodologies for fostering critical thinking skills from university students’ points of view. Education Sciences, 13(2), 132. https://doi.org/10.3390/educsci13020132

De Bie, H., Wilhelm, P., & van der Meij, H. (2015). The Halpern Critical Thinking Assessment: Towards a Dutch appraisal of critical thinking. Thinking Skills and Creativity, 17, 33–44. https://doi.org/10.1016/j.tsc.2015.04.001

Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399–412. https://doi.org/10.1111/bjop.12046

Ferrando, P. J., & Anguiano-Carrasco, C. (2010). El análisis factorial como técnica de investigación en psicología [Factor analysis as a research technique in psychology]. Papeles del Psicólogo, 31(1), 18–33. https://www.redalyc.org/articulo.oa?id=77812441003

Fuentes, C. (2010). Halpern's Critical Thinking Test, version adapted to the Chilean context [Manuscript no publicado]. Universidad Diego Portales.

Furr, R. M. (2017). Psychometrics: An introduction (3rd ed.). SAGE Publications.

Gómez-Velasco NY, Jiménez-González AE, Ortiz-Padilla ME. (2024). Methodological proposal for analyzing critical thinking test scores as a tool for development in higher education. Journal of Infrastructure, Policy and Development. 8(15): 10089. https://doi.org/10.24294/jipd10089

Green, S. B., & Yang, Y. (2015). Evaluation of dimensionality in the assessment of internal consistency reliability: Coefficient alpha and omega coefficients. Educational Measurement: Issues and Practice, 34(4), 14–20. https://doi.org/10.1111/emip.12100

Halpern, D. F. (1998). Teaching critical thinking for transfer across domains: Disposition, skills, structure training, and metacognitive monitoring. American Psychologist, 53(4), 449–455. https://doi.org/10.1037/0003-066X.53.4.449

Halpern, D. F. (2014). Thought and knowledge: An introduction to critical thinking (5th ed.). Psychology Press.

Halpern, D. F. (2016). Halpern Critical Thinking Assessment: Test manual HCTA, Version 51. Schuhfried GmbH.

Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115–135. https://doi.org/10.1007/s11747-014-0403-8

Hill, C. J., Bloom, H. S., Black, A. R., & Lipsey, M. W. (2008). Empirical benchmarks for interpreting effect sizes in research. Child Development Perspectives, 2(3), 172–177. https://doi.org/10.1111/j.1750-8606.2008.00061.x

Indrašienė, V., Jegelevičienė, V., Merfeldaitė, O., Penkauskienė, D., Pivorienė, J., Railienė, A., Sadauskas, J., & Valavičienė, N. (2021). The value of critical thinking in higher education and the labor market: The voice of stakeholders. Social Sciences, 10(8), 286. https://doi.org/10.3390/socsci10080286

Jöreskog, K. G., Olsson, U. H., & Wallentin, F. Y. (2016). Multivariate analysis with LISREL. Springer. https://doi.org/10.1007/978-3-319-33153-9

Lai, K. (2018). Estimating standardized SEM parameters given nonnormal data and incorrect model: Methods and comparison. Structural Equation Modeling: A Multidisciplinary Journal, 25(4), 600–620. https://doi.org/10.1080/10705511.2017.1392248

Liang, X., & Yang, Y. (2013). Confirmatory factor analysis under violations of distributional and structural assumptions. International Journal of Quantitative Research in Education, 1(1), 61–84. https://doi.org/10.1504/IJQRE.2013.055642

Marsh, H. W., Guo, J., Dicke, T., Parker, P. D., & Craven, R. G. (2020). Confirmatory factor analysis (CFA), exploratory structural equation modeling (ESEM), and set-ESEM: Optimal balance between goodness of fit and parsimony. Multivariate Behavioral Research, 55(1), 102–119. https://doi.org/10.1080/00273171.2019.1602503

Maturana-Moreno, G. A., & Lombo-Sánchez, M. L. (2020). Naturalistic intelligence: Effects on critical thinking and cognition needs. Praxis & Saber, 11(25), 177–204. https://doi.org/10.19053/22160159.v11.n25.2020.9094

Mulaik, S. A. (2009). Linear causal modeling with structural equations. CRC Press/Taylor & Francis Group. https://doi.org/10.1201/9781439800393

Muñiz, J., Elosua, P., & Hambleton, R. K. (2013). Directrices para la traducción y adaptación de los tests: Segunda edición [Guidelines for test translation and adaptation: Second edition]. Psicothema, 25(2), 151–157. https://doi.org/10.7334/psicothema2013.24

Nieto, A. M., Saiz, C., & Orgaz, B. (2009). Análisis de las propiedades psicométricas de la versión española del HCTAES-Test de Halpern para la evaluación del pensamiento crítico mediante situaciones cotidianas [Analysis of the psychometric properties of the Spanish version of Halpern's HCTAES-Test for assessing critical thinking using everyday situations]. Revista Electrónica de Metodología Aplicada, 14(1), 1–15. https://doi.org/10.17811/rema.14.1.2009.1-15

Ortega-Quevedo, V., Gil-Puente, C., Vallés-Rapp, C., & López-Luengo, M. A. (2020). Diseño y validación de instrumentos de evaluación del pensamiento crítico en Educación Primaria [Design and validation of instruments for assessing critical thinking in primary education]. Tecné, Episteme y Didaxis: TED, 48, 91–110. https://doi.org/10.17227/ted.num48-10557

Peters, G.-J. Y. (2018). The alpha and the omega of scale reliability and validity: Why and how to abandon Cronbach’s alpha and the route towards more comprehensive assessment of scale quality. European Health Psychologist, 16(2), 56–69. https://doi.org/10.31234/osf.io/h47fv

Rivas, S. F., Saiz, C., & Almeida, L. S. (2023). The role of critical thinking in predicting and improving academic performance. Sustainability, 15(2), 1527. https://doi.org/10.3390/su15021527

Rodríguez, Á. M. M., González, A. E. J., & Velasco, N. Y. G. (2025). Fases para el diseño y validación de instrumentos de investigación: alcance de la inteligencia artificial. Ciencia en Desarollo, (1), 105-109.

Rodrigues, A., Soares, P., & da Silva, L. (2018). Translation, adaptation, and validation of the Halpern Critical Thinking Assessment to Portugal: Effect of disciplinary area and academic level on critical thinking. Anales de Psicología, 34(2), 292–298. https://doi.org/10.6018/analesps.34.2.272401

Saiz, C., Almeida, L. S., & Rivas, S. F. (2022). Can critical thinking be assessed briefly? *Psico-USF, 26*(esp.), 139–148. https://doi.org/10.1590/1413-8271202126nesp13

Saldarriaga-Zambrano, P. J., Gómez-Vergara, L. G., & Giraldo-Gutiérrez, J. A. (2022). Probabilidad e incertidumbre: Un aporte a la comprensión del pensamiento crítico en estudiantes universitarios [Probability and uncertainty: A contribution to the understanding of critical thinking in university students]. En M. D. García-Álvarez (Ed.), Educational Revolution in the New Era (Vol. 1, pp. 428-442). Redine.

Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107–120. https://doi.org/10.1007/s11336-008-9101-0

Song, H., & Cai, L. (2024). Interactive learning environment as a source of critical thinking skills for college students. BMC Medical Education, 24(1), 270. https://doi.org/10.1186/s12909-024-05247-y

Stensen, K., & Lydersen, S. (2022). Internal consistency: From alpha to omega. Tidsskrift for Den Norske Legeforening, 142(12). https://doi.org/10.4045/tidsskr.22.0288

Taseer, N. A., Siddique, A., Rabi, S. K., & Maqsood, A. (2023). Effect of emotional intelligence on students' academic performance in Pakistan. Journal of Arts & Social Sciences, 10(1), 179–190.

Tarasova, K., Gracheva, D., Talov, D., Orel, E., & Dementiev, A. (2025). Measuring changes in critical thinking skills among university economics students: Insights from domain-specific assessment. Studies in Higher Education, 50(1), 1–16. https://doi.org/10.1080/03075079.2023.2284000

Ventura-León, J., & Caycho-Rodríguez, T. (2017). El coeficiente omega: Un método alternativo para la estimación de la confiabilidad [The Omega coefficient: An alternative method for estimating reliability]. Revista Latinoamericana de Ciencias Sociales, Niñez y Juventud, 15(1), 625–627. https://www.redalyc.org/pdf/773/77349627039.pdf

Wood, P. (2008). Confirmatory factor analysis for applied research. The American Statistician, 62(1), 91–92. https://doi.org/10.1198/tas.2008.s98

Xia, Y., & Yang, Y. (2019). RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods. Behavior Research Methods, 51(1), 409–428. https://doi.org/10.3758/s13428-018-1055-2

Announcement

EDUPIJ Citation Metrics

EDUPIJ News!