Development and Validation of a Competency Framework for Thai External Quality Assessment Supervisory Units

Background/purpose: Effective supervision is critical for credible educational quality assurance, yet a validated framework for External Quality Assessment Supervisory Units (EQASUs) remains absent in Thailand. This study aimed to develop and validate a competency framework of components and quality indicators to address this gap and enhance EQASU effectiveness.

Materials/methods: The research was conducted in two sequential stages, in which the first stage used 18 experts to analyze the framework's core components and indicators. Second, the model was empirically validated through a survey of 434 external quality assessors (EQAs) in 2023, selected via proportional stratified random sampling. Data were analyzed using Confirmatory Factor Analysis (CFA) to validate the construct and the Mann–Whitney U test to examine performance differences.

Results: The validation process confirmed a framework of four core components: organizational management (OM), personnel performance development (PPD), productivity and results (PAR), and cooperative networks (CN). All components demonstrated strong predictive power on EQASU effectiveness, with PPD emerging as the most crucial (β = 0.94), followed by OM (β = 0.81), CN (β = 0.79), and PAR (β = 0.78). A meta-evaluation of the framework's application confirmed the feasibility of its policy recommendations. Among the evaluation dimensions, utility was highest, followed by feasibility, accuracy, and propriety. Furthermore, high-performing EQASUs showed a significantly closer fit to the validated framework than the general group at the 0.01 level.

Conclusion: The verified EQASU framework addresses a critical need in the Thai educational landscape. It contributes to the literature on assessment supervision and offers a practical, evidence-based tool for strengthening external quality assurance in Thailand and similar international contexts.

Keywords: Educational quality indicators; evaluation methods; program evaluation; school-based evaluation; Thailand

References

Alam, L., Alam, M., Shafi, M. N. U. H., Khan, S., & Khan, Z. M. (2022). Meaningful in-training and end-of-training assessment: The need for implementing a continuous workplace-based formative assessment system in our training programs. Pakistan Journal of Medical Sciences, 38(5), 1132. https://doi.org/10.12669%2Fpjms.38.5.5921

Asampong, E., Kamau, E. M., Teg-Nefaah Tabong, P., Glozah, F., Nwameme, A., Opoku-Mensah, K., ... & Dako-Gyeke, P. (2023). Capacity building through comprehensive implementation, research, training, and mentorship: an approach for translating knowledge into practice. Globalization and Health, 19(1), 35. https://doi.org/10.1186/s12992-023-00935-8

Ayeni, A. J. (2012). Assessment of Principals' Supervisory Roles for Quality Assurance in Secondary Schools in Ondo State, Nigeria. World Journal of Education, 2(1), 62-69. https://eric.ed.gov/?id=EJ1158929

Awasthy, R., Flint, S., Sankarnarayana, R., & Jones, R. L. (2020). A framework to improve university-industry collaboration. Journal of Industry-University Collaboration, 2(1), 49-62. https://doi.org/10.1108/JIUC-09-2019-0016

Behravesh, E., Abubakar, A. M., & Tanova, C. (2021). Participation in decision-making and work outcomes: evidence from a developing economy. Employee Relations: The International Journal, 43(3), 704-723. https://doi.org/10.1108/ER-08-2018-0228

Brennan, J., & Shah, T. (2000). Quality assessment and institutional change: Experiences from 14 countries. Higher education, 40(3), 331-349. https://doi.org/10.1023/A:1004159425182

Brady, N., & Bates, A. (2016). The standards paradox: How quality assurance regimes can subvert teaching and learning in higher education. European Educational Research Journal, 15(2), 155-174. https://doi.org/10.1177/1474904115617484

Burgess, M., & Phillips, H. (2021). Authentic assessments and the external examiner. In The Role of External Examining in Higher Education: Challenges and Best Practices (Vol. 38, pp. 43-59). Emerald Publishing Limited. https://doi.org/10.1108/S2055-364120210000038004

Byrne, B. M. (2013). Structural equation modeling with Mplus: Basic concepts, applications, and programming. Routledge. https://doi.org/10.4324/9780203807644

Castanelli, D. J., Weller, J. M., Molloy, E., & Bearman, M. (2022). How trainees come to trust supervisors in workplace-based assessment: a grounded theory study. Academic Medicine, 97(5), 704-710. https://doi.org/10.1097/ACM.0000000000004501

Chaiya, C., & Ahmad, M. M. (2021). Success or failure of the Thai higher education development—Critical factors in the policy process of quality assurance. Sustainability, 13(17), 9486. https://doi.org/10.3390/su13179486

Chikazinga, W. W. (2025). University Actors Responding to the National Quality Assurance Regime for Higher Education in Malawi: A Case of Instrumentality, Pragmatism and Symbolic Compliance. Higher Education Quarterly, 79(1), e12577. https://doi.org/10.1111/hequ.12577

Child, S., & Ellis, P. (2021). The what, why, and how of assessment: A guide for teachers and school leaders. Sage Publications. https://www.torrossa.com/en/resources/an/5018839

Dart, J., Twohig, C., Anderson, A., Bryce, A., Collins, J., Gibson, S., ... & Palermo, C. (2021). The value of programmatic assessment in supporting educators and students to succeed: A qualitative evaluation. Journal of the Academy of Nutrition and Dietetics, 121(9), 1732-1740. https://doi.org/10.1016/j.jand.2021.01.013

Dill, D. D. (2020). Enhancing academic quality and collegial control: Insights from U.S. policy on the ethical conduct of human subjects' research. Higher Education Policy, 33(1), 45-64. https://doi.org/10.1057/s41307-018-0093-9.

Duarte, N., & Vardasca, R. (2023). Literature review of accreditation systems in higher education. Education Sciences, 13(6), 582. https://doi.org/10.3390/educsci13060582

El-Khawas, E. (2014). Inside quality reform: Early results on using outcomes for improvement. Quality in Higher Education, 20(2), 183-194. https://doi.org/10.1080/13538322.2014.926083

Galeazzo, A., Furlan, A., & Vinelli, A. (2021). The role of employees' participation and managers' authority on continuous improvement and performance. International Journal of Operations & Production Management, 41(13), 34-64. https://doi.org/10.1108/IJOPM-07-2020-0482

Geisinger, K. F. (1994). Cross-cultural normative assessment: translation and adaptation issues influencing the normative interpretation of assessment instruments. Psychological Assessment, 6(4), 304. https://doi.org/10.1037//1040-3590.6.4.304

Grifoll, J., Hopbach, A., McClaran, A., Valeikiene, A., Sanchez, T., & Ranne, P. (2015). Quality Procedures in the European higher education area and beyond–internationalisation of Quality assurance agencies. 4th ENQA survey. Brussels: ENQA. https://www.enqa.eu/wp-content/uploads/enqa_oc_22.pdf

Harris, P., Bhanji, F., Topps, M., Ross, S., Lieberman, S., Frank, J. R., ... & ICBME Collaborators. (2017). Evolving concepts of assessment in a competency-based world. Medical Teacher, 39(6), 603-608. https://doi.org/10.1080/0142159X.2017.1315071

Harvey, L., & Newton, J. (2007). Transforming quality evaluation: moving on. In Quality assurance in higher education: Trends in regulation, translation and transformation (pp. 225-245). Springer Netherlands. https://doi.org/10.1007/978-1-4020-6012-0

Henderson, C. (2025). Teacher wellbeing and the shaping of teacher shortages in crisis-affected contexts. Paper commissioned for the 2024 Global Report on Teachers. https://tinyurl.com/4uh24fej

Hu, L. T., & Bentler, P. M. (1999). Cut-off criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1 – 55. http://doi.org/10.1080/10705519909540118

Jöreskog, K. G., Olsson, U. H., & Wallentin, F. Y. (2016). Multivariate analysis with LISREL. Springer. https://doi.org/10.1007/978-3-319-33153-9

Kaiser, F., Melo, A. I., & Hou, A. Y. C. (2022). Are quality assurance and rankings useful tools to measure ‘new’ policy issues in higher education? The practices in Europe and Asia. European Journal of Higher Education, 12(sup1), 391-415. https://doi.org/10.1080/21568235.2022.2094816

Kehm, B. M. (2013). To Be or Not to Be? The Impacts of the Excellence Initiative on the German System of Higher Education. In Shin, J., Kehm, B. (Eds.), Institutionalization of World-Class University in Global Competition. The Changing Academy – The Changing Academic Profession in International Comparative Perspective, vol 6. Springer. https://doi.org/10.1007/978-94-007-4975-7_6

Kohoutek, J., & Westerheijden, D. (2014). 13. Opening up the black box. In Eggins, H. (Ed.), Drivers and barriers to achieving quality in higher education, 167. https://doi.org/10.1007/978-94-6209-494-9_13

Krooi, M., Whittingham, J., & Beausaert, S. (2024). Introducing the 3P conceptual model of internal quality assurance in higher education: A systematic literature review. Studies in Educational Evaluation, 82, 101360. https://doi.org/10.1016/j.stueduc.2024.101360

Lee, J., Lee, J. S., & Ellis, N. (2023). Public Opinion about National Large-Scale Student Assessment: A case of NAPLAN. Educational Assessment, 28(3), 137-155. https://doi.org/10.1080/10627197.2023.2215977

Lee, U., Jung, H., Jeon, Y., Sohn, Y., Hwang, W., Moon, J., & Kim, H. (2024). Few-shot is enough: exploring ChatGPT prompt engineering method for automatic question generation in English education. Education and Information Technologies, 29(9), 11483-11515. https://doi.org/10.1007/s10639-023-12249-8

Leiber, T., Stensaker, B., & Harvey, L. (2015). Impact evaluation of quality assurance in higher education: methodology and causal designs. Quality in Higher Education, 21(3), 288-311. https://doi.org/10.1080/13538322.2015.1111007

Loukkola, T., & Zhang, T. (2010). Examining quality culture: Part 1-Quality assurance processes in higher education institutions. Brussels: European University Association. https://aic.lv/bolona/2010/Reports/Examining%20Quality%20Culture%20Part%201.pdf

Lucander, H., & Christenson, C. (2020). Engagement for quality development in higher education: A process for quality assurance of assessment. Quality in Higher Education, 26(2), 135-155. https://doi.org/10.1080/13538322.2020.1761008

Ma, M. (2023). Exploring employee participation in corporate volunteering in China: Affecting factors and outcomes (Doctoral dissertation, Hong Kong Baptist University). China. https://tinyurl.com/bdef56zv

Mala, D. (2018, August 11). After delays, Onesqa to begin evaluations. Bangkok Post. https://tinyurl.com/y3jnxvvt

Mutluer, C. (2023). Development of the scientific literacy scale: A study of validity and reliability. International Journal of Educational Researchers, 14(3). https://doi.org/10.1016/10.29329/ijer.2023.595.4

Mwinyipembe, M. M., & Orodho, A. J. (2014). Effectiveness of quality assurance and standards officers school supervisory roles in enhancing students’ academic performance in national examinations in Nakuru District, Kenya. Journal of Education and Practice, 5(16), 69-80. https://tinyurl.com/2vy5cktb

Nadeem, M. (2023). Leadership's Role in Effective Quality Assurance Implementation in Higher Education Institutions. Education Quarterly Reviews, 6(3). https://doi.org/10.31014/aior.1993.06.03.778

Narayanamma, P. L., & Lalitha, K. (2016). Balanced scorecard-The learning & growth perspective. Aweshkar Research Journal, 21(2), 59.

Naz, N., Gulab, F., & Aslam, M. (2022). Development of qualitative semi-structured interview guide for case study research. Competitive Social Science Research Journal, 3(2), 42-52. https://cssrjournal.com/index.php/cssrjournal/article/view/170

Newton, J. (2010). A tale of two ‘qualitys’: reflections on the quality revolution in higher education. Quality in Higher Education, 16(1), 51-53. https://doi.org/10.1080/13538321003679499

Nguyen, Q. L. H. T. T., CHU, N. N. M., & Van Hong, T. R. A. N. (2020). Application of total quality management in developing quality assessment model: The case of Vietnamese higher education. The Journal of Asian Finance, Economics and Business, 7(11), 1049-1057. https://doi.org/10.13106/jafeb.2020.vol7.no11.1049

ONESQA. (2013, October). Manual for Educational Institutions: The third round of external quality assessment (2011-2015). https://tinyurl.com/39ndn6xs

ONESQA. (2017). About ONESQA. http://www.onesqa.or.th/en/profile/973/

ONESQA. (2023, June 23). Guidelines for the Fourth Round EQA for Higher Education. https://www.onesqa.or.th/en/profile/1291/

Pimdee, P., Sukkamart, A., Nantha, C., Kantathanawat, T., & Leekitchwatana, P. (2024). Enhancing Thai student-teacher problem-solving skills and academic achievement through a blended problem-based learning approach in online flipped classrooms. Heliyon, 10(7). https://doi.org/10.1016/j.heliyon.2024.e29172

Pitiyanuwat, S. (2011). Quality assurance in South-East Asian higher education. In Brock, C., & Pe Symaco, L. (Eds.). Education in South-East Asia. https://tinyurl.com/27n2tzf9

Pitiyanuwat, S., Phanphruk, S., & Pitiyanuwat, T. (2018). Educational testing, assessment, and quality assurance. In Fry, G. (Ed.). Education in Thailand. Education in the Asia-Pacific Region: Issues, Concerns and Prospects, vol 42. Springer. https://doi.org/10.1007/978-981-10-7857-6_24

Pizzolato, D., & Dierickx, K. (2023). Research integrity supervision practices and institutional support: A qualitative study. Journal of Academic Ethics, 21(3), 427-448. https://doi.org/10.1007/s10805-022-09468-y

Praneetpolkrung, K., & Supakicco, P. S. (2023). Empowering Teachers' Learning to Enhance Students' Change Leadership Skills. World Journal of Education, 13(3), 1-12. https://doi.org/10.5430/wje.v13n3p1

Preston, R., Gratani, M., Owens, K., Roche, P., Zimanyi, M., & Malau-Aduli, B. (2020). Exploring the impact of assessment on medical students’ learning. Assessment & Evaluation in Higher Education, 45(1), 109-124. https://doi.org/10.1080/02602938.2019.1614145

Phuangsuwan, P., Limna, P., & Siripipatthanakul, S. (2024). Ethics in the Social Sciences Research. Advance Knowledge for Executives, 3(4), No. 49. https://tinyurl.com/4a8b2e34

Ra-ngubtook, W., & Bhongsatiern, J. (2022). Basic education in Thailand. In International Handbook on Education in South East Asia (pp. 1-25). Springer Nature. https://doi.org/10.1007/978-981-16-8136-3_21-1

Riina, M. D., Stambaugh, C., Stambaugh, N., & Huber, K. E. (2023). Continuous variable analyses: t-test, Mann–Whitney, and Wilcoxon rank. In Translational Radiation Oncology (pp. 153-163). Academic Press. https://doi.org/10.1016/B978-0-323-88423-5.00070-4

Roengsumran, A. (2015). A Study of External School Evaluators' Competencies for Grade Levels 1-12 Quality Assessment in Thailand (Doctoral dissertation, University of Minnesota). Minnesota. https://tinyurl.com/4zp986kt

Saengpassa, C. (2014, May 19). ONESQA director rejects the criticism. The Nation. https://www.nationthailand.com/in-focus/30233944

Schwarz, S., & Westerheijden, D. F. (Eds.). (2007). Accreditation and evaluation in the European higher education area (Vol. 5). Springer Science & Business Media.

Sempere, A. G., Mohn, C., & Pieroni, R. (2009). Improving educational effectiveness and promoting internal and external information—sharing through student learning outcomes assessment. Toward useful program evaluation in college foreign language education, 57-81. https://tinyurl.com/3fjwv4fe

Shawyun, T. (2012). Strategic performance management system: An integrated framework. International Edition. Center for Excellence, Assumption University of Thailand. https://tinyurl.com/3684fzxv

Sottiyotin, T., Uitrakul, S., Sakdiset, P., Sukkarn, B., Sangfai, T., Chuaboon, L., & Palee, P. (2023). Effective formative assessment for pharmacy students in Thailand: Lesson learns from a school of pharmacy in Thailand. BMC Medical Education, 23(1), 300. https://doi.org/10.1186/s12909-023-04232-1

Sriarunrasmee, J., & Seekheio, D. (2024). The development of a support system for the follow-up on basic education quality assessment and accreditation. Educational Administration: Theory and Practice, 30(7), 532-539. https://doi.org/10.53555/kuey.v30i7.6717

Sriputtarin, S. (2008). Developing a model for certifying external assessors in educational quality assessment of basic education. Education Journal of Thailand, 2(1), 73-83. https://ojs.lib.buu.ac.th/index.php/Education3/article/view/1630

Stensaker, B. R. (2008). Outcomes of quality assurance: A discussion of knowledge, methodology and validity. Quality in Higher Education, 14(1), 3-13. https://doi.org/10.1080/13538320802011532

Stensaker, B., & Leiber, T. (2015). Assessing the organisational impact of external quality assurance: Hypothesising key dimensions and mechanisms. Quality in Higher Education, 21(3), 328-342. https://doi.org/10.1080/13538322.2015.1111009

Stufflebeam, D. L. (2015). CIPP evaluation model checklist: A tool for applying the CIPP model to assess projects and programs. Western Michigan University Evaluation Center. http://tinyurl.com/mucdxtm7

Tangwarasittichai, K., Leekitchwatana, P., & Pimdee, P. (2021). Thai early childhood education external assessor competency indicator development. PalArch's Journal of Archaeology of Egypt/Egyptology, 18(3), 120-132. https://tinyurl.com/un2f6883

Toosi, M., Modarres, M., Amini, M., & Geranmayeh, M. (2021). Context, Input, Process, and Product Evaluation Model in medical education: A systematic review. Journal of Education and Health Promotion, 10(1), 199. https://doi.org/10.4103/jehp.jehp_1115_20

United Nations Educational, Scientific, & Cultural Organization. (2016). Reviews of National Policies for Education in Thailand An OECD-UNESCO Perspective: An OECD-UNESCO Perspective. OECD Publishing. https://tinyurl.com/3hbfrxdx

Valentine, N., Shanahan, E. M., Durning, S. J., & Schuwirth, L. (2021). Making it fair: Learners’ and assessors’ perspectives of the attributes of fair judgement. Medical Education, 55(9), 1056-066. https://doi.org/10.1111/medu.14574

Wang, Q., Hou, H., & Li, Z. (2022). Participative leadership: A literature review and prospects for future research. Frontiers in Psychology, 13, 924357. https://doi.org/10.3389/fpsyg.2022.924357

Weldearegay, C. A. (2020, December). External quality assurance in Ethiopian higher education: Comparing practices in public and private universities. (Doctoral thesis-Addis Ababa University). Ethiopia. https://tinyurl.com/394weya9

Wongrugsa, A., Kanjanawasee, S., & Ratchusanti, S. (2022). The efficiency of an evaluation model for undergraduate vocational education programs: An application of multiple evaluation approaches. Kasetsart Journal of Social Sciences, 43(3), 769-776. https://doi.org/10.34044/j.kjss.2022.43.3.32

Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2010). The program evaluation standards: A guide for evaluators and evaluation users. (3rd ed.), Sage Publications. http://tinyurl.com/39txh4v6

Zheng, L., Zhang, X., & Cui, P. (2020). The role of technology-facilitated peer assessment and supporting strategies: A meta-analysis. Assessment & Evaluation in Higher Education, 45(3), 372-386. https://doi.org/10.1080/02602938.2019.1644603

Announcement

EDUPIJ Citation Metrics

EDUPIJ News!