Adaptive Design for Critical Literacy: Validation of Readability-Based Reading Materials and HOTS Assessment in Indonesian Education (Grades 4–12)
Article Number: e2026073 | Available Online: May 2026 | DOI: 10.22521/edupij.2026.23.73
Subyantoro , Muhammad Badrus Siroj , Lisa Virdinarti Putra , Endang Kusrini , Sisca Septiani
Full text PDF |
292 |
146
Abstract
|
Background/purpose. This study developed and validated adaptive reading materials based on readability integrated with Higher Order Thinking Skills (HOTS) assessment for Indonesian elementary and secondary school students (grades 4–12). Materials/methods. This study used a Research and Development (R&D) design. The materials were reconstructed based on the Flesch–Kincaid Grade Level (FKGL) linear progression for grades 4–12, and the HOTS (C3–C6) assessment was designed to be calibrated. Validation was conducted by various stakeholders (N≈3000 students, N≈250 teachers, N=12 experts). Effectiveness was tested using a quasi-experimental design (Paired Sample T-test) with a sample of N=30, supplemented by item psychometric analysis (Difficulty and Discrimination Indices). Using a Research and Development (R&D) design based on the Borg & Gall model, this study involved multi-stakeholder validation with approximately 3,000 students, 250 teachers, and 12 experts, followed by a quasi-experimental effectiveness test (N=30). Results. The results show high design validity and successful text calibration with Flesch Reading Ease scores ranging from 85.2 (Grade 4) to 45.1 (Grade 12). High acceptance from stakeholders (average score >4.0/5.0). Psychometric analysis confirmed the difficulty level of most HOTS items (C3–C5). Effectiveness testing proved a significant improvement in critical reading competency (average increase of 12.567 points; p=0.000). |
Conclusion. This integrated adaptive design model has proven to be valid, feasible, and significantly effective in optimising Critical Reading Competency. This product offers a data-driven and scalable benchmark for improving critical literacy in Indonesian primary and secondary education.
Keywords: Higher-Order Thinking Skills, Literacy, Readability
ReferencesAdam, P., Kleemans, T., Droop, M., Amorim, A. N., & Segers, E. (2025). A digital boost of the home literacy environment positively impacts early reading and vocabulary development of Indonesian 1 1st graders. The Journal of Educational Research, 118(5), 425–436. https://doi.org/10.1080/00220671.2025.2490963
Adiredja, R. K., Hartati, T., & Riyana, C. (2023). Development of Integrated Writing Materials Based on Multiliteracies and High-Order Thinking Skills. Jurnal Kependidikan: Jurnal Hasil Penelitian Dan Kajian Kepustakaan Di Bidang Pendidikan, Pengajaran Dan Pembelajaran, 9(3), 882. https://doi.org/10.33394/jk.v9i3.8757
Afriani, N. R. (2022). Effectiveness Of The Problem Based Learning Model On Critical Thinking Ability About Science Subject For Fifth Grade Elementary School Students. EduHumaniora | Jurnal Pendidikan Dasar Kampus Cibiru, 14(1), 46–58. https://doi.org/10.17509/eh.v14i1.36791
Afrilyasanti, R., Basthomi, Y., & Zen, E. L. (2023). EFL students’ participations and teachers’ roles in online discussion forum for critical media literacy learning. Contemporary Educational Technology, 15(2), ep414. https://doi.org/10.30935/cedtech/12965
Amorim, C., & Ribeiro-Silva, E. (2025). Preservice physical education teachers’ perceptions of initial teacher education. European Physical Education Review, 31(3), 355–373. https://doi.org/10.1177/1356336X241276825
Ardhian, T., Ummah, I., Anafiah, S., & Rachmadtullah, R. (2020). Reading and Critical Thinking Techniques on Understanding Reading Skills for Early Grade Students in Elementary School. International Journal of Instruction, 13(2), 107–118. https://doi.org/10.29333/iji.2020.1328a
Aryana, S., Sobari, T., Wikanengsih, Fauziya, D. S., & Mustika, R. I. (2025). The Effectiveness of 21st Century Competency Assessment Model and HOTS in Improving Writing Skills: An Experimental Research. Journal of Language Teaching and Research, 16(4), 1115–1128. https://doi.org/10.17507/jltr.1604.07
Badolo, M., Malik, M. A., Nur, R., & Latifa, A. (2025). The Impact of Metacognitive Strategy Training on Higher-Order Thinking Skills (HOTS) in High School Mathematics: A Quasi-Experimental Study. International Journal of Environment, Engineering and Education, 7(2), 146–157. https://doi.org/10.55151/ijeedu.v7i2.302
Barton, G., Riddle, S., & Lowien, N. (2024). ““The last bastion of democracy”: teachers’ perceptions of the democratic potential of English curriculum”. English Teaching: Practice & Critique, 23(4), 475–491. https://doi.org/10.1108/ETPC-08-2024-0127
Beck, K. B., MacKenziem, K. T., Kirby, A. V., McDonald, K., Moura, I., Breitenfeldt, K., Rutenberg, E., Kumar, T., Mancino, J., Sabatello, M., Roth, S., & Nicolaidis, C. (2025). Guidelines for Creating Accessible Consent Materials and Procedures: Lessons from Research with Autistic People and People with Intellectual Disability. Autism in Adulthood. https://doi.org/10.1089/aut.2024.0263
Begimbetova, G. A., Retnawati, H., Kassymova, G. K., Triyono, M. B., Azman, M. N. A., & Sodikin, R. A. (2025). Developing Students’ HOTS Digital Literacy Profile by Using IRT. Journal of Curriculum Studies Research, 7(2), 155–176. https://doi.org/10.46303/jcsr.2025.16
Belda-Medina, J. (2024). Inclusive Education through Digital Comic Creation in Higher Learning Environments. Social Sciences, 13(5), 272. https://doi.org/10.3390/socsci13050272
Burke, E., Harkins, P., & Arumugasamy, M. (2024). A Critical Assessment of the Quality and Readability of Information on Gastric Cancer Available on the Internet for Patients. Cureus. https://doi.org/10.7759/cureus.71691
Ceballos, H., Bogaart, T. van de, van Ginkel, S., Spandaw, J., & Drijvers, P. (2026). How collaborative problem solving promotes higher-order thinking skills: A systematic review of design features and processes. Thinking Skills and Creativity, 59, 102001. https://doi.org/10.1016/j.tsc.2025.102001
Chipamaunga, S. R., Prozesky, D. R., Kafumukache, E., Katowa-Mukwato, P., Dithole, K. S., van der Merwe, L. J., Kagawa, M. N., & Gwini, R. (2025). Needs for Technology-Enhanced Health Professions Education in Eastern and Southern Africa: Protocol for a Descriptive, Cross-Sectional Survey. JMIR Research Protocols, 14, e67331. https://doi.org/10.2196/67331
Clark, C. E. J., & Post, G. (2021). Preparation and synchronous participation improve student performance in a blended learning experience. Australasian Journal of Educational Technology, 187–199. https://doi.org/10.14742/ajet.6811
Ekinci, F., Bektaş, O., Karaca, M., & Yiğit, K. N. (2023). The flipped learning perception scale: A validity and reliability study. Education and Information Technologies, 28(11), 14139–14166. https://doi.org/10.1007/s10639-023-11731-7
Foo, S. Y. (2025). Investigating gifted students’ higher-order thinking skills in a differentiated learning environment: A case study. Gifted Education International, 41(2), 236–259. https://doi.org/10.1177/02614294241305766
Gil-Rojas, B., Castellanos-Alvarenga, L. M., Gaitán-Angulo, M., Gómez-Caicedo, M. I., & Pérez-Pertuz, J. (2025). Critical Reading and Human Rights in Police Students: Development of Citizen Competencies to Prevent Crimes. SAGE Open, 15(2). https://doi.org/10.1177/21582440251335847
Hartini, S., Liliasari, L., & Sinaga, P. (2023). Students’ problem solving skill in nuclear physics course through NPIRL. Momentum: Physics Education Journal, 7(2), 279–289. https://doi.org/10.21067/mpej.v7i2.8567
Hartnett, D. A., Philips, A. P., Daniels, A. H., & Blankenhorn, B. D. (2025). Readability of Online Foot and Ankle Surgery Patient Education Materials. Foot & Ankle Specialist, 18(1), 9–18. https://doi.org/10.1177/19386400221116463
Indriani, R. P., Kurniati, T. H., & Ristanto, R. H. (2024). Implementing ARVi media to enhance students high order cognitive skills. JPBI (Jurnal Pendidikan Biologi Indonesia), 10(2), 572–579. https://doi.org/10.22219/jpbi.v10i2.32446
Istifadah, I., Usriyah, L., & Rahmawati, R. (2023). Triggering Interest in Learning through Environmentally Based Space Management. Al-Tanzim: Jurnal Manajemen Pendidikan Islam, 7(4), 1161–1176. https://doi.org/10.33650/al-tanzim.v7i4.6841
Kenny Tirtayasa, I. K., Nitiasih, P. K., & Budiarta, L. G. R. (2022). HOTS-Based Worksheet in English Learning Activity for Eight Grade Students of Junior High School. Jurnal Pendidikan Bahasa Inggris Undiksha, 10(1), 73–81. https://doi.org/10.23887/jpbi.v10i1.49824
Kim, H. J., Yi, P., & Hong, J. I. (2020). Students’ Academic Use of Mobile Technology and Higher-Order Thinking Skills: The Role of Active Engagement. Education Sciences, 10(3), 47. https://doi.org/10.3390/educsci10030047
Kwangmuang, P., Jarutkamolpong, S., Sangboonraung, W., & Daungtod, S. (2021). The development of learning innovation to enhance higher-order thinking skills for students in Thailand junior high schools. Heliyon, 7(6), e07309. https://doi.org/10.1016/j.heliyon.2021.e07309
Li, J. W., Leung, S. K. Y., Wang, M. D., & Yuen, M. (2024). Developing Talented Children’s Computational Thinking Through Multimodal Literacies in Pop-Up Storybooks: A Case Study in Hong Kong. Education Sciences, 14(12), 1377. https://doi.org/10.3390/educsci14121377
Lu, K., Yang, H. H., Shi, Y., & Wang, X. (2021). Examining the key influencing factors on college students’ higher-order thinking skills in the smart classroom environment. International Journal of Educational Technology in Higher Education, 18(1), 1. https://doi.org/10.1186/s41239-020-00238-7
M. Mujiya Ulkhaq. (2024). Factors Influencing Indonesian Students’ Performance on PISA 2018. Proceeding of The International Conference of Inovation, Science, Technology, Education, Children, and Health, 4(1), 74–87. https://doi.org/10.62951/icistech.v4i1.78
Mardiana, N., Maulina, B., Mardiani, N., Sabar, S., & Collantes, L. M. (2025). Enhancing Critical Thinking Skills Through Android-Assisted Virtual Physics Learning: A Focus On HOTS Development. Jurnal Ilmiah Ilmu Terapan Universitas Jambi, 9(2), 513–526. https://doi.org/10.22437/jiituj.v9i2.34373
Maxnun, L., Kristiani, K., & Sulistyaningrum, C. D. (2024). Development of hots-based cognitive assessment instruments: ADDIE model. Journal of Education and Learning (EduLearn), 18(2), 489–498. https://doi.org/10.11591/edulearn.v18i2.21079
Mukhlis, M., Suwandi, S., Rohmadi, M., & Setiawan, B. (2023). Higher Order Thinking Skills in Reading Literacy Questions at Vocational High Schools in Indonesia. International Journal of Language Education, 7(4). https://doi.org/10.26858/ijole.v7i4.37603
Nurherdiati, R. (2022). An Instructional Design to Improve Students’ Higher Order Thinking Skills in Linear Programming. J-PiMat : Jurnal Pendidikan Matematika, 4(1), 459–465. https://doi.org/10.31932/j-pimat.v4i1.1619
Palupi, T. M. (2024). Developing a learning management system for critical literacy course: A need analysis. Bahasa Dan Seni: Jurnal Bahasa, Sastra, Seni, Dan Pengajarannya, 51(1). https://doi.org/10.17977/um015v51i12023p91
Panigrahi, A. R., & Bakshi, S. G. (2021). Preparation, validation, and evaluation of an information leaflet for patients undergoing day-care surgeries under general anesthesia at a busy tertiary care hospital. Journal of Anaesthesiology Clinical Pharmacology, 37(2), 243–248. https://doi.org/10.4103/joacp.JOACP_264_19
Pujawan, I. G. N., Rediani, N. N., Antara, I. G. W. S., Putri, N. N. C. A., & Bayu, G. W. (2022). Revised Bloom Taxonomy-Oriented Learning Activities to Develop Scientific Literacy and Creative Thinking Skills. Jurnal Pendidikan IPA Indonesia, 11(1), 47–60. https://doi.org/10.15294/jpii.v11i1.34628
Puspitadani, E., Yudea, F., & Loo, F. (2022). Educational Leadership and Learning Quality: The Influence of the Principal’s Leadership Style on Teacher Performance. Jurnal Ilmu Pendidikan Dan Humaniora, 11(3), 206–220. https://doi.org/10.35335/jiph.v11i3.24
Rahmawati, T., Wahyuningrum H, M. M., Bustari, M., Lestari, S., Ernawati, R. D., & Selviana, S. (2023). Science Learning Management Based on Higher Order Thinking Skills (HOTS). Jurnal Penelitian Pendidikan IPA, 9(1), 533–541. https://doi.org/10.29303/jppipa.v9i1.3152
Randell, R. L., Wilson, H. P., Ragavan, M. I., Collins, A. B., Vail, J., Ramirez, S., Amodei, J., Mickievicz, E., Krieger, M. S., Macon, E. C., & Hornik, C. P. (2025). Communicating Health Research With Plain Language. INQUIRY: The Journal of Health Care Organization, Provision, and Financing, 62. https://doi.org/10.1177/00469580251357755
Sahhar, M., Singh, M., Mehta, T., Laud, A., Nassar, J. E., Farias, M. J., Diebo, B. G., & Daniels, A. H. (2025). Lost in Translation: Preoperative Orthopaedic Education Materials Significantly Exceed Recommended Reading Levels. JBJS Open Access, 10(3). https://doi.org/10.2106/JBJS.OA.25.00143
Simons, M., Smits, T. F. H., & Janssenswillen, P. (2020). Newspapers as teaching tools for media literacy education: what makes teachers use newspapers in their classrooms? Educational Media International, 57(4), 332–352. https://doi.org/10.1080/09523987.2020.1848510
Stepanenko, O., Suriak, M., Trubacheva, S., Sirenko, P., & Vlasiuk, I. (2023). Formation of Students’ Reading Competence in the Conditions of Integrated Learning. Cadernos de Educação, Tecnologia e Sociedade, 16(3), 648–656. https://doi.org/10.14571/brajets.v16.n3.648-656
Sukawati, F., Saude, S., Yusra, Y., & Ubadah, U. (2023). Inquiry Discovery Learning Method To Improve Student Learning Achievement In Islamic Religious Education Subjects. International Journal Of Contemporary Islamic Education, 5(2), 46–56. https://doi.org/10.24239/ijcied.Vol5.Iss2.80
Sutika, I. M., Winaya, I. M. A., Rai, I. B., Sila, I. M., Sudiarta, I. N., Kartika, I. M., & Sujana, I. G. (2023). The Effectiveness of the Problem-Based Learning Model in Improving Higher-Order Thinking Skills and Character of Elementary School Students. Jurnal Pendidikan Dan Pengajaran, 55(3), 688–702. https://doi.org/10.23887/jpp.v55i3.57636
Weng, T. (2023). Creating Critical Literacy Praxis: Bridging the Gap between Theory and Practice. RELC Journal, 54(1), 197–207. https://doi.org/10.1177/0033688220982665
Wu, T., Sarwono, E., & Huang, Y. (2025). Incorporating Computational Thinking Into Virtual Laboratories to Enhance Learning Motivation, Engagement, and Higher‐Order Thinking Skills. Journal of Computer Assisted Learning, 41(2). https://doi.org/10.1111/jcal.70017
Yang, Y., Li, Y., Fu, J., Guo, D., & Xue, J. (2025). Effectiveness of a Four-Stage Death Education Model Based on Constructivist Learning Theory for Trainee Nursing Students. Journal of Multidisciplinary Healthcare, Volume 18, 1371–1380. https://doi.org/10.2147/JMDH.S500169
Yüceer-Çetiner, E., Kazan, D., Nesiri, M., & Basa, S. (2025). Evaluating the Competence of AI Chatbots in Answering Patient-Oriented Frequently Asked Questions on Orthognathic Surgery. Healthcare, 13(17), 2114. https://doi.org/10.3390/healthcare13172114