Volume 23 (2026) - In progress Download Cover Page

Adaptive Design for Critical Literacy: Validation of Readability-Based Reading Materials and HOTS Assessment in Indonesian Education (Grades 4–12)

Article Number: e2026073  |  Available Online: May 2026  |  DOI: 10.22521/edupij.2026.23.73

Subyantoro , Muhammad Badrus Siroj , Lisa Virdinarti Putra , Endang Kusrini , Sisca Septiani

Abstract

Background/purpose. This study developed and validated adaptive reading materials based on readability integrated with Higher Order Thinking Skills (HOTS) assessment for Indonesian elementary and secondary school students (grades 4–12).

Materials/methods. This study used a Research and Development (R&D) design. The materials were reconstructed based on the Flesch–Kincaid Grade Level (FKGL) linear progression for grades 4–12, and the HOTS (C3–C6) assessment was designed to be calibrated. Validation was conducted by various stakeholders (N≈3000 students, N≈250 teachers, N=12 experts). Effectiveness was tested using a quasi-experimental design (Paired Sample T-test) with a sample of N=30, supplemented by item psychometric analysis (Difficulty and Discrimination Indices). Using a Research and Development (R&D) design based on the Borg & Gall model, this study involved multi-stakeholder validation with approximately 3,000 students, 250 teachers, and 12 experts, followed by a quasi-experimental effectiveness test (N=30).

Results. The results show high design validity and successful text calibration with Flesch Reading Ease scores ranging from 85.2 (Grade 4) to 45.1 (Grade 12). High acceptance from stakeholders (average score >4.0/5.0). Psychometric analysis confirmed the difficulty level of most HOTS items (C3–C5). Effectiveness testing proved a significant improvement in critical reading competency (average increase of 12.567 points; p=0.000).

Conclusion. This integrated adaptive design model has proven to be valid, feasible, and significantly effective in optimising Critical Reading Competency. This product offers a data-driven and scalable benchmark for improving critical literacy in Indonesian primary and secondary education.

Keywords: Higher-Order Thinking Skills, Literacy, Readability

References

Adam, P., Kleemans, T., Droop, M., Amorim, A. N., & Segers, E. (2025). A digital boost of the home literacy environment positively impacts early reading and vocabulary development of Indonesian 1 1st graders. The Journal of Educational Research, 118(5), 425–436. https://doi.org/10.1080/00220671.2025.2490963

Adiredja, R. K., Hartati, T., & Riyana, C. (2023). Development of Integrated Writing Materials Based on Multiliteracies and High-Order Thinking Skills. Jurnal Kependidikan: Jurnal Hasil Penelitian Dan Kajian Kepustakaan Di Bidang Pendidikan, Pengajaran Dan Pembelajaran, 9(3), 882. https://doi.org/10.33394/jk.v9i3.8757

Afriani, N. R. (2022). Effectiveness Of The Problem Based Learning Model On Critical Thinking Ability About Science Subject For Fifth Grade Elementary School Students. EduHumaniora | Jurnal Pendidikan Dasar Kampus Cibiru, 14(1), 46–58. https://doi.org/10.17509/eh.v14i1.36791

Afrilyasanti, R., Basthomi, Y., & Zen, E. L. (2023). EFL students’ participations and teachers’ roles in online discussion forum for critical media literacy learning. Contemporary Educational Technology, 15(2), ep414. https://doi.org/10.30935/cedtech/12965

Amorim, C., & Ribeiro-Silva, E. (2025). Preservice physical education teachers’ perceptions of initial teacher education. European Physical Education Review, 31(3), 355–373. https://doi.org/10.1177/1356336X241276825

Ardhian, T., Ummah, I., Anafiah, S., & Rachmadtullah, R. (2020). Reading and Critical Thinking Techniques on Understanding Reading Skills for Early Grade Students in Elementary School. International Journal of Instruction, 13(2), 107–118. https://doi.org/10.29333/iji.2020.1328a

Aryana, S., Sobari, T., Wikanengsih, Fauziya, D. S., & Mustika, R. I. (2025). The Effectiveness of 21st Century Competency Assessment Model and HOTS in Improving Writing Skills: An Experimental Research. Journal of Language Teaching and Research, 16(4), 1115–1128. https://doi.org/10.17507/jltr.1604.07

Badolo, M., Malik, M. A., Nur, R., & Latifa, A. (2025). The Impact of Metacognitive Strategy Training on Higher-Order Thinking Skills (HOTS) in High School Mathematics: A Quasi-Experimental Study. International Journal of Environment, Engineering and Education, 7(2), 146–157. https://doi.org/10.55151/ijeedu.v7i2.302

Barton, G., Riddle, S., & Lowien, N. (2024). ““The last bastion of democracy”: teachers’ perceptions of the democratic potential of English curriculum”. English Teaching: Practice & Critique, 23(4), 475–491. https://doi.org/10.1108/ETPC-08-2024-0127

Beck, K. B., MacKenziem, K. T., Kirby, A. V., McDonald, K., Moura, I., Breitenfeldt, K., Rutenberg, E., Kumar, T., Mancino, J., Sabatello, M., Roth, S., & Nicolaidis, C. (2025). Guidelines for Creating Accessible Consent Materials and Procedures: Lessons from Research with Autistic People and People with Intellectual Disability. Autism in Adulthood. https://doi.org/10.1089/aut.2024.0263

Begimbetova, G. A., Retnawati, H., Kassymova, G. K., Triyono, M. B., Azman, M. N. A., & Sodikin, R. A. (2025). Developing Students’ HOTS Digital Literacy Profile by Using IRT. Journal of Curriculum Studies Research, 7(2), 155–176. https://doi.org/10.46303/jcsr.2025.16

Belda-Medina, J. (2024). Inclusive Education through Digital Comic Creation in Higher Learning Environments. Social Sciences, 13(5), 272. https://doi.org/10.3390/socsci13050272

Burke, E., Harkins, P., & Arumugasamy, M. (2024). A Critical Assessment of the Quality and Readability of Information on Gastric Cancer Available on the Internet for Patients. Cureus. https://doi.org/10.7759/cureus.71691

Ceballos, H., Bogaart, T. van de, van Ginkel, S., Spandaw, J., & Drijvers, P. (2026). How collaborative problem solving promotes higher-order thinking skills: A systematic review of design features and processes. Thinking Skills and Creativity, 59, 102001. https://doi.org/10.1016/j.tsc.2025.102001

Chipamaunga, S. R., Prozesky, D. R., Kafumukache, E., Katowa-Mukwato, P., Dithole, K. S., van der Merwe, L. J., Kagawa, M. N., & Gwini, R. (2025). Needs for Technology-Enhanced Health Professions Education in Eastern and Southern Africa: Protocol for a Descriptive, Cross-Sectional Survey. JMIR Research Protocols, 14, e67331. https://doi.org/10.2196/67331

Clark, C. E. J., & Post, G. (2021). Preparation and synchronous participation improve student performance in a blended learning experience. Australasian Journal of Educational Technology, 187–199. https://doi.org/10.14742/ajet.6811

Ekinci, F., Bektaş, O., Karaca, M., & Yiğit, K. N. (2023). The flipped learning perception scale: A validity and reliability study. Education and Information Technologies, 28(11), 14139–14166. https://doi.org/10.1007/s10639-023-11731-7

Foo, S. Y. (2025). Investigating gifted students’ higher-order thinking skills in a differentiated learning environment: A case study. Gifted Education International, 41(2), 236–259. https://doi.org/10.1177/02614294241305766

Gil-Rojas, B., Castellanos-Alvarenga, L. M., Gaitán-Angulo, M., Gómez-Caicedo, M. I., & Pérez-Pertuz, J. (2025). Critical Reading and Human Rights in Police Students: Development of Citizen Competencies to Prevent Crimes. SAGE Open, 15(2). https://doi.org/10.1177/21582440251335847

Hartini, S., Liliasari, L., & Sinaga, P. (2023). Students’ problem solving skill in nuclear physics course through NPIRL. Momentum: Physics Education Journal, 7(2), 279–289. https://doi.org/10.21067/mpej.v7i2.8567

Hartnett, D. A., Philips, A. P., Daniels, A. H., & Blankenhorn, B. D. (2025). Readability of Online Foot and Ankle Surgery Patient Education Materials. Foot & Ankle Specialist, 18(1), 9–18. https://doi.org/10.1177/19386400221116463

Indriani, R. P., Kurniati, T. H., & Ristanto, R. H. (2024). Implementing ARVi media to enhance students high order cognitive skills. JPBI (Jurnal Pendidikan Biologi Indonesia), 10(2), 572–579. https://doi.org/10.22219/jpbi.v10i2.32446

Istifadah, I., Usriyah, L., & Rahmawati, R. (2023). Triggering Interest in Learning through Environmentally Based Space Management. Al-Tanzim: Jurnal Manajemen Pendidikan Islam, 7(4), 1161–1176. https://doi.org/10.33650/al-tanzim.v7i4.6841

Kenny Tirtayasa, I. K., Nitiasih, P. K., & Budiarta, L. G. R. (2022). HOTS-Based Worksheet in English Learning Activity for Eight Grade Students of Junior High School. Jurnal Pendidikan Bahasa Inggris Undiksha, 10(1), 73–81. https://doi.org/10.23887/jpbi.v10i1.49824

Kim, H. J., Yi, P., & Hong, J. I. (2020). Students’ Academic Use of Mobile Technology and Higher-Order Thinking Skills: The Role of Active Engagement. Education Sciences, 10(3), 47. https://doi.org/10.3390/educsci10030047

Kwangmuang, P., Jarutkamolpong, S., Sangboonraung, W., & Daungtod, S. (2021). The development of learning innovation to enhance higher-order thinking skills for students in Thailand junior high schools. Heliyon, 7(6), e07309. https://doi.org/10.1016/j.heliyon.2021.e07309

Li, J. W., Leung, S. K. Y., Wang, M. D., & Yuen, M. (2024). Developing Talented Children’s Computational Thinking Through Multimodal Literacies in Pop-Up Storybooks: A Case Study in Hong Kong. Education Sciences, 14(12), 1377. https://doi.org/10.3390/educsci14121377

Lu, K., Yang, H. H., Shi, Y., & Wang, X. (2021). Examining the key influencing factors on college students’ higher-order thinking skills in the smart classroom environment. International Journal of Educational Technology in Higher Education, 18(1), 1. https://doi.org/10.1186/s41239-020-00238-7

M. Mujiya Ulkhaq. (2024). Factors Influencing Indonesian Students’ Performance on PISA 2018. Proceeding of The International Conference of Inovation, Science, Technology, Education, Children, and Health, 4(1), 74–87. https://doi.org/10.62951/icistech.v4i1.78

Mardiana, N., Maulina, B., Mardiani, N., Sabar, S., & Collantes, L. M. (2025). Enhancing Critical Thinking Skills Through Android-Assisted Virtual Physics Learning: A Focus On HOTS Development. Jurnal Ilmiah Ilmu Terapan Universitas Jambi, 9(2), 513–526. https://doi.org/10.22437/jiituj.v9i2.34373

Maxnun, L., Kristiani, K., & Sulistyaningrum, C. D. (2024). Development of hots-based cognitive assessment instruments: ADDIE model. Journal of Education and Learning (EduLearn), 18(2), 489–498. https://doi.org/10.11591/edulearn.v18i2.21079

Mukhlis, M., Suwandi, S., Rohmadi, M., & Setiawan, B. (2023). Higher Order Thinking Skills in Reading Literacy Questions at Vocational High Schools in Indonesia. International Journal of Language Education, 7(4). https://doi.org/10.26858/ijole.v7i4.37603

Nurherdiati, R. (2022). An Instructional Design to Improve Students’ Higher Order Thinking Skills in Linear Programming. J-PiMat : Jurnal Pendidikan Matematika, 4(1), 459–465. https://doi.org/10.31932/j-pimat.v4i1.1619

Palupi, T. M. (2024). Developing a learning management system for critical literacy course: A need analysis. Bahasa Dan Seni: Jurnal Bahasa, Sastra, Seni, Dan Pengajarannya, 51(1). https://doi.org/10.17977/um015v51i12023p91

Panigrahi, A. R., & Bakshi, S. G. (2021). Preparation, validation, and evaluation of an information leaflet for patients undergoing day-care surgeries under general anesthesia at a busy tertiary care hospital. Journal of Anaesthesiology Clinical Pharmacology, 37(2), 243–248. https://doi.org/10.4103/joacp.JOACP_264_19

Pujawan, I. G. N., Rediani, N. N., Antara, I. G. W. S., Putri, N. N. C. A., & Bayu, G. W. (2022). Revised Bloom Taxonomy-Oriented Learning Activities to Develop Scientific Literacy and Creative Thinking Skills. Jurnal Pendidikan IPA Indonesia, 11(1), 47–60. https://doi.org/10.15294/jpii.v11i1.34628

Puspitadani, E., Yudea, F., & Loo, F. (2022). Educational Leadership and Learning Quality: The Influence of the Principal’s Leadership Style on Teacher Performance. Jurnal Ilmu Pendidikan Dan Humaniora, 11(3), 206–220. https://doi.org/10.35335/jiph.v11i3.24

Rahmawati, T., Wahyuningrum H, M. M., Bustari, M., Lestari, S., Ernawati, R. D., & Selviana, S. (2023). Science Learning Management Based on Higher Order Thinking Skills (HOTS). Jurnal Penelitian Pendidikan IPA, 9(1), 533–541. https://doi.org/10.29303/jppipa.v9i1.3152

Randell, R. L., Wilson, H. P., Ragavan, M. I., Collins, A. B., Vail, J., Ramirez, S., Amodei, J., Mickievicz, E., Krieger, M. S., Macon, E. C., & Hornik, C. P. (2025). Communicating Health Research With Plain Language. INQUIRY: The Journal of Health Care Organization, Provision, and Financing, 62. https://doi.org/10.1177/00469580251357755

Sahhar, M., Singh, M., Mehta, T., Laud, A., Nassar, J. E., Farias, M. J., Diebo, B. G., & Daniels, A. H. (2025). Lost in Translation: Preoperative Orthopaedic Education Materials Significantly Exceed Recommended Reading Levels. JBJS Open Access, 10(3). https://doi.org/10.2106/JBJS.OA.25.00143

Simons, M., Smits, T. F. H., & Janssenswillen, P. (2020). Newspapers as teaching tools for media literacy education: what makes teachers use newspapers in their classrooms? Educational Media International, 57(4), 332–352. https://doi.org/10.1080/09523987.2020.1848510

Stepanenko, O., Suriak, M., Trubacheva, S., Sirenko, P., & Vlasiuk, I. (2023). Formation of Students’ Reading Competence in the Conditions of Integrated Learning. Cadernos de Educação, Tecnologia e Sociedade, 16(3), 648–656. https://doi.org/10.14571/brajets.v16.n3.648-656

Sukawati, F., Saude, S., Yusra, Y., & Ubadah, U. (2023). Inquiry Discovery Learning Method To Improve Student Learning Achievement In Islamic Religious Education Subjects. International Journal Of Contemporary Islamic Education, 5(2), 46–56. https://doi.org/10.24239/ijcied.Vol5.Iss2.80

Sutika, I. M., Winaya, I. M. A., Rai, I. B., Sila, I. M., Sudiarta, I. N., Kartika, I. M., & Sujana, I. G. (2023). The Effectiveness of the Problem-Based Learning Model in Improving Higher-Order Thinking Skills and Character of Elementary School Students. Jurnal Pendidikan Dan Pengajaran, 55(3), 688–702. https://doi.org/10.23887/jpp.v55i3.57636

Weng, T. (2023). Creating Critical Literacy Praxis: Bridging the Gap between Theory and Practice. RELC Journal, 54(1), 197–207. https://doi.org/10.1177/0033688220982665

Wu, T., Sarwono, E., & Huang, Y. (2025). Incorporating Computational Thinking Into Virtual Laboratories to Enhance Learning Motivation, Engagement, and Higher‐Order Thinking Skills. Journal of Computer Assisted Learning, 41(2). https://doi.org/10.1111/jcal.70017

Yang, Y., Li, Y., Fu, J., Guo, D., & Xue, J. (2025). Effectiveness of a Four-Stage Death Education Model Based on Constructivist Learning Theory for Trainee Nursing Students. Journal of Multidisciplinary Healthcare, Volume 18, 1371–1380. https://doi.org/10.2147/JMDH.S500169

Yüceer-Çetiner, E., Kazan, D., Nesiri, M., & Basa, S. (2025). Evaluating the Competence of AI Chatbots in Answering Patient-Oriented Frequently Asked Questions on Orthognathic Surgery. Healthcare, 13(17), 2114. https://doi.org/10.3390/healthcare13172114