AI-Enhanced Research Coaching for Graduate Education: A Mixed-Methods Evaluation of Learning Outcomes and Educational Design Effectiveness

Background/purpose. This study addresses challenges in graduate research supervision by developing and evaluating an AI chatbot research coaching system designed through Design Thinking and Educational Design Research. The research aimed to examine system usability, perceived value, and alignment between user experiences and original design principles.

Materials/methods. Thirty graduate students from Learning Management Innovation programs at a Thai university participated in a mixed-methods evaluation. The system, grounded in the GROW coaching model and delivered via LINE and Dialogflow, provided structured guidance, templates, and automated feedback. Data collection included Likert-scale questionnaires, open-ended reflections, and focus group discussions, which the researchers analyzed using a convergent mixed-methods approach.

Results. The system achieved high satisfaction (M = 4.52) and strong emotional engagement across diverse levels of digital literacy. Qualitative analysis revealed six themes, including enhanced understanding of research, increased confidence, and perceived social presence. Participants developed unexpected emotional attachments, describing the system as a "thinking partner." A strong alignment emerged between user feedback and design principles, yielding notable outcomes in conceptual understanding, skill application, and milestone completion.

Conclusion. Results demonstrate that emotionally intelligent AI coaching systems can effectively support graduate research development by addressing both cognitive and affective learning dimensions. The study provides a validated framework for developing culturally sensitive, user-centered educational technologies in resource-constrained contexts.

Keywords: Graduate education, educational technology, AI-powered learning support, research mentoring, educational design research, chatbot

References

Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher, 41(1), 16–25. https://doi.org/10.3102/0013189X11428813

Bailey, D. R., Almusharraf, N., & Almusharraf, A. (2022). Video conferencing in the e-learning context: explaining learning outcome with the technology acceptance model. Education and Information Technologies, 27(6), 7679-7698. https://doi.org/10.1007/s10639-022-10949-1

Banjar, H. R., Alsefri, L., Alshomrani, A., Hamdhy, M., Alahmari, S., & Sharaf, S. (2024). Activating the Mobile User Interface With a Rule‐Based Chatbot and EEG‐Based Emotion Recognition to Aid in Coping With Negative Emotions. Human Behavior and Emerging Technologies, 2024(1), 7499554. https://doi.org/10.1155/2024/7499554

Bayne, S. (2015). Posthumanism and research in digital education. Discourse: Studies in the

Cultural Politics of Education, 36(1), 4–15. https://doi.org/10.1080/01596306.2013.849484

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Brown, T. (2009). Change by design: How design thinking creates new alternatives for business and society. Harvard Business Press.

Creswell, J. W., & Plano Clark, V. L. (2017). Designing and conducting mixed methods research (3rd ed.). Sage.

Fetters, M. D., Curry, L. A., & Creswell, J. W. (2013). Achieving integration in mixed methods designs—principles and practices. Health services research, 48(6pt2), 2134–2156. https://doi.org/10.1111/1475-6773.12117

Gardner, S. K. (2009). The development of doctoral students: Phases of challenge and support. ASHE Higher Education Report, 34(6), 1–127. http://dx.doi.org/10.1002/aehe.3406

Gessala, N., Kwangmuang, P., Kanjanasorn, W., & Srikoon, S. (2025). Assessing design thinking proficiency in pre-service teachers: A comprehensive validation study. Cogent Education, 12(1), 2521156. https://doi.org/10.1080/2331186X.2025.2521156

Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? Field Methods, 18(1), 59–82. https://doi.org/10.1177/1525822X05279903

Hassenzahl, M. (2010). Experience design: Technology for all the right reasons. Morgan & Claypool.

Hassenzahl, M., & Tractinsky, N. (2006). User experience – a research agenda. Behaviour & Information Technology, 25(2), 91–97. https://doi.org/10.1080/01449290500330331

Holmes, W., Bialik, M., & Fadel, C. (2019). Artificial intelligence in education: Promises and implications for teaching and learning. Center for Curriculum Redesign.

Kim, C., Park, H., & Cozart, J. (2022). Affective and cognitive dimensions of the online learning experience. Educational Technology Research and Development, 70(1), 13–31. https://doi.org/10.1007/s11423-021-10044-4

Kim, H. J., Yi, P., & Ko, B. W. (2023). Deepening students' experiences with problem identification and definition in an empathetic approach: lessons from a university design-thinking program. Journal of Applied Research in Higher Education, 15(3), 852-865. https://doi.org/10.1108/JARHE-03-2022-0083

Lee, A. (2008). How are doctoral students supervised? Concepts of doctoral research supervision. Studies in Higher Education, 33(3), 267–281. https://doi.org/10.1080/03075070802049202

Lee, J., Lee, H., & Wang, Y. (2022). Chatbots in education: A systematic review. Computers & Education: Artificial Intelligence, 3, 100060. https://doi.org/10.1016/j.caeai.2022.100060

McKenney, S., & Reeves, T. C. (2012). Conducting educational design research. Routledge.

Mohebbi, A. (2025). Enabling learner independence and self-regulation in language education using AI tools: a systematic review. Cogent Education, 12(1), 2433814. https://doi.org/10.1080/2331186X.2024.2433814

Moretti, A. E., & Simmons, C. A. (2023). Psychology doctoral students’ satisfaction with virtual supervision during COVID-19. Training and Education in Professional Psychology, 17(2), 176–184. https://doi.org/10.1037/tep0000411

Norman, D. A. (2004). Emotional design: Why we love (or hate) everyday things. Basic Books.

OECD. (2021). The future of education and skills 2030: OECD learning compass. OECD Publishing.

OECD. (2023). Education at a Glance 2023: OECD Indicators. OECD Publishing. https://doi.org/10.1787/69096873-en

Pergantis, P., Bamicha, V., Skianis, C., & Drigas, A. (2025). Ai chatbots and cognitive control: Enhancing executive functions through chatbot interactions: A systematic review. Brain Sciences, 15(1), 47. https://doi.org/10.3390/brainsci15010047

Pérez-Marín, D. (2021). A review of the practical applications of pedagogic conversational agents. British Journal of Educational Technology, 52(3), 778–796. https://doi.org/10.1111/bjet.13058

Plomp, T. (2007). Educational design research: An introduction. In T. Plomp & N. Nieveen (Eds.), An introduction to educational design research (pp. 9–35). Netherlands Institute for Curriculum Development (SLO).

Pumyoch, N., & Srikoon, S. (2024). Executive function training curriculum to enhance emotional intelligence in early childhood: Theory adaptation in educational design research. Teaching and Teacher Education: Leadership and Professional Development, 8, 101673. https://doi.org/10.1016/j.tsc.2024.101673

Pyhältö, K., Stubb, J., & Lonka, K. (2009). Developing scholarly communities as learning environments for doctoral students. International Journal for Academic Development, 14(3), 221–232. https://doi.org/10.1080/13601440903106551

Salah, M., Alhalbusi, H., Ismail, M. M., & Abdelfattah, F. (2024). Chatting with ChatGPT: decoding the mind of Chatbot users and unveiling the intricate connections between user perception, trust and stereotype perception on self-esteem and psychological well-being. Current Psychology, 43(9), 7843-7858. https://doi.org/10.1007/s12144-023-04989-0

Srisa-ard, B., & Sangsiri, T. (2022). Challenges in postgraduate research: A study of Thai graduate students. Kasetsart Journal of Social Sciences, 43(2), 148–160.

Turner, R. C., & Carlson, L. (2003). Indexes of item-objective congruence for multidimensional items. International journal of testing, 3(2), 163-171. https://doi.org/10.1207/S15327574IJT0302_5

UNESCO. (2018). ICT competency framework for teachers: Version 3. UNESCO Publishing.

UNESCO. (2021). Reimagining our futures together: A new social contract for education. UNESCO Publishing.

Van den Akker, J. (1999). Principles and methods of development research. In J. van den Akker, R. M. Branch, K. Gustafson, N. Nieveen, & T. Plomp (Eds.), Design approaches and tools in education and training (pp. 1–14). Springer. https://doi.org/10.1007/978-94-011-4255-7_1

Venkatesh, V., & Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46(2), 186–204. https://doi.org/10.1287/mnsc.46.2.186.11926

Wang, Y., Yu, L., & Fesenmaier, D. R. (2021). Transforming education with intelligent agents: From traditional learning to smart learning. Educational Technology Research and Development, 69(4), 2231–2250. https://doi.org/10.1007/s11423-021-10058-y

Whitmore, J. (2009). Coaching for performance (4th ed.). Nicholas Brealey Publishing.

Winkler, R., & Söllner, M. (2018). Unleashing the potential of chatbots in education: A state- of-the-art analysis. ACM Transactions on Human-Computer Interaction, 25(3), 1–46. https://doi.org/10.1145/3158661

Wongwanich, S. (2020). Design-based research for educational innovation. Chulalongkorn University Press.

Zeithaml, V. A. (1988). Consumer perceptions of price, quality, and value: A means–end model and synthesis of evidence. Journal of Marketing, 52(3), 2–22. https://doi.org/10.1177/002224298805200302

Zhou, M., Xu, Q., & Wang, D. (2023). Exploring the role of empathy in intelligent tutoring systems. Computers & Education, 199, 104744. https://doi.org/10.1016/j.compedu.2023.104744

Announcement

EDUPIJ Citation Metrics

EDUPIJ News!