Are Academics Satisfied with the Measurement and Evaluation Practices Applied During Emergency Remote Teaching due to COVID-19?
pp. 27-52 | Published Online: December 2022 | DOI: 10.22521/edupij.2022.114.2
Hasan Fehmi Özdemir, Çetin Toraman, Güneş Korkmaz
Background/purpose – The aim of this research is to examine the meanings through metaphors that academics derive from their experiences regarding the measurement and evaluation practices when emergency remote teaching was conducted due to the COVID-19 pandemic, to identify the problems they experienced with the measurement and evaluation practices, and to present their solution recommendations.
Materials/methods – The study was structured as a qualitative research. The problems experienced by academics in measurement and evaluation practices, together with their solution suggestions were examined according to the “basic qualitative research” pattern. On the other hand, the “phenomenological qualitative research” design was used since the meanings attributed to the measurement and evaluation practice experiences were examined through metaphors. The participants of the study were 2,321 academics teaching at state and private universities in Turkey.
Results – The findings reveal that most of the problems experienced were related to “cheating, test security, fair exam environment, plagiarism, inability to measure whether learning objectives had been achieved, inappropriate online measurement and evaluation methods, lack of quality in assessment practices, and technical issues regarding learning management systems.”
Conclusion – The solutions proposed by the participant academics were the use of alternative assessment methods, conducting face-to-face rather than online exams, asking different questions for each student by mixing up the questions in the exam, and using webcams for online exam invigilation. Metaphors attributed to the measurement and evaluation practices were grouped under three categories; as positive metaphors (e.g., lighthouse, life buoy), negative metaphors (e.g., hallucination, digging a well with a needle), and metaphors implying that such practices played a key role (e.g., water in desert, surgery) in emergency.
Keywords: Emergency remote teaching, measurement and evaluation, academics, qualitative study, metaphorsReferences
Adedoyin, O. B., & Soykan, E. (2020). Covid-19 pandemic and online learning: the challenges and opportunities. Interactive Learning Environments. Advance online publication. https://doi.org/10.1080/10494820.2020.1813180
Agostinelli, M. D., Jr. (2019). From distance education to online education: a review of the literature. Graduate Student Theses, Dissertations, & Professional Papers. 11335. https://scholarworks.umt.edu/etd/11335
Aksoy, D. A., Bozkurt, A., & Kurşun, E. (2021). Yükseköğretim öğrencilerinin Koronavirüs (Covid-19) pandemi sürecinde uzaktan eğitime yönelik algıları. Anadolu Üniversitesi Eğitim Fakültesi Dergisi (AUJEF), 5(3), 285-308. https://dergipark.org.tr/en/download/article-file/1651651
Almpanis, T., & Joseph-Richard, P. (2022). Lecturing from home: Exploring academics’ experiences of remote teaching during a pandemic. International Journal of Educational Research Open, 3, Article 100133. https://doi.org/10.1016/j.ijedro.2022.100133
Armstrong‐Carter, E., & Telzer, E. H. (2021). Advancing measurement and research on youths’ prosocial behavior in the digital age. Child Development Perspectives, 15(1), 31-36. https://doi.org/10.1111/cdep.12396
Baniamer, Z., & Muhamed, B. (2022). Cheating in Online Exams: Motives, Methods and Ways of Preventing from the Perceptions of Business Students in Bahrain. In A. Hamdan, A. E. Hassanien, T. Mescon, & B. Alareeni (Eds.), Technologies, Artificial Intelligence and the Future of Learning Post-COVID-19 (pp. 267-282). Springer. https://doi.org/10.1007/978-3-030-93921-2_16
Bekova, S. K., Terentev, E. A., & Maloshonok, N. G. (2021). Educational Inequality and COVID‑19 Pandemic: Relationship between the Family Socio-Economic Status and Student Experience of Remote Learning. Educational Studies Moscow, 1, 74-92. https://doi.org/10.17323/1814-9545-2021-1-74-92
Bizami, N. A., Tasir, Z., & Kew, S. N. (2022). Innovative pedagogical principles and technological tools capabilities for immersive blended learning: a systematic literature review. Education and Information Technologies. Advance online publication. https://doi.org/10.1007/s10639-022-11243-w
Bozkurt, A. (2020). Koronavirüs (COVID-19) pandemisi sırasında ilköğretim öğrencilerinin uzaktan eğitime yönelik imge ve algıları: Bir metafor analizi. Uşak Üniversitesi Eğitim Araştırmaları Dergisi, 6(2), 1-23. https://doi.org/10.29065/usakead.777652
Bozkurt, A., Jung, I., Xiao, J., Vladimirschi, V., Schuwer, R., Egorov, G., & Rodes, V. (2020). A global outlook to the interruption of education due to COVID-19 Pandemic: Navigating in a time of uncertainty and crisis. Asian Journal of Distance Education, 15(1), 1-126. https://discovery.ucl.ac.uk/id/eprint/10101679/
Bozkurt, A., & Sharma, R. C. (2020). Emergency remote teaching in a time of global crisis due to CoronaVirus pandemic. Asian Journal of Distance Education, 15(1), i-vi. https://doi.org/10.5281/zenodo.3778083
Christensen, L. B., Johnson, R. B., & Turner, L. A. (2014). Research methods, design, and analysis. Pearson.
Creswell, J. W. (2007). Qualitative inquiry & research design, choosing among five approaches. Sage.
Creswell, J. W. (2012). Educational research: Planning conducting and evaluating quantitative and qualitative research. Pearson.
Creswell, J. W. (2014). Araştırma deseni: Nitel, nicel ve karma yöntem yaklaşımları (S. Beşir Demir, Trans./Ed.). Eğiten Kitap.
Çelik, B., Bilgin, R., & Yıldız, Y. (2022). The views of instructors in foreign language teaching with distance education model during the Covid 19 pandemic process: A study at Tishk International University in Erbil, Iraq. International Journal of Social Sciences & Educational Studies, 9(1), 148-176. https://ijsses.tiu.edu.iq/index.php/volume-9-issue-1-article-9/
Çetinkaya Aydın, G. (2020). COVID-19 Salgını Sürecinde Öğretmenler. TEDMEM. https://tedmem.org/covid-19/covid-19-salgini-surecinde-ogretmenler
Daniel, J. (2014). Foreword to the special section on massive open online courses MOOCs Evolution or revolution? Journal of Online Learning and Teaching, 10(1), i–iv. https://jolt.merlot.org/vol10no1/daniel_foreword_0314.pdf
Demir, E. (2015). Uzaktan eğitime genel bir bakış. Dumlupınar Üniversitesi Sosyal Bilimler Dergisi, 39, 203-212. https://dergipark.org.tr/tr/download/article-file/55935
Dimopoulos, K., Koutsampelas, C., & Tsatsaroni, A. (2021). Home schooling through online teaching in the era of COVID-19: Exploring the role of home-related factors that deepen educational inequalities across European societies. European Educational Research Journal, 20(4), 479-497. https://doi.org/10.1177/14749041211023331
El-Sakran, A., Salman, R., & Alzaatreh, A. (2022). Impacts of Emergency Remote Teaching on College Students Amid COVID-19 in the UAE. International Journal of Environmental Research and Public Health, 19(5), Article 2979. https://doi.org/10.3390/ijerph19052979
Forgács, B. (2022). The pragmatic functions of metaphorical language. In J. Gervain, G. Csibra, & K. Kovács (Eds.), A Life in Cognition (pp. 41-57). Springer.
Gunawardena, C. N., & LaPointe, D. K. (2003). Planning and management of student assessment. In S. Panda (Ed.), Planning & Management in Distance Education (pp. 195-205). Kogan.
Hagstrom, D., Hubbard, R., Hurtig, C., Mortola, P., Ostrow, J., & White, V. (2000). Teaching is like..? Educational Leadership, 57(8), 24-27. https://www.ascd.org/el/articles/teaching-is-like-.-.-.-
Henderson, M., Chung, J., Awdry, R., Mundy, M., Bryant, M., Ashford, C., & Ryan, K. (2022). Factors associated with online examination cheating. Assessment & Evaluation in Higher Education. Advance online publication. https://doi.org/10.1080/02602938.2022.2144802
Hoban, G. (2000). Using a reflective framework to study teaching–learning relationships. Reflective Practice, 1(2), 165-182. https://doi.org/10.1080/713693148
Hodges, C., Moore, S., Lockee, B., Trust, T., & Bond, A. (2020, March 27). The difference between emergency remote teaching and online learning. Educause Review. https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teaching-and-online-learning
Janke, S., Rudert, S. C., Petersen, Ä., Fritz, T. M., & Daumiller, M. (2021). Cheating in the wake of COVID-19: How dangerous is ad-hoc online testing for academic integrity? Computers and Education Open, 2, Article 100055. https://doi.org/10.1016/j.caeo.2021.100055
Junus, K., Santoso, H. B., Putra, P. O. H., Gandhi, A., & Siswantining, T. (2021). Lecturer readiness for online classes during the pandemic: A survey research. Education Sciences, 11(3), Article 139. https://doi.org/10.3390/educsci11030139
Kamble, K. P., & Ghorpade, V. R. (2021). Video Interpretation for Cost-Effective Remote Proctoring to Prevent Cheating. In Proceeding of First Doctoral Symposium on Natural Computing Research (pp. 259-269). Springer, Singapore. https://doi.org/10.1007/978-3-030-93921-2_16
Karaköse, T. (2021). Emergency remote teaching due to COVID-19 pandemic and potential risks for socioeconomically disadvantaged students in higher education. Educational Process: International Journal, 10(3), 53-62. https://dx.doi.org/10.22521/edupij.2021.103.4
Kemp, E. (1999). Metaphors as a tool for evaluation. Assessment and Evaluation in Higher Education, 24(1), 81-90. https://doi.org/10.1080/0260293990240107
Kilinc, H., Okur, M. R., & Usta, İ. (2021). The Opinions of Field Experts on Online Test Applications and Test Security During the COVID-19 Pandemic. International Journal of Assessment Tools in Education, 8(4), 975-990. https://doi.org/10.21449/ijate.875293
Korkmaz, G., & Toraman, Ç. (2020). Are we ready for the post-COVID-19 educational practice? An investigation into what educators think as to online learning. International Journal of Technology in Education and Science (IJTES), 4(4), 293-309. https://doi.org/10.46328/ijtes.v4i4.110
Krippendorff, K. (2004). Content analysis an introduction to its methodology. Sage.
Liberman, J., Levin, V., & Luna-Bazaldua, D. (2020, April 27). Are students still learning during COVID-19? Formative assessment can provide the answer. Education for Global Development. https://blogs.worldbank.org/education/are-students-still-learning-during-covid-19-formative-assessment-can-provide-answer
Linden, K., & Gonzalez, P. (2021). Zoom invigilated exams: A protocol for rapid adoption to remote examinations. British Journal of Educational Technology, 52(4), 1323-1337. https://doi.org/10.1111/bjet.13109
Mahajan, R., Saiyad, S., Virk, A., Joshi, A., & Singh, T. (2021). Blended programmatic assessment for competency based curricula. Journal of Postgraduate Medicine, 67(1), 18-23. https://doi.org/10.4103/jpgm.JPGM_1061_20
Mata, J. R. (2021). How to Teach Online? Recommendations for the assessment of online exams with University students in the USA in times of pandemic. International Journal of Educational Research and Innovation, 15, 188-202. https://doi.org/10.46661/ijeri.5003
Maxwell, J. A. (1996). Qualitative research design: An interactive approach. Sage.
Mayer, R. E. (2019). Thirty years of research on online learning. Applied Cognitive Psychology, 33(2), 152-159. https://doi.org/10.1002/acp.3482
Mayring, P. (2015). Qualitative content analysis: Theoretical background and procedures. In A. Bikner-Ahsbahs, C. Knipping, & N. Presmeg (Eds.), Approaches to Qualitative Research in Mathematics Education – Examples of Methodology and Methods (pp. 365-380). Springer. https://doi.org/10.1007/978-94-017-9181-6
Merriam, S. B., & Tisdell, E. J. (2016). Qualitative research, a guide to design and implementation. Jossey-Bass, Wiley.
Moore, M., & Kearsley, G. (1996). Distance Education: A Systems View. Wadsworth.
Nigam, A., Pasricha, R., Singh, T., & Churi, P. (2021). A systematic review on AI-based proctoring systems: Past, present and future. Education and Information Technologies, 26(5), 6421-6445. https://doi.org/10.1007/s10639-021-10597-x
Noorbehbahani, F., Mohammadi, A., & Aminazadeh, M. (2022). A systematic review of research on cheating in online exams from 2010 to 2021. Education and Information Technologies, 27, 8413-8460. https://doi.org/10.1007/s10639-022-10927-7
Olatunde-Aiyedun, T. G., Ogunode, N. J., & Eyiolorunse Aiyedun, C. T. (2021). Effectiveness of virtual learning during COVID-19 Lockdown in Nigerian universities. Academicia Globe: Inderscience Research, 2(5), 159-175. https://doi.org/10.17605/OSF.IO/S6N2Q
Patton, M. Q. (2002). Qualitative evaluation & research methods. Sage.
Pekcan, N., & Toraman, Ç. (2022). Covid-19 pandemisinde çevrimiçi ölçme ve değerlendirme uygulamalarinin öğretmen-öğrenci görüşlerine göre incelenmesi. Erzincan Üniversitesi Eğitim Fakültesi Dergisi, 24(1), 120-129. https://doi.org/10.17556/erziefd.862654
Rojas-de-Gracia, M. M., Casado-Molina, A. M., & Alarcón-Urbistondo, P. (2021). Relationship between reputational aspects of companies and their share price in the online environment. Technology in Society, 64, Article 101500. https://doi.org/10.1016/j.techsoc.2020.101500
Saban, A. (2008). Metaphors about School. Educational Administration: Theory and Practice, 55, 459-496. https://dergipark.org.tr/en/download/article-file/108283
Saleh, A. M., & Meccawy, Z. (2022). Teaching in Tough Times: Examining EFL Teachers’ Perceptions of Online Learning Challenges in the Context of Higher Education in Saudi Arabia. Journal of Education and Learning, 11(3), 47-57. https://doi.org/10.5539/jel.v11n3p47
Salta, K., Paschalidou, K., Tsetseri, M., & Koulougliotis, D. (2022). Shift from a Traditional to a Distance Learning Environment during the COVID-19 Pandemic. Science & Education, 31(1), 93-122. https://doi.org/10.1007/s11191-021-00234-x
Senel, S., & Senel, H. C. (2021). Remote Assessment in Higher Education during COVID-19 Pandemic. International Journal of Assessment Tools in Education, 8(2), 181-199. https://doi.org/10.21449/ijate.820140
Sharma, N. K., Gautam, D. K., Rathore, S., & Khan, M. R. (2021). CNN Implementation for Detect Cheating in Online Exams During COVID-19 Pandemic: A CVRU Perspective. Materials Today: Proceedings. https://doi.org/10.1016/j.matpr.2021.05.490
Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22, 63-75. https://doi.org/10.3233/EFI-2004-22201
Shohel, M. M. C., Shams, S., Ashrafuzzaman, M., Alam, A. S., Al Mamun, M. A., & Kabir, M. M. (2022). Emergency remote teaching and learning: Digital competencies and pedagogical transformation in resource-constrained contexts. In M. R. Islam, S. Kumar Behera, & L. Naibaho (Eds.), Handbook of Research on Asian Perspectives of the Educational Impact of COVID-19 (pp. 175-200). IGI Global. https://doi.org/10.4018/978-1-7998-8402-6.ch011
Simonson, M., Zvacek, S. M., & Smaldino, S. (2019). Teaching and Learning at a Distance: Foundations of Distance Education (7th ed.). IAP.
Spitzer, M. W. H., & Musslick, S. (2021). Academic performance of K-12 students in an online-learning environment for mathematics increased during the shutdown of schools in wake of the COVID-19 pandemic. PloS One, 16(8), Article e0255629. https://doi.org/10.1371/journal.pone.0255629
Steer, K., Gang, L., Layfield, C. & Borg, J. (2016, December 28-29). Assessment in e-learning for higher education [Conference presentation]. 2nd International Conference on Computers and Management (ICCM 2016). Kota (Rajasthan), India
Stephens, V., Matthews, L., Cornelissen, J. P., & Rowlands, H. (2022). Building Novel Supply Chain Theory Using “Metaphorical Imagination”. Journal of Supply Chain Management, 58(1), 124-139. https://doi.org/10.1111/jscm.12257
Sumner, J. (2010). Serving the system: A critical history of distance education. Open Learning: The Journal of Open, Distance and e-Learning, 15(3), 267-285. https://doi.org/10.1080/713688409
Surahman, E., & Wang, T. H. (2022). Academic dishonesty and trustworthy assessment in online learning: a systematic literature review. Journal of Computer Assisted Learning, 38, 1535-1553. https://doi.org/10.1111/jcal.12708
Tarigan, R. N., Nadlifatin, R., & Subriadi, A. P. (2021). Academic Dishonesty (Cheating) In Online Examination: A Literature Review. In Slamin, A. C. Prihandoko, F. Adnan, R. R. Suryono, E. Sulistiyani, & E. M. Rini (Eds.), International Conference on Computer Science, Information Technology, and Electrical Engineering (ICOMITEE) 2021 Proceeding (pp. 148-153). IEEE. https://doi.org/10.1109/ICOMITEE53461.2021.9650082
Tremblay, K., Lalancette, D., & Roseveare, D. (2012). Assessment of higher education learning outcomes: Feasibility study report, volume 1 design and implementation. OECD- Organisation for Economic Co-operation and Development. https://www.oecd.org/education/skills-beyond-chool/AHELOFSReportVolume1.pdf
UNESCO. (2020). School closures caused by Coronavirus (Covid-19). https://en.unesco.org/covid19/educationresponse#schoolclosures
Woon, J., & Ho, Y. (2005). Metaphorical construction of self in teachers’ narratives. Language and Education, 19(5), 359-379. https://doi.org/10.1080/09500780508668691
Yıldırım, A., & Şimşek, H. (2011). Sosyal bilimlerde nitel araştırma yöntemleri. Seçkin.
Yıldız, E. P., Alkan, A., & Çengel, M. (2021). Students’ metaphoric perceptions regarding the concept of distance education during the pandemic. Educational Research and Reviews, 16(6), 247-255. https://doi.org/10.5897/ERR2021.4159
Yüksek Öğretim Kurumu. (2020a). Basın açıklaması. https://www.yok.gov.tr/Sayfalar/Haberler/2020/YKSErtelenmesiBas%C4%B1nA%C3%A7%C4%B1klamas%C4%B1.aspx
Yüksek Öğretim Kurumu. (2020b). Koronavirüs (Covid-19) bilgilendirme notu: 1. https://www.yok.gov.tr/Sayfalar/Haberler/2020/coronavirus_bilgilendirme_1.aspx
Zhang, Z., Cao, T., Shu, J., & Liu, H. (2022). Identifying key factors affecting college students’ adoption of the e-learning system in mandatory blended learning environments. Interactive Learning Environments, 30(8), 1388-1401. https://doi.org/10.1080/10494820.2020.1723113
Zhao, Y., Lei, J., Yan, B., Lai, C., & Tan, H. S. (2005). What makes the difference? A practical analysis of research on the effectiveness of distance education. Teachers College Record, 107(8), 18-36. http://dx.doi.org/10.1111/j.1467-9620.2005.00544.x