Volume 11 Issue 4 (2022)

Are Academics Satisfied with the Measurement and Evaluation Practices Applied During Emergency Remote Teaching due to COVID-19?

pp. 27-52  |  Published Online: December 2022  |  DOI: 10.22521/edupij.2022.114.2

Hasan Fehmi Özdemir, Çetin Toraman, Güneş Korkmaz

Abstract

Background/purpose – The aim of this research is to examine the meanings through metaphors that academics derive from their experiences regarding the measurement and evaluation practices when emergency remote teaching was conducted due to the COVID-19 pandemic, to identify the problems they experienced with the measurement and evaluation practices, and to present their solution recommendations.

Materials/methods – The study was structured as a qualitative research. The problems experienced by academics in measurement and evaluation practices, together with their solution suggestions were examined according to the “basic qualitative research” pattern. On the other hand, the “phenomenological qualitative research” design was used since the meanings attributed to the measurement and evaluation practice experiences were examined through metaphors. The participants of the study were 2,321 academics teaching at state and private universities in Turkey.

Results – The findings reveal that most of the problems experienced were related to “cheating, test security, fair exam environment, plagiarism, inability to measure whether learning objectives had been achieved, inappropriate online measurement and evaluation methods, lack of quality in assessment practices, and technical issues regarding learning management systems.”

Conclusion – The solutions proposed by the participant academics were the use of alternative assessment methods, conducting face-to-face rather than online exams, asking different questions for each student by mixing up the questions in the exam, and using webcams for online exam invigilation. Metaphors attributed to the measurement and evaluation practices were grouped under three categories; as positive metaphors (e.g., lighthouse, life buoy), negative metaphors (e.g., hallucination, digging a well with a needle), and metaphors implying that such practices played a key role (e.g., water in desert, surgery) in emergency.

Keywords: Emergency remote teaching, measurement and evaluation, academics, qualitative study, metaphors

References

Adedoyin, O. B., & Soykan, E. (2020). Covid-19 pandemic and online learning: the challenges and opportunities. Interactive Learning Environments. Advance online publication. https://doi.org/10.1080/10494820.2020.1813180

Agostinelli, M. D., Jr. (2019). From distance education to online education: a review of the literature. Graduate Student Theses, Dissertations, & Professional Papers. 11335. https://scholarworks.umt.edu/etd/11335

Aksoy, D. A., Bozkurt, A., & Kurşun, E. (2021). Yükseköğretim öğrencilerinin Koronavirüs (Covid-19) pandemi sürecinde uzaktan eğitime yönelik algıları. Anadolu Üniversitesi Eğitim Fakültesi Dergisi (AUJEF), 5(3), 285-308. https://dergipark.org.tr/en/download/article-file/1651651

Almpanis, T., & Joseph-Richard, P. (2022). Lecturing from home: Exploring academics’ experiences of remote teaching during a pandemic. International Journal of Educational Research Open, 3, Article 100133. https://doi.org/10.1016/j.ijedro.2022.100133

Armstrong‐Carter, E., & Telzer, E. H. (2021). Advancing measurement and research on youths’ prosocial behavior in the digital age. Child Development Perspectives15(1), 31-36. https://doi.org/10.1111/cdep.12396

Baniamer, Z., & Muhamed, B. (2022). Cheating in Online Exams: Motives, Methods and Ways of Preventing from the Perceptions of Business Students in Bahrain. In A. Hamdan, A. E. Hassanien, T. Mescon, & B. Alareeni (Eds.), Technologies, Artificial Intelligence and the Future of Learning Post-COVID-19 (pp. 267-282). Springer. https://doi.org/10.1007/978-3-030-93921-2_16

Bekova, S. K., Terentev, E. A., & Maloshonok, N. G. (2021). Educational Inequality and COVID‑19 Pandemic: Relationship between the Family Socio-Economic Status and Student Experience of Remote Learning. Educational Studies Moscow, 1, 74-92. https://doi.org/10.17323/1814-9545-2021-1-74-92

Bizami, N. A., Tasir, Z., & Kew, S. N. (2022). Innovative pedagogical principles and technological tools capabilities for immersive blended learning: a systematic literature review. Education and Information Technologies. Advance online publication. https://doi.org/10.1007/s10639-022-11243-w

Bozkurt, A. (2020). Koronavirüs (COVID-19) pandemisi sırasında ilköğretim öğrencilerinin uzaktan eğitime yönelik imge ve algıları: Bir metafor analizi. Uşak Üniversitesi Eğitim Araştırmaları Dergisi, 6(2), 1-23. https://doi.org/10.29065/usakead.777652

Bozkurt, A., Jung, I., Xiao, J., Vladimirschi, V., Schuwer, R., Egorov, G., & Rodes, V. (2020). A global outlook to the interruption of education due to COVID-19 Pandemic: Navigating in a time of uncertainty and crisis. Asian Journal of Distance Education, 15(1), 1-126. https://discovery.ucl.ac.uk/id/eprint/10101679/

Bozkurt, A., & Sharma, R. C. (2020). Emergency remote teaching in a time of global crisis due to CoronaVirus pandemic. Asian Journal of Distance Education, 15(1), i-vi. https://doi.org/10.5281/zenodo.3778083

Christensen, L. B., Johnson, R. B., & Turner, L. A. (2014). Research methods, design, and analysis. Pearson.

Creswell, J. W. (2007). Qualitative inquiry & research design, choosing among five approaches. Sage.

Creswell, J. W. (2012). Educational research: Planning conducting and evaluating quantitative and qualitative research. Pearson.

Creswell, J. W. (2014). Araştırma deseni: Nitel, nicel ve karma yöntem yaklaşımları (S. Beşir Demir, Trans./Ed.). Eğiten Kitap.

Çelik, B., Bilgin, R., & Yıldız, Y. (2022). The views of instructors in foreign language teaching with distance education model during the Covid 19 pandemic process: A study at Tishk International University in Erbil, Iraq. International Journal of Social Sciences & Educational Studies, 9(1), 148-176. https://ijsses.tiu.edu.iq/index.php/volume-9-issue-1-article-9/

Çetinkaya Aydın, G. (2020). COVID-19 Salgını Sürecinde Öğretmenler. TEDMEM. https://tedmem.org/covid-19/covid-19-salgini-surecinde-ogretmenler

Daniel, J. (2014). Foreword to the special section on massive open online courses MOOCs Evolution or revolution? Journal of Online Learning and Teaching, 10(1), i–iv. https://jolt.merlot.org/vol10no1/daniel_foreword_0314.pdf

Demir, E. (2015). Uzaktan eğitime genel bir bakış. Dumlupınar Üniversitesi Sosyal Bilimler Dergisi, 39, 203-212. https://dergipark.org.tr/tr/download/article-file/55935

Dimopoulos, K., Koutsampelas, C., & Tsatsaroni, A. (2021). Home schooling through online teaching in the era of COVID-19: Exploring the role of home-related factors that deepen educational inequalities across European societies. European Educational Research Journal20(4), 479-497. https://doi.org/10.1177/14749041211023331

El-Sakran, A., Salman, R., & Alzaatreh, A. (2022). Impacts of Emergency Remote Teaching on College Students Amid COVID-19 in the UAE. International Journal of Environmental Research and Public Health, 19(5), Article 2979. https://doi.org/10.3390/ijerph19052979

Forgács, B. (2022). The pragmatic functions of metaphorical language. In J. Gervain, G. Csibra, & K. Kovács (Eds.), A Life in Cognition (pp. 41-57). Springer.

Gunawardena, C. N., & LaPointe, D. K. (2003). Planning and management of student assessment. In S. Panda (Ed.), Planning & Management in Distance Education (pp. 195-205). Kogan.

Hagstrom, D., Hubbard, R., Hurtig, C., Mortola, P., Ostrow, J., & White, V. (2000). Teaching is like..? Educational Leadership, 57(8), 24-27. https://www.ascd.org/el/articles/teaching-is-like-.-.-.-

Henderson, M., Chung, J., Awdry, R., Mundy, M., Bryant, M., Ashford, C., & Ryan, K. (2022). Factors associated with online examination cheating. Assessment & Evaluation in Higher Education. Advance online publication. https://doi.org/10.1080/02602938.2022.2144802

Hoban, G. (2000). Using a reflective framework to study teaching–learning relationships. Reflective Practice, 1(2), 165-182. https://doi.org/10.1080/713693148

Hodges, C., Moore, S., Lockee, B., Trust, T., & Bond, A. (2020, March 27). The difference between emergency remote teaching and online learning. Educause Review. https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teaching-and-online-learning

Janke, S., Rudert, S. C., Petersen, Ä., Fritz, T. M., & Daumiller, M. (2021). Cheating in the wake of COVID-19: How dangerous is ad-hoc online testing for academic integrity? Computers and Education Open, 2, Article 100055. https://doi.org/10.1016/j.caeo.2021.100055

Junus, K., Santoso, H. B., Putra, P. O. H., Gandhi, A., & Siswantining, T. (2021). Lecturer readiness for online classes during the pandemic: A survey research. Education Sciences, 11(3), Article 139. https://doi.org/10.3390/educsci11030139

Kamble, K. P., & Ghorpade, V. R. (2021). Video Interpretation for Cost-Effective Remote Proctoring to Prevent Cheating. In Proceeding of First Doctoral Symposium on Natural Computing Research (pp. 259-269). Springer, Singapore. https://doi.org/10.1007/978-3-030-93921-2_16

Karaköse, T. (2021). Emergency remote teaching due to COVID-19 pandemic and potential risks for socioeconomically disadvantaged students in higher education. Educational Process: International Journal10(3), 53-62. https://dx.doi.org/10.22521/edupij.2021.103.4

Kemp, E. (1999). Metaphors as a tool for evaluation. Assessment and Evaluation in Higher Education, 24(1), 81-90. https://doi.org/10.1080/0260293990240107

Kilinc, H., Okur, M. R., & Usta, İ. (2021). The Opinions of Field Experts on Online Test Applications and Test Security During the COVID-19 Pandemic. International Journal of Assessment Tools in Education, 8(4), 975-990. https://doi.org/10.21449/ijate.875293

Korkmaz, G., & Toraman, Ç. (2020). Are we ready for the post-COVID-19 educational practice? An investigation into what educators think as to online learning. International Journal of Technology in Education and Science (IJTES), 4(4), 293-309. https://doi.org/10.46328/ijtes.v4i4.110

Krippendorff, K. (2004). Content analysis an introduction to its methodology. Sage.

Liberman, J., Levin, V., & Luna-Bazaldua, D. (2020, April 27). Are students still learning during COVID-19? Formative assessment can provide the answer. Education for Global Development. https://blogs.worldbank.org/education/are-students-still-learning-during-covid-19-formative-assessment-can-provide-answer

Linden, K., & Gonzalez, P. (2021). Zoom invigilated exams: A protocol for rapid adoption to remote examinations. British Journal of Educational Technology52(4), 1323-1337. https://doi.org/10.1111/bjet.13109

Mahajan, R., Saiyad, S., Virk, A., Joshi, A., & Singh, T. (2021). Blended programmatic assessment for competency based curricula. Journal of Postgraduate Medicine, 67(1), 18-23. https://doi.org/10.4103/jpgm.JPGM_1061_20

Mata, J. R. (2021). How to Teach Online? Recommendations for the assessment of online exams with University students in the USA in times of pandemic. International Journal of Educational Research and Innovation, 15, 188-202. https://doi.org/10.46661/ijeri.5003

Maxwell, J. A. (1996). Qualitative research design: An interactive approach. Sage.

Mayer, R. E. (2019). Thirty years of research on online learning. Applied Cognitive Psychology, 33(2), 152-159. https://doi.org/10.1002/acp.3482

Mayring, P. (2015). Qualitative content analysis: Theoretical background and procedures. In A. Bikner-Ahsbahs, C. Knipping, & N. Presmeg (Eds.), Approaches to Qualitative Research in Mathematics Education – Examples of Methodology and Methods (pp. 365-380). Springer. https://doi.org/10.1007/978-94-017-9181-6

Merriam, S. B., & Tisdell, E. J. (2016). Qualitative research, a guide to design and implementation. Jossey-Bass, Wiley.

Moore, M., & Kearsley, G. (1996). Distance Education: A Systems View. Wadsworth.

Nigam, A., Pasricha, R., Singh, T., & Churi, P. (2021). A systematic review on AI-based proctoring systems: Past, present and future. Education and Information Technologies26(5), 6421-6445. https://doi.org/10.1007/s10639-021-10597-x

Noorbehbahani, F., Mohammadi, A., & Aminazadeh, M. (2022). A systematic review of research on cheating in online exams from 2010 to 2021. Education and Information Technologies, 27, 8413-8460. https://doi.org/10.1007/s10639-022-10927-7

Olatunde-Aiyedun, T. G., Ogunode, N. J., & Eyiolorunse Aiyedun, C. T. (2021). Effectiveness of virtual learning during COVID-19 Lockdown in Nigerian universities. Academicia Globe: Inderscience Research2(5), 159-175. https://doi.org/10.17605/OSF.IO/S6N2Q

Patton, M. Q. (2002). Qualitative evaluation & research methods. Sage.

Pekcan, N., & Toraman, Ç. (2022). Covid-19 pandemisinde çevrimiçi ölçme ve değerlendirme uygulamalarinin öğretmen-öğrenci görüşlerine göre incelenmesi. Erzincan Üniversitesi Eğitim Fakültesi Dergisi, 24(1), 120-129. https://doi.org/10.17556/erziefd.862654

Rojas-de-Gracia, M. M., Casado-Molina, A. M., & Alarcón-Urbistondo, P. (2021). Relationship between reputational aspects of companies and their share price in the online environment. Technology in Society, 64, Article 101500. https://doi.org/10.1016/j.techsoc.2020.101500

Saban, A. (2008). Metaphors about School. Educational Administration: Theory and Practice, 55, 459-496. https://dergipark.org.tr/en/download/article-file/108283

Saleh, A. M., & Meccawy, Z. (2022). Teaching in Tough Times: Examining EFL Teachers’ Perceptions of Online Learning Challenges in the Context of Higher Education in Saudi Arabia. Journal of Education and Learning, 11(3), 47-57. https://doi.org/10.5539/jel.v11n3p47

Salta, K., Paschalidou, K., Tsetseri, M., & Koulougliotis, D. (2022). Shift from a Traditional to a Distance Learning Environment during the COVID-19 Pandemic. Science & Education, 31(1), 93-122. https://doi.org/10.1007/s11191-021-00234-x

Senel, S., & Senel, H. C. (2021). Remote Assessment in Higher Education during COVID-19 Pandemic. International Journal of Assessment Tools in Education, 8(2), 181-199. https://doi.org/10.21449/ijate.820140

Sharma, N. K., Gautam, D. K., Rathore, S., & Khan, M. R. (2021). CNN Implementation for Detect Cheating in Online Exams During COVID-19 Pandemic: A CVRU Perspective. Materials Today: Proceedings. https://doi.org/10.1016/j.matpr.2021.05.490

Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22, 63-75. https://doi.org/10.3233/EFI-2004-22201

Shohel, M. M. C., Shams, S., Ashrafuzzaman, M., Alam, A. S., Al Mamun, M. A., & Kabir, M. M. (2022). Emergency remote teaching and learning: Digital competencies and pedagogical transformation in resource-constrained contexts. In M. R. Islam, S. Kumar Behera, & L. Naibaho (Eds.), Handbook of Research on Asian Perspectives of the Educational Impact of COVID-19 (pp. 175-200). IGI Global. https://doi.org/10.4018/978-1-7998-8402-6.ch011

Simonson, M., Zvacek, S. M., & Smaldino, S. (2019). Teaching and Learning at a Distance: Foundations of Distance Education (7th ed.). IAP.

Spitzer, M. W. H., & Musslick, S. (2021). Academic performance of K-12 students in an online-learning environment for mathematics increased during the shutdown of schools in wake of the COVID-19 pandemic. PloS One, 16(8), Article e0255629. https://doi.org/10.1371/journal.pone.0255629

Steer, K., Gang, L., Layfield, C. & Borg, J. (2016, December 28-29). Assessment in e-learning for higher education [Conference presentation]. 2nd International Conference on Computers and Management (ICCM 2016). Kota (Rajasthan), India

Stephens, V., Matthews, L., Cornelissen, J. P., & Rowlands, H. (2022). Building Novel Supply Chain Theory Using “Metaphorical Imagination”. Journal of Supply Chain Management, 58(1), 124-139. https://doi.org/10.1111/jscm.12257

Sumner, J. (2010). Serving the system: A critical history of distance education. Open Learning: The Journal of Open, Distance and e-Learning, 15(3), 267-285. https://doi.org/10.1080/713688409

Surahman, E., & Wang, T. H. (2022). Academic dishonesty and trustworthy assessment in online learning: a systematic literature review. Journal of Computer Assisted Learning, 38, 1535-1553. https://doi.org/10.1111/jcal.12708

Tarigan, R. N., Nadlifatin, R., & Subriadi, A. P. (2021). Academic Dishonesty (Cheating) In Online Examination: A Literature Review. In Slamin, A. C. Prihandoko, F. Adnan, R. R. Suryono, E. Sulistiyani, & E. M. Rini (Eds.), International Conference on Computer Science, Information Technology, and Electrical Engineering (ICOMITEE) 2021 Proceeding (pp. 148-153). IEEE. https://doi.org/10.1109/ICOMITEE53461.2021.9650082

Tremblay, K., Lalancette, D., & Roseveare, D. (2012). Assessment of higher education learning outcomes: Feasibility study report, volume 1 design and implementation. OECD- Organisation for Economic Co-operation and Development. https://www.oecd.org/education/skills-beyond-chool/AHELOFSReportVolume1.pdf

UNESCO. (2020). School closures caused by Coronavirus (Covid-19). https://en.unesco.org/covid19/educationresponse#schoolclosures

Woon, J., & Ho, Y. (2005). Metaphorical construction of self in teachers’ narratives. Language and Education, 19(5), 359-379. https://doi.org/10.1080/09500780508668691

Yıldırım, A., & Şimşek, H. (2011). Sosyal bilimlerde nitel araştırma yöntemleri. Seçkin.

Yıldız, E. P., Alkan, A., & Çengel, M. (2021). Students’ metaphoric perceptions regarding the concept of distance education during the pandemic. Educational Research and Reviews, 16(6), 247-255. https://doi.org/10.5897/ERR2021.4159

Yüksek Öğretim Kurumu. (2020a). Basın açıklaması. https://www.yok.gov.tr/Sayfalar/Haberler/2020/YKSErtelenmesiBas%C4%B1nA%C3%A7%C4%B1klamas%C4%B1.aspx

Yüksek Öğretim Kurumu. (2020b). Koronavirüs (Covid-19) bilgilendirme notu: 1. https://www.yok.gov.tr/Sayfalar/Haberler/2020/coronavirus_bilgilendirme_1.aspx

Zhang, Z., Cao, T., Shu, J., & Liu, H. (2022). Identifying key factors affecting college students’ adoption of the e-learning system in mandatory blended learning environments. Interactive Learning Environments, 30(8), 1388-1401. https://doi.org/10.1080/10494820.2020.1723113

Zhao, Y., Lei, J., Yan, B., Lai, C., & Tan, H. S. (2005). What makes the difference? A practical analysis of research on the effectiveness of distance education. Teachers College Record, 107(8), 18-36. http://dx.doi.org/10.1111/j.1467-9620.2005.00544.x

Announcement

EDUPIJ News!

► Journal Metrics

  • 9% acceptance rate
  • 1.8 (2022) CiteScore (Scopus)
  • Q3 (2022) CiteScore Best Quartile
  • 0.294 (2022) Scimago Journal & Country Rank (SJR)
  • 0.612 (2022) Source-Normalized Impact per Paper (SNIP) 

EDUPIJ Statistics from Scopus

CiteScore: 1.8, view Scopus page

SCImago Journal & Country Rank

► Educational Process: International Journal is member of the Committee on Publication Ethics (COPE). 

► New issue coming soon! (Volume 13 Issue 2, 2024)