Introduction
Distance education is a planned educational activity that utilizes various technological tools to facilitate interactive or non-interactive teaching methods between the teacher and learner, even when they are not in the same physical environment. While unforeseen circumstances may necessitate the replacement of face-to-face education with distance education, it is crucial to emphasize that this should always be a carefully planned and executed activity. The Covid-19 pandemic, which emerged in late 2019, has caused a global shift from face-to-face to distance education at all levels. This transition has had a significant impact on educational activities worldwide with diverse implementations in higher education (Stewart, 2021). In contrast to the learning experiences that were planned and designed online from the beginning of the process, the transition to an alternative delivery mode of teaching activities considering the crisis conditions was defined as ‘Emergency Remote Teaching’ (ERT) (Hodges et al., 2020). Since ERT lacked online learning design, which is one of the most important elements of distance education, some problems were observed in the implementation process. The solution developed in the short term was to transfer the education and training activities, which are mostly carried out face-to-face, to the digital environment through video conferencing software (Sezgin, 2021). However, this has brought many problems and uncertainties, especially in assessment and evaluation activities. The fact that face-to-face exams, which constituted a large part of the assessment and evaluation practices carried out in higher education institutions before the pandemic, could not be carried out in this period, led to the emphasis on practices such as homework, projects and online assessment and evaluation. In addition to the advantages of online exams such as providing students with flexibility in terms of time, collecting data more quickly, determining the results quickly, alleviating the workload of instructors such as printing and distributing questions, creating question pools and providing instant feedback, there are also limitations such as the need for hardware and internet connection and requiring special online assessment and evaluation software. Regardless of the method of implementation, it is important that assessment and evaluation practices reflect the actual performance of the student. Therefore, ensuring the reliability of online assessment and evaluation practices becomes very important. Studies on online assessment and evaluation practices in distance education have shown that students have been observed to commit academic irregularities in online exams and therefore academic insecurity is felt more in online assessment and evaluation processes than in face-to-face assessment and evaluation processes (Bozkurt & Uçar, 2018; Newton & Essex, 2022; Valizadeh, 2022). Accordingly, it has become necessary to take some security measures in online assessment and evaluation practices (Bozkurt & Uçar, 2018).
In order to better manage the online assessment and evaluation process by preventing cheating and plagiarism, academic staff and faculties should be informed about the necessary procedures (Gamage et al., 2020). Some of the important recommendations for instructors include informing students in advance about the way the exam will be administered and the ethical rules, monitoring students’ behavior during the exam, recording camera footage and obtaining students’ explicit consent in accordance with the personal data protection for transactions related to this data, and organizing mock exams after decisions on how the exam will be conducted. Reminding students of the ethical rules by instructors to make them aware of the moral and ethical seriousness of this practice is one of the important factors in increasing deterrence (Meccawy et al., 2021). In addition, it has been observed that the tendency to cheat can be prevented in camera-recorded online exams (Golden & Kohlbeck, 2020; Kılınç et al., 2021).
The Covid-19 pandemic made the transition to distance education compulsory and all stakeholders of the education process were caught unprepared for this situation. In addition to technical issues such as the inadequacy of distance education infrastructure in universities, the lack of readiness of students and lecturers for distance education has also posed a problem and difficulties have arisen in ensuring exam security. Although the steps for conducting the courses are provided quickly with live lecture platforms, how the exams will be conducted remotely has been a source of concern. In this process, technology has played a dual role in both facilitating and detecting academic misconduct (Gamage et al., 2020). Instructors have experienced various problems in ensuring exam security for a fair evaluation of online exam processes (Bozkurt & Uçar, 2018; Newton & Essex, 2022; Valizadeh, 2022). As a matter of fact, many decisions regarding exams have been taken by both universities and The Council of Higher Education (CoHE) in the process. This study discusses what kind of problems can be experienced in online exams and how measures can be taken against these problems.
One of the most significant issues associated with online examinations is the maintenance of academic integrity. The administration of exams in an online environment presents a number of challenges in terms of preventing academic dishonesty. This gives rise to questions regarding the reliability of the resulting examination grades. In contrast to traditional face-to-face examinations, where students are subject to monitoring in a controlled setting, it has been asserted that instances of academic dishonesty are more prevalent in online exams (Hillier, 2014; Lanier, 2006). Consequently, greater vigilance is required to prevent such malpractice. In this context, the reliability of academic outcomes in online assessment processes is more susceptible to challenge than in face-to-face assessments. This underscores the importance of robust and effective implementation of exam security measures. In relation to this, the purpose of this study is to determine the exam security measures regarding online assessment and evaluation practices carried out at higher education in the ERT conditions during the Covid-19 global pandemic and to provide suggestions regarding exam security measures in distance education in future including emergency situations. Using a case study approach, the venue has been a higher education institution in the midwest of Türkiye. Considering the problems experienced in assessment and evaluation practices in distance education conducted during the Covid-19 process, it is thought that the findings to be obtained from the study will contribute to the literature. In addition, in the current study, by examining the experiences of the instructors regarding the online exams conducted in the ERT process, an evaluation of the security measures in these exams and guiding future studies in this field is intended. In this context, the research seeks answers to the following questions for the case:
What are the measures taken to ensure the security of online exams?
What are the challenges faced in ensuring the security of online exams conducted?
What are the solutions suggested by the instructors to ensure the security of online exams conducted?
It is anticipated that by examining the comprehensive approach to exam security practices at a university during ERT, this study can provide insights that can inform future efforts to enhance similar practices across all institutions. In addition, it is thought that the solutions proposed in the study will guide universities and instructors who experience similar problems in the context of online exams.
Background and Related Literature
The security of online exams in distance education is a highly debated issue (Garg & Goel, 2022), especially when it comes to documenting qualifications such as certificates and diplomas. The main reasons include privacy concerns, technological issues, potential biases in proctoring algorithms, differences in access to resources, and increased test anxiety among students during online exams (Kuleva ve Miladinov, 2024). In addition, issues such as student authentication and remembering strong passwords, coupled with the risk of biometric spoofing, further complicate the security of online exams (Nguyen et al., 2023). To eliminate these potential problems, educational institutions generally prefer to conduct supervised face-to-face exams (Aksoy, 2018). In Karadağ and Özgür’s (2020) study, in which they examined the assessment and evaluation practices of mega universities before the pandemic, they found that a low number of online assessment practices were used. The use of online technology in assessment systems was not common, and the reason for this situation could be the disadvantages of online environments in terms of security. Bozkurt and Uçar (2018) argue that a significant proportion of assessment and evaluation in e-learning takes place through real-time, face-to-face exams and student-generated homework and projects, which contradicts the concept of distance education, which aims to provide education anytime and anywhere.
Following the declaration of a global pandemic by the World Health Organisation (WHO) on 11 March 2020, ERT was adopted as a solution to the risks posed by face-to-face education. As a result, online examinations were conducted to limit interpersonal interaction and protect against the risk of transmission during the pandemic. Maintaining academic integrity has become a critical issue, including the design of assessments and technological safeguards (Amrane-Cooper et al., 2021). The widespread use of online exams during the pandemic has led to a number of studies and proposals for different technological solutions to protect academic integrity.
In their systematic literature review, Muzaffar et al. (2021) examined online exam research from 25 countries with 53 online exam solutions in e-learning studies from January 2016 to July 2020. The authors identified four key factors for global adoption: Network infrastructure, hardware requirements, implementation complexity and training requirements. Butler-Henderson and Crawford’s (2020) review of 36 articles on online exams also highlights the importance of online exam security and student and staff preferences. In their study of 730 medical students’ exam preferences and views on academic integrity during Covid-19, Elsalem et al (2021) found that only about a third of students preferred remote e-exams and that this preference was related to exam preparation time, question quality and academic performance. The duration of exams was also reported by Aksu Dünya et al. (2021), as students found an hour duration very short and a whole day or week period more reliable in their learning. To overcome the challenges of online proctoring and to make online examination processes more effective and accessible, Oeding et al. (2024) suggested that universities should develop online examination procedures, experiment with proctoring in live classes, and provide computer labs equipped with hardware and software for students without their own devices.
Proctoring software was also found to be used in practice. Zhiguo Zheng et al. (2024) proposed intelligent monitoring systems that incorporate technologies such as head posture detection, eye tracking, screenshot prevention and clipboard deactivation to improve security in online examinations. They emphasize that these systems provide an effective solution to ensure exam security and fairness by preventing cheating. Elhiny et al. (2023) developed a multimodal approach to mitigate cheating in online exams, focusing on proactively preventing cheating with strategies that target specific misconduct behaviors. The solution is designed to be flexible enough to adapt to new cheating methods and includes modules such as authentication, IP monitoring, suspicious behavior detection and exam analysis that combine detection and mitigation strategies. Ong et al. (2023) used a clustering approach that monitors students’ faces, eyes and devices with CCTV to detect cheating in online exams. They aimed to increase the reliability of online assessment processes by analyzing cheating behavior with 83% accuracy. Kaddoura ve Gumaei (2022) developed a system to prevent cheating in online exams using deep learning models, specifically deep convolutional neural networks (CNNs) and Gaussian-based discrete fourier transform (DFT) statistical methods. This system analyzes recorded audio and video data to provide real-time detection. Gudiño Paredes et al. (2021) studied the impact of remotely proctored exams on online graduate students’ learning and academic integrity. Remote proctored exams had a significant impact on academic integrity, with students citing a sense of obligation and being watched. Lack of privacy and anxiety are concerns for students. The study also highlighted students’ recommendations for remote proctored exams, emphasizing the importance of assessment for learning. Ahmed et al. (2021) examined the experiences of educational institutions in implementing e-exams and e-evaluation as critical components of e-learning in different countries. They focused on the difficulties encountered during the global Covid-19 pandemic. It is advisable to assess students through rigorous and continuous assessment, which includes the use of e-exams supported by authentication mechanisms to identify and minimize instances of student misconduct. The authors noted that e-examination centers face challenges such as limited internet speed, high costs and ensuring authenticity. In conclusion, the study suggests that facial recognition technology could be considered as a reliable method of authenticating e-exams in the future.
Upon analysis of the studies, it becomes evident that systems with flexible designs are being developed with the objective of proactively preventing cheating. Online assessment systems are undergoing continuous evolution in response to advances in ICT, including the integration of artificial intelligence, image processing for data analysis, data mining techniques, widespread Internet access and network infrastructure development (Topuz et al., 2022). However, while online surveillance has the potential to provide secure assessments, it also raises ethical and practical concerns that need to be addressed to ensure a fair and effective assessment. Furthermore, academic integrity policies and practices vary from country to country, so it is not practical to expect a universal model (Gamage, 2020). Therefore, further research is required in this field.
Methods
Context and Study Area
In response to the Covid-19 pandemic, higher education in Turkey has transitioned to distance education, with some exceptions, following global trends. As part of the post-pandemic process, Distance Education Centers were established in all universities to facilitate the distance education process.
CoHE (YÖK, 2020) has outlined the basic principles for exams that can be conducted in digital environments at universities. Consequently, universities have begun conducting online exams by establishing procedures and principles for electronic exams.
Eskişehir Technical University utilised Canvas Learning Management System and live lecture platforms (Zoom, BigBlueButton and Microsoft Teams) for exam preparation and implementation, as well as for learning environments. Online exams through Canvas LMS were widely used for assessment and evaluation, contributing to fair and reliable exam administration.
The Implementation Guide and Proposal Guide prepared by Eskişehir Technical University Open and Distance Education Application and Research Centre, contain technical specifications for exam security. An informative meeting was held for instructors and suggestions were presented on the issues to be considered in online exams, and the arrangements that can be made through Canvas LMS such as providing an appropriate and challenging duration, providing each student with a single answer, displaying one question at a time, organizing questions with a locking feature, and monitoring quiz logs. According to the guidelines, exam sessions and procedures can be conducted by a single proctor. However, if the relevant unit/department committees determine that this approach is not feasible or effective, a sufficient number of proctors should be assigned based on the number of students. Accordingly, it has been suggested that in large classes, it is advisable to open multiple virtual classroom sessions and assign a supervisor for every 40–50 students. Additionally, chat settings should be organized to prevent communication between students.
Research Model
In this study, online exams during the pandemic period are discussed as a case study. Case studies are a methodological approach that allows researchers to gain an in-depth understanding of a particular subject within its real-life context. They are particularly useful for exploring complex phenomena and can provide rich qualitative data, although they may face limitations in terms of generalizability and potential researcher bias (Bolton & Stolcis, 2003). The opinions and experiences of instructors about the security of the final exams held remotely at Eskişehir Technical University in the fall semester of the 2020–2021 academic year were investigated. A convergent parallel design, one of the mixed research methods, is used in the study. Convergent parallel design is defined as a method in which both qualitative and quantitative data are collected, analyzed separately, and conclusions are drawn by integrating them with each other (Creswell, 2019). Mixed research is a methodology that compensates for the weaknesses of quantitative and qualitative research, providing a more robust evidence base than either approach alone. It is a preferred methodology because it minimizes the limitations of both approaches, offering researchers a broader perspective (Creswell, 2019; Creswell & Clark, 2011). The quantitative and qualitative data of this study were obtained through the Questionnaire of Exam Security for the End-of-Semester Exams.
Research Group
The study group consisted of 163 individuals who participated in the survey of the 649 instructors who provided distance education at Eskişehir Technical University during the fall semester of the 2020–2021 academic year. The questionnaire, which was developed as part of the study, was distributed to all lecturers who taught during the specified period. Participation in the study was voluntary. Ethical approval for the study was granted by the Social and Human Sciences Scientific Research and Publication Ethics Committee of Eskişehir Technical University on 02/25/2021(decision no. 4/9).
Eskişehir Technical University comprises five faculties, two vocational schools, and three institutes. The highest level of participation in the research was observed among the Faculty of Science (n = 36), the Faculty of Engineering (n = 32), and the Faculty of Aviation and Space Sciences (n = 31). Approximately one-third of the participants were assistant professors. A total of 38 professors and 30 associate professors participated in the study.
Data Collection
The data of this study were collected using the “Questionnaire of Exam Security for the End-of-Semester Exams”. The questionnaire was administered online via Google Forms after the fall semester of the 2020–2021 academic year to collect data on the assessment and evaluation practices used by instructors in the exams. In the first part of it, participants were informed that their identities would be kept confidential throughout the study, that their responses would be used solely for research purposes, and that any opinions shared would be included anonymously if needed. It included both required and optional items. Questions about basic academic information about the instructors, information about the methods they use in their exam practices, exam security measures, challenges they face, and solutions they have developed. Information about the items in the questionnaire can be found in Appendix-1. A total of 165 forms were collected, but two duplicate forms were removed during data analysis and cleaning, resulting in a final data set of 163 forms. In the research, data were collected online and securely stored with backups.
When designing the data collection tool, the relevant literature were reviewed and items were created specifically for the features allowed by the LMS used at ESTU. In addition, considering the documents on distance assessment and evaluation practices published by CoHE (YÖK, 2020), items were also prepared to determine the extent to which instructors comply with these recommendations. The items developed for the questionnaire are in accordance with the three research questions sought to be answered within the scope of the research. To answer the first research question, items about the LMS module used to create and administer the assessment and evaluation application, the exam settings and options, the types of questions and course settings used, the exam preparation and administration phases, exam video surveillance, and the reasons for different implementations across courses were created. To answer the second research question, items to determine the difficulties experienced before, during, and after the exam, whether there were ethical violations, and the adequacy of the security measures recommended by the university were created. To answer the third research question, an open-ended item was created to get the instructors’ suggestions to increase exam security.
Reliability refers to the consistency of questionnaire results over time or across raters and indicates that the questionnaire consistently measures what it is intended to measure (Downing, 2004). Questionnaire results were collected through an online form and quantitative data were reported. Experts in the field of distance education reached a consensus in the interpretation of qualitative data and the discovery of themes. Validity refers to the fact that the results obtained from the measurement tool accurately represent the concept being measured. In the development phase of the measurement tool, items were created that would provide answers to the research questions. The scope of the related field was evaluated by two distance education experts, one educational technology expert, and one educational measurement and evaluation expert, and thus the questionnaire was finalized.
Data Analysis
Due to the nature of the questionnaire items, descriptive statistics were calculated to make sense of the quantitative data. Qualitative data from open-ended questions were analyzed using content analysis. Independent researchers coded the data to ensure the reliability of the findings from the qualitative data. The researchers discussed the codes, and categories and themes were determined by consensus. Reliability in qualitative research is enhanced by having more than one researcher code the data and resolve any discrepancies. Because the data were collected in writing in an online environment, there was no problem with data verification in this study. It has been argued that the use of quotations in qualitative research serves as evidence and strengthens the findings by helping the reader to assess the accuracy of the analysis (Corden & Sainsbury, 2006). Therefore, direct quotes from participants were used in the findings.
Findings
Measures Taken to Ensure the Security of Online Exams
The first research question investigated the security measures implemented by instructors for online exams at the university. To this end, the LMS module used for assessment, online exam settings, question types used, settings of the course in which the exams are conducted, procedures applied before and during the exam, video surveillance, and the use of different applications were examined.
The LMS utilized by the university allows for the implementation of online exams through the “Quiz” and “Assignment” modules. Of the 563 exams administered by instructors at the university, 424 (75.3%) were conducted using the Quiz module, while 139 (24.7%) employed the Assignment module. This indicates that the inclination to construct exams comprising disparate question types is more prevalent among instructors. Indeed, the assessments created in the assignment module employ a task-based logic in the form of text entry or file upload.
In the exams administered on the aforementioned modules, instructors are able to implement some settings pertaining to the exams. The exam settings utilized by the instructors in these exams via the LMS modules are illustrated in Figure 1.
Figure 1
Usage rates of exam settings in the LMS.
The data demonstrate that the most prevalent exam settings are as follows: displaying one question on the screen at the same time (74.8%), making the exam duration realistic and challenging (74.2%), having one opportunity to answer the exam (68.1%), mixing options (67.5%), and preventing students from going back to previous questions (62.6%). The least utilized settings are creating groups within courses (7.4%), using a plagiarism checker (11.7%), and creating a pool of questions to draw from when creating exams (28.2%). This shows that LMS features are mostly used by instructors to ensure exam security.
In online exams, a variety of question types may be employed. Within the scope of the questionnaire, the question types used by the instructors in the exams were also examined. As anticipated, the most frequently utilized question type is the multiple-choice questions (%70,6). Furthermore, questions with file upload were also frequently utilized (%62). Especially task-based exams under surveillance explain the frequent use of questions with file upload.
The LMS is also equipped with the functionality to restrict access to course materials in modules other than the exam module. This is a measure that can be taken to prevent students from utilising the resources in the course during the exam period. Indeed, the questionnaire also inquired of the instructors what kind of restrictions they employ in their course settings. It was found that the most prevalent setting among the course settings was to prevent students from accessing files before the exam period. However, only half of the instructors employed this measure. Nevertheless, approximately one-third of the instructors did not utilize any settings for the restricted access of files or learning materials.
In terms of security measures, the actions undertaken by instructors in the context of exam preparation were also investigated. Accordingly, the overwhelming majority of instructors (90.4%) reported informing students before administering the exams. However, it was observed that some of the instructors were not inclined to obtain confirmation from students that they accepted the exam rules (22.9%) or to conduct a mock exam/practice session (12.7%). Such practices may have a negative impact on the exam results of students who are not conversant with the utilization of the exam module within the LMS.
It is undoubtedly the case that proctoring represents one of the most frequently taken security measures during the conduct of online exams. In this context, Figure 2 presents information about the actions taken on the LMS by instructors who used at least one method of proctoring in their exams at the university.
Figure 2
Operations during the execution of the exam in the LMS.
Figure 3 reveals that in 23% of the exam administrations conducted by the instructors, a combination of proctor assignment, students’ quiz logs, and video surveillance was utilized. Additionally, video surveillance was employed as the sole proctoring method in 21.9% of exam administrations. While 18.2% of exams utilized video surveillance and students’ quiz logs concurrently, 15% of exams employed a proctor assignment and video surveillance. These findings illustrate the pivotal role of video surveillance in exam practices. Additionally, it was indicated that video surveillance was predominantly conducted via the Zoom application.
Figure 3
Preferred video surveillance methods.
Video surveillance is an emerging security measure that is widely used among instructors. Information on the method of video surveillance was also collected through the questionnaire. The results of this questionnaire are presented in Figure 3, which shows how the instructors performed video surveillance during the exam practices.
As illustrated in Figure 3, the majority of instructors employed the use of a camera for the purpose of video surveillance. A total of 29.8% of the instructors did not utilize a microphone, while 27.7% employed both a microphone and a camera. A mere 11.8% of the instructors utilized more than one camera. This indicates that most of the instructors believe the use of a single camera is adequate for surveillance purposes. The use of microphones is similarly uncommon.
Opinions of Instructors on Exam Security
The study identified themes from the instructors’ opinions on exam security. These themes are shown in Figure 4 and their sub-themes are shown in Figure 5, Figure 6 and Figure 9 below.
Figure 4
Themes related to exam security practices.
Figure 5
Exam Security Measures.
Figure 6
Challenges in Ensuring Examination Security.
One of the themes used by trainers in relation to exam security was surveillance. Surveillance generally took the form of camera monitoring of the examination process. In addition to the camera, other tools used included the use of microphones and headphones. Mobile phones were used as external cameras to prevent the external use of mobile phones. Instructor 132 commented as follows “…the camera angle was adjusted to show the hands, the top of the table and the screen. The microphone was left on. After logging into Zoom with the mobile phone, the exam was held in Mergen (Canvas LMS)”.
Another issue related to security was diversifying the formats of the questions asked to students and multiplying the questions so that different questions are sent to each student. Another practice was to prepare the answers in a way that requires comments and to send the answers in handwritten form. In this way, instructors aimed to prevent answers being taken from other places, while ensuring that each student answered a different question. Instructor 26 commented on this issue as follows “…I used a variety of question techniques including scenarios, analytical information and interpretation”. In another respect, Instructor 68 explained his approach as follows: “Each student drew shapes in their own handwriting. They wrote the answers to the questions in their own handwriting. It was possible to compare the handwriting they wrote before the exam with the handwriting in the answers. Since each student’s comments and handwriting were different, it also prevented plagiarism.
Another theme that emerged was notifications and reminders. Instructors warned and reminded students of the rules before and at various times during the exams. Instructor 117 stated the following: “I always emphasized that cheating in class is unethical behaviour”. Instructor 125 stated: “Attendance was taken twenty minutes before the start of the exam and students were verbally reminded of the exam rules”.
On the topic of marking, it was noted that some instructors marked exams with the jury rather than on their own. It is understood that this was done to ensure academic honesty and to make the marking of the exams fairer. Instructor 20 commented on this issue as follows: “Instead of doing the exams myself, I did them as a jury evaluation”.
In the themes of scope and content and number of students, it was concluded that it would not be right to conduct exams in the same way for each course. It was felt that different assessments based on the characteristics and content of each course would give better results. Instructor 13 expressed his thoughts on the subject as follows: “In terms of content, some courses included open-ended questions due to traditional course formats, while others were completely multiple choice. …exams (including both multiple choice and open-ended questions and file uploads) were introduced for courses that included projects and practical applications”. Instructor 163 commented on the same issue: “In some courses, it may not be appropriate to limit questions in terms of purpose and content. In other cases, open-ended questions may be necessary. However, when dealing with open-ended questions, it’s essential to allocate enough time and structure them in a way that invites thoughtful interpretation. In my applied laboratory course, exams were conducted during live lectures along with the production of report assignments. Oral exams were supervised by assistants.
Regarding the number of students, it was concluded that this generally influenced the format of the exam and the number of questions. Instructor 154 commented as follows: “In accordance with the students’ expectations from the course, I give exams in the form of tests, open-ended written exams and homework”. Instructor 33 expressed his thoughts as follows: “…we have not appointed a proctor for postgraduate courses with a small number of students and we have appointed a proctor for courses with a large number of students. Having a proctored exam is extremely important, especially in courses with large numbers of students”.
Challenges in Ensuring Examination Security
Under the theme of problems encountered by instructors with the LMS, subheadings emerged such as not being able to enter questions into the question bank, problems with uploading homework to the system, and difficulties with evaluating exams via the LMS. It can be concluded that the main problem underlying these themes is due to various technical problems arising from the software. However, it can be assumed that technical infrastructure facilities also have an impact on this process. Looking at the instructors’ statements on the themes, instructor 18 says the following “It was very difficult to read the questions from the LMS after the exam. I read the first exam of one of my courses in this way. It would have been useful if there was a feature in the LMS that allowed you to make notes on the answer sheets, e.g. using a pen”. Instructor 94 states: “…when extra time was given before the end of the exam, this process was not valid for some students and the students had to leave the exam within the first allotted time. The work of one student who said he had uploaded the exam was not visible in the system. Students stated that they did not receive confirmation that their exams had been sent after uploading them to the system and therefore were not sure”. As can be seen from these statements, problems with the LMS or some other technology have made the examination process difficult for both the instructor and the students.
Under the theme of external technical issues, the sub-themes included internet/electricity shortage, technical difficulties faced by students, and Lack of Zoom application license are the sub-themes. With regards to the internet/electricity shortage, Instructor 151 reported: “Power and internet disruptions were experienced during the examination. It was attempted to ensure that students submitted their own documents using the method of answering questions via email while keeping track of the submission time.” Instructor 132 stated that internet and power shortages are the most common problems experienced by students and that (s)he conducts make-up exams for those who experience these problems. Therefore, it can be seen that various problems that occurred outside the system also had a negative impact on the process.
Another theme, surveillance issues, is the difficulty in using cameras and the lack of penalties. It was observed that some students did not use their cameras, some had different difficulties with the cameras and the lack of clarity of penalties caused difficulties. Regarding the issue, Instructor 103 said: “…We had difficulties in positioning the cameras to capture the student’s exam paper, hands and face, as well as in providing adequate lighting around them.” Instructor 44 said: “As a solution to the difficulties students had in adjusting the camera angles during the exam, a visual guide was added to the exam instructions for adjusting the camera angles. In other words, a special surveillance system was established to ensure exam security and a guide was created regarding camera use, including camera position, number and angle.” Regarding the lack of penalties, Instructor 42 said: “…I created a report for students who were warned that they had left the exam screen during the supervised exam. However, it was thought that the student may have entered the exam with a mobile phone and that the screen lock was active, and the report was not evaluated. Of course, students who saw that no action was taken continued to leave the exam screen during the exam.”
It was noted that there were some problems related to student issues. Due to the use of handwriting in some exams, there were difficulties in reading students’ writing. On the other hand, it was also noted that some students used different applications to communicate with each other and in groups. Instructors tried to take various measures to prevent such problems. But the emphasis on the need for a legal basis for such situations is noticeable. Instructor 12 had the following to say on the subject: “…the act of reading handwritten files submitted by students is a difficult one. This is due to the lack of a standardised format in the students’ handwriting. We have given the rules about what to look out for in the next exam and how to assess which situations will be assessed. However, I think these should be added to the rules and should be based on a foundation.” For students, instructor 143 stated: “The most important problem in online exams is that students communicate with each other through the groups they form among themselves.” Similarly, instructor 118 stated: “During the exam, it was noticed that students were using communication groups like WhatsApp to communicate with each other. So arrangements were made in the examination process for future examinations. In order to prevent students from communicating with others during the exam, cameras were set to record the student’s workspace, face and computer screen.”
Instructors’ opinions on whether ethical violations occur in their exams
The study also investigated instructors’ perceptions of unethical behavior in online exams. It is thought that the opinions of instructors who oversee the exam processes of their students from beginning to end and compare them with their previous experiences are important. The results of this investigation are presented in Figure 7.
Figure 7
Instructors’ views on ethical violations.
Figure 7 illustrates that 22.7% of instructors held the view that ethical violations, such as cheating and plagiarism, did not occur in their exams. Conversely, 62% of instructors expressed the opinion that a significant amount of ethical violations had taken place, while 15.3% of instructors believed that some ethical violations had occurred. The limited number of instructors who believe that no ethical violations occur indicates that this issue is a significant concern for instructors.
Despite concerns regarding potential ethical violations, Figure 8 illustrates the opinions of instructors on the sufficiency of security measures proposed by the university.
Figure 8
Instructors’ views on the recommended security measures.
As illustrated in Figure 8, 39.3% of instructors indicate that the exam security measures recommended by the university are sufficient, while 47.2% believe that they are only partially sufficient. A total of 13.5% of instructors expressed the opinion that the exam security measures recommended by the university are not sufficient. The responses of approximately half of the instructors indicate that they believe the measures are, at least to some extent, adequate. This suggests that they perceive the system as having some shortcomings, rather than as entirely inadequate.
Suggestions for Improving Exam Security
The study analyzed the recommendations made by the instructors to improve the security of remote examinations. The themes identified for these recommendations are shown in Figure 9 as standardization of supervision, use of learning management systems and software, ethical principles’, and legal framework’.
Figure 9
Suggestions for improving exam security.
Regarding the standardisation of surveillance, instructors generally made suggestions on the use of cameras, browser locking programs/extensions, centralised exam practices and post-exam evaluation. One of the highlighted suggestions can be considered as ensuring security and surveillance in exams. In particular, measures to prevent ethical violations and cheating attempts during the exam were expressed. However, the high cost of some software and the lack of regulations are also obstacles.
Instructor 18: “…regarding the use of cameras, the university should inform the students in written form before the exams that the cameras will be activated and that the cameras should be positioned in such a way that they show their faces, workspaces and computer screens. As some students may object to this requirement and only show their faces, it is important to provide clear instructions on how to capture images during exams. It is helpful to make generalised arrangements to improve clarity and impartiality. Also, clear guidance should be provided.” Instructor 25 stated the following: “Students should be asked to connect their mobile phones to a secondary camera via Zoom and view their actions and computer screens with this camera during the exam.” Regarding browser locking software, instructor 14 stated the following: “I have heard of a service called Browser Locker.”
Regarding the sub-theme of centralised exam administration, Instructor 145: “Exams should be administered by a centralised Examination Unit at university or faculty level, not by the course instructor. This will enable the assistants to take a more proactive approach and take more responsibility throughout the process.” Instructor 151 made the following statements on the sub-theme of post-exam evaluation: “The evaluation of the answers to the questions after the exam were done via live connection with the students and the similarities in the papers were opened to peer review, which resulted in positive results to the extent that the students accepted that they were in contact with each other. It can be recommended for all contexts where this type of application is appropriate.”
Moreover, recommendations were made for the efficient use of LMS functionalities. These included terminating the test when the candidate exits the exam screen, optimizing the use of the question bank, and setting time limitations based on questions. Regarding the automatic termination of the test if the student exits the exam screen, Instructor 22 commented: “I agree that the test should be terminated immediately once the student switches the screen. During Open Education High School exams conducted by the Ministry of National Education the system automatically terminates the exam of any student who attempts to change their screen or use their mobile phone, among other prohibited actions.” It is recommended that a similar system be implemented across all exam sessions. Instructor 42 suggests: “Movement tracking (monitoring computer activity using the quiz recording feature) is carried out and if a student engages in more than five ‘suspicious’ processes during the exam, their exam will be marked as invalid”. In terms of the effective use of the question bank, Instructor 165 stated that: “T he question bank and formulated question system in LMS are largely sufficient.” (S)he suggested that the inclusion of a more advanced mathematical module, such as derivatives, integration, and stepped question options, would increase its effectiveness in engineering branches. Instructor 143 expressed a different opinion: “In exams administered through the LMS, question groups can be formed to ensure that different students receive varied questions. It would also be beneficial to randomise the sequence in which questions appear within the exam”. Another proposal for the learning management system is to enforce a timed limit for each individual question. Instructor 16 suggests that: “Implementing time limits based on the type of questions can improve exam security by preventing question jumping and ensuring fair time allocation.”
Another aspect that has been raised to improve exam security is the incorporation of ethical principles. Many educators believe that securing exams should involve imparting ethical values to students. Instructor 12 stated: “ By frequently reminding students that honesty and trustworthiness are learned human behaviors, and by explicitly rewarding such behaviors (even though they should already be expected), we can enhance exam security. ” Instructor 28 emphasized “ Frankly, I think that people should be raised with ethical and core principles throughout their education.” Instructor 103 said: “In my opinion, ensuring exam security at the desired level cannot be guaranteed if students have not adopted ethical standards.” and pointed out that therefore, promoting ethical conduct among students is the solution.
Regarding the sub-theme of “including original/creative works,” Instructor 45 suggested that: “Conducting an open book exam can be a method to achieve high-level measurement. This allows students to access books, articles, and online content. When designing such an exam, students should be given time to conduct research and analysis after seeing the questions.”
In regards to “the policy of disallowing students who do not accept the rules from taking the exam,” Instructor 69 believes that “students who adhere to the procedures and regulations should be protected. Those who do not comply, however, should be excluded from the exam or have their exam invalidated through software arrangement if they do not respond or respond negatively to the questionnaire (about the ethical and other rules regarding the exam). After giving a positive answer, students should receive a mild verbal or written warning if cheating or plagiarism is detected. This will be communicated to all students and is important for compliance with rules and increasing exam security. It demonstrates the university’s standard attitude towards academic integrity.”
Taking all the evaluations together, it can be seen that the efforts made by the instructors to ensure the security of the exams are intensive. In addition, it can be said that the measures taken or attempted by the instructors alone are not sufficient. This can be interpreted in terms of various technical problems that cause this situation, such as the interruption of electricity and Internet connection, the lack of sufficient legal basis, the fact that not all higher education institutions have the same level of practices, the existence of negative experiences of students, the existence of technological inadequacies and, finally, the importance of ethics. Finally, in order to prevent the recurrence of problems in the future, it is possible to ensure that general and comprehensive legal bases are established and implemented. The development of exam systems and the definition of supervisory tools can also be seen as components that will make a significant contribution.
Discussion
Exam security has become increasingly important during the Covid-19 global pandemic, as ERT has become necessary. This study examined the issue of exam security in the context of a higher education institution during the epidemic period. The findings have shown that there are several common and distinguished measures taken by instructors although they still faced challenges.
It is seen that the measures and challenges are mostly based on the capabilities of the learning management system used by the institution. Within the limits of technical capabilities, the instructors organized the display of exam questions, response times, question types etc. ensuring that exam security was prioritized. Restricting access to teaching materials uploaded to the learning management system was also one of the measures taken. For proctoring, only a small portion of instructors have not used several strategies such as video surveillance with or without microphone use. Also, it has been the case that the number of students, content and characteristics of the courses can be decisive factors in the way supervision is to be carried out. A review of the literature reveals that the use of video surveillance in various forms during online exams has become increasingly widespread. While such surveillance may cause discomfort among students, it plays a crucial role in deterring unethical behavior. In terms of assessment, strategies such as presenting questions and answers in various formats, using personalized questions, requiring handwritten responses, and reinforcing exam rules have gained prominence. Additionally, the inclusion of juries in certain courses to evaluate student performance represents an emerging approach. Instructors often informed students prior to exams. Online pre-exam briefings provide information about the exam, including its content, duration, and any technological issues to be considered. These briefings have become increasingly important during the ERT period (Ocak & Karakuş, 2021). In the study on students’ perspectives on online exams during distance education, Afacan Adanır (2021) found that 59% of the students reported receiving sufficient information prior to taking online exams, which was important for them. Around 90% of participant instructors have stated to inform students before online exams in the current study. This finding suggests that student information is adequately provided. However, it is important to test the technological situations in advance to avoid problems during the exam process. It was observed that the rate of accepting the exam rules and conducting a trial session/exam was quite low in this study. The instructors’ reluctance to conduct a trial session may be due to the students’ familiarity with the LMS and online exams during the initial period of the pandemic.
Regarding the problems experienced during the exam, it is seen that the most frequently mentioned problems by the instructors were electricity/internet shortages, problems arising from the LMS, technical challenges experienced by the students, and students’ unwillingness to turn on the camera and interact. In Balaman and Hanbay Tiryaki’s (2021) study, one of the disadvantages of distance education was found to be that not all students have the same conditions in terms of technical facilities and infrastructure. The issue of inadequate technical facilities of students, which is shown as the most fundamental problem in the reports and articles published on the pandemic process, stands out as one of the most fundamental obstacles in the distance education process (Batubara, 2021; Savaş, 2021) and stands out as the primary element of student characteristics, which are included as the main success factor in the literature (Alhabeeb & Rowley, 2018).
One measure taken to address the issue was the restrictions of exam durations. However, several studies have reported students’ dissatisfaction and negative opinions regarding this restriction. For instance, Aksu Dünya et al. (2021) found that the limitation had a negative impact on students’ satisfaction, which in turn affected their perceptions of honesty and led to increased cheating behavior. As stated by Aksu Dünya et al. (2021), the principle of usefulness is very important and increasing the reliability of the exam by restricting the exam time may have an impact on its usefulness. To ensure reliability and usefulness in online exams, it is recommended to determine an appropriate duration for each question and for the exam as a whole. The estimation of the sufficient time for each online exam can be determined by the coordination of the relevant department heads and the e-assessment committee (Elzainy et al. 2020) or online assessment and evaluation experts. Within the scope of this study, no recommended or adopted method was found in the literature to determine appropriate question durations. It may be possible to test the duration by assigning test users to the instructors or by pre-testing.
A substantial majority of instructors believe that there are some or many ethical violations in online exams. According to a study by Watson and Sottile (2010), students are four times more likely to commit ethical violations in online exams than in face-to-face exams. This suggests that instructors’ concerns about ethical violations are not unfounded and that they should implement strict exam security measures. Most instructors have a favourable view of the security measures proposed by the university. Based on these findings, it can be concluded that although many security measures were taken, there are still risks for ethical violations. Therefore, given the continued lack of institutional provisions for the management of academic integrity in the context of distance education (Gamage et al., 2020), it can be said that policy makers should develop policies in the context of ethical rules.
To ensure exam security in online exams, another effective method is to provide each student with a unique set of data for the questions. This can be achieved in universities using Canvas-based systems as LMSs through the formula question feature.
The study also analyzed the instructors’ suggestions for ensuring online exam security. The majority of instructors recommended standardizing camera surveillance in online exams. In this way, it is thought that the issues of students objecting to the use of cameras in online exams or how the camera positions should be will be resolved. Despite all measures taken, the majority of the participants responded that there may still have been ethical violations.
Some challenges in ensuring exam security were related to the use of the LMS by both instructors and students. Difficulties arose in areas such as the effective use of question banks, uploading assignments, and conducting assessments within the system. Instructors also noted that some students experienced difficulties with camera usage and positioning. Also, the absence of sanctions for students who fail or do not turn on their cameras during exams, or who communicate with each other on other platforms during the exam, highlights legal gaps. Adapting assessment practices and other pedagogical needs is crucial during emergency or crisis situations. Therefore, ongoing professional development of educators is highly necessary (Stewart & Lowenthal, 2023). The current study emphasizes the importance of supporting not only instructors but also students on the ethical principles of assessment in this process. Amrane-Cooper et al. (2023) highlighted the importance of providing technological and pedagogical training resources, as well as orienting students to online assessment, in support of this argument. Therefore, it is critical to implement policies that design and implement training for both students and instructors.
Integration of additional measures to improve these limitations with the system can be stated as a situation that needs to be addressed.
Conclusion and Recommendations
This study sought to present a comprehensive account of the exam security measures implemented by a higher education institution during the ERT period. The findings of the study indicated that the online exam security measures were primarily based on the existing features of the LMS. The fundamental prerequisite for effective exam security was identified as the provision of adequate hardware and software, as well as access to the requisite infrastructure, for both learners and instructors. This is considered a fundamental requirement for distance education, and has been once again highlighted during the ERT period with this study.
The impact of exam duration restrictions, as a significant measure highlighted in the current study for ensuring exam security, warrants further examination to ascertain their efficacy and optimal implementation. The potential for ethical violations represents a significant threat and a major challenge in this study. Therefore, it is of the utmost importance to identify potential solutions and integrate them into the online examination processes. In addition, the potential ethical risks inherent in or associated with exam security measures is another issue that requires further investigation.
The emergency situation in the current study highlighted the need for the rapid adaptation of assessment practices. This process has also prompted educators to reevaluate and enhance their measurement and evaluation practices, whether in face-to-face or remote settings. Additionally, the emergence of ethical issues in examinations has underscored the necessity of reconsidering assessment strategies to accommodate diverse environments and scenarios. Surveillance systems, in this regard, play a crucial role. These results highlight once again that online exam security measures necessitate both technical and pedagogical support. It has been observed that some of the measures and precautions implemented and attempted to be implemented in the examinations require a legal basis and sanctions. In some cases, instructors and institutions encounter difficulties due to the absence of legal sanctions. In this respect, it can be considered important to establish laws, regulations and directives that encompass all higher education institutions.
There are several limitations to this study. First, the participants in this study were limited to instructors. The perspectives of learners and other stakeholders, including administrators and support groups, may yield differing insights when consulted. Since it is a case study, it is hard to generalize the findings to all institutions. Future studies can focus on the long term studies with a larger set of data.
Data Accessibility Statement
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Additional File
The additional file for this article can be found as follows:
Ethics and Consent
The ethics approval was gathered on 25.02.2021 from the Ethics Committee of Eskisehir Technical University with the number 4/9.
Competing Interests
The authors have no competing interests to declare.
Author Contributions (CRediT)
Esra Pınar Uça Güneş: Supervision, Conceptualization, Methodology, Writing – original draft, Writing – review & editing. Nuray Gedik: Conceptualization, Methodology, Writing – original draft, Writing – review & editing. Mehmet Ali Işıkoğlu: Conceptualization, Methodology, Quantitative data analysis, Qualitative data analysis, Visualization, Writing – original draft, Writing – review & editing. Barış Yiğit: Qualitative data analysis, Visualization, Writing – original draft, Writing – review & editing. İhsan Güneş: Conceptualization, Methodology, Writing – original draft, Writing – review & editing. Ayfer Beylik: Qualitative data analysis, Visualization, Writing – original draft, Writing – review & editing. All authors have read and agreed to the published version of the manuscript.
Author Notes
This paper was reviewed and refined with the assistance of Google Translate and DeepL (Version as of December 1, 2024), complementing the human editorial process. The human author critically assessed and validated the content to maintain academic rigor. The author also assessed and addressed potential biases inherent in AI-generated content. The final version of the paper is the sole responsibility of the human author (Adopted from: Bozkurt, 2024).