source: ELZ
Wiktoria Allan and Mariana Aguilar Ramirez from the Language Centre first contacted us (ELZ) in March 2024 to request information on implementing the UNIcert® exam via Moolde quizzes in a secure PC environment (implemented with the Safe Exam Browser – we reported). UNIcert® is a seal of quality for language teaching and examination procedures at universities and allows language certificates of the same name to be awarded. The exam tests various competences, such as listening comprehension, reading comprehension and writing comprehension.
After the two of them worked diligently on their questions for the exam in winter semester 2024/25, we took care of the technical challenges. The various competences to be tested resulted in more demanding configuration scenarios for the Safe Exam Browser compared to the everyday e-exams. Due to different time requirements and the lack of setting options in the Moodle quiz, a separate test activity was also created for each competence and these were linked to each other via prerequisites in order to be able to guide students easily through the examination process on the one hand and to create access requirements or barriers on the other.
From our point of view, the two of them really went the extra mile, worked with a wide variety of question types and tried to utilise all the possibilities on Moodle. In the preliminary communication and also on the day of the retreat, we benefited from their excellent Moodle skills, as they were able to understand our processes immediately and implement them independently. The collaboration was very positive, especially as we had to overcome a few stumbling blocks here and there due to the tight time management. We asked them to tell us about their initial experiences, which they kindly agreed to do and you can read about them in the following interview-style paragraphs:
What are the advantages of this format?
“E-assessments generate detailed data on student/exam performance, including time spent on questions and response patterns. This data can help us improve the quality of the exams. We can know which questions are good questions, and which missed the mark. We did not have that information before.
The tasks are also easier to read, therefore easier and faster to grade. Additionally, I (Wiktoria) can be sure that the points have been added up correctly, because the system does it for me, leaving no room for human error. “
Are there any disadvantages?
“The upfront work is considerable. Transferring a test into Moodle does get easier with time and practice, though. Some more creative task types can be harder to transfer, as Moodle doesn’t always have exactly the right question, but we’ve always been able to find appropriate solutions so far by making slight modifications.“
How did you feel about the support?
“We had an initial consultation with the e-learning team where we learned what an e-assessment entails, and how it is typically set up. As we were both comfortable using Moodle and creating quizzes, it was reassuring to know that with some extra set up, an e-assessment could be something we can do.
The team provided ongoing support, and were extremely patient with us as we finalized the exams. The team also helped us white list links that students were allowed to use during the exam.
On exam day, the e-learning team are there to set everything up, from the computers to a handy presentation with passwords for the tests and instructions. I (Mariana) was initially very nervous about administering the exam to a group of 60 students. However, once I learned that Agata and Doreen would be there to support and troubleshoot anything that needed to be fixed, I felt like I could breathe again.”
What additional features would you like to see?
“It would be amazing if we could provide students with headphones to do the listening on their own. Self-paced listening is more inclusive and can be more authentic than listening in a big group. This way, the three parts of the test (listening, reading, and writing) could also be more self-guided and students could work through them at their own paces, rather than dictated by us.
It is almost impossible to automate grading for any type of question where students need to type, e.g., short answers, cloze, etc. We have been using AI to try to come up with as many correct answers as possible, but many are still marked as “incorrect” because Moodle does not allow for spelling mistakes in correct answers.“
What do you feel is the biggest hurdle, or what is your biggest concern regarding this format?
“Some students mentioned they had never taken an e-assessment. So there needs to be more training done in class.
I (Wiktoria) am always concerned about the accessibility and inclusion aspects of the test, so it’s important that we make sure that providing an e-assessment doesn’t disadvantage whole groups of students.“
Do you have examples of how this format improved or changed the learning process?
“Not yet. We have only done e-assessments once, but we will report back!“
Would you go back to the original version with less support?
“I don’t think we would go back to regular pen-and-paper exams. We have briefly discussed the option of letting students decide between a paper exam AND an e-assessment. However, creating exams that are equally functional in both formats requires additional effort to ensure fairness and comparability between paper-based and online versions. How do we deal with managing different grading systems if one format is perceived as easier or harder than the other? Exams designed to work in both formats may not leverage the full potential of the mode, leading to a less optimized experience for both formats.“
source: Wiktoria Allan & Mariana Aguilar Ramirez
Was there feedback from the students? If so, what was it?
“Some students noted that they were able to work much faster through the questions and appreciated not having to write anything by hand. That being said, many students appreciated having a printed text for the reading section that they could annotate by hand.
We do believe that some less tech-savvy students may have been slightly disadvantaged (for example, some students did not know that they could use a split screen to see the text AND the exam at the same time- nor did they know HOW to do it). Not everyone feels more comfortable typing and this is something which needs to be taken into consideration.
We also heard that many students found it difficult to concentrate, particularly during the listening exam, when 65 other students were typing their answers at the same time (this is another case in which headphones for each student would be useful).
Finally, the sound quality is not the same in all parts of the room (CS B 4.04), some students reported they could barely hear while others mentioned it was too loud, meaning some parts of the listening exam are particularly difficult to understand.”
Authors:
Wiktoria Allan (Language Centre)
Mariana Aguilar Ramirez (Language Centre)
Doreen Degen (ELZ),
Agata Warpas (ELZ)
More information about the language centre
0 Comments