In the academic year 2020/2021, I was a module coordinator of the large module (250 students) “Data Mining and Visualisation” at the Department of Computer Science at the University of Liverpool.
The module was new to me, but it was already taught several years at the department, and I had teaching materials that could be re-used. Those materials were used for the delivery of the module in the face-to-face environment. My goal was to adapt the module to the online environment in such a way as to support students learning in the challenging and unexpected online setting.
Since the module was run in the second semester, there was an opportunity to learn from the experience of my colleagues who delivered their modules in the same environment in the first semester. I also had an opportunity to learn students’ experience from one-to-one meetings with my tutees and from the pulse survey run by the department in the middle of the first semester.
Among students’ difficulties of learning in the online environment most frequently mentioned were:
- Lack of interaction and feel of isolation. In normal years, students often get together to work on assignments and help each other to understand teaching materials. The restrictions of the academic year 2020/2021 made it impossible for many students to meet physically and therefore considerably limited social learning. This problem was voiced by many students.
- Problems with self-discipline. Some students found it difficult to start working on teaching materials when there are no scheduled sessions. For example, in the year 2020/2021, 3 weekly face-to-face lectures were commonly replaced by 1-3 pre-recorded videos of the total duration of 1.5-2 hours. Students were supposed to watch these videos independently during the week. Since there were no centrally fixed time slots to watch the videos, each of the students needed to manage their own time individually and be sufficiently disciplined to work out all weekly materials for each of their modules. Quite a few students found this challenging.
- Problems with prioritisation. Some students reported that it is not clear what to do and in which order. They complained that in some modules there are a lot of materials provided, but it is not clear what they should concentrate on in order to progress successfully with the module. No priorities were explicitly set by the module coordinators.
In order to partially address the above-mentioned difficulties that students face in the online environment, and after literature review, it was decided to focus on improving the level of communication in the module (to mitigate the lack of interaction and feel of isolation) and on students’ understanding on what they are supposed to learn every week (to address the problems of self-discipline and prioritisation).
Communication
In order to improve the quality of interaction in the module, it was decided to introduce the following measures:
-
Use the Q&A dashboard Campuswire.com. The main reason for choosing this specific Q&A dashboard was that it allows anonymous questions and responses. The students were encouraged to use Campuswire to ask their questions personally or anonymously, reply to questions of others, 'Like' questions, and 'Endorse' answers. Nearly every module page contained signposting to Campuswire.
Signposting to Campuswire - Use breakout rooms during the lab sessions. It was supposed that during every lab session, in the first 10 min of the session a demonstrator would explain the tasks and answer questions, if any. Then the students will be invited to work on the tasks in groups of 5-7 students in breakout rooms for 30-35 min. During that time the demonstrator would join every group for up to 5 minutes to provide help and/or feedback. After the group work, students were invited to join the main room for a concluding discussion with the demonstrator.
-
Lecturer presence on the screen in the pre-recorded videos. In this module, every video was recorded with the lecturer present on the screen.
A typical picture in the pre-recorded videos with the lecturer present on the screen
Clear expectations
The main measure that I used to improve students’ understanding of the expectations is weekly Learning Objectives. Learning Objectives—explicit statements of the types of tasks the students should be able to complete if they learn what the teacher intends to teach them.
In my module, the introductory page of each week contained the week’s Learning Objectives
The Learning Objectives were used to design lessons, class activities, lab sessions, assignments, and the exam. The purpose of the Learning Objectives was explained to the students at the first Q&A session and was regularly reminded afterward.
Evaluation of the introduced measures
The effectiveness and students’ perception of the introduced measures were assessed via an anonymous end-of-module survey. The main findings and conclusions are presented below.
We start with the general assessment of the level of communication and clarity of expectations.
Communication. Most respondents assessed the level of communication in the module at least as high as above average compared to the other modules:
- 92% of the participants assessed the level of communication with the module coordinator as one of the best or above average;
- 65% of the participants assessed the level of communication with their demonstrators as one of the best or above average; 96% assessed as average or higher;
- 69% of the participants assessed the level of communication with the other students as one of the best or above average; 95% assessed as average or higher.
Clarity of expectations. 57% of the respondents indicated that the level of clarity in what they are expected to learn every week is one of the best compared to the other modules, and 97% of the respondents assessed the level as one of the best or above average.
Next, we present evaluation of each of the introduced measures.
Campuswire
Campuswire was actively used by the students during the module: There were 203 discussion threads created by the students. Campuswire provides several benefits that positively contributed to the level of communication in the module.
Streamlined communication. The three most useful channels of communication with the module coordinator were Campuswire, the online Q&A sessions, and one-to-one MS Teams chats. In 34% of the responses, Campuswire was ranked as the most useful channel, and in 68% of responses, it was among the top 3 most useful channels. In many cases, I was able to answer email/chat questions of the students by directing them to an already answered question in Campuswire. Furthermore, some students checked existing Campuswire questions before contacting the module coordinator, as was mentioned by one of the students
I did not post many questions campus wire because I had good contact with other students who I could ask questions too but it was still a useful resource to refer to before asking others/ reaching out to the module coordinator.
All these greatly optimized my communication workload while keeping the quality of communication with the students at a very high level.
Anonymity. One of the advantages of Campuswire is that it allows asking questions anonymously. This advantage seems to be important as 28% (57 out of 203) of all discussion threads were created anonymously. As one of the students put it
The ability to ask questions anonymously on Campuswire was very helpful, as was the ability to see other student's questions.
Social aspect. Another benefit of Campuswire is an added element of social learning. In many cases, students answered the questions of their peers. Thus, in addition to the Q&A sessions, lab sessions, and direct contacts, Campuswire provided a place where the students can get in touch with each other. In the responses, 62% of the participants indicated that Campuswire was one of the most useful channels of communication with the other students. The fact that the students learn from each other via Campuswire was also mentioned by several students, e.g.
I've found it useful many times to browse if somebody has already had a similar problem and asked the question, and to post it if not.
Campuswire was a good platform to centralise the information from the module. I found that other students who are ahead on course material / assignments ask questions which I intended to ask or was struggling with and it really speeds up progress as well as other insightful questions which help enhance understanding. Furthermore, the sharing of resources and information was very useful.
Breakout rooms
Campuswire contributed positively to the communication level between the module coordinator and the students, and among the students. Another introduced measure to improve communication among the students and with the demonstrators was the use of breakout rooms during the lab sessions.
Unfortunately, breakout rooms did not work very well as was indicated by some of the students
demonstrator-led lab sessions … lacked a sense of engagement between students and demonstrators, especially in breakout rooms, in which I found that I might be the only one with a microphone and/or camera turned on.
I very much did not like the break-out rooms and avoided the lab sessions because of this. Bigger classes worked better because you could hear other student's questions.
The breakout rooms were a waste of time and the lab felt useless to be honest, it was a waste of time there was no connection built there i had to seek help outside the lab from friends to understand the concept of the course
The issue with the breakout rooms was revealed after the first 5 weeks of the module during discussions with the demonstrators and from the students’ feedback in the Mid Term Survey. I also learned that many students regularly contacted their demonstrators directly via email or MS Teams to get help with the lab tasks.
Based on this information, I decided to change the format of the lab sessions in the second half of the module. Now instead of the breakout rooms part, the students were given 30-35 minutes to contact the demonstrator directly via MS Teams to ask their individual questions.
According to the survey results, MS Teams chats worked better for more students than the breakout rooms. 51% of the respondents indicated that MS Teams chats were one of the most useful channels of communication with the demonstrators. On the other hand, the breakout rooms were indicated as one the most useful channels of communication only by 31% of the participants.
Based on the results, there seems to be no definite preference for one channel over the other. In fact, given the survey and module setup, it is hard to draw any plausible conclusion, as the student’s preference may depend on a demonstrator (due to a large number of students, there were 5 demonstrators on the module) as well as on the personality of the student.
Therefore, the question of effective channels of communication during lab sessions in the online setting requires further study.
Presence of the lecturer on the pre-recorded videos
The lecture materials in the module were in the form of pre-recorded videos. This significantly reduced the amount of synchronous interaction between the students and the lecturer compared to ‘normal’ years. In order to partially compensate for this missed communication, the videos were recorded so that the lecturer is present on the videos and the students can clearly see the lecturer’s facial expressions and body language.
81% of the respondents liked this format of the videos and saw the benefit of having the lecturer on the videos.
In particular, 31% of the students mentioned that this improved the level of engagement.
Also, a positively unexpected outcome of the video format was that 41% of the respondents indicated that it improves concentration and helps to focus.
However, there were 18% of students who think that seeing the lecturer on the videos does not make a difference.
Weekly Learning Objectives
88% of the students indicated that the weekly Learning Objectives helped them clearly understand what they are supposed to learn every week.
In the free text responses, 44% of the respondents mentioned the weekly Learning Objectives as one of the most useful things that helped them to understand what they are expected to learn every week. Some typical examples of the responses are as follows
Having the objectives clearly outlined was very useful as it provided a checklist of the concepts that should be understood.
Knowing the full learning objectives of the week made understanding each topic easier, as it is easier to understand a topic when you know the full context.
In addition, some students mentioned the weekly Learning Objectives as useful material for preparation for the final exam. As some students put it
The weekly learning objectives were particularly helpful as a checklist for measuring understanding during revision for the final exam.
The weekly learning objectives have been really useful and I'll come back to those when revising for the exam.
It was interesting to discover that splitting lectures into topic videos with clearly defined names was also helpful in the understanding of what the students are expected to learn. This was indicated by 27% of respondents.
Takeaway conclusions
- The use of Q&A dashboard Campuswire with an option of anonymous questions positively contributed to the level of communication in the module.
- The use of breakout rooms during the lab sessions worked for some students, but didn’t work for others; more students preferred individual contacts with their demonstrator via MS Teams. However, no single channel of communication with the demonstrators was identified as the best by most of the students, and therefore further investigation is required.
- The presence of the lecturer on the pre-recorded videos improved engagement and concentration of the students.
- The use of weekly Learning Objectives improved students’ understanding of what they are expected to learn every week.
- Some students considered Learning Objectives as a useful resource for preparation for the exam.
- Some students indicated that splitting lecture material into topic videos with clearly defined names helped them to understand what they are expected to learn.