From code to conversation: integrating IOAs into a large undergraduate IT subject
Teaching contextEducator Name(s): Ms Hasti Samadi & Dr Geela Chee |
Overview of the IOA
Background and Rationale
The IOA was introduced in Semester 2, 2023 to achieve three specific objectives:
- enhance academic integrity by mitigating the misuse of generative AI tools and contract cheating;
- improve accountability in group assignments by ensuring that each student has meaningfully contributed to the task, and understood both their own work and that of their team; and
- develop employability skills, including the ability to explain technical decisions, communicate effectively, and engage in constructive peer discussions, as commonly expected in professional environments.
Context and Scenario
From Week 2, students are divided into groups of 3–4 to collaborate on two assignments, each of which are artefacts linked to an IOA. Assignment 1 (A1), which requires students to develop code and answer analytical questions, is submitted at the end of Week 6. Assessors then have one week to review and mark the code and develop questions for each student before the IOAs are conducted in Week 8. The focus of the A1 IOA is on students' coding and analysis.
Assignment 2 (A2) is an open-ended data analysis project comprising both coding and a 12 to 15-page technical report in which students define research questions and conduct analysis. This assignment is submitted at the end of Week 10. Assessors then have one week to review and grade the submissions, and develop questions for students, before the IOAs are conducted in Week 12. The IOA for A2 covers the report, including design decisions, methods, and conclusions.
During the IOAs, students are expected to demonstrate their individual understanding of both personal and group contributions to the submitted artefacts. They are also asked questions designed to assess their ability to explain their design and implementation decisions, defend their methods and interpretation of results, and reflect on limitations. These IOAs simulate authentic workplace practices such as technical presentations, team meetings, code walkthroughs, and internal evaluations.
Design Features
Students undertake the IOAs individually, but within their assigned groups of 3–4 peers. This ensures accountability for both personal contributions and group work.
The A1 IOA:
- is conducted via 20-minute Zoom sessions outside usual workshop hours;
- focuses on individual coding work and shared analytical results;
- requires each student to answer at least one question about their personal coding and at least two about the group analysis; and
- contributes 5 out of 15 marks for A1.
The A2 IOA:
- is conducted in-person during workshops (with additional sessions as needed);
- comprises a group presentation (students submit their slides in Week 11) followed by individual viva-style questioning;
- features two-hour sessions, each accommodating three groups (9–12 students), led by two assessors;
- requires students to answer at least one question about their own section of the report and at least two questions about other report sections;
- focuses on explaining and justifying methodology, design choices, analytical processes, conclusions, and limitations; and
- contributes 13 out of 35 marks for A2.
For both IOAs, each student has approximately five minutes of focused questioning, plus additional time for group-level interactions. Collectively the two assignments and IOAs are worth 50% of the total subject mark.
Implementation Details
Each IOA session is conducted by two independent assessors, which helps ensure fairness and consistency in marking. Immediate feedback is not provided during the session; instead, students receive written feedback afterwards alongside their marks. Final marks are calculated as the average of the two assessors' independent evaluations.
To ensure assessment consistency, all assessors undergo 2-hour training sessions prior to each IOA. All sessions are also recorded for quality assurance and potential review. A structured process is in place to address significant discrepancies between a student's oral responses and their written report. Such discrepancies trigger a formal misconduct investigation, an approach that has proven highly effective in detecting otherwise hidden breaches of academic integrity.
Reflections and outcomes
According to the educator, the IOAs have led to significant improvements in academic integrity, student accountability, and depth of understanding. Students have reported feeling more engaged with the material, better prepared to discuss and defend their work, and have increased awareness of the importance of genuine collaboration. The process has also been particularly effective in identifying cases of misconduct, especially instances where students relied on AI tools or contributed minimally to group work, issues that were previously difficult to detect with written submissions alone. Additionally, the IOA experience simulates real-world technical discussions and reviews, equipping students with communication and critical thinking skills valued by employers.
The teaching team views the IOA process as iterative, with each delivery providing new insights into improving question design, examiner calibration, and student support. They have also published a short paper which features additional details about the IOA assessment design, available for download here: https://publications.ascilite.org/index.php/APUB/article/view/1282
Recommendations for other educators
- Align the IOA explicitly with the subject’s learning outcomes, focusing on reasoning, understanding, and communication, not just knowledge recall.
- Link the IOA to meaningful artefacts like reports or code to maintain authenticity.
- Provide students with clear expectations, examples, and preparation guidelines.
- Invest in examiner training, covering not only marking consistency but also techniques for asking effective follow-up questions and minimising bias related to accent, language proficiency, or confidence.
- Carefully plan logistics, including grouping, scheduling, and workload balancing, particularly for in-person components with large cohorts.
- Implement a structured protocol for investigating discrepancies between written and oral performance to strengthen integrity outcomes.