Assessment and Generative AI

Dr Lenka Ucnik, acting Director of TEQSA’s Higher Education Integrity Unit

It’s been over 2 years since the launch of ChatGPT and almost one year since TEQSA’s AI request for information. In that time institutions have grappled with what this new technology means for teaching, learning and assessment. The good news is Australian institutions are leading the world in their actions to mitigate the risks and embrace the benefits gen AI offers. Yet, the journey is only beginning and there is still work to be done. So what are some different strategies institutions have taken and what are some key considerations to maintaining and protecting award integrity.

Associate Professor Gergely Nyilasy & Dr Tom Whitford, Department of Management & Marketing, Faculty of Business & Economics

This presentation explores an innovative initiative embedding Generative AI (GenAI) into undergraduate entrepreneurship assessments. Supported by a Learning and Teaching Innovation Grant, the project redesigned tasks to promote critical engagement with GenAI affordances. The approach utilises experiential learning and scaffolded activities to develop key competencies. Findings reveal enhanced critical thinking, metacognitive awareness, and ethical understanding among students. However, challenges such as increased workload and varying AI literacy and engagement depth are noted. The study offers a practical, scalable model for integrating AI in higher education without compromising learning outcomes. It addresses literature gaps and challenges narratives about AI undermining academic integrity. The session includes interactive discussions on assessment design and discipline-specific applications.

Dr Paul Beuchat (Presenting Author), Suhail Najeeb (Presenting Author), Dr Matt Dos Santos Xavier, Department of Electrical and Electronic Engineering, Faculty of Engineering and Information Technology

We present a modular AI feedback pipeline designed to deliver instructor-quality feedback on engineering reports using rubric-aligned, multi-shot prompting with large language models. Developed to address time constraints in formative assessment, the system extracts and evaluates report segments via tailored prompts and delivers structured feedback through a web interface. Initial evaluations with synthetic reports showed strong alignment with instructor expectations. Our presentation will include a live demo, audience reflection on their own feedback practices, and discussion of how AI can enhance rather than replace human judgement. This work contributes to a scalable framework for AI-augmented feedback in engineering education.

Dr Stella Peng & Dr Winn Chow, School of Computing and Information Systems, Faculty of Engineering and Information Technology

This study addresses the challenges of generative AI in academic assessments by developing customisable AI use scales for various assessment types. Unlike previous generic approaches, these scales provide tailored guidance for different formats like oral presentations and project reports. The scales help instructors set clear boundaries for AI use and assist students in understanding appropriate usage. The research employs a mixed-methods approach, using surveys and interviews based on the Theory of Planned Behaviour to evaluate effectiveness. Preliminary findings suggest the scales are easy to implement and clearly communicate expectations. The study contributes to literature on guiding GenAI use in assessments and aims to promote a culture of academic integrity.

Dr Eduardo Oliveira (Presenting Author) & Cory dal Ponte, School of Computing and Information Systems, Faculty of Engineering and Information Technology

This presentation introduces the development of a self-assessment instrument for genAI literacy and fluency, designed to facilitate personalised, self-determined learning at scale. Grounded in Bloom’s revised taxonomy, the Knowledge Dimension, and the UNESCO AI Competency Framework, the instrument comprises 28 behavioural indicators mapped across cognitive and knowledge domains. It enables learners to reflect on their current capabilities, identify personalised goals, and access developmentally aligned learning pathways. The proposed self-assessment instrument supports heutagogical approaches to education and provides learners with a structured, theoretically grounded framework for embedding genAI capability development into curriculum and practice.

Christopher DF Honig (Presenting Author), Douglas Mann, & Bronson Hill, Department of Chemical Engineering, Faculty of Engineering and Information Technology

We report on an assessment design with unrestricted AI-usage for capstone students within the Masters of Chemical Engineering. Students completed three technical reports (totalling approximately 200 pages) with explicit permission and encouragement to use any generative AI tools (e.g., ChatGPT, Claude, Gemini). Post-assignment surveys captured self-reported AI practices, perceived usefulness and confidence, while subject marks normalized against prior WAM provided performance data. Thematic analysis of >200 qualitative responses revealed AI was used primarily for research, formalisation and grammar polishing. Unrestricted use neither inflated grades nor advantaged particular demographics; instead, lower-achieving students reported greater perceived AI benefit, consistent with Vygotsky’s Zone of Proximal Development. Implications for AI-ready, integrity-aware assessment are discussed.

Dr Hesaam Haji Aboutorab Kashi, Faculty of Education

This paper proposes reframing critical thinking in higher education assessments in light of generative AI. Drawing on Bourdieu's Theory of Practice, it reconceptualises critical thinking as students’ capacity to meaningfully engage with AI. Through a pilot case study in a postgraduate course, the paper explores how students’ use of AI can be understood as new forms of capital and habitus. The presentation invites discussion on how assessment design might evolve when AI becomes a central focus, contributing to the broader reimagining of pedagogical practices in the AI era.

Dr Kim Allison, Faculty of Medicine, Dentistry and Health Sciences (Presenting Author), Dr Christopher DF Honig, Faculty of Engineering and Information Technology &  Dr Mark Merolli, Faculty of Medicine, Dentistry and Health Sciences

This presentation explores the integration of AI-powered chatbots in physiotherapy education to enhance communication and subjective assessment skills. The project developed a GenAI-based chatbot simulating patient encounters for first-year Doctor of Physiotherapy students. The tool allows free-text conversations with virtual patients, complemented by a GPT-based virtual supervisor providing immediate feedback. Preliminary results from over 130 participants indicate high engagement and perceived educational value, particularly in supporting structured questioning and reflective practice. The study contributes to the growing field of AI in health professional education, offering a scalable, flexible tool to complement traditional teaching methods. The session includes a live demonstration, interactive polling, and discussions on pedagogical design and implementation.

Melissa Slamet (Presenting Author), Associate Professor Julie Choi (Presenting Author), & Dr Jessica Gannaway, Faculty of Education

This presentation explores the integration of AI-powered translation tools in multilingual students' academic literacy practices. The study, set in an Australian university, examines how digital technologies mediate complex meaning-making processes, drawing on students' full linguistic and cultural repertoires. Using Cowley's distributed language view, the research reveals a meshwork of ecological, dialogical, and non-local elements in students' learning. The findings propose three movements to reframe digital tool use: dismantling assumptions about multilingual learning, understanding meaning-making processes, and designing supportive environments. The session challenges the view of AI tools as threats to academic integrity, instead advocating for pedagogical transformation that embraces students' diverse expertise and fosters collaborative learning.

Dr Eduardo Oliveira (Presenting Author) & Di Wu, School of Computing and Information Systems, Faculty of Engineering and Information Technology

This presentation introduces a novel approach to personalised, consistent and large scaled support using stylometric fingerprints: interpretable profiles of students’ writing styles. By extracting linguistic, syntactic, and rhetorical features, these individual writing fingerprints enable tailored feedback, adaptive communication, and developmental monitoring. Aligned with the ASE Strategy, this work supports inclusive, student-centred education by helping educators adapt tone, complexity, and messaging to learners’ needs. Applications include monitoring writing development, supporting multilingual learners, and enhancing academic communication and feedback.