At this year’s Learning and Teaching Festival, Paul Borge, a Senior Lecturer in Music Technology at the London College of Music, shared his experience of integrating AI technologies into the assessment processes of their music technology programs. With the assistance of GoCreate AI, Borge and his team have made significant strides in enhancing the assessment framework for their students.
Background
Paul Borge runs a core Level 4 module called Recording Theory, which forms the bedrock of all music technology programs at the college. This module covers a wide array of topics crucial to the field of music technology, supporting the practical aspects of various courses. Its primary aim is to provide students with a thorough understanding of the theory and terminology associated with contemporary acoustic and digital recording practices, as well as ensuring a fundamental grasp of the basic physics of sound and sound propagation. Each academic year, this module is undertaken by 140 to 180 students.
The Challenge with Traditional Assessments
Initially, the core assessment for this module involved a 2000-word essay in which students designed and equipped a theoretical recording studio. While this framework allowed students to demonstrate their understanding, it had several drawbacks:
- Writing Skills vs. Subject Knowledge: It assumed that students were proficient in academic writing from the start, shifting the focus from key topics to writing skills.
- Over-Conceptualization: The essay format allowed for too much interpretation, complicating the assessment of students’ actual understanding.
- Time-Consuming Reviews: Evaluating a large number of essays was labor-intensive and delayed feedback.
In response, the assessment was changed to a traditional multiple-choice exam, which offered several advantages:
- Focused Lecture Delivery: Lectures could be tailored to ensure students grasped essential concepts and theory.
- Efficient Marking: Grading 180 papers became a quick process, enhancing feedback efficiency.
However, this method also had its issues:
- Student Anxiety: Not all students thrived in test environments, leading to stress and anxiety.
- Feedback Challenges: Providing effective feedback was still problematic, and the environmental impact of printing numerous test papers was significant.
The New Assessment Approach
To address these issues, the assessment was divided into three tasks:
- Task A1 (30%): An analytical and conceptual task in Week 5, designed to build confidence and secure early points.
- Tasks A2 and A3 (35% each): Two short in-class multiple-choice tests in Weeks 9 and 14, invigilated in a less stressful environment.
This approach resulted in greater consistency across the marking bands, with a mean average of around 56% and a respectable distribution of firsts, thirds, and fails. However, providing effective feedback and the environmental impact of printing numerous test papers remained concerns.
Leveraging Blackboard
From the beginning, Blackboard’s automated multiple-choice tests were used as mock exams. These tests allowed students to practice with questions in the same format as the actual exam and provided automated feedback, directing students to further resources without compromising the test.
While the intention was to use this system for the actual exams, technical challenges related to the large cohort size and test security prevented full implementation.
Solution with GoCreate AI
Recently, GoCreate AI tests were trialed for recording theory resets, effectively addressing many issues:
- Secure Online Environment: Provided a secure, online, distance-based exam environment.
- Automated Grading and Feedback: Allowed for instant grading and feedback, with AI invigilation and live support.
- Immediate Data Review: Enabled assessors to review data almost immediately after the exam.
Key Considerations
When implementing these assessments, several key factors were considered:
- Preparation: Creating a sufficient pool of multiple-choice questions took significant time, about a year, to develop.
- Question Design: Questions were designed to encourage problem-solving based on intelligent reasoning rather than tricking students.
- Collaboration: Engaging fellow experts and experienced teaching staff was crucial for composing and testing questions.
- Feedback: Generic yet informative and constructive feedback was prepared in advance.
- Timetable: A clear assessment timetable was necessary to accommodate late sittings, mitigation attempts, and resets.
- Instructions: Detailed instructions were provided well in advance to ensure clarity.
Conclusion
While this assessment structure may not be suitable for all practical courses, the integration of AI technologies through GoCreate AI has shown promising results in supporting fact-based theoretical modules. The London College of Music’s experience demonstrates how AI can enhance assessment processes, making them more efficient and effective while reducing the environmental impact.
For more detailed data or questions, Paul Borge encourages interested parties to get in touch.
