As colleagues and students at Bolton College make greater use of GenAi chatbots and GenAi feedback on FirstPass, a platform which supports students and teachers with the formative assessment of open-ended questions, we will be better placed to assess the design, development, use and governance of this service. It also places the College in a better position to address the many technical and pedagogical challenges that accompany the design and use of GenAi services within an education setting. On the technical front, they include scaling the use of conversational services, tempering operational costs, ensuring that they are safe to use and more. With regard to supporting learning, the challenges include academic integrity, alogorithmic authority or the friction that arises as older norms and practices meet the new. If these challenges can be addressed through a collaborative, participatory and co-design process, there is every chance that a further education college like Bolton can offer all its students and employees access to services that make use of generative Ai chatbots; especially if it can take advantage of the affordances that arise through the use of open-source large language models.

The College has a number of reasons for making use of open-source large language models for the FirstPass platform. The first centres on our desire to have greater control and governance over the design, production and use of generative Ai chatbots. The second reason is affordability, which supports our desire to create generative Ai services which can be accessed equitably by all our students and staff. The third reason centres on our desire to co-design Ai services with our students, teachers and campus support teams. This is particularly important to us, because at the present moment in time, there are few examples of co-designed AiED services which are operational and used on a day-to-day basis by schools, colleges and universities. And the fourth reason is regulatory, as we seek to design and use AiED services which comply with UK GDPR and responsible Ai frameworks. Having control over the design of generative Ai conversational services thus enables the College to have complete control over how they are used in a safe and ethical manner.

A dialogical approach informs the development of the conversational feedback interfaces on FirstPass. As a formative assessment platform, these conversational interfaces remind us that learning takes place in a discursive space (Ajjawi and Boud, 2017 p.254). The first of these is between student and teacher which allows for asynchronous and synchronous dialogue between a student and a teacher. And the second conversational interface allows for synchronous dialogue between the student and a chatbot. The chatbot is trained by teachers as they upload the open-ended question, mark scheme and model answers to the chatbot associated with the open-ended question that students are addressing on FirstPass. In addition, teachers can direct the behaviour of their chatbots before they begin to respond to student questions and prompts relating to the open-ended questions.

The project critically assesses if the use of computer mediated conversational feedback impacts positively on the teacher's ability deliver effective feedback at scale, and if it leads to improved learning outcomes. Its absence is particularly acute when students articulate their responses to open-ended questions remotely and through a digital medium. Our work also assesses if the lack of timely effective feedback at the closest point of action is a significant barrier to raising learning outcomes.

Current online digital platforms temper the ability to engage in real-time conversational feedback. This impacts negatively on learning and assessment for several reasons. Firstly, without real-time conversations, feedback is delayed, which may cause students to lose context or interest in the material, and their thought processes are interrupted as they wait for feedback from teachers. Secondly, conversations foster a sense of connection and engagement between students and teachers. In its absence, students feel isolated, less motivated, and disconnected. This sense of isolation may impact negatively on motivation and their willingness to seek help. Thirdly, real-time interactions allow teachers to adapt their teaching based on the immediate responses they receive from their students. Teachers can gauge understanding, misconceptions or gaps in knowledge and they can immediately adjust their instruction or feedback. Without this interaction, teaching becomes less responsive to the immediate needs of students. Fourthly, conversations encourage students to think critically and articulate their thoughts clearly. The process of formulating questions, responding to feedback, and engaging in discussion promotes deeper understanding and helps students to develop their communication and critical thinking skills. The absence of such exchanges limits opportunities for students to practice and refine these skills. Finally, conversation exchanges are a key component of active learning. The absence of such exchanges can lead to a more passive learning experience, which is often less effective. 

Services like FirstPass are currently few in number. However, this will no doubt change if vendors of learning management systems start to embed GenAi feedback into their platforms. Indeed, GenAi feedback will become common place in all EdTech services; including student information systems. However, there are concerns regarding the delegation of feedback in an overt way to GenAi services. For example, will teachers concede to the feedback that GenAi services give to their students, or will the quality of teacher feedback deteriorate with time? Students may welcome on-demand feedback from a GenAi service, but will they react negatively to receiving feedback from computers that are not embodied in the world that they reside in, and from machines which have no comprehension or compassion for their thoughts and words?

References
Ajjawi, R. & Boud, D. 2017, "Researching feedback dialogue.", Assessment and evaluation in higher education, vol. 42, no. 2, pp. 252-265.