I was inspired by the use of technology probes in action research (Madden et al., 2014), that is, creating a digital intervention and then evaluating users’ responses to it in their own environment. In this case, the “Rubber Duck” and the GenAI checklist are the probes to be introduced to the learners, seen as probes in a field-testing setting, taken to the learners’ workspace, to investigate their responses.
Participants
Students in the BSc Year 1 Mathematics and Statistics for Data Science (Math&Stats) unit who gave consent to join the research project.
Intervention
The GenAI checklist and the Rubber Duck chat mode (described in the GenAI Checklist and the Rubber Duck Chat Mode blog post) will be added to the Math&Stats class repository at the beginning of the unit. The GenAI checklist will be a compulsory submission element for all students, the chat mode will not be compulsory, but an encouraged tool to use for coding. Data will only be collected from the participants.
GenAI Workshop
A GenAI workshop (~60mins) will take place during the class time of Math&Stats, using the last 60mins of the class, which is usually a flexible time for Q&As, debugging, and self-study. Workshop schedule:
- 0 – 15mins: I’ll give a brief on the use of AI and AI coding tools
- 15 – 30mins: I’ll walk through the principles and guidelines of using GenAI coding tools in class, the adapted GenAI checklist, and examples of how to keep track of the use of AI for coding.
- 30 – 40mins: Practical notes on AI coding tools, including how they work, how to set up, applying for educational benefits (free access for students), and limitations of AI coding tools.
- 40 – 60mins: Several programming tasks will be prepared. In this activity, students will get into groups of two, choose a task, use the Rubber Duck chat mode and prompt the AI to do the task, keep a chat log of their use according to the GenAI checklist, and put their results in Miro and share with the class.
Slides for the GenAI workshop:
Data Collection
Group discussion on participants’ perceptions toward AI in academia, including anonymised answers to:
- What comes to mind when you hear the term AI? (Words/Feelings, what excites you about using AI? What worries you about using AI)
- What did you use AI for? (What was the model? What questions/prompts did you ask? How did it go?)
Sample chat logs shared on the Miro board by participants, and chat logs included in the final assignment submitted by participants.
Data Analysis
I’ll deliver the Miro discussion in the form of a word cloud. I’ll also reflect on the submitted chat logs to discuss whether the GenAI checklist has been well-received by participants.