Project SokratesT

AI chatbots, under the right conditions, have the potential to strengthen personalised learning, boost learner success and motivation, and enhance students' sense of self-efficacy and competence. 

However, unregulated usage can have significant consequences for the effectiveness and legitimacy of teaching and assessment practices. For example, if students upload materials or textbooks to ChatGPT, potential copyright issues could arise. In some cases, uninformed AI usage can even degrade performance—without the need for the tools to 'hallucinate'. Even factually correct AI-generated content can hinder learning if it focuses on irrelevant topics or incorrectly links concepts. Moreover, AI can encourage students to generate answers without truly understanding the connections, applying what they have learned to new problems, or synthesising the information—crucial steps in competence development according to Bloom and Krathwohl.

The key challenge is finding a way to harness the creative potential of generative AI while ensuring compliance with privacy, didactic, and exam regulations, all while maintaining the quality of course-relevant content. The goal is to develop a bot that helps students engage with content at their own pace, according to their prior knowledge and interests, within a course that has clear learning objectives and assessment requirements. This requires a well-defined and validated information corpus. To achieve this, we are using the proven method of Retrieval Augmented Generation (RAG), which enables the language model's responses to be largely based on a database of previously validated knowledge graphs, documents, and media.

The challenge goes beyond technical issues: a factually accurate answer is only valuable from a pedagogical perspective if it is part of an analytical thought process that extends beyond simple reproduction. It would be far more effective if the AI encouraged students to engage in a Socratic dialogue, prompting them with targeted questions to reflect on knowledge gaps, rather than simply providing the "right" answer. The Socratic method has long been used in higher education to promote critical thinking, making it an ideal didactic principle for an AI tutor.

Methodology
Sokratest is being developed in collaboration with students and teachers. We are adopting an agile, iterative approach, progressing through four development phases based on feedback from stakeholders. This feedback allows for flexible development while maintaining focus on the project's main goal.

In the initial stages, we will define the technical framework for the RAG bot and the design of the process model. At the same time, a psychological study will be developed to examine students’ expectations and experiences.

We will test how best to integrate the data corpus and the language model, analysing factors such as performance, prevention of manipulation, didactic effectiveness, and data privacy. The code will be made available via GitHub for interested users. Additionally, we will create a process model and a survey tool as an Open Educational Resource (OER), allowing potential users to generate a tailored bot.

The data corpus and research questions will be defined within the module, and the real-world laboratory phase will begin. We anticipate increased intrinsic motivation and improved AI literacy among the participants and will analyse pseudonymised queries and dialogues accordingly. Once we have gathered enough data, we will consider adjusting certain aspects, such as the response style.

Usage data will be systematically analysed, and the development phases will be documented. A process model will be prepared to facilitate both internal and external knowledge transfer (see section 1.4). Phase 4 will conclude with an analysis and presentation of the results, which will form the basis for scaling the project.

For copyright reasons, it is expected that no paid large language models (e.g., LLMs such as ChatGPT) will be used. Instead, open-source models for smaller language models will be employed. Open-source models (e.g., Mistral’s “Mixtral”) offer greater flexibility and can be more securely executed, especially when applying RAG with protected content. As the expected knowledge corpus will include not only texts but also audiobooks, videos, and other media formats, multimodal aspects will be explored and utilised in both the data analysis and in interactions with the bot.

Scalability, Application, and Transfer
To promote early scalability, internal and external information sessions have already been planned. The student body has been actively involved in designing a comprehensive survey on AI usage. Based on the positive experiences, we will also foster systematic exchanges with student representatives to make students key drivers of the scaling process.The pilot application has since been launched in two English-language course modules — Business Studies (Bachelor’s) and Psychology (Master’s) — as part of the summer semester 2025.