Augmenting AI with Curated Learning Analytics Literature: Building and Initial Exploration of a Local RAG for Supporting Teachers (LARAG)

Abstract

Though LLMs have completely taken the world by storm, their use in academic settings still faces significant challenges. One of these challenges is that LLMs sometimes “hallucinate” when they do not have the necessary information to reply to the user prompt and, even when they do, they fail to provide a trusted source to back up their claims. In this article, we explore the use of retrieval-augmented generation (RAG) as a way to overcome the aforementioned limitation and enable evidence-based LLM-generated insights. Specifically, we provide the results of our initial exploration of LARAG, a RAG-based system aimed at providing learning analytics recommendations based on the existing literature. Our initial impressions about the system are that it may offer some benefits over traditional LLMs. However, these initial benefits are far from groundbreaking or very accurate. © 2024 Copyright for this paper by its authors.