Mohammed Saqr, PhD

AI explanations are always computed from aggregate data of all the students to offer the “average” picture. Whereas the average may work for most students, it does not reflect or capture the individual differences or the variability among students. Therefore, instance-level predictions –where explanations for each particular student are presented according to their own data– may help understand how and why predictions were estimated and how a student or teacher may act or make decisions. This study aims to examine the utility of individualized instance-level AI, its value in informing decision-making, and –more importantly– how they can be used to offer personalized feedback. Furthermore, the study examines mispredictions, their explanations and how they offer explanations or affect decision making. Using data from a full course with 126 students, five ML algorithms were implemented with explanatory mechanisms, compared and the best performing algorithm (Random Forest) was therefore selected. The results show that AI explanations, while useful, cannot achieve their full potential without a nuanced human involvement (i.e., hybrid human AI collaboration). Instance-level explainability may allow us to understand individual algorithmic decisions but may not very helpful for personalization or individualized support. In case of mispredictions, the explanations show that algorithms decide based on the “wrong predictors” which underscores the fact that a full data-driven approach cannot be fully trusted with generating plausible recommendations completely on its own and may require human assistance.

Existing predictive learning analytics models have exclusively relied on aggregate data which not only have obfuscated individual differences but also made replicability and generalizability difficult. Therefore, this study takes a radical departure and uses a person-specific approach to predicting and explaining students’ self-regulation (SRL). A person-specific approach entails developing a predictive algorithm for each individual using their own data (i.e., idiographic, single-subject or N=1) . We also use explainable and interpretable AI models that allow us to identify the variables that explain students‘ SRL and guide data-informed decisions. Our study has shown that idiographic single-subject models are tenable, informative, and can accurately capture the individualized students’ SRL mechanisms. Predictions varied vastly across students regarding accuracy and predictors. Furthermore, the traditional average model did not match any student regarding the predictors’ order. These findings are a testimony that the often hypothesized “average” is rare and often does not match any student, let alone the majority of students as always claimed. This stark difference between students, as well as with the average model speaks to the role of individual peculiarities and indicates that no single model can accurately and reliably capture all students precisely. In other words, traditional models –while they may capture general trends– they cannot capture individual students‘ unique personal learning processes, for that, idiographic methods may be the answer.

A central assumption of the scientific method is that inferences derived from group-level analysis align with and generalize to the individual level. This study was conducted to put this assumption to the test to examine if and to what extent our analysis, inferences, and assumptions hold, and which variables generalize from the group to the individual level. We use engagement as the underpinning of this study. However, the same methods and questions apply elsewhere. The study included 238 students over six courses and applied the latest advances in psychological networks. Two networks were estimated using the same data: a between-person model that captures the group-level engagement and a within-person model that captures the within-person processes. The results showed that there were significant differences between the two networks and a lack of generalizability regarding regularity, academic achievement, and online disengagement. Such findings cast doubts on inferences drawn from group-level data about our understanding of learners’ performance or engagement, or to design personalized interventions. More attention and efforts are needed to further model within-person processes to understand, and possibly deliver precise personalized support and interventions that are more generalizable and truly personalized.

There is a paucity of longitudinal studies in online learning across courses or throughout programs. Our study intends to add to this emerging body of research by analyzing the longitudinal trajectories of interaction between student engagement and achievement over a full four-year program. We use learning analytics and life-course methods to study how achievement and engagement are intertwined and how such relationship evolves over a full program for 106 students. Our findings have indicated that the association between engagement and achievement varies between students and progresses differently between such groups over time. Our results showed that online engagement at any single time-point is not a consistent indicator for high achievement. It takes more than a single point of time to reliably forecast high achievement throughout the program. Longitudinal high grades, or longitudinal high levels of engagement (either separately or combined) were indicators of a stable academic trajectory in which students remained engaged –at least on average– and had a higher level of achievement. On the other hand, disengagement at any time point was consistently associated with lower achievement among low-engaged students. Improving to a higher level of engagement was associated with –at least– acceptable achievement levels and rare dropouts. Lack of improvement or “catching up” may be a more ominous sign that should be proactively addressed.

The pace of our knowledge on online engagement has not been at par with our need to understand the temporal dynamics of online engagement, the transitions between engagement states, and the factors that influence a student being persistently engaged, transitioning to disengagement, or catching up and transitioning to an engaged state. Our study addresses such a gap and investigates how engagement evolves or changes over time, using a person-centered approach to identify for whom the changes happen and when. We take advantage of a novel and innovative multistate Markov model to identify what variables influence such transitions and with what magnitude, i.e., to answer the why. We use a large data set of 1428 enrollments in six courses (238 students). The findings show that online engagement changes differently –across students– and at different magnitudes –according to different instructional variables and previous engagement states. Cognitively engaging instructions helped cognitively engaged students stay engaged while negatively affecting disengaged students. Lectures –a resource that requires less mental energy– helped improve disengaged students. Such differential effects point to the different ways interventions can be applied to different groups, and how different groups may be supported. A balanced, carefully tailored approach is needed to design, intervene, or support students’ engagement that takes into account the diversity of engagement states as well as the varied response magnitudes that intervention may incur across diverse students’ profiles.

A prevailing trend in CSCL literature has been the study of students‘ participatory roles. The majority of existing studies examine a single collaborative task or, at most, a complete course. This study aims to investigate the presence –or the lack thereof– of a more enduring disposition that drives student participation patterns across courses. Based on data from a 4-year program where 329 students used CSCL to collaborate in 10 successive courses (amounting up to 84,597 interactions), we identify the emerging roles using centrality measures and latent profile analysis (LPA) and trace the unfolding of roles over the entire duration of the program. Thereafter, we use Mixture Hidden Markov Models (MHMM) –methods that are particularly useful in detecting “latent traits” in longitudinal data– to identify how students‘ roles, transition, persist or evolve over time. Relevant covariates were also examined to explain students’ membership of different trajectories. We identified three different roles (leader, mediator, isolate) at the course level. At the program level, we found three distinct trajectories: an intense trajectory with mostly leaders, a fluctuating trajectory with mostly mediators, and a wallowing-in-the-mire trajectory with mostly isolates. Our results show that roles re-emerge consistently regardless of the task or the course over extended periods of time and in a predictable manner. For instance, isolates “assumed” such a role in almost all of their courses over four years. © 2022 The Authors

Student engagement has a trajectory (a timeline) that unfolds over time and can be shaped by different factors including learners’ motivation, school conditions, and the nature of learning tasks. Such factors may result in either a stable, declining or fluctuating engagement trajectory. While research on online engagement is abundant, most authors have examined student engagement in a single course or two. Little research has been devoted to studying online longitudinal engagement, i.e., the evolution of student engagement over a full educational program. This learning analytics study examines the engagement states (sequences, successions, stability, and transitions) of 106 students in 1396 course enrollments over a full program. All data of students enrolled in the academic year 2014–2015, and their subsequent data in 2015–2016, 2016–2017, and 2017–2018 (15 courses) were collected. The engagement states were clustered using Hidden Markov Models (HMM) to uncover the hidden engagement trajectories which resulted in a mostly-engaged (33% of students), an intermediate (39.6%), and a troubled (27.4%) trajectory. The mostly-engaged trajectory was stable with infrequent changes, scored the highest, and was less likely to drop out. The troubled trajectory showed early disengagement, frequent dropouts and scored the lowest grades. The results of our study show how to identify early program disengagement (activities within the third decile) and when students may drop out (first year and early second year). © 2021 The Author(s)

In recent years, using game concepts for educational purposes in digital environments has become continually more popular and relevant. Games can be used to motivate and engage users in regular system use and, in the end, support learners in achieving better learning outcomes. In this context, different kinds of game concepts exist, such as gamification or serious games, each with a different perspective and usefulness in digital learning environments. Because developing and using with game concepts in digital learning environments has recently become more important, and developing them is still not fully established, questions arise about future research directions involving games in digital learning. Therefore, this study aims to identify the state of the field and determine what is relevant when using game concepts in digital learning. To achieve this goal, we present the results of a bibliometric analysis considering more than 10,000 articles between 2000 and 2019 and summarize them to develop a research agenda. This agenda supports researchers and practitioners in identifying avenues for future research. We contribute to theory by providing a detailed understanding of the relevance of game concepts in digital learning. We propose a research agenda to assist researchers in planning future approaches with and about gamification concepts in digital learning. Practical implications are proposed by demonstrating what should be considered when using game concepts in learning environments. © 2021 The Authors

Predictive modelling of academic success and retention has been a key research theme in Learning Analytics. While the initial work on predictive modelling was focused on the development of general predictive models, portable across different learning settings, later studies demonstrated the drawbacks of not considering the specificities of course design and disciplinary context. This study builds on the methods and findings of related earlier studies to further explore factors predictive of learners‘ academic success in blended learning. In doing so, it differentiates itself by (i) relying on a larger and homogeneous course sample (15 courses, 50 course offerings in total), and (ii) considering both internal and external conditions as factors affecting the learning process. We apply mixed effect linear regression models, to examine: i) to what extent indicators of students‘ online learning behaviour can explain the variability in the final grades, and ii) to what extent that variability is attributable to the course and students‘ internal conditions, not captured by the logged data. Having examined different types of behaviour indicators (e.g., indicators of the overall activity level, those indicative of regularity of study, etc), we found little difference, if any, in their predictive power. Our results further indicate that a low proportion of variance is explained by the behaviour-based indicators, while a significant portion of variability stems from the learners‘ internal conditions. Hence, when variability in external conditions is largely controlled for (the same institution, discipline, and nominal pedagogical model), students’ internal state is the key predictor of their course performance. © 2021 The Authors

This study maps the evolution and state of Computer-Supported Collaborative Learning (CSCL) research, analyzing 6,388 documents published between 1990 and 2022. The findings highlight the sustained engagement of a core group of scholars and the field’s geographic concentration in Western countries, particularly the USA and Europe. While the field remains productive and diverse, recent trends reflect a growing emphasis on integrating emerging technologies such as learning analytics, augmented and virtual reality, and artificial intelligence (AI) into collaborative learning contexts. The study uncovers a tension within CSCL between its epistemological and methodological diversity and the need for theoretical coherence. This diversity has allowed the field to adapt and innovate but raises concerns about fragmentation and the risk of losing a unified identity. For example, while scripting remains a foundational topic, debates persist on balancing instructional guidance with learner agency to foster productive collaboration. Looking ahead, the study underscores the need for CSCL to develop integrative theoretical frameworks that bridge its rich historical foundations with the challenges posed by large-scale, distributed, and technology-mediated collaboration. Addressing these challenges, such as aligning AI innovations with existing theories and ensuring cumulative knowledge-building, will be critical for the field’s ability to sustain its relevance and influence in understanding collaborative learning in complex educational environments.

Proper self-regulating skills are essential in the new reality of digital learning in higher education. Research has shown that the trace data of students’ learning management system activity can identify various online learning tactics and strategies, but also their transitional dynamics, which are linked to academic performance. This study builds on this work by examining how learning tactics and strategies change within individual courses and how this relates to academic performance. A substantial dataset of 41 courses over two academic years at one university is analyzed. Employing Markov models on trace data, we identify prevalent tactics and strategies students use throughout courses. Our study examines shifts in strategy usage, comparing patterns between the initial and latter stages of the courses. The results reveal distinct clusters of learning strategies and their impact on academic achievement. Notably, deep learning strategies show significantly superior performance to surface approaches, especially when maintained over time. Moreover, students who consistently apply the same strategy score higher than those who are inconsistent. However, consistent surface learners score significantly lower than inconsistent learners. Underscoring such longitudinal trends could help interventions, aiding educators in targeting students with weaker strategies at specific times to boost their effectiveness and efficiency. This research contributes to a nuanced understanding of self-regulated learning behaviors in online educational contexts by showing the importance of dynamic transition of learning strategies for educators, instructional designers, and policymakers to enhance student learning experiences and outcomes.

Complex dynamic systems offer a rich platform for understanding the individual or the person-specific mechanisms. Yet, in learning analytics research and education at large, a complex dynamic system has rarely been framed, developed, or used to understand the individual student where the learning process takes place. Individual (or person-specific) methods can accurately and precisely model the individual person, create person-specific models, and devise unique parameters for each individual. Our study used the latest advances in complex systems dynamics to study the differences between group-based and individual self-regulated learning (SRL) dynamics. The findings show that SRL is a complex, dynamic system where different sub-processes influence each other resulting in the emergence of non-trivial patterns that vary across individuals and time scales, and as such far from the uniform picture commonly theorized. We found that the average SRL process does not reflect the individual SRL processes of different people. Therefore, interventions derived from the group-based SRL insights are unlikely to be effective in personalization. We posit that, if personalized interventions are needed, modeling the person with person-specific methods should be the guiding principle. Our study offered a reliable solution to model the person-specific self-regulation processes which can serve as a ground for understanding and improving individual learning and open the door for precision education.