Publications

My 188 research publications include 57 conference papers, 80 journal articles, 33 book chapters, 12 editorials, 5 other publications, and 1 thesis.

Journal Articles

2024

Saqr M. (2024). Group-level analysis of engagement poorly reflects individual students’ processes: Why we need idiographic learning analytics. Computers in Human Behavior, art. no. 107991. doi: 10.1016/j.chb.2023.107991.

A central assumption of the scientific method is that inferences derived from group-level analysis align with and generalize to the individual level. This study was conducted to put this assumption to the test to examine if and to what extent our analysis, inferences, and assumptions hold, and which variables generalize from the group to the individual level. We use engagement as the underpinning of this study. However, the same methods and questions apply elsewhere. The study included 238 students over six courses and applied the latest advances in psychological networks. Two networks were estimated using the same data: a between-person model that captures the group-level engagement and a within-person model that captures the within-person processes. The results showed that there were significant differences between the two networks and a lack of generalizability regarding regularity, academic achievement, and online disengagement. Such findings cast doubts on inferences drawn from group-level data about our understanding of learners’ performance or engagement, or to design personalized interventions. More attention and efforts are needed to further model within-person processes to understand, and possibly deliver precise personalized support and interventions that are more generalizable and truly personalized.
learning analytics, idiographic engagement, within-person, psychological networks, precision education
Mertala P., López-Pernas S., Vartiainen H., Saqr M., Tedre M. (2024). Digital natives in the scientific literature: A topic modeling approach. Computers in Human Behavior, art. no. 108076. doi: 10.1016/j.chb.2023.108076.

The term “digital natives” was introduced in 2001 to describe a generation that has grown up surrounded by technology and the internet. The accompanying claims of a new way of thinking among digital natives were influential in shaping educational policy. Still, they were challenged by research that found no evidence of generation-wide cognitive changes in learners. Yet, the digital natives narrative persists in popular media and the education discourse. This study set out to investigate the reasons for the persistence of the digital native myth. It analyzed the metadata from 1886 articles related to the term between 2001 and 2022 using bibliometric methods and structural topic modeling. The results show that the concept of “digital native” is still both warmly embraced and fiercely criticized by scholars mostly from western and high income countries, and the volume of research on the topic is growing. However, the results suggest that what appears as the persistence of the idea is actually evolution and complete reinvention: The way the “digital native” concept is operationalized has shifted over time through a series of (metaphorical) mutations. The concept of digital native is one (albeit a highly successful) mutation of the generational gap discourse dating back to the early 1900s. While the initial digital native literature relied on Prensky's unvalidated claims and waned upon facing empirical challenges, subsequent versions have sought more nuanced interpretations. Notably, a burgeoning third mutation now co-opts the “digital native” terminology for diverse purposes, often completely decoupled from the foundational literature and its critiques. This study explains the concept's persistence as dynamic evolution of the digital native discourse in contemporary academic and public spheres.
digital nativesbibliometricsstructured topic modelingdigital immigrants
Bobrowicz K., López-Pernas S., Teuber Z., Saqr M., Greiff S. (2024). Prospects in the field of learning and individual differences: Examining the past to forecast the future using bibliometrics. Learning and Individual Differences, art. no. 102399. doi: 10.1016/j.lindif.2023.102399.

Over two centuries, research has delved into individual differences in learning across educational and professional contexts. This commentary conducts a bibliometric analysis of 6556 articles, identifying key research keywords, topics and themes, and their historical evolution. The findings revealed a longstanding emphasis on educational psychology, particularly motivation and achievement, rather than cross-curricular competencies and learner's well-being and socio-economic background. Notably, self-regulated learning (SRL) emerged as an overarching research subject in terms of motivation and achievement, but, surprisingly, not (meta)cognition. Prospects for the field build on cross-disciplinary research, theoretical refinement, and methodological advances. Further, the field is expected to maintain academic rigor, address diversity among learners, foster global collaboration, and focus on underprivileged populations.
bibliometric analysislearning and individual differenceseducation trends
López-Pernas S., Saqr M. (2024). How the dynamics of engagement explain the momentum of achievement and the inertia of disengagement: A complex systems theory approach. Computers in Human Behavior (in-press), art. no. 108126. doi: 10.1016/j.chb.2023.108126.

Engagement can be understood as a complex dynamic process that unfolds over time and interacts with variables within the student, school, and environment. Most of the research on the dynamics of engagement comes from classroom settings and it is so far inconclusive how and why engagement and disengagement evolve over time. Using person-centered methods, sequence, transition, and covariate analysis, we examined a large dataset of 18 consecutive courses of 245 students over a full program. We identified three engagement states (active, average, and disengaged), as well as three distinct longitudinal engagement trajectories (engaged, fluctuating, and mostly disengaged). Taken together, our results showed that engagement trajectories are rather stable over time conforming to the universal dynamics of complex systems. Engaged students were driven by course materials, their achievement, and their previous engaged states (momentum). Most importantly, our results offer a novel theoretical grounding for the understanding of disengagement which has so far remained unexplained. According to our results, disengagement follows the dynamics of a complex system where stability does not require a hard-wired causal mechanism but rather, it is an attractor state that pulls the system to settle in (inertia). Thus, disengagement becomes an equilibrium state for those students that is hard to change (or a stuck state).
learning analyticsengagementmarkov modelingsequence analysislongitudinal studycomplex dynamic systems
López-Pernas S., Saqr M., Conde M.Á., Apiola M., Tedre M. (2024). Mapping computer engineering education research: A topic analysis. Computer (in-press). doi: 10.1109/MC.2024.3349913.

The field of computer engineering has evolved into a separate entity from electrical engineering and computer science. Emerging technologies such as IoT and cloud computing have made their way into computer engineering programs and, as a result, computer engineering education has become increasingly relevant. To ensure that computer engineering students receive a comprehensive education, research is necessary to identify the key areas of focus and determine the current state of the field. In this article, we applied structural topic modeling to identify the themes of computer engineering education research using bibliometric metadata. We analyze the trends in research and the relationships between themes. Our findings reveal that research mainly focuses on subjects that are not unique to computer engineering education (e.g., mathematics and programming). Furthermore, pedagogy and teaching practices play an increasingly central role in connecting research themes. Lastly, there is increasing attention to learning analytics and psychological research.
structural topic modelingcomputer engineeringengineering educationbibliometrics
Saqr M., López-Pernas S. (2024). Mapping the self in self-regulation using complex dynamic systems and idiographic methods. British Journal of Educational Technology. doi: 10.1111/bjet.13452.

Complex dynamic systems offer a rich platform for understanding the individual or the person-specific mechanisms. Yet, in learning analytics research and education at large, a complex dynamic system has rarely been framed, developed, or used to understand the individual student where the learning process takes place. Individual (or person-specific) methods can accurately and precisely model the individual person, create person-specific models, and devise unique parameters for each individual. Our study used the latest advances in complex systems dynamics to study the differences between group-based and individual self-regulated learning (SRL) dynamics. The findings show that SRL is a complex, dynamic system where different sub-processes influence each other resulting in the emergence of non-trivial patterns that vary across individuals and time scales, and as such far from the uniform picture commonly theorized. We found that the average SRL process does not reflect the individual SRL processes of different people. Therefore, interventions derived from the group-based SRL insights are unlikely to be effective in personalization. We posit that, if personalized interventions are needed, modeling the person with person-specific methods should be the guiding principle. Our study offered a reliable solution to model the person-specific self-regulation processes which can serve as a ground for understanding and improving individual learning and open the door for precision education.
Saqr M., López-Pernas S., Murphy K. (2024). How group structure, members' interactions and teacher facilitation explain the emergence of roles in collaborative learning. Learning and Individual Differences (in-press). doi: 10.1016/j.lindif.2024.102463.

The existing research on emerging roles in computer-supported collaborative learning (CSCL) has mostly focused on who did what rather than why, i.e., which variables led to the emergence of certain roles. Therefore, we aimed to bridge such a gap and investigate the variables that explain the emergence of roles. We used a large dataset of 173,838 interactions by 7,054 students in 787 small groups. Two groups of variables were investigated: those related to other collaborators in the group —group size, cohesion, effort, dominance, distribution of participation and replies— as well as teacher factors —effort, influence, replies, collaborators size (ego), and uptake. The study used a novel person-centered method: mixture of experts model framework that incorporates the covariates into the model to quantify their magnitude of explanation of the emergence of the identified roles. Three roles were identified: leaders, mediators, and isolates. Our results show that leaders were likely to emerge regardless of the number of students per group and contribute to better participatory environments where more students are involved, and more posts are contributed by others and further discussed by diverse members. Mediators were more likely to emerge in averagely interactive and balanced groups, whereas isolates “lurked” in active groups which are dominated by few active students. We use our findings and a review of the literature, both in CSCL and in social sciences at large, to propose a framework —which updates the decade-old framework— for operationalization and understanding of the social roles and the factors that drive their emergence.
computer-supported collaborative learning (cscl)model-based clusteringsocial network analysisemerging roleslearning analyticsproblem-based learning
Deriba F., Saqr M., Tukiainen T. (2024). Assessment of Accessibility in Virtual Laboratories: A Systematic Review. Frontiers in Education (in-press). doi: 10.3389/feduc.2024.1351711.

In an era of rapid evolution in educational technologies, Virtual Labs (V-Labs) have emerged as a promising solution, fundamentally altering how learners engage with scientific concepts and experiments. Despite their potential, ensuring their effectiveness and inclusivity in terms of accessibility to diverse students remains a challenging task. Currently, there is limited insight into the accessibility of V-Labs, a gap that our study aims to address. This study seeks to ascertain the effectiveness of V-Labs in terms of accessibility and inclusivity. We synthesized empirical studies, reviewing 36 articles published between 2000 and 2023. Of these articles, 69% of the studies were conducted in higher education and covered a wide range of learning environments. Our study revealed that 47.3% of the studies focused on various engineering subjects. Our findings provide insight into V-Labs' accessibility from four key perspectives: a) students with limited abilities, b) diverse cultural and linguistic backgrounds, c) instructional design features and content availability, and d) interaction supporting features. We also identified existing gaps in the accessibility of the V-Labs from the four perspectives. Furthermore, we examined the assessment methods of V-Labs, shed light on the aspects that are evaluated, and underscored the need for future work on assessment strategies
virtual laboratoryaccessible educational environment;inclusive technologiesonline learningonline experiment
Albadarin Y., Saqr M., Tukiainen M., Pope N. (2024). A systematic literature review of empirical research on ChatGPT in education. Discover Education (in-press). doi: 10.1007/s44217-024-00138-2.

Over the last four decades, studies have investigated the incorporation of Artificial Intelligence (AI) into education. A recent prominent AI-powered technology that has impacted the education sector is ChatGPT. This article provides a systematic review of 14 empirical studies incorporating ChatGPT into various educational settings, published in 2022 and before the 10th of April 2023—the date of conducting the search process. It carefully followed the essential steps outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2020) guidelines, as well as Okoli's (2015) steps for conducting a rigorous and transparent systematic review. In this review, we aimed to explore how students and teachers have utilized ChatGPT in various educational settings, as well as the primary findings of those studies. By employing Creswell's (2015) coding techniques for data extraction and interpretation, we sought to gain insight into their initial attempts at ChatGPT incorporation into education. This approach also enabled us to extract insights and considerations that can facilitate its effective and responsible use in future educational contexts. The results of this review show that learners have utilized ChatGPT as a virtual intelligent assistant, where it offered instant feedback, on-demand answers, and explanations of complex topics. Additionally, learners have used it to enhance their writing and language skills by generating ideas, composing essays, summarizing, translating, paraphrasing texts, or checking grammar. Moreover, learners turned to it as an aiding tool to facilitate their directed and personalized learning by assisting in understanding concepts and homework, providing structured learning plans, and clarifying assignments and tasks. However, the results of specific studies (n=3, 21.4%) show that overuse of ChatGPT may negatively impact innovative capacities and collaborative learning competencies among learners. Educators, on the other hand, have utilized ChatGPT to create lesson plans, generate quizzes, and provide additional resources, which helped them enhance their productivity and efficiency and promote different teaching methodologies. Despite these benefits, the majority of the reviewed studies recommend the importance of conducting structured training, support, and clear guidelines for both learners and educators to mitigate the drawbacks. This includes developing critical evaluation skills to assess the accuracy and relevance of information provided by ChatGPT, as well as strategies for integrating human interaction and collaboration into learning activities that involve AI tools. Furthermore, they also recommend ongoing research and proactive dialogue with policymakers, stakeholders, and educational practitioners to refine and enhance the use of AI in learning environments. This review could serve as an insightful resource for practitioners who seek to integrate ChatGPT into education and stimulate further research in the field.
artificial intelligencelarge language modelschatgpteducationeducational technology;systematic review
Zahn E., Schöbel S., Saqr M., Söllner M. (2024). Mapping Soft Skills and Further Research Directions for Higher Education: A Bibliometrics Approach with Structural Topic Modelling. Studies in Higher Education (in-press). doi: 10.1080/03075079.2024.2361831.

The global landscape and labour market are rapidly evolving in the twenty-first century, requiring individuals to adapt flexibly to new situations. Soft skills, such as problem-solving and communication skills, are becoming increasingly important for preparing students in higher education for future job opportunities. We conducted a bibliometric analysis of 4,358 research studies with the aim of guiding future research and practice. We sought to answer the following research question: What research themes involving soft skills are discussed in the literature, and what further research avenues can inform us about teaching soft skills? Given the interdisciplinary, cross-cutting nature of soft skills, we analysed research from various fields to uncover the relevance of and relationships among different soft skill topics and to identify which soft skill topics are particularly important in higher education settings. To help researchers develop a holistic approach to soft skills education, we provide a summary of existing soft skills research and identify directions for future research. In addition, we outline the evolution of keyword clusters to illuminate the future directions. This approach facilitates a deeper and more nuanced exploration of the relevant concepts. Our research supports practitioners and fosters a better understanding of the meaning of soft skills and how they can be taught in higher education, especially in digital environments.
soft skillshigher educationbibliometric studytwenty-first century learning
Saqr M., Vogelsmeier L.V.D.E., López-Pernas S. (2024). Capturing where the learning process takes place: A person-specific and person-centered primer. Learning and Individual Differences, vol. 113, art. no. 102492. doi: 10.1016/j.lindif.2024.102492.

Research conducted using variable-centered methods uses data from a “group of others” to derive generalizable laws. The average is considered a “norm” where everyone is supposed to be homogeneous and to fit the average yardstick. Deviations from the average are viewed as irregularities rather than natural manifestations of individual differences. However, this homogeneity assumption is theoretically and empirically flawed, leading to inaccurate generalizations about students' behavior based on averages. Alternatively, heterogeneity is a more plausible and realistic characteristic of human functioning and behavior. In this paper, we review the limitations of variable-centered methods and introduce—with empirical examples—person-centered and person-specific methods as alternatives. Person-centered methods are designed with the foundational assumption that humans are heterogeneous, and such heterogeneity can be captured with statistical methods into patterns (or clusters). Person-specific (or idiographic) methods aim to accurately and precisely model the individual person (at the resolution of the single subject sample size). The implications of this paradigm shift are significant, with potential benefits including improved research validity, more effective interventions, and a better understanding of individual differences in learning, and, more importantly, personalization that is tethered to personalized analysis.
learning analyticsidiographicperson-specificheterogeneityperson-centered
Saqr M., Cheng R., López-Pernas S., Beck E. (2024). Idiographic artificial intelligence to explain students' self-regulation: Toward precision education. Learning and Individual Differences, vol. 114, art. no. 102499. doi: 10.1016/j.lindif.2024.102499.

Existing predictive learning analytics models have exclusively relied on aggregate data which not only have obfuscated individual differences but also made replicability and generalizability difficult. Therefore, this study takes a radical departure and uses a person-specific approach to predicting and explaining students’ self-regulation (SRL). A person-specific approach entails developing a predictive algorithm for each individual using their own data (i.e., idiographic, single-subject or N=1) . We also use explainable and interpretable AI models that allow us to identify the variables that explain students' SRL and guide data-informed decisions. Our study has shown that idiographic single-subject models are tenable, informative, and can accurately capture the individualized students’ SRL mechanisms. Predictions varied vastly across students regarding accuracy and predictors. Furthermore, the traditional average model did not match any student regarding the predictors’ order. These findings are a testimony that the often hypothesized “average” is rare and often does not match any student, let alone the majority of students as always claimed. This stark difference between students, as well as with the average model speaks to the role of individual peculiarities and indicates that no single model can accurately and reliably capture all students precisely. In other words, traditional models —while they may capture general trends— they cannot capture individual students' unique personal learning processes, for that, idiographic methods may be the answer.
idiographicartificial intelligencemachine learninglearning analyticsself-regulated learningperson-specificpersonalized learningexplainable aiwithin-person
Elmoazen R., Saqr M., Hirsto L., Tedre M. (2024). Capturing Temporal Pathways of Collaborative Roles: A Multilayered Analytical Approach Using Community of Inquiry. International Journal of Computer-Supported Collaborative Learning, vol. in-press. doi: 10.1007/s11412-024-09431-6.

In collaborative learning, students may follow different trajectories that evolve over time. This study used a multilayered approach to map the temporal dynamics of online problem-based learning (PBL) and the transition of students’ roles across time over a full year duration. Based on data from 135 dental students across four consecutive courses throughout a full academic year, the students’ discourses were coded based on the Community of Inquiry (CoI). A mixture model was used to identify students' roles. The roles identified were leaders, social mediators, and peripheral explorer roles, and they were visualized using epistemic network analysis (ENA). Similar trajectories were identified and visualized using sequence and process mining. The results showed varying activity levels across three trajectories. Students in the active-constructive trajectory took on leadership roles, while the students in the social interactive trajectory were mostly social mediators, and the free rider trajectory showed a predominant peripheral explorer role. The students in all trajectories returned to their initial roles, showing features typical of stable collaborative dispositions. Both active trajectories (active constructive and social interactive) had very close levels of achievement, whereas the free riders demonstrated lower grades compared to their peers. This research suggests that understanding role dynamics and their evolving trajectories can help teachers better design future collaborative activities, assign roles, form groups, distribute tasks, and, more importantly, be able to support students.
computer-supported collaborative learningroleslearning analyticssequence miningprocess miningepistemic network analysisproblem-based learning
Heikkinen S., Cristea T., Saqr M., Malmberg J., Kleingeld A., Snijders D., Matzat U., Tedre M. (2024). Sequence analysis and process mining perspectives to goal setting: What distinguishes business students with high and low self-efficacy beliefs?. Smart Learning Environments, vol. 11, art. no. 40. doi: 10.1186/s40561-024-00327-4.

This study investigates the relationship between students' self-efficacy beliefs, goal-setting, and learning tactics in an online business course. Using sequence analysis and process mining techniques, we analyzed log data from 209 students to identify distinct learning tactics and their association with self-efficacy beliefs, inferred from initial goal-setting and final grades. Four learning tactics were identified, with significant differences observed between students with high and low self-efficacy beliefs. High self-efficacy students demonstrated more varied and adaptive tactics, including greater use of quizzes and project-focused activities. In contrast, low self-efficacy students and those who didn't set goals showed less engagement and fewer monitoring activities. The project-focused tactic had the strongest correlation with final grades and goal achievement. Goal-setting at the course's start was linked to more effective learning behaviors and better outcomes. The study reveals how students' online learning behavior changes based on initial goal-setting and subsequent reflection. These findings contribute to research on self-regulated learning in online environments and offer practical implications for designing online courses and learning analytics interventions. Understanding these differences enables the development of targeted interventions to improve learning tactics and self-efficacy beliefs among students, ultimately enhancing their ability to achieve learning goals in online educational settings.
sequence analysis, process mining, self-efficacy beliefs, learning analytics, project management
Saqr M., López-Pernas S. (2024). Why Explainable AI May Not Ne Enough and How Predictions and Mispredictions Affect Decision Making in Education. Smart Learning Environments, vol. 11(52). doi: 10.1186/s40561-024-00343-4.

AI explanations are always computed from aggregate data of all the students to offer the “average” picture. Whereas the average may work for most students, it does not reflect or capture the individual differences or the variability among students. Therefore, instance-level predictions —where explanations for each particular student are presented according to their own data— may help understand how and why predictions were estimated and how a student or teacher may act or make decisions. This study aims to examine the utility of individualized instance-level AI, its value in informing decision-making, and —more importantly— how they can be used to offer personalized feedback. Furthermore, the study examines mispredictions, their explanations and how they offer explanations or affect decision making. Using data from a full course with 126 students, five ML algorithms were implemented with explanatory mechanisms, compared and the best performing algorithm (Random Forest) was therefore selected. The results show that AI explanations, while useful, cannot achieve their full potential without a nuanced human involvement (i.e., hybrid human AI collaboration). Instance-level explainability may allow us to understand individual algorithmic decisions but may not very helpful for personalization or individualized support. In case of mispredictions, the explanations show that algorithms decide based on the “wrong predictors” which underscores the fact that a full data-driven approach cannot be fully trusted with generating plausible recommendations completely on its own and may require human assistance.
Yasir Mustafa M., Tlili A., Lampropoulos G., Huang R., Jandrić P., Zhao J.; Salha S., Xu, L., Panda S., Kinshuk, López-Pernas S., Saqr M. (2024). A systematic review of literature reviews on Artificial Intelligence in Education (AIED): A roadmap to a future research agenda. Smart Learning Environments, vol. 11(59). doi: 10.1186/s40561-024-00350-5.

Despite the increased adoption of Artificial Intelligence in Education (AIED), several concerns are still associated with it. This has motivated researchers to conduct (systematic) reviews aiming at synthesizing the AIED findings in the literature. However, these AIED reviews are diversified in terms of focus, stakeholders, educational level and region, and so on. This has made the understanding of the overall landscape of AIED challenging. To address this research gap, this study proceeds one step forward by systematically meta-synthesizing the AIED literature reviews. Specifically, 143 literature reviews were included and analyzed according to the technology-based learning model. It is worth noting that most of the AIED research has been from China and the U.S. Additionally, when discussing AIED, strong focus was on higher education, where less attention is paid to special education. The results also reveal that AI is used mostly to support teachers and students in education with less focus on other educational stakeholders (e.g. school leaders or administrators). The study provides a possible roadmap for future research agenda on AIED, facilitating the implementation of effective and safe AIED.
artificial intelligencegenerative aieducationsmart learningliterature reviewmeta-synthesisfuture research

2023

Saqr M., Matcha W., Ahmad Uzir N-A., Jovanovic J., Gasevic. G, López-Pernas S. (2023). Transferring effective learning strategies across learning contexts matters: A study in problem-based learning. Australasian Journal of Educational Technology, vol. 39(3), pp. 35-57. doi: https://doi.org/10.14742/ajet.8303.

Learning strategies are important catalysts of students’ learning. Research has shown that students with effective learning strategies are more likely to have better academic achievement and complete the program. Given the importance of learning strategies in problem-based learning —and in education in general— for student’s learning and achievement, this study aimed to investigate students’ adoption of learning strategies in different course implementations as well as the transfer of learning strategies between courses and relations to performance. We took advantage of recent advances in learning analytics methods, namely sequence and process mining, as well as statistical methods and visualizations to study how students regulate their online learning through using learning strategies. The study included 81,739 log traces of students’ learning related activities from two different problem-based learning medical courses. The results revealed a relation between the adopted learning strategies, course implementation and scaffolding. Students who applied deep learning strategies were more likely to score high grades, and students who applied surface learning strategies were more likely to score lower grades in either course. More importantly, students who were able to transfer deep learning strategies or continue to use effective strategies between courses proved to be the ones with the highest scores, and the least likely to adopt surface strategies in the subsequent course. These results highlight the need for supporting the development of effective learning strategies in PBL curricula so that students adopt and transfer effective strategies as they advance through the program.
learning strategiesproblem-based learninglearning analyticssequence miningprocess mining
Saqr M., López-Pernas S., Jovanovic J., Gasevic. G (2023). Intense, turbulent, or wallowing in the mire: A longitudinal study of cross-course online tactics, strategies, and trajectories. The Internet and Higher Education, art. no. 100902. doi: 10.1016/j.iheduc.2022.100902.

Research has repeatedly demonstrated that students with effective learning strategies are more likely to have better academic achievement. Existing research has mostly focused on a single course or two, while longitudinal studies remain scarce. The present study examines the longitudinal sequence of students’ strategies, their succession, consistency, temporal unfolding, and whether students tend to retain or adapt strategies between courses. We use a large dataset of online traces from 135 students who completed 10 successive courses (i.e., 1350 course enrollments) in a higher education program. The methods used in this study have shown the feasibility of using trace data recorded by learning management systems to unobtrusively trace and model the longitudinal learning strategies across a program. We identified three program-level strategy trajectories: a stable and intense trajectory related to deep learning where students used diverse strategies and scored the highest grades; a fluctuating interactive trajectory, where students focused on course requirements, scored average grades, and were relatively fluctuating; and a light trajectory related to surface learning where students invested the least effort, scored the lowest grades, and had a relatively stable pathway. Students who were intensely active were more likely to transfer the intense strategies and therefore, they were expected to require less support or guidance. Students focusing on course requirements were not as effective self-regulators as they seemed and possibly required early guidance and support from teachers. Students with consistent light strategies or low effort needed proactive guidance and support.
learning analyticslearning strategiessequence analysislongitudinal studies
Saqr M., López-Pernas S. (2023). The temporal dynamics of online problem-based learning: Why and when sequence matters. International Journal of Computer-Supported Collaborative Learning (in-press). doi: 10.1007/s11412-023-09385-1.

Early research about online PBL explored students’ satisfaction, effectiveness, and design. The temporal aspect of online PBL have rarely been addressed. Thus, a gap exists in how online PBL unfolds: when, and for how long a group engages in collaborative discussions. Similarly, little is known about if and what order or sequence of interactions could predict higher achievement. This study aims to bridge such a gap, by implementing the latest advances in temporal learning analytics to analyze the sequential and temporal phases of online PBL across a large sample (n=204 students) of qualitatively coded interactions (8,009 interactions). We analyzed the group level —to understand the group dynamics across the whole problem discussions— and at the students’ level —to understand the students’ contribution dynamics across different episodes. We followed such analysis by examining the association of interaction types and the sequences thereof with students’ performance using multilevel linear regression models. The analysis of the interactions reflected that the scripted PBL process is followed in sequence, yet often lacked enough depth. When cognitive interactions (e.g., arguments, questions, and evaluations) happened, they kindled high cognitive interactions, when low cognitive and social interactions dominated, they kindled low cognitive interaction. Order and sequence of interactions were more predictive of performance with higher explanatory power than frequencies. Starting or initiating interactions (even with low cognitive content) showed the highest association with performance, which points to the importance of timing and sequence.
Saqr M. (2023). Modeling within-person idiographic variance could help explain and individualize learning. The British Journal of Educational Technology (in-press). doi: 10.1111/bjet.13309.

Learning analytics is a fast-growing discipline. Institutions and countries alike are racing to harness the power of using data to support students, teachers and stakeholders. Research in the field has proven that predicting and supporting underachieving students is worthwhile. Nonetheless, challenges remain unre-solved, for example, lack of generalizability, porta-bility and failure to advance our understanding of students' behaviour. Recently, interest has grown in modelling individual or within-person behaviour, that is, understanding the person-specific changes. This study applies a novel method that combines within-person with between-person variance to better understand how changes unfolding at the individual level can explain students' final grades. By modelling the within-person variance, we directly model where the process takes place, that is the student. Our study finds that combining within-and between-person variance offers a better explanatory power and a better guidance of the variables that could be targeted for intervention at the personal and group levels. Furthermore , using within-person variance opens the door for person-specific idiographic models that work on individual student data and offer students support based on their own insights. K E Y W O R D S idiographic, learning analytics, personalized, predictive, within-person
idiographiclearning analyticspersonalizedpredictivewithin-person
Schöbel S., Schmitt A., Benner D., Saqr M., Janson A., Leimeister J.M. (2023). Charting the Evolution and Future of Conversational Agents: A Research Agenda Along Five Waves and New Frontiers. Information Systems Frontiers, vol. 4(20), pp. 1-26. doi: 10.1007/s10796-023-10375-9.

Conversational agents have come a long way from their first appearance in the 1960s to today’s generative AI omnipresent smart personal assistants. Continuous technological advancements such as statistical computing and large language models allow for an increasingly natural and effortless interaction and have contributed to the evolution of conversational agents. Ultimately, these advancements beg the questions of how technical capabilities have and will develop, how the nature of work is changed through humans’ interaction with such agents, as well as how research has and will frame dominant perceptions and depictions ofsuch agents. To address these questions, we conducted a bibliometric study including over 5000 research articles on conversational agents. Based on a systematic analysis of keywords, topics, and author networks, we derive “ five waves of conversational agent research” that describe the past, present, and potential future of research on conversational agents. Our results highlight fundamental evolution CA research, the strengthening role of big tech and novel technological advancements like OpenAI GPT or BLOOM NLU that mark the next frontier of CA research. We contribute to theory by laying out central research streams in the research domain of conversational agents and offer practical implications by highlighting the design and deployment opportunities of conversational agents.
bibliometric analysischatbotconversational agentvoice assistantchatgptlarge language modelsgenerative artificial intelligence
Elmoazen R., Saqr M., Khalil M., Wasson B. (2023). Learning analytics in virtual laboratories: a systematic literature review of empirical research. Smart Learning Environments (in-press). doi: 10.1186/s40561-023-00244-y.

Remote learning has advanced from the theoretical to the practical sciences with the advent of virtual labs. Although virtual labs allow students to conduct their experiments remotely, it is a challenge to evaluate student progress and collaboration using learning analytics. So far, a study that systematically synthesizes the status of research on virtual laboratories and learning analytics does not exist, which is a gap our study aimed to fill. This study aimed to synthesize the empirical research on learning analyt-ics in virtual labs by conducting a systematic review. We reviewed 21 articles that were published between 2015 and 2021. The results of the study showed that 48% of studies were conducted in higher education, with the main focus on the medical field. There is a wide range of virtual lab platforms, and most of the learning analytics used in the reviewed articles were derived from student log files for students' actions. Learning analytics was utilized to measure the performance, activities, perception, and behavior of students in virtual labs. The studies cover a wide variety of research domains, platforms , and analytical approaches. Therefore, the landscape of platforms and applications is fragmented, small-scale, and exploratory, and has thus far not tapped into the potential of learning analytics to support learning and teaching. Therefore, educators may need to find common standards, protocols, or platforms to build on each others' findings and advance our knowledge.
virtual laboratoryremote laboratorieslearning analyticsdistanceeducationonline learning
Saqr M., López-Pernas S., Hrastinski S., Helske S. (2023). The longitudinal association between engagement and achievement varies by time, students’ profiles, and achievement state: A full program study. Computers and Education, vol. 199.0, art. no. 104787. doi: 10.1016/j.compedu.2023.104787.

There is a paucity of longitudinal studies in online learning across courses or throughout programs. Our study intends to add to this emerging body of research by analyzing the longitudinal trajectories of interaction between student engagement and achievement over a full four-year program. We use learning analytics and life-course methods to study how achievement and engagement are intertwined and how such relationship evolves over a full program for 106 students. Our findings have indicated that the association between engagement and achievement varies between students and progresses differently between such groups over time. Our results showed that online engagement at any single time-point is not a consistent indicator for high achievement. It takes more than a single point of time to reliably forecast high achievement throughout the program. Longitudinal high grades, or longitudinal high levels of engagement (either separately or combined) were indicators of a stable academic trajectory in which students remained engaged —at least on average— and had a higher level of achievement. On the other hand, disengagement at any time point was consistently associated with lower achievement among low-engaged students. Improving to a higher level of engagement was associated with —at least— acceptable achievement levels and rare dropouts. Lack of improvement or “catching up” may be a more ominous sign that should be proactively addressed.
learning analyticsmulti-channel sequence analysislongitudinal engagementacademic achievementmixture hidden markov modelsstudent profilesperson-centered methods
Sointu E., Saqr M., Valtonen T., Hallberg S., Väisänen S., Kankaanpää J., Tuominen V., Hirsto L. (2023). Understanding Emotional Behavior with Learning Analytics to Support Pre-service Teachers’ Learning in Challenging Content Area. Journal of Technology and Teacher Education, vol. 31, pp. 67-87. https://www.learntechlib.org/primary/p/221721/.

Pre-service teacher training is research intensive in Finland. Additionally, teaching as profession is highly valued among young people. However, quantitative methods courses are challenging for teacher students from many reasons. Particularly, this is due to previous negative experiences and emotions (among other things). Thus, new approaches for teaching quantitative methods are warranted. In this research we used Flipped Learning, online teaching and learning analytics to support the content learning. The aim of this research was to investigate teacher students’ (N = 40) emotional profiles (i.e., cluster) based on their emotional level (anxiety, boredom and enjoyment) towards quantitative research methods studies and online behavior. For creating profiles, we used questionnaire data. These profiles were then further analyzed with learning analytics data, more precisely, time-ordered data of teacher students’ interactions (i.e., frequencies). Based on the results, three distinct profiles were found: “medium”, “pro quantitative”, and “scared” teacher students towards quantitative research methods. Further investigation revealed that scared students demonstrated statistically significant transitions between different learning materials and activities within the learning management systems compared to other profiles. Interestingly, pro quantitative had the lowest and medium teacher students had no difference in these results. The results are discussed further in the conclusions.
Kahila J., Valtonen T., López Pernas S., Saqr M., Vartiainen H., Kahila S., Tedre M. (2023). A typology of metagamers: Identifying player types based on beyond the game activities. Games & Culture (in-press). doi: 10.1177/15554120231187758.

Previous research on player types is based on players' in-game behaviors and their motivations to play games. However, there are many other activities related to digital games beyond playing the games proper. This study investigates the prevalence and interconnections between these different metagame activities, and classifies gamers based on their use thereof. The results show that digital game-related information-seeking activities are key metagame activities with connections to other metagame activities. Three distinct groups of players were identified based on their metagame activities: versatile metagamers, strategizers, and casual metagamers. The results contribute to the existing literature on metagaming and provide insights into game studies, game design and marketing, and into digital games and learning.
digital gamesmetagamingmetagameplayer typesmodel-based clusteringepistemic network analysis
Kleimola R., López-Pernas S. Väisänen S., Saqr M., Sointu E., Hirsto L. (2023). Learning Analytics to Explore the Motivational Profiles of Non-Traditional Practical Nurse Students: A Mixed-Methods Approach. Empirical Research in Vocational Education and Training (in-press). doi: 10.1186/s40461-023-00150-0.

Learning analytics provides a novel means to examine various aspects of students’ learning and to support them in their individual endeavors. The purpose of this study was to explore the potential of learning analytics to provide insights into non-traditional, vocational practical nurse students’ (N = 132) motivational profiles for choosing their studies, using a mixed-methods approach. Non-traditional students were somewhat older learners than those following a more straightforward educational pathway and had diverse educational or professional backgrounds. Institutional admission data and analytics were used to identify their specific study motives and distinct motivational profiles, and to illustrate the connections between the motives emerging in the motivational profiles. Furthermore, the association between the motivational profiles and study performance was examined. The results of qualitative content analysis indicated that non-traditional practical nurse students pursued such specialized training for various reasons, and that pragmatic, professional rationales were emphasized over prosocial, altruistic factors. Through the adoption of person-centered latent class analysis, three motivational profiles were identified: self-aware goal-achievers, qualification attainers, and widely oriented humanitarians. Additionally, the analyses of epistemic networks for the profiles showed the complex interplay between the motives, confirming that some motive connections appear to be more prominent than others. Moreover, the findings indicated that study motives reported at admission did not seem to dictate students’ later study performance, as no statistically significant associations were found between the motivational profile and the students’ final grade point average or study dropout. This investigation paves the way for more-targeted motivational support and the use of learning analytics in the context of vocational education and training.
learning analyticsstudy motivesmotivational profilespractical nurse studentsnon-traditional studentsvocational education and trainingmixed-methods approachlatent class analysisepistemic network analysis
Saqr M., Raspopovic Milic M., Pancheva K., Peltekova E.V., Conde M.Á. (2023). A multimethod synthesis of Covid-19 education research: the tightrope between covidization and meaningfulness. Universal Access in the Information Society. doi: 10.1007/s10209-023-00989-w.

This study ofers a comprehensive analysis of COVID-19 research in education. A multi-methods approach was used to capture the full breadth of educational research. As such, a bibliometric analysis, structural topic modeling, and qualitative synthesis of top papers were combined. A total of 4,201 articles were retrieved from Scopus, mostly published from 2019 to 2021. In this work special attention is paid to analyzing and synthesizing fndings about: (i) status of research about COVID-19 regarding frequencies, venues, publishing countries, (ii) identifcation of main topics in the COVID-19 research, and (iii) identifcation of the major themes in most cited articles and their impact on the educational community. Structural topic modeling identifed three main groups of topics that related to education in general, moving to online education, or diverse topics (e.g., perceptions, inclusion, medical education, engagement and motivation, well-being, and equality). A deeper analysis of the papers that received most attention revealed that problem understanding was the dominating theme of papers, followed by challenges, impact, guidance, online migration, and tools and resources. A vast number of papers were produced. However, thoughtful, well-planned, and meaningful research was hard to conceptualize or implement, and a sense of urgency led to a deluge of research with thin contributions in a time of dire need to genuine insights.
bibliometrics analysiscovid19
Hrastinski S., Stenbom S., Saqr M., Jansson M., Viberg O. (2023). Examining the Development of K-12 Students' Cognitive Presence Over Time: The Case of Online Mathematics Tutoring. Online Learning Journal, vol. 27(3), pp. 252-270. doi: 10.24059/olj.v27i3.3481.

In this article, we focus on the cognitive presence element of the Community of Inquiry (CoI) framework. Cognitive presence consists of four categories: Triggering Event, Exploration, Integration, and Resolution. These categories have been described as phases following an idealized logical sequence, although the phases should not be seen as immutable. Few studies have empirically examined how the four categories develop over time during the inquiry process. This article uses learning analytics methods to study transitions between the categories in K-12 online mathematics tutoring. It was statistically most probable that the tutoring sessions started with Triggering Event (95%) and then transitioned to Exploration (51%). The transitions from Exploration to Integration (18%) and Integration to Resolution (21%) achieved statistical significance but were less likely. In fact, it was more likely that the tutoring sessions transitioned from Integration to Exploration (39%) and Resolution to Exploration (36%). In conclusion, the findings suggest that the idealized logical sequence is evident in the data but that other transitions occur as well; especially Exploration recurs throughout the sessions. It seems challenging for students to reach the Integration and Resolution categories. As the CoI framework is commonly adopted in practice, it is important that tutors and educators understand that the categories of cognitive presence will often not play out in idealized ways, underlining their role in supporting how the inquiry process unfolds. In order to gain an improved understanding of the inquiry process, future research is suggested to investigate how the presences and categories of the CoI framework develop over time in different educational settings.
cognitive presencecommunity of inquirytimeonline mathematics tutoring
Saqr M., López-Pernas S., Vogelsmeier L.V.D.E. (2023). When, how and for whom changes in engagement happen: A transition analysis of instructional variables. Computers and Education, vol. 207, art. no. 104934. doi: 10.1016/j.compedu.2023.104934.

The pace of our knowledge on online engagement has not been at par with our need to understand the temporal dynamics of online engagement, the transitions between engagement states, and the factors that influence a student being persistently engaged, transitioning to disengagement, or catching up and transitioning to an engaged state. Our study addresses such a gap and investigates how engagement evolves or changes over time, using a person-centered approach to identify for whom the changes happen and when. We take advantage of a novel and innovative multistate Markov model to identify what variables influence such transitions and with what magnitude, i.e., to answer the why. We use a large data set of 1428 enrollments in six courses (238 students). The findings show that online engagement changes differently —across students— and at different magnitudes —according to different instructional variables and previous engagement states. Cognitively engaging instructions helped cognitively engaged students stay engaged while negatively affecting disengaged students. Lectures —a resource that requires less mental energy— helped improve disengaged students. Such differential effects point to the different ways interventions can be applied to different groups, and how different groups may be supported. A balanced, carefully tailored approach is needed to design, intervene, or support students’ engagement that takes into account the diversity of engagement states as well as the varied response magnitudes that intervention may incur across diverse students’ profiles.
learning analyticstransition analysisonline engagementlongitudinal engagementlatent markov modeling

2022

Saqr M., López-Pernas S. (2022). How CSCL roles emerge, persist, transition, and evolve over time: A four-year longitudinal study. Computers and Education, vol. 189, art. no. 104581. doi: 10.1016/j.compedu.2022.104581.

A prevailing trend in CSCL literature has been the study of students' participatory roles. The majority of existing studies examine a single collaborative task or, at most, a complete course. This study aims to investigate the presence —or the lack thereof— of a more enduring disposition that drives student participation patterns across courses. Based on data from a 4-year program where 329 students used CSCL to collaborate in 10 successive courses (amounting up to 84,597 interactions), we identify the emerging roles using centrality measures and latent profile analysis (LPA) and trace the unfolding of roles over the entire duration of the program. Thereafter, we use Mixture Hidden Markov Models (MHMM) —methods that are particularly useful in detecting “latent traits” in longitudinal data— to identify how students' roles, transition, persist or evolve over time. Relevant covariates were also examined to explain students’ membership of different trajectories. We identified three different roles (leader, mediator, isolate) at the course level. At the program level, we found three distinct trajectories: an intense trajectory with mostly leaders, a fluctuating trajectory with mostly mediators, and a wallowing-in-the-mire trajectory with mostly isolates. Our results show that roles re-emerge consistently regardless of the task or the course over extended periods of time and in a predictable manner. For instance, isolates “assumed” such a role in almost all of their courses over four years. © 2022 The Authors
computer-supported collaborative learninglearning analyticsrolesperson-centeredlongitudinal analysis
Saqr M., Peeters W. (2022). Temporal networks in collaborative learning: A case study. British Journal of Educational Technology, vol. 53(5), pp. 1283-1303. doi: 10.1111/bjet.13187.

Social Network Analysis (SNA) has enabled researchers to understand and optimize the key dimensions of collaborative learning. A majority of SNA research has so far used static networks, ie, aggregated networks that compile interactions without considering when certain activities or relationships occurred. Compressing a temporal process by discarding time, however, may result in reductionist oversimplifications. In this study, we demonstrate the potentials of temporal networks in the analysis of online peer collaboration. In particular, we study: (1) social interactions by analysing learners' collaborative behaviour, part of a case study in which they worked on academic writing tasks, and (2) cognitive interactions through the analysis of students' self-regulated learning tactics. The study included 123 students and 2550 interactions. By using temporal networks, we show how to analyse the longitudinal evolution of a collaborative network visually and quantitatively. Correlation coefficients with grades, when calculated with time-respecting temporal measures of centrality, were more correlated with learning outcomes than traditional centrality measures. Using temporal networks to analyse the co-temporal and longitudinal development, reach, and diffusion patterns of students' learning tactics has provided novel insights into the complex dynamics of learning, not commonly offered through static networks. © 2022 The Authors. British Journal of Educational Technology published by John Wiley & Sons Ltd on behalf of British Educational Research Association.
learning analyticssequence miningprocess miningcsclproblem-based learning
Viberg O., Engström L., Saqr M., Hrastinski S. (2022). Exploring students’ expectations of learning analytics: A person-centered approach. Education and Information Technologies, vol. 27(6), pp. 8561-8581. doi: 10.1007/s10639-022-10980-2.

In order to successfully implement learning analytics (LA), we need a better understanding of student expectations of such services. Yet, there is still a limited body of research about students’ expectations across countries. Student expectations of LA have been predominantly examined from a view that perceives students as a group of individuals representing homogenous views. This study examines students’ ideal (i.e., representing their wanted outcomes) and predicted expectations (i.e., unveiling what they realistically expect the LA service is most likely to be) of LA by employing a person-centered approach that allows exploring the heterogeneity that may be found in student expectations. We collected data from 132 students in the setting of Swedish higher education by means of an online survey. Descriptive statistics and Latent Class Analysis (LCA) were used for the analysis. Our findings show that students’ ideal expectations of LA were considerably higher compared to their predicted expectations. The results of the LCA exhibit that the Swedish students’ expectations of LA were heterogeneous, both regarding their privacy concerns and their expectations of LA services. The findings of this study can be seen as a baseline of students’ expectations or a cross-sectional average, and be used to inform student-centered implementation of LA in higher education. © 2022, The Author(s).
higher educationimpactlatent class analysislearning analyticsperson-centered approachstudents’ expectations
Törmänen T., Järvenoja H., Saqr M., Malmberg J., Järvelä S. (2022). A Person-Centered Approach to Study Students’ Socio-Emotional Interaction Profiles and Regulation of Collaborative Learning. Frontiers in Education, vol. 7, art. no. 866612. doi: 10.3389/feduc.2022.866612.

Emotions in collaborative learning both originate from and are externalized in students’ socio-emotional interactions, and individual group members evidently contribute to these interactions to varying degrees. Research indicates that socio-emotional interactions within a group are related with the occurrence of co- and socially shared regulation of learning, which poses a need to study individual contributions to these interactions via a person-centered approach. This study implements multimodal data (video and electrodermal activity) and sequence mining methods to explore how secondary school students’ (n = 54, 18 groups) participation in socio-emotional interactions evolved across a series of collaborative tasks. On this basis, it identifies subgroups of students with distinct longitudinal profiles. Furthermore, it investigates how students with different socio-emotional interaction profiles contributed to their groups’ regulation of learning. Three profiles were identified: negative, neutral, and diverse. Each profile represents a particular socio-emotional interaction pattern with unique characteristics regarding the emotional valence of participation and physiological emotional activation. The profiles relate to students’ contributions to group regulation of learning. Students with the diverse profile were more likely to contribute to regulation, whereas the neutral profile students were less likely to contribute. The results highlight the importance of person-centered methods to account for individual differences and participation dynamics in collaborative learning and consequently clarify how they relate to and influence group regulation of learning. Copyright © 2022 Törmänen, Järvenoja, Saqr, Malmberg and Järvelä.
collaborative learningemotionsmultimodal dataperson-centered approachself-regulated learningsocio-emotional interaction
Saqr M., Tuominen V., Valtonen T., Sointu E., Väisänen S., Hirsto L. (2022). Teachers’ Learning Profiles in Learning Programming: The Big Picture!. Frontiers in Education, vol. 7, art. no. 840178. doi: 10.3389/feduc.2022.840178.

Currently there is a need for studying learning strategies within Massive Open Online Courses | (MOOCs), especially in the context of in-service teachers. This study aims to bridge this gap and try to understand how in-service teachers approach and regulate their learning in MOOCs. In particular, it examines the strategies used by the in-service teachers as they study a course on how to teach programming. The study implemented a combination of unsupervised clustering and process mining in a large MOOC (n = 27,538 of which 8,547 completed). The results show similar trends compared to previous studies conducted within MOOCs, indicating that teachers are similar to other groups of students based on their learning strategies. The analysis identified three subgroups (i.e., clusters) with different strategies: (1) efficient (n = 3596, 42.1%), (2) clickers (n = 1785, 20.9%), and (3) moderates (n = 3,166, 37%). The efficient students finished the course in a short time, spent more time on each lesson, and moved forward between lessons. The clickers took longer to complete the course, repeated the lessons several times, and moved backwards to revise the lessons repeatedly. The moderates represented an intermediate approach between the two previous clusters. As such, our findings indicate that a significant fraction within teachers poorly regulate their learning, and therefore, teacher education should emphasize learning strategies and self-regulating learning skills so that teacher can better learn and transfer their skills to students. Copyright © 2022 Saqr, Tuominen, Valtonen, Sointu, Väisänen and Hirsto.
clusteringeducational data miningin-service teacherslearning analyticslearning strategiesmoocsprocess miningself-regulated learning
Saqr M., López-Pernas S. (2022). The Curious Case of Centrality Measures: A Large-Scale Empirical Investigation. Journal of Learning Analytics, vol. 9(1), pp. 13-31. doi: 10.18608/jla.2022.7415.

There has been extensive research using centrality measures in educational settings. One of the most common lines of such research has tested network centrality measures as indicators of success. The increasing interest in centrality measures has been kindled by the proliferation of learning analytics. Previous works have been dominated by single-course case studies that have yielded inconclusive results regarding the consistency and suitability of centrality measures as indicators of academic achievement. Therefore, large-scale studies are needed to overcome the multiple limitations of existing research (limited datasets, selective and reporting bias, as well as limited statistical power). This study aims to empirically test and verify the role of centrality measures as indicators of success in collaborative learning. For this purpose, we attempted to reproduce the most commonly used centrality measures in the literature in all the courses of an institution over five years of education. The study included a large dataset (n=3,277) consisting of 69 course offerings, with similar pedagogical underpinnings, using meta-analysis as a method to pool the results of different courses. Our results show that degree and eigenvector centrality measures can be a consistent indicator of performance in collaborative settings. Betweenness and closeness centralities yielded uncertain predictive intervals and were less likely to replicate. Our results have shown moderate levels of heterogeneity, indicating some diversity of the results comparable to single laboratory replication studies. © 2022, Society for Learning Analytics Research. All rights reserved.
centrality measureslearning analyticsmeta-analysisreplicabilityreproducibilitysocial network analysis
Valtonen T., López-Pernas S., Saqr M., Vartiainen H., Sointu E.T., Tedre M. (2022). The nature and building blocks of educational technology research. Computers in Human Behavior, vol. 128, art. no. 107123. doi: 10.1016/j.chb.2021.107123.

Supporting teaching and learning with different technologies has a long and broad history. The theories of learning have changed in recent decades, and new technologies have been invented that provide possibilities for supporting learning processes based on different learning theories. Recently, this field has been studied using bibliometric methods with large datasets to gain an overview of this research area. This paper continues this approach by targeting and analyzing the most relevant journals in the field, covering 30,632 articles. The aim was to deepen the results gained from previous bibliometric studies by focusing on the journals within the educational technology field, the keywords used, the most-cited papers, and especially by outlining the theoretical backgrounds of the analyzed articles. Results show new journals with increasing numbers of articles published. By analyzing the publications used as the articles' backgrounds, we can identify the large entities within the field. Articles targeted how technology can support learning processes based on different learning theories. Along with building an understanding of technology-related learning processes, the second large area of research targets the integration and factors affecting the use of technology for teaching and learning practices. The third large perspective focuses on learners’ characteristics, especially their learning skills, motivation, and self-efficacy. The results show that new technologies by themselves do not cause fast changes in research goals and topics; rather, significant changes in the research field evolve slowly. © 2021
Ismail I.I., Saqr M. (2022). A Quantitative Synthesis of Eight Decades of Global Multiple Sclerosis Research Using Bibliometrics. Frontiers in Neurology, vol. 13, art. no. 845539. doi: 10.3389/fneur.2022.845539.

Bibliometric studies on the field of multiple sclerosis (MS) research are scarce. The aim of this study is to offer an overarching view of the body of knowledge about MS research over eight decades–from 1945 to 2021–by means of a bibliometric analysis. We performed a quantitative analysis of a massive dataset based on Web of Science. The analysis included frequencies, temporal trends, collaboration networks, clusters of research themes, and an in-depth qualitative analysis. A total of 48,356 articles, with 1,766,086 citations were retrieved. Global MS research showed a steady increase with an annual growth rate of 6.4%, with more than half of the scientific production published in the last decade. Published articles came from 98 different countries by 123,569 authors in 3,267 journals, with the United States ranking first in a number of publications (12,770) and citations (610,334). A co-occurrence network analysis formed four main themes of research, covering the pathophysiological mechanisms, neuropsychological symptoms, diagnostic modalities, and treatment of MS. A noticeable increase in research on cognition, depression, and fatigue was observed, highlighting the increased attention to the quality of life of patients with MS. This bibliometric analysis provided a comprehensive overview of the status of global MS research over the past eight decades. These results could provide a better understanding of this field and help identify new directions for future research. Copyright © 2022 Ismail and Saqr.
articlesbibliometricscitationscountry productivityimpact factormultiple sclerosispublication trendsscientific collaboration
Saqr M., Elmoazen R., Tedre M., López-Pernas S., Hirsto L. (2022). How well centrality measures capture student achievement in computer-supported collaborative learning? – A systematic review and meta-analysis. Educational Research Review, vol. 35, art. no. 100437. doi: 10.1016/j.edurev.2022.100437.

Research has shown the value of social collaboration and the benefits it brings to learners. In this study, we investigate the worth of Social Network Analysis (SNA) in translating students' interactions in computer-supported collaborative learning (CSCL) into proxy indicators of achievement. Previous research has tested the correlation between SNA centrality measures and achievement. Some results indicate a positive association, while others do not. To synthesize research efforts, investigate which measures are of value, and how strong of an association exists, this article presents a systematic review and meta-analysis of 19 studies that included 33 cohorts and 16 centrality measures. Achievement was operationalized in most of the reviewed studies as final course or task grade. All studies reported that one or more centrality measures had a positive and significant correlation with, or a potential for predicting, achievement. Every centrality measure in the reviewed sample has shown a positive correlation with achievement in at least one study. In all the reviewed studies, degree centralities correlated with achievement in terms of final course grades or other achievement measure with the highest magnitude. Eigenvector-based centralities (Eigenvector, PageRank, hub, and authority values) were also positively correlated and mostly statistically significant in all the reviewed studies. These findings emphasize the robustness and reliability of degree- and eigenvector-based centrality measures in understanding students’ interactions in relation to achievement. In contrast, betweenness and closeness centralities have shown mixed or weak correlations with achievement. Taken together, our findings support the use of centrality measures as valid proxy indicators of academic achievement and their utility for monitoring interactions in collaborative learning settings. © 2022 The Authors
achievementcentrality measurescscleducational data mininglearning analyticspredicting performancesocial network analysis
Nissinen M., Silvennoinen E., Saqr M. (2022). Monivalintakysymykset oikeustieteellisen alan yhteisvalintakokeessa - hitti vai huti?. Edilex. https://www.edilex.fi/artikkelit/25527.

Oikeustieteellisellä alalla valtakunnallinen suomenkielinen valintakoe otettiin käyttöön vuonna 2018. Samalla valintakokeeseen osallistujien määrä kasvoi ja tarkistus muutettiin kaksivaiheiseksi siten, että ensimmäisessä vaiheessa hakijat karsitaan monivalintatehtävien perusteella. Kaikilta kokeeseen osallistuneilta tarkistetaan monivalintatehtävät, ja hakijat asetetaan niiden perusteella paremmuusjärjestykseen. Vain toiseen vaiheeseen edenneiltä hakijoilta tarkistetaan kokeen esseetehtävät. Monivalintakysymysten käyttöönoton myötä ja hakijoita karsivan luonteen vuoksi monivalintakysymysten käyttöä ja toimivuutta on tärkeää arvioida. Tutkimuksen on tarkoitus auttaa ymmärtämään paremmin monivalintakysymysten hyötyjä ja haittoja oikeustieteen valintakokeessa. Tutkimuksessa vastataan kysymykseen siitä, voitaisiinko opiskelijavalinta tehdä luotettavasti ja riittävän erottelevasti pelkästään monivalintakysymysten perusteella. Tutkimuksella saadaan tietoa, jonka avulla voidaan arvioida koulutusjärjestelmän tavoitteita, laatua ja tasapuolisuutta, sekä sen haasteita ja kehityskohteita.
muutoksenhakuoikeustieteellinen tutkintoyhdenvertaisuus
Merikko J., Ng K., Saqr M., Ihantola P. (2022). To Opt in or to Opt Out? Predicting Student Preference for Learning Analytics-Based Formative Feedback. IEEE Access, vol. 10, pp. 99195-99204. doi: 10.1109/ACCESS.2022.3207274.

Teachers' work is increasingly augmented with intelligent tools that extend their pedagogical abilities. While these tools may have positive effects, they require use of students' personal data, and more research into student preferences regarding these tools is needed. In this study, we investigated how learning strategies and study engagement are related to students' willingness to share data with learning analytics (LA) applications and whether these factors predict students' opt-in for LA-based formative feedback. Students (N = 158) on a self-paced online course set their personal completion goals for the course and chose to opt in for or opt out of personalized feedback based on their progress toward their goal. We collected self-reported measures regarding learning strategies, study engagement, and willingness to share data for learning analytics through a survey (N = 73). Using a regularized partial correlation network, we found that although willingness to share data was weakly connected to different aspects of learning strategies and study engagement, students with lower self-efficacy were more hesitant to share data about their performance. Furthermore, we could not sufficiently predict students' opt-in decisions based on their learning strategies, study engagement, or willingness to share data using logistic regression. Our findings underline the privacy paradox in online privacy behavior: theoretical unwillingness to share personal data does not necessarily lead to opting out of interventions that require the disclosure of personal data. Future research should look into why students opt in for or opt out of learning analytics interventions. © 2013 IEEE.
feedbacklearning strategiesopt-inprivacyself-regulationstudy engagementteaching augmentation
Heikkinen S., Saqr M., Malmberg J., Tedre M. (2022). Supporting self-regulated learning with learning analytics interventions – a systematic literature review. Education and Information Technologies. doi: 10.1007/s10639-022-11281-4.

During the past years scholars have shown an increasing interest in supporting students' self-regulated learning (SRL). Learning analytics (LA) can be applied in various ways to identify a learner’s current state of self-regulation and support SRL processes. It is important to examine how LA has been used to identify the need for support in different phases of SRL cycle, which channels are used to mediate the intervention and how efficient and impactful the intervention is. This will help the learners to achieve the anticipated learning outcomes. The systematic literature review followed PRISMA 2020 statement to examine studies that applied LA interventions to enhance SRL. The search terms used for this research identified 753 papers in May 2021. Of these, 56 studies included the elements of LA, SRL, and intervention. The reviewed studies contained various LA interventions aimed at supporting SRL, but only 46% of them revealed a positive impact of an intervention on learning. Furthermore, only four studies reported positive effects for SRL and covered all three SRL phases (planning, performance, and reflection). Based on the findings of this literature review, the key recommendation is for all phases of SRL to be considered when planning interventions to support learning. In addition, more comparative research on this topic is needed to identify the most effective interventions and to provide further evidence on the effectiveness of interventions supporting SRL. © 2022, The Author(s).
interventionlearning analyticsself-regulated learningsystematic literature review
Apiola M., Lopez-Pernas S., Saqr M., Pears A., Daniels M., Malmi L., Tedre M. (2022). From a National Meeting to an International Conference: A Scientometric Case Study of a Finnish Computing Education Conference. IEEE Access, vol. 10, pp. 66576-66588. doi: 10.1109/ACCESS.2022.3184718.

Computerisation and digitalisation are shaping the world in fundamental and unpredictable ways, which highlights the importance of computing education research (CER). As part of understanding the roots of CER, it is crucial to investigate the evolution of CER as a research discipline. In this paper we present a case study of a Finnish CER conference called Koli Calling, which was launched in 2001, and which has become a central publication venue of CER. We use data from 2001 to 2020, and investigate the evolution of Koli Calling's scholarly communities and zoom in on it's publication habits and internalisation process. We explore the narrative of the development and scholarly agenda behind changes in the conference submission categories from the perspective of some of the conference chairs over the years. We then take a qualitative perspective, analysing the conference publications based on a comprehensive bibliometric analysis. The outcomes include classification of important research clusters of authors in the community of conference contributors. Interestingly, we find traces of important events in the historical development of CER. In particular, we find clusters emerging from specific research capacity building initiatives and we can trace how these connect research spanning the world CER community from Finland to Sweden and then further to the USA, Australia and New Zealand. This paper makes a strategic contribution to the evolution of CER as a research discipline, from the perspective of one central event and publication venue, providing a broad perspective on the role of the conference in connecting research clusters and establishing an international research community. This work contributes insights to researchers in one specific CER community and how they shape the future of computing education © 2013 IEEE.
computer science educationcomputing educationcomputing education researchreviewscience mappingscientometrics
Törmänen T., Järvenoja H., Saqr M., Malmberg J., Järvelä S. (2022). Affective states and regulation of learning during socio-emotional interactions in secondary school collaborative groups. British Journal of Educational Psychology. doi: 10.1111/bjep.12525.

Background: Group affective states for learning are constantly formed through socio-emotional interactions. However, it remains unclear how the affective states vary during collaboration and how they occur with regulation of learning. Appropriate methods are needed to track both group affective states and these interaction processes. Aims: The present study identifies different socio-emotional interaction episodes during groups' collaborative learning and examines how group affective states fluctuate with regulation of learning during these episodes. Sample: The participants were 54 secondary school students working in groups across four science learning sessions. Methods: Multichannel process data (video, electrodermal activity [EDA]) were collected in an authentic classroom. Groups' affective states were measured with emotional valence captured from video data, and activation captured as sympathetic arousal from EDA data. Regulation of learning was observed from the videotaped interactions. Results: The study disclosed four clusters of socio-emotional interaction episodes (positive, negative, occasional regulation, frequent regulation), which differed in terms of fluctuation of affective states and activated regulation of learning. These clustered episodes confirm how affective states are constantly reset by socio-emotional interactions and regulation of learning. The results also show that states requiring regulation do not automatically lead to its activation. Conclusions: By advancing existing understanding of how group level socio-emotional processes contribute to regulation of learning, the study has implications for educational design and psychological practice. Methodologically, it contributes to collaborative learning research by employing multiple data channels (including biophysiological measures) to explore the various dimensions of socio-emotional processes in groups. © 2022 The British Psychological Society.
affectco-regulationcollaborative learningself-regulated learningsocially shared regulationsocio-emotional interaction
Malmberg J., Saqr M., Järvenoja H., Järvelä S. (2022). How the Monitoring Events of Individual Students Are Associated With Phases of Regulation — A Network Analysis Approach. Journal of Learning Analytics, vol. 9(1), pp. 77-92. doi: 10.18608/jla.2022.7429.

The current study uses a within-person temporal and sequential analysis to understand individual learning processes as part of collaborative learning. Contemporary perspectives of self-regulated learning acknowledge monitoring as a crucial mechanism for each phase of the regulated learning cycle, but little is known about the function of the monitoring of these phases by individual students in groups and the role of motivation in this process. This study addresses this gap by investigating how monitoring coexists temporally and progresses sequentially during collaborative learning. Twelve high school students participated in an advanced physics course and collaborated in groups of three for twenty 90-minute learning sessions. Each student’s monitoring events were first identified from the videotaped sessions and then associated with the regulation phase. In addition, the ways in which students acknowledged each monitoring event were coded. The results showed that cyclical phases of regulation do not coexist. However, when we examined temporal and sequential aspects of monitoring, the results showed that the monitoring of motivation predicts the monitoring of task definition, leading to task enactment. The results suggest that motivation is embedded in regulation phases. The current study sheds light on idiographic methods that have implications for individual learning analytics. © 2022, Society for Learning Analytics Research. All rights reserved.
collaborative learningidiographic network analysisnetwork analysispsychological networksself-regulated learning
Saqr M., Jovanovic J., Viberg O., Gašević D. (2022). Is there order in the mess? A single paper meta-analysis approach to identification of predictors of success in learning analytics. Studies in Higher Education, vol. 47(12), pp. 2370-2391. doi: 10.1080/03075079.2022.2061450.

Predictors of student academic success do not always replicate well across different learning designs, subject areas, or educational institutions. This suggests that characteristics of a particular discipline and learning design have to be carefully considered when creating predictive models in order to scale up learning analytics. This study aimed to examine if and to what extent frequently used predictors of study success are portable across a homogenous set of courses. The research was conducted in an integrated blended problem-based curriculum with trace data (n = 2,385 students) from 50 different course offerings across four academic years. We applied the statistical method of single paper meta-analysis to combine correlations of several indicators with students’ success. Total activity and the forum indicators exhibited the highest prediction intervals, where the former represented proxies of the overall engagement with online tasks, and the latter with online collaborative learning activities. Indicators of lecture reading (frequency of lecture view) showed statistically insignificant prediction intervals and, therefore, are less likely to be portable across course offerings. The findings show moderate amounts of variability both within iterations of the same course and across courses. The results suggest that the use of the meta-analytic statistical method for the examination of study success indicators across courses with similar learning design and subject area can offer valuable quantitative means for the identification of predictors that reasonably well replicate and consequently can be reliably portable in the future. © 2022 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.
learning analyticsmeta-analysisportabilityreproducibilitystudent success
Tyni J., Tarkiainen A., Lopez-Pernas S., Saqr M., Kahila J., Bednarik R., Tedre M. (2022). Games and Rewards: A Scientometric Study of Rewards in Educational and Serious Games. IEEE Access, vol. 10, pp. 31578-31585. doi: 10.1109/ACCESS.2022.3160230.

In this study we provide a new viewpoint on the body of literature regarding rewards in serious and educational games. The study includes a quantitative bibliometric analysis of literature in this context from 1969 to 2020. The dataset from the Scopus abstract and citation database was analyzed with the Bibliometrix R library. The data set was manually cleaned to contain only the relevant articles and conference papers. The data was then categorized to match the common themes. From the remaining documents, the amount of annual numbers of publications is presented and the most contributing countries are shown. The most frequent terms from the abstracts and keywords set by the authors are presented, and a co-occurrence network is drawn from the same data. The results of this study reveal that the most occurring topics in this dataset are gamification, physical activity, health, game design, and game-based learning. New directions for research are provided as the most commonly used media appear to be video games and mobile devices in addition to the literature being mostly focused on theory and not practical application. © 2022 IEEE.
bibliometricseducational gamesrewardsscientometric analysisserious games
Saqr M., Poquet O., Lopez-Pernas S. (2022). Networks in Education: A Travelogue Through Five Decades. IEEE Access, vol. 10, pp. 32361-32380. doi: 10.1109/ACCESS.2022.3159674.

For over five decades, researchers have used network analysis to understand educational contexts, spanning diverse disciplines and thematic areas. The wealth of traditions and insights accumulated through these interdisciplinary efforts is a challenge to synthesize with a traditional systematic review. To overcome this difficulty in reviewing 1791 articles researching the intersection of networks and education, this study combined a scientometric approach with a more qualitative analysis of metadata, such as keywords and authors. Our analysis shows rapidly growing research that employs network analysis in educational contexts. This research output is produced by researchers in a small number of developed countries. The field has grown more recently, through the surge in the popularity of data-driven methods, the adoption of social media, and themes as teacher professional development and the now-declining MOOC research. Our analysis suggests that research combining networks and educational phenomena continues to lack an academic home, as well as remains dominated by descriptive network methods that depict phenomena such as interpersonal friendship or patterns of discourse-based collaboration. We discuss the gaps in existing research, the methodological shortcomings, the possible future directions and most importantly how network research could help advance our knowledge of learning, learners, and contribute to our knowledge and to learning theories. © 2013 IEEE.
bibliometricseducationlearning analyticsnetwork sciencesocial network analysis
Apiola M., Saqr M., Lopez-Pernas S., Tedre M. (2022). Computing Education Research Compiled: Keyword Trends, Building Blocks, Creators, and Dissemination. IEEE Access, vol. 10, pp. 27041-27068. doi: 10.1109/ACCESS.2022.3157609.

The need for organized computing education efforts dates back to the 1950s. Since then, computing education research (CER) has evolved and matured from its early initiatives and separation from mathematics education into a respectable research specialization of its own. In recent years, a number of meta-research papers, reviews, and scientometric studies have built overviews of CER from various perspectives. This paper continues that approach by offering new perspectives on the past and present state of CER: Analyses of influential papers throughout the years, of the theoretical backgrounds of CER, of the institutions and authors who create CER, and finally of the top publication venues and their citation practices. The results reveal influential contributions from early curriculum guidelines to rigorous empirical research of today, the prominence of computer programming as a topic of research, evolving patterns of learning-Theory usage, the dominance of high-income countries and a cluster of 52 elite institutions, and issues regarding citation practices within the central venues of dissemination. © 2013 IEEE.
computer science educationcomputing educationcomputing education researchreviewscience mappingscientometrics
López-Pernas S., Saqr M., Gordillo A., Barra E. (2022). A learning analytics perspective on educational escape rooms. Interactive Learning Environments. doi: 10.1080/10494820.2022.2041045.

Learning analytics methods have proven useful in providing insights from the increasingly available digital data about students in a variety of learning environments, including serious games. However, such methods have not been applied to the specific context of educational escape rooms and therefore little is known about students' behavior while playing. The present work aims to fill the gap in the existing literature by showcasing the power of learning analytics methods to reveal and represent students' behavior when participating in a computer-supported educational escape room. Specifically, we make use of sequence mining methods to analyze the temporal and sequential aspects of the activities carried out by students during these novel educational games. We further use clustering to identify different player profiles according to the sequential unfolding of students' actions and analyze how these profiles relate to knowledge acquisition. Our results show that students' behavior differed significantly in their use of hints in the escape room and resulted in differences in their knowledge acquisition levels. © 2022 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.
computer science educationeducational escape roomsgame-based learninglearning analyticssequence miningserious games
Elmoazen R., Saqr M., Tedre M., Hirsto L. (2022). A Systematic Literature Review of Empirical Research on Epistemic Network Analysis in Education. IEEE Access, vol. 10, pp. 17330-17348. doi: 10.1109/ACCESS.2022.3149812.

Over the past decade, epistemic network analysis (ENA) has emerged as a quantitative ethnography tool for modeling discourse in different types of human behaviors. This article offers a comprehensive systematic review of ENA educational applications in empirical studies ({n}=76 ) published between 2010 and 2021. We review the ENA methods that research has relied on, the use of educational theories, their method of application, comparisons across groups and the main findings. Our results show that ENA has helped visually model the coded interactions and illustrate the connection strength among elements of network models. The applications of ENA have expanded beyond discourse analysis to several new areas of inquiry such as modeling surveys, log files or game play. Most of the reviewed articles used ENA based on educational theories and frameworks ({n}=53 , 69.7%), with one or more theories per article, while 23 articles (30.3%) did not report theoretical grounding. The implementation of ENA has enabled comparisons across groups and helped augment the insights of other methods such as process mining, however there is little evidence that studies have exploited the quantitative potential of ENA. Most of the reviewed studies used ENA on small sample size with manually coded interactions with few examples of large samples and automated coding. © 2013 IEEE.
epistemic network analysislearning analyticsnetwork analysisquantitative ethnographysystematic review
Sointu E., Valtonen T., Hallberg S., Kankaanpää J., Väisänen S., Heikkinen L., Saqr M., Tuominen V., Hirsto L. (2022). Learning analytics and Flipped Learning in online teaching for supporting preservice teachers’ learning of quantitative research methods. Seminar.net, vol. 18(1). doi: 10.7577/seminar.4686.

Research methods, including those of a quantitative nature, are an important part of preservice teacher training in Finland. However, quantitative research methods are considered challenging, often feared, and even hated among preservice teachers. This may be due to previous negative experiences and emotions associated with their use, which also influence other aspects of learning such as self-regulation, self-efficacy, and orientations. Given such circumstances, new ways to teach and support the learning of quantitative methods are needed. Here, we investigate the self-regulation, self-efficacy, orientations, and emotions of preservice teachers (N = 38) enrolled in a quantitative methods online course incorporating learning analytics and a flipped learning approach. Dispositional learning analytics data from five measurement points were used, and data were analyzed via descriptive statistics, internal consistency (Cronbach alpha), bootstrapped paired sample t-test (between first and final measurement point), and profiles based on mean. The results demonstrate that in this teaching context, preservice teachers’ time management skills can be improved, and task avoidance, anxiety, and boredom towards quantitative methods decreased. The meaning of these results from the teaching context perspective are also examined, as are the limitations and implications of this study.
research methodsquantitative methodspreservice teacherlearning analyticsflipped learningonline teaching

2021

Saqr M., López-Pernas S. (2021). Modelling diffusion in computer-supported collaborative learning: a large scale learning analytics study. International Journal of Computer-Supported Collaborative Learning, vol. 16(4), pp. 441-483. doi: 10.1007/s11412-021-09356-4.

This study empirically investigates diffusion-based centralities as depictions of student role-based behavior in information exchange, uptake and argumentation, and as consistent indicators of student success in computer-supported collaborative learning. The analysis is based on a large dataset of 69 courses (n = 3,277 students) with 97,173 total interactions (of which 8,818 were manually coded). We examined the relationship between students’ diffusion-based centralities and a coded representation of their interactions in order to investigate the extent to which diffusion-based centralities are able to adequately capture information exchange and uptake processes. We performed a meta-analysis to pool the correlation coefficients between centralities and measures of academic achievement across all courses while considering the sample size of each course. Lastly, from a cluster analysis using students’ diffusion-based centralities aimed at discovering student role-taking within interactions, we investigated the validity of the discovered roles using the coded data. There was a statistically significant positive correlation that ranged from moderate to strong between diffusion-based centralities and the frequency of information sharing and argumentation utterances, confirming that diffusion-based centralities capture important aspects of information exchange and uptake. The results of the meta-analysis showed that diffusion-based centralities had the highest and most consistent combined correlation coefficients with academic achievement as well as the highest predictive intervals, thus demonstrating their advantage over traditional centrality measures. Characterizations of student roles based on diffusion centralities were validated using qualitative methods and were found to meaningfully relate to academic performance. Diffusion-based centralities are feasible to calculate, implement and interpret, while offering a viable solution that can be deployed at any scale to monitor students’ productive discussions and academic success. © 2021, The Author(s).
centrality measurescomputer-supported collaborative learningdiffusionlearning analyticssocial network analysisstudents’ rolesstudy success
Saqr M., Peeters W., Viberg O. (2021). The relational, co-temporal, contemporaneous, and longitudinal dynamics of self-regulation for academic writing. Research and Practice in Technology Enhanced Learning, vol. 16(1), art. no. 29. doi: 10.1186/s41039-021-00175-7.

Writing in an academic context often requires students in higher education to acquire a new set of skills while familiarising themselves with the goals, objectives and requirements of the new learning environment. Students’ ability to continuously self-regulate their writing process, therefore, is seen as a determining factor in their learning success. In order to study students’ self-regulated learning (SRL) behaviour, research has increasingly been tapping into learning analytics (LA) methods in recent years, making use of multimodal trace data that can be obtained from students writing and working online. Nevertheless, little is still known about the ways students apply and govern SRL processes for academic writing online, and about how their SRL behaviour might change over time. To provide new perspectives on the use of LA approaches to examine SRL, this study applied a range of methods to investigate what they could tell us about the evolution of SRL tactics and strategies on a relational, co-temporal, contemporaneous and longitudinal level. The data originates from a case study in which a private Facebook group served as an online collaboration space in a first-year academic writing course for foreign language majors of English. The findings show that learners use a range of SRL tactics to manage their writing tasks and that different tactic can take up key positions in this process over time. Several shifts could be observed in students’ behaviour, from mainly addressing content-specific topics to more form-specific and social ones. Our results have also demonstrated that different methods can be used to study the relational, co-temporal, contemporaneous, and longitudinal dynamics of self-regulation in this regard, demonstrating the wealth of insights LA methods can bring to the table. © 2021, The Author(s).
academic writingepistemic network analysislearning analyticsprocess miningself-regulated learningsequence miningsocial network analysistemporal networks
Saqr M., López-Pernas S. (2021). The longitudinal trajectories of online engagement over a full program. Computers and Education, vol. 175, art. no. 104325. doi: 10.1016/j.compedu.2021.104325.

Student engagement has a trajectory (a timeline) that unfolds over time and can be shaped by different factors including learners’ motivation, school conditions, and the nature of learning tasks. Such factors may result in either a stable, declining or fluctuating engagement trajectory. While research on online engagement is abundant, most authors have examined student engagement in a single course or two. Little research has been devoted to studying online longitudinal engagement, i.e., the evolution of student engagement over a full educational program. This learning analytics study examines the engagement states (sequences, successions, stability, and transitions) of 106 students in 1396 course enrollments over a full program. All data of students enrolled in the academic year 2014–2015, and their subsequent data in 2015–2016, 2016–2017, and 2017–2018 (15 courses) were collected. The engagement states were clustered using Hidden Markov Models (HMM) to uncover the hidden engagement trajectories which resulted in a mostly-engaged (33% of students), an intermediate (39.6%), and a troubled (27.4%) trajectory. The mostly-engaged trajectory was stable with infrequent changes, scored the highest, and was less likely to drop out. The troubled trajectory showed early disengagement, frequent dropouts and scored the lowest grades. The results of our study show how to identify early program disengagement (activities within the third decile) and when students may drop out (first year and early second year). © 2021 The Author(s)
learning analyticslongitudinal engagementsequence miningsurvival analysistrajectories of engagement
Schöbel S., Saqr M., Janson A. (2021). Two decades of game concepts in digital learning environments – A bibliometric study and research agenda. Computers and Education, vol. 173, art. no. 104296. doi: 10.1016/j.compedu.2021.104296.

In recent years, using game concepts for educational purposes in digital environments has become continually more popular and relevant. Games can be used to motivate and engage users in regular system use and, in the end, support learners in achieving better learning outcomes. In this context, different kinds of game concepts exist, such as gamification or serious games, each with a different perspective and usefulness in digital learning environments. Because developing and using with game concepts in digital learning environments has recently become more important, and developing them is still not fully established, questions arise about future research directions involving games in digital learning. Therefore, this study aims to identify the state of the field and determine what is relevant when using game concepts in digital learning. To achieve this goal, we present the results of a bibliometric analysis considering more than 10,000 articles between 2000 and 2019 and summarize them to develop a research agenda. This agenda supports researchers and practitioners in identifying avenues for future research. We contribute to theory by providing a detailed understanding of the relevance of game concepts in digital learning. We propose a research agenda to assist researchers in planning future approaches with and about gamification concepts in digital learning. Practical implications are proposed by demonstrating what should be considered when using game concepts in learning environments. © 2021 The Authors
bibliometric studygame conceptsgamesgamificationmobile learningsimulations
Jovanović J., Saqr M., Joksimović S., Gašević D. (2021). Students matter the most in learning analytics: The effects of internal and instructional conditions in predicting academic success. Computers and Education, vol. 172, art. no. 104251. doi: 10.1016/j.compedu.2021.104251.

Predictive modelling of academic success and retention has been a key research theme in Learning Analytics. While the initial work on predictive modelling was focused on the development of general predictive models, portable across different learning settings, later studies demonstrated the drawbacks of not considering the specificities of course design and disciplinary context. This study builds on the methods and findings of related earlier studies to further explore factors predictive of learners' academic success in blended learning. In doing so, it differentiates itself by (i) relying on a larger and homogeneous course sample (15 courses, 50 course offerings in total), and (ii) considering both internal and external conditions as factors affecting the learning process. We apply mixed effect linear regression models, to examine: i) to what extent indicators of students' online learning behaviour can explain the variability in the final grades, and ii) to what extent that variability is attributable to the course and students' internal conditions, not captured by the logged data. Having examined different types of behaviour indicators (e.g., indicators of the overall activity level, those indicative of regularity of study, etc), we found little difference, if any, in their predictive power. Our results further indicate that a low proportion of variance is explained by the behaviour-based indicators, while a significant portion of variability stems from the learners' internal conditions. Hence, when variability in external conditions is largely controlled for (the same institution, discipline, and nominal pedagogical model), students’ internal state is the key predictor of their course performance. © 2021 The Authors
data science applications in educationdistance education and online learning
Bermo M., Saqr M., Hoffman H., Patterson D., Sharar S., Minoshima S., Lewis D.H. (2021). Utility of SPECT Functional Neuroimaging of Pain. Frontiers in Psychiatry, vol. 12, art. no. 705242. doi: 10.3389/fpsyt.2021.705242.

Functional neuroimaging modalities vary in spatial and temporal resolution. One major limitation of most functional neuroimaging modalities is that only neural activation taking place inside the scanner can be imaged. This limitation makes functional neuroimaging in many clinical scenarios extremely difficult or impossible. The most commonly used radiopharmaceutical in Single Photon Emission Tomography (SPECT) functional brain imaging is Technetium 99 m-labeled Ethyl Cysteinate Dimer (ECD). ECD is a lipophilic compound with unique pharmacodynamics. It crosses the blood brain barrier and has high first pass extraction by the neurons proportional to regional brain perfusion at the time of injection. It reaches peak activity in the brain 1 min after injection and is then slowly cleared from the brain following a biexponential mode. This allows for a practical imaging window of 1 or 2 h after injection. In other words, it freezes a snapshot of brain perfusion at the time of injection that is kept and can be imaged later. This unique feature allows for designing functional brain imaging studies that do not require the patient to be inside the scanner at the time of brain activation. Functional brain imaging during severe burn wound care is an example that has been extensively studied using this technique. Not only does SPECT allow for imaging of brain activity under extreme pain conditions in clinical settings, but it also allows for imaging of brain activity modulation in response to analgesic maneuvers whether pharmacologic or non-traditional such as using virtual reality analgesia. Together with its utility in extreme situations, SPECTS is also helpful in investigating brain activation under typical pain conditions such as experimental controlled pain and chronic pain syndromes. © Copyright © 2021 Bermo, Saqr, Hoffman, Patterson, Sharar, Minoshima and Lewis.
brainecdfunctional imagingpainspect
Pernas S.L., Saqr M., Viberg O. (2021). Putting it all together: Combining learning analytics methods and data sources to understand students’ approaches to learning programming. Sustainability (Switzerland), vol. 13(9), art. no. 4825. doi: 10.3390/su13094825.

Learning programming is a complex and challenging task for many students. It involves both understanding theoretical concepts and acquiring practical skills. Hence, analyzing learners’ data from online learning environments alone fails to capture the full breadth of students’ actions if part of their learning process takes place elsewhere. Moreover, existing studies on learning analytics applied to programming education have mainly relied on frequency analysis to classify students according to their approach to programming or to predict academic achievement. However, frequency analysis provides limited insights into the individual time‐related characteristics of the learning process. The current study examines students’ strategies when learning programming, combining data from the learning management system and from an automated assessment tool used to support students while solving the programming assignments. The study included the data of 292 engineering students (228 men and 64 women, aged 20–26) from the two aforementioned sources. To gain an in‐depth understanding of students’ learning process as well as of the types of learners, we used learning analytics methods that account for the temporal order of learning actions. Our results show that students have special preferences for specific learning resources when learning programming, namely, slides that support search, and copy and paste. We also found that videos are relatively less consumed by students, especially while working on programming assignments. Lastly, students resort to course forums to seek help only when they struggle. © 2021 by the authors. Licensee MDPI, Basel, Switzerland.
automated assessmentcomputer sciencelearning analyticsprocess miningprogrammingsequence mining
Lopez-Pernas S., Saqr M. (2021). Bringing Synchrony and Clarity to Complex Multi-Channel Data: A Learning Analytics Study in Programming Education. IEEE Access, vol. 9, pp. 166531-166541. doi: 10.1109/ACCESS.2021.3134844.

Supporting teaching and learning programming with learning analytics is an active area of inquiry. Most data used for learning analytics research comes from learning management systems. However, such systems were not developed to support learning programming. Therefore, educators have to resort to other systems that support the programming process, which can pose a challenge when it comes to understanding students' learning since it takes place in different contexts. Methods that support the combination of different data sources are needed. Such methods would ideally account for the time-ordered sequence of students' learning actions. In this article, we use a novel method (multi-channel sequence mining with Hidden Markov Models, HMMs) that allows the combination of multiple data sources, accounts for the temporal nature of students' learning actions, and maps the transitions between different learning tactics. Our study included 291 students enrolled in a higher education programming course. Students' trace-log data were collected from the learning management system and from a programming automated assessment tool. Data were analyzed using multi-channel sequence mining and HMM. The results reveal different patterns of students' approaches to learning programming. High achievers start earlier to work on the programming assignments, use more independent strategies and consume learning resources more frequently, while the low achievers procrastinate early in the course and rely on help forums. Our findings demonstrate the potentials of multi-channel sequence mining and how this method can be analyzed using HMM. Furthermore, the results obtained can be of use for educators to understand students' strategies when learning programming. © 2013 IEEE.
automated assessmentcomputer science educationhidden markov modelslearning analyticsprogrammingsequence mining
Saqr M., Ng K., Oyelere S.S., Tedre M. (2021). People, Ideas, Milestones: A Scientometric Study of Computational Thinking. ACM Transactions on Computing Education, vol. 21(3), art. no. 20, pp. 1-17. doi: 10.1145/3445984.

The momentum around computational thinking (CT) has kindled a rising wave of research initiatives and scholarly contributions seeking to capitalize on the opportunities that CT could bring. A number of literature reviews have showed a vibrant community of practitioners and a growing number of publications. However, the history and evolution of the emerging research topic, the milestone publications that have shaped its directions, and the timeline of the important developments may be better told through a quantitative, scientometric narrative. This article presents a bibliometric analysis of the drivers of the CT topic, as well as its main themes of research, international collaborations, influential authors, and seminal publications, and how authors and publications have influenced one another. The metadata of 1,874 documents were retrieved from the Scopus database using the keyword “computational thinking.” The results show that CT research has been US-centric from the start, and continues to be dominated by US researchers both in volume and impact. International collaboration is relatively low, but clusters of joint research are found between, for example, a number of Nordic countries, lusophone- and hispanophone countries, and central European countries. The results show that CT features the computing’s traditional tripartite disciplinary structure (design, modeling, and theory), a distinct emphasis on programming, and a strong pedagogical and educational backdrop including constructionism, self-efficacy, motivation, and teacher training.
computational thinkingbibliometric researchhistoryscientometricsliterature reviewcomputing education researchcomputer science education

2020

Saqr M., Nouri J., Vartiainen H., Tedre M. (2020). Robustness and rich clubs in collaborative learning groups: a learning analytics study using network science. Scientific Reports, vol. 10(1), art. no. 14445. doi: 10.1038/s41598-020-71483-z.

Productive and effective collaborative learning is rarely a spontaneous phenomenon but rather the result of meeting a set of conditions, orchestrating and scaffolding productive interactions. Several studies have demonstrated that conflicts can have detrimental effects on student collaboration. Through the application of network science, and social network analysis in particular, this learning analytics study investigates the concept of group robustness; that is, the capacity of collaborative groups to remain functional despite the withdrawal or absence of group members, and its relation to group performance in the frame of collaborative learning. Data on all student and teacher interactions were collected from two phases of a course in medical education that employed an online learning environment. Visual and mathematical analysis were conducted, simulating the removal of actors and its effect on the group’s robustness and network structure. In addition, the extracted network parameters were used as features in machine learning algorithms to predict student performance. The study contributes findings that demonstrate the use of network science to shed light on essential elements of collaborative learning and demonstrates how the concept and measures of group robustness can increase understanding of the conditions of productive collaborative learning. It also contributes to understanding how certain interaction patterns can help to promote the sustainability or robustness of groups, while other interaction patterns can make the group more vulnerable to withdrawal and dysfunction. The finding also indicate that teachers can be a driving factor behind the formation of rich clubs of well-connected few and less connected many in some cases and can contribute to a more collaborative and sustainable process where every student is included. © 2020, The Author(s).
Saqr M., Viberg O., Vartiainen H. (2020). Capturing the participation and social dimensions of computer-supported collaborative learning through social network analysis: which method and measures matter?. International Journal of Computer-Supported Collaborative Learning, vol. 15(2), pp. 227-248. doi: 10.1007/s11412-020-09322-6.

The increasing use of digital learning tools and platforms in formal and informal learning settings has provided broad access to large amounts of learner data, the analysis of which has been aimed at understanding students’ learning processes, improving learning outcomes, providing learner support as well as teaching. Presently, such data has been largely accessed from discussion forums in online learning management systems and has been further analyzed through the application of social network analysis (SNA). Nevertheless, the results of these analyses have not always been reproducible. Since such learning analytics (LA) methods rely on measurement as a first step of the process, the robustness of selected techniques for measuring collaborative learning activities is critical for the transparency, reproducibility and generalizability of the results. This paper presents findings from a study focusing on the validation of critical centrality measures frequently used in the fields of LA and SNA research. We examined how different network configurations (i.e., multigraph, weighted, and simplified) influence the reproducibility and robustness of centrality measures as indicators of student learning in CSCL settings. In particular, this research aims to contribute to the provision of robust and valid methods for measuring and better understanding of the participation and social dimensions of collaborative learning. The study was conducted based on a dataset of 12 university courses. The results show that multigraph configuration produces the most consistent and robust centrality measures. The findings also show that degree centralities calculated with the multigraph methods are reliable indicators for students’ participatory efforts as well as a consistent predictor of their performance. Similarly, Eigenvector centrality was the most consistent centrality that reliably represented social dimension, regardless of the network configuration. This study offers guidance on the appropriate network representation as well as sound recommendations about how to reliably select the appropriate metrics for each dimension. © 2020, The Author(s).
centrality measurescomputer-supported collaborative learninglearning analyticsnetwork configurationsparticipatory and social dimensionssocial network analysisvalidity
Schöbel S., Janson A., Jahn K., Kordyaka B., Turetken O., Djafarova N., Saqr M., Wu D., Söllner M., Adam M., Gad P.H., Wesseloh H., Leimeister J.M. (2020). A research agenda for the why, what, and how of gamification designs: Outcomes of an ecis 2019 panel. Communications of the Association for Information Systems, vol. 46, art. no. 30, pp. 706-721. doi: 10.17705/1CAIS.04630.

This report summarizes a panel session on gamification designs at the 2019 European Conference on Information Systems in Stockholm, Sweden. The panel explored a research agenda for gamification design. The panel considered the “what, why, and how” to analyze state-of-the-art gamification research. We present an adapted definition of gamification as one outcome of the workshop to better describe what gamification is and what it can be used for. We discuss “why” and “how” to employ gamification for different contexts. Researchers and practitioners can use the report’s research questions and insights to gamify information systems, identity outcomes that gamification concepts address, and explore new ways to gamify. Overall, we present new areas for future research and practice by identifying innovative ways to bring existing gamification concepts to a more impactful level. © 2020 by the Association for Information Systems.
future researchgame design elementsgamificationgamification designs
Saqr M., Nouri J., Vartiainen H., Malmberg J. (2020). What makes an online problem-based group successful? A learning analytics study using social network analysis. BMC Medical Education, vol. 20(1), art. no. 80. doi: 10.1186/s12909-020-01997-7.

Background: Although there is a wealth of research focusing on PBL, most studies employ self-reports, surveys, and interviews as data collection methods and have an exclusive focus on students. There is little research that has studied interactivity in online PBL settings through the lens of Social Network Analysis (SNA) to explore both student and teacher factors that could help monitor and possibly proactively support PBL groups. This study adopts SNA to investigate how groups, tutors and individual student's interactivity variables correlate with group performance and whether the interactivity variables could be used to predict group performance. Methods: We do so by analyzing 60 groups' work in 12 courses in dental education (598 students). The interaction data were extracted from a Moodle-based online learning platform to construct the aggregate networks of each group. SNA variables were calculated at the group level, students' level and tutor's level. We then performed correlation tests and multiple regression analysis using SNA measures and performance data. Results: The findings demonstrate that certain interaction variables are indicative of a well-performing group; particularly the quantity of interactions, active and reciprocal interactions among students, and group cohesion measures (transitivity and reciprocity). A more dominating role for teachers may be a negative sign of group performance. Finally, a stepwise multiple regression test demonstrated that SNA centrality measures could be used to predict group performance. A significant equation was found, F (4, 55) = 49.1, p < 0.01, with an R2 of 0.76. Tutor Eigen centrality, user count, and centralization outdegree were all statistically significant and negative. However, reciprocity in the group was a positive predictor of group improvement. Conclusions: The findings of this study emphasized the importance of interactions, equal participation and inclusion of all group members, and reciprocity and group cohesion as predictors of a functioning group. Furthermore, SNA could be used to monitor online PBL groups, identify important quantitative data that helps predict and potentially support groups to function and co-regulate, which would improve the outcome of interacting groups in PBL. The information offered by SNA requires relatively little effort to analyze and could help educators get valuable insights about their groups and individual collaborators. © 2020 The Author(s).
data analyticslearning analyticsonline learningproblem-based learningsmall groupssocial network analysissocial networking
Saqr M., Al-Mohaimeed A., Rasheed Z. (2020). Tear down the walls: Disseminating open access research for a global impact. International Journal of Health Sciences, vol. 14(5), pp. 43-49. https://pubmed.ncbi.nlm.nih.gov/32952504.

Objective: Publications are the cornerstone of the dissemination of scientific innovation and scholarly work, but published works are mostly behind paywalls. Therefore, many researchers and institutions are searching for alternative models for disseminating scholarly work that bypasses the current structure of paywalls. This study aimed to determine whether a self-published open access (OA) journal, the International Journal of Health Sciences (IJHS), has been able to reach a global audience in terms of authorship, readership, and impact using the OA model. Methods: All IJHS articles were retrieved and analyzed using scientometric methods. Using the keywords from abstracts and titles, unsupervised clustering was performed to map research trends. Network analysis was used to chart the network of collaboration. The analysis of articles’ metadata and the visualizations was performed using R programming language. Results: Using Google Scholar as a source, the general statistics of IJHS from inception to 2019 showed that the average citation per article was 11.29, and the impact factor of the journal was 2.28. The results demonstrate the obvious local and global impact of a locally published journal that allows unrestricted OA and uses an open source publishing platform. The journal’s success at attracting diverse topics, authors, and readers is a testament to the power of the OA model. Conclusions: Open source is feasible and rewarding and enables a global reach for research from under-represented regions. Local journals can help the Global South disseminate their scholarly work, which is frequently ignored by commercial and established publications.
global impactinternational journal of health sciencesopen access journalsscientometric methods
Bergdahl N., Nouri J., Afzaal M., Karunaratne T., Saqr M. (2020). Learning Analytics for Blended Learning: A Systematic Review of Theory, Methodology, and Ethical Consideration. International Journal of Learning Analytics and Artificial Intelligence for Education, vol. 2(2), pp. 46-79. doi: 10.3991/ijai.v2i2.17887.

Learning Analytics (LA) approaches in Blended Learning (BL) research is becoming an established field. In the light of previous critiqued toward LA for not being grounded in theory, the General Data Protection and a renewed focus on individuals’ integrity, this review aims to explore the use of theories, the methodological and analytic approaches in educational settings, along with surveying ethical and legal considerations. The review also maps and explores the outcomes and discusses the pitfalls and potentials currently seen in the field. Journal articles and conference papers were identified through systematic search across relevant databases. 70 papers met the inclusion criteria: they applied LA within a BL setting, were peer-reviewed, full-papers, and if they were in English. The results reveal that the use of theoretical and methodological approaches was disperse, we identified approaches of BL not included in categories of BL in existing BL literature and suggest these may be referred to as hybrid blended learning, that ethical considerations and legal requirements have often been overlooked. We highlight critical issues that contribute to raise awareness and inform alignment for future research to ameliorate diffuse applications within the field of LA.
literature reviewlearning analyticsblended learningsystematic review

2019

Saqr M., Alamro A. (2019). The role of social network analysis as a learning analytics tool in online problem based learning. BMC Medical Education, vol. 19(1), art. no. 160. doi: 10.1186/s12909-019-1599-6.

Background: Social network analysis (SNA) might have an unexplored value in the study of interactions in technology-enhanced learning at large and in online (Problem Based Learning) PBL in particular. Using SNA to study students' positions in information exchange networks, communicational activities, and interactions, we can broaden our understanding of the process of PBL, evaluate the significance of each participant role and learn how interactions can affect academic performance. The aim of this study was to study how SNA visual and mathematical analysis can be sued to investigate online PBL, furthermore, to see if students' position and interaction parameters are associated with better performance. Methods: This study involved 135 students and 15 teachers in 15 PBL groups in the course of "growth and development" at Qassim University. The course uses blended PBL as the teaching method. All interaction data were extracted from the learning management system, analyzed with SNA visual and mathematical techniques on the individual student and group level, centrality measures were calculated, and participants' roles were mapped. Correlation among variables was performed using the non-parametric Spearman rank correlation test. Results: The course had 2620 online interactions, mostly from students to students (89%), students to teacher interactions were 4.9%, and teacher to student interactions were 6.15%. Results have shown that SNA visual analysis can precisely map each PBL group and the level of activity within the group as well as outline the interactions among group participants, identify the isolated and the active students (leaders and facilitators) and evaluate the role of the tutor. Statistical analysis has shown that students' level of activity (outdegree rs(133) = 0.27, p = 0.01), interaction with tutors (rs (133) = 0.22, p = 0.02) are positively correlated with academic performance. Conclusions: Social network analysis is a practical method that can reliably monitor the interactions in an online PBL environment. Using SNA could reveal important information about the course, the group, and individual students. The insights generated by SNA may be useful in the context of learning analytics to help monitor students' activity. © 2019 The Author(s).
blended learningblended problem-based learninglearning analyticsproblem-based learningsocial network analysis
Saqr M., Nouri J., Fors U. (2019). Time to focus on the temporal dimension of learning: A learning analytics study of the temporal patterns of students’ interactions and self-regulation. International Journal of Technology Enhanced Learning, vol. 11(4), pp. 398-412. doi: 10.1504/IJTEL.2019.102549.

In this learning analytics study, we attempt to understand the role of temporality measures for the prediction of academic performance. The study included four online courses over a full-year duration. Temporality was studied on daily, weekly, course-wise and year-wise. Visualising the activities has highlighted certain patterns. On the week level, early participation was a consistent predictor of high achievement. This finding was consistent from course to course and during most periods of the year. On course level, high achievers were also likely to participate early and consistently. With a focus on temporal measures, we were able to predict high achievers with reasonable accuracy in each course. These findings highlight the idea that temporality dimension is a significant source of information about learning patterns and has the potential to inform educators about students’ activities and to improve the accuracy and reproducibility of predicting students’ performance. Copyright © 2019 Inderscience Enterprises Ltd.
collaborative learninglearning analyticsproblem-based learningself-regulationsocial network analysistemporalitytime
Nouri J., Saqr M., Fors U. (2019). Predicting performance of students in a flipped classroom using machine learning: Towards automated data-driven formative feedback. Journal of Systemics, Cybernetics and Informatics, pp. 79-82. https://www.iiisci.org/journal/pdv/sci/pdfs/EB614LI19.pdf.

Learning analytics (LA) is a relatively new research discipline that uses data to try to improve learning, optimizing the learning process and develop the environment in which learning occurs. One of the objectives of LA is to monitor students' activities and early predict performance to improve retention, offer personalized feedback and facilitate the provision of support to the students. Flipped classroom is one of the pedagogical methods that find strength in the combination of physical and digital environments - i.e. blended learning environments. Flipped classroom often make use of learning management systems in which video-recorded lectures and digital material is made available, which thus generates data about students' interactions with these materials. In this paper, we report on a study conducted with focus on a flipped learning course in research methodology. Based on data regarding how students interact with course material (video recorded lectures and reading material), how they interact with teachers and other peers in discussion forums, and how they perform on a digital assessment (digital quiz), we apply machine learning methods (i.e. Neural Networks, Naïve Bayes, Random Forest, kNN, and Logistic regression) in order to predict students' overall performance on the course. The final predictive model that we present in this paper could with fairly high accuracy predict low- and high achievers in the course based on activity and early assessment data. Using this approach, we are given opportunities to develop learning management systems that provide automatic data-driven formative feedback that can help students to self-regulate as well as inform teachers where and how to intervene and scaffold students. Copyright © 2019 by the International Institute ofInformatics and Systemics.
assessmentfeedbackflipped classroomlearning analyticsmachine learning
Alsuhaibani M., Alharbi A., Inam S.N.B., Alamro A., Saqr M. (2019). Research education in an undergraduate curriculum: Students perspective. International Journal of Health Sciences, vol. 13(2), pp. 30-34. https://pubmed.ncbi.nlm.nih.gov/30983943.

Objective: This study aims to investigate the attitude and practice toward undergraduate research studies among medical students at Qassim University in Buraydah, Saudi Arabia. Methods: An online cross-sectional survey developed based on previous studies. It was announced to all registered medical students who have active college’s email (n = 448) at Qassim University in Buraydah, Saudi Arabia during the academic year of 2016. Results: The response rate was 56.6% (n = 252). Less than half of the students have started their research projects (41.6%). Students complained about the lack of free time and the unavailability of a university hospital: 92.4% and 97.1%, respectively. One-third of students participated in extra-curriculum research, and female students were more involved. Only 15.2% have published their research and 26.7% have presented it in conferences. Male students have more journal publication in compared to their female collages while the females have presented their projects more often in conferences. To improve their curriculum vitae, 95.2% stated they are going to participate in extra-curriculum research in the future. Conclusions: Students believe in the importance of research for improving their future work life. The main reason for not participating in research, beyond deficiency of research activities, is lack of free time. Students are unsatisfied with research skills gained through academic life, although their interest toward research increases and they plan on participating in future research.
research curriculumresearch educationstudentsundergraduate research
Nouri J., Larsson K., Saqr M. (2019). Bachelor thesis analytics: Using machine learning to predict dropout and identify performance factors. International Journal of Learning Analytics and Artificial Intelligence for Education, vol. 1(1), pp. 116-131. doi: 10.3991/ijai.v1i1.11065.

The bachelor thesis is commonly a necessary last step towards the first graduation in higher education and constitutes a central key to both further studies in higher education and employment that requires higher education degrees. Thus, completion of the thesis is a desirable outcome for individual students, academic institutions and society, and non-completion is a significant cost. Unfortunately, many academic institutions around the world experience that many thesis projects are not completed and that students struggle with the thesis process. This paper addresses this issue with the aim to, on the one hand, identify and explain why thesis projects are completed or not, and on the other hand, to predict non-completion and completion of thesis projects using machine learning algorithms. The sample for this study consisted of bachelor students’ thesis projects (n=2436) that have been started between 2010 and 2017. Data were extracted from two different data systems used to record data about thesis projects. From these systems, thesis project data were collected including variables related to both students and supervisors. Traditional statistical analysis (correlation tests, t-tests and factor analysis) was conducted in order to identify factors that influence non-completion and completion of thesis projects and several machine learning algorithms were applied in order to create a model that predicts completion and non-completion. When taking all the analysis mentioned above into account, it can be concluded with confidence that supervisors’ ability and experience play a significant role in determining the success of thesis projects, which, on the one hand, corroborates previous research. On the other hand, this study extends previous research by pointing out additional specific factors, such as the time supervisors take to complete thesis projects and the ratio of previously unfinished thesis projects. It can also be concluded that the academic title of the supervisor, which was one of the variables studied, did not constitute a factor for completing thesis projects. One of the more novel contributions of this study stems from the application of machine learning algorithms that were used in order to – reasonably accurately – predict thesis completion/non-completion. Such predictive models offer the opportunity to support a more optimal matching of students and supervisors.
thesisbachelorcompletionmachine learningretentionperformancelearning analyticspredictiondropout
Nouri J., Ebner M., Ifenthaler D., Saqr M., Malmberg J., Khalil M., Bruun J., Viberg O., Conde-González M.Á., Papamitsiou Z. (2019). Efforts in Europe for Data-Driven Improvement of Education–A review of learning analytics research in six countries. International Journal of Learning Analytics and Artificial Intelligence for Education, vol. 1(1), pp. 8-27. doi: 10.3991/ijai.v1i1.11053.

Information and communication technologies are increasingly mediating learning and teaching practices as well as how educational institutions are handling their administrative work. As such, students and teachers are leaving large amounts of digital footprints and traces in various educational apps and learning management platforms, and educational administrators register various processes and outcomes in digital administrative systems. It is against such a background we in recent years have seen the emergence of the fast-growing and multi-disciplinary field of learning analytics. In this paper, we examine the research efforts that have been conducted in the field of learning analytics in Austria, Denmark, Finland, Norway, Germany, Spain, and Sweden. More specifically, we report on developed national policies, infrastructures and competence centers, as well as major research projects and developed research strands within the selected countries. The main conclusions of this paper are that the work of researchers around Europe has not led to national adoption or European level strategies for learning analytics. Furthermore, most countries have not established national policies for learners’ data or guidelines that govern the ethical usage of data in research or education. We also conclude, that learning analytics research on pre-university level to high extent have been overlooked. In the same vein, learning analytics has not received enough focus form national and European national bodies. Such funding is necessary for taking steps towards data-driven development of education.
learning analyticseuropedata-driven improvementeducation

2018

Saqr M., Fors U., Nouri J. (2018). Using social network analysis to understand online problem-based learning and predict performance. PLoS ONE, vol. 13(9), art. no. e0203590. doi: 10.1371/journal.pone.0203590.

Social network analysis (SNA) may be of significant value in studying online collaborative learning. SNA can enhance our understanding of the collaborative process, predict the under-achievers by means of learning analytics, and uncover the role dynamics of learners and teachers alike. As such, it constitutes an obvious opportunity to improve learning, inform teachers and stakeholders. Besides, it can facilitate data-driven support services for students. This study included four courses at Qassim University. Online interaction data were collected and processed following a standard data mining technique. The SNA parameters relevant to knowledge sharing and construction were calculated on the individual and the group level. The analysis included quantitative network analysis and visualization, correlation tests as well as predictive and explanatory regression models. Our results showed a consistent moderate to strong positive correlation between performance, interaction parameters and students’ centrality measures across all the studied courses, regardless of the subject matter. In each of the studied courses, students with stronger ties to prominent peers (better social capital) in small interactive and cohesive groups tended to do better. The results of correlation tests were confirmed using regression tests, which were validated using a next year dataset. Using SNA indicators, we were able to classify students according to achievement with high accuracy (93.3%). This demonstrates the possibility of using interaction data to predict underachievers with reasonable reliability, which is an obvious opportunity for intervention and support. © 2018 Saqr et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Saqr M., Fors U., Tedre M., Nouri J. (2018). How social network analysis can be used to monitor online collaborative learning and guide an informed intervention. PLoS ONE, vol. 13(3), art. no. e0194777. doi: 10.1371/journal.pone.0194777.

To ensure online collaborative learning meets the intended pedagogical goals (is actually collaborative and stimulates learning), mechanisms are needed for monitoring the efficiency of online collaboration. Various studies have indicated that social network analysis can be particularly effective in studying students’ interactions in online collaboration. However, research in education has only focused on the theoretical potential of using SNA, not on the actual benefits they achieved. This study investigated how social network analysis can be used to monitor online collaborative learning, find aspects in need of improvement, guide an informed intervention, and assess the efficacy of intervention using an experimental, observational repeated-measurement design in three courses over a full-term duration. Using a combination of SNA-based visual and quantitative analysis, we monitored three SNA constructs for each participant: the level of interactivity, the role, and position in information exchange, and the role played by each participant in the collaboration. On the group level, we monitored interactivity and group cohesion indicators. Our monitoring uncovered a non-collaborative teacher-centered pattern of interactions in the three studied courses as well as very few interactions among students, limited information exchange or negotiation, and very limited student networks dominated by the teacher. An intervention based on SNA-generated insights was designed. The intervention was structured into five actions: increasing awareness, promoting collaboration, improving the content, preparing teachers, and finally practicing with feedback. Evaluation of the intervention revealed that it has significantly enhanced student-student interactions and teacher-student interactions, as well as produced a collaborative pattern of interactions among most students and teachers. Since efficient and communicative activities are essential prerequisites for successful content discussion and for realizing the goals of collaboration, we suggest that our SNA-based approach will positively affect teaching and learning in many educational domains. Our study offers a proof-of-concept of what SNA can add to the current tools for monitoring and supporting teaching and learning in higher education. © 2018 Saqr et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Saqr M., Fors U., Tedre M. (2018). How the study of online collaborative learning can guide teachers and predict students' performance in a medical course. BMC Medical Education, vol. 18(1), art. no. 24. doi: 10.1186/s12909-018-1126-1.

Background: Collaborative learning facilitates reflection, diversifies understanding and stimulates skills of critical and higher-order thinking. Although the benefits of collaborative learning have long been recognized, it is still rarely studied by social network analysis (SNA) in medical education, and the relationship of parameters that can be obtained via SNA with students' performance remains largely unknown. The aim of this work was to assess the potential of SNA for studying online collaborative clinical case discussions in a medical course and to find out which activities correlate with better performance and help predict final grade or explain variance in performance. Methods: Interaction data were extracted from the learning management system (LMS) forum module of the Surgery course in Qassim University, College of Medicine. The data were analyzed using social network analysis. The analysis included visual as well as a statistical analysis. Correlation with students' performance was calculated, and automatic linear regression was used to predict students' performance. Results: By using social network analysis, we were able to analyze a large number of interactions in online collaborative discussions and gain an overall insight of the course social structure, track the knowledge flow and the interaction patterns, as well as identify the active participants and the prominent discussion moderators. When augmented with calculated network parameters, SNA offered an accurate view of the course network, each user's position, and level of connectedness. Results from correlation coefficients, linear regression, and logistic regression indicated that a student's position and role in information relay in online case discussions, combined with the strength of that student's network (social capital), can be used as predictors of performance in relevant settings. Conclusion: By using social network analysis, researchers can analyze the social structure of an online course and reveal important information about students' and teachers' interactions that can be valuable in guiding teachers, improve students' engagement, and contribute to learning analytics insights. © 2018 The Author(s).
blended learningcase discussionsclinicalcollaborative learningcomputer-supported collaborative learninge-learninglearning analyticssocial network analysis
Saqr M. (2018). A literature review of empirical research on learning analytics in medical education. International Journal of Health Sciences, vol. 12(2), pp. 80-85. https://pubmed.ncbi.nlm.nih.gov/29599699.

The number of publications in the field of medical education is still markedly low, despite recognition of the value of the discipline in the medical education literature, and exponential growth of publications in other fields. This necessitates raising awareness of the research methods and potential benefits of learning analytics (LA). The aim of this paper was to offer a methodological systemic review of empirical LA research in the field of medical education and a general overview of the common methods used in the field in general. Search was done in Medline database using the term "LA." Inclusion criteria included empirical original research articles investigating LA using qualitative, quantitative, or mixed methodologies. Articles were also required to be written in English, published in a scholarly peer-reviewed journal and have a dedicated section for methods and results. A Medline search resulted in only six articles fulfilling the inclusion criteria for this review. Most of the studies collected data about learners from learning management systems or online learning resources. Analysis used mostly quantitative methods including descriptive statistics, correlation tests, and regression models in two studies. Patterns of online behavior and usage of the digital resources as well as predicting achievement was the outcome most studies investigated. Research about LA in the field of medical education is still in infancy, with more questions than answers. The early studies are encouraging and showed that patterns of online learning can be easily revealed as well as predicting students' performance.
learning analyticsmedical educationreview

2017

Saqr M., Fors U., Tedre M. (2017). How learning analytics can early predict under-achieving students in a blended medical education course. Medical Teacher, vol. 39(7), pp. 757-767. doi: 10.1080/0142159X.2017.1309376.

Aim: Learning analytics (LA) is an emerging discipline that aims at analyzing students’ online data in order to improve the learning process and optimize learning environments. It has yet un-explored potential in the field of medical education, which can be particularly helpful in the early prediction and identification of under-achieving students. The aim of this study was to identify quantitative markers collected from students’ online activities that may correlate with students’ final performance and to investigate the possibility of predicting the potential risk of a student failing or dropping out of a course. Methods: This study included 133 students enrolled in a blended medical course where they were free to use the learning management system at their will. We extracted their online activity data using database queries and Moodle plugins. Data included logins, views, forums, time, formative assessment, and communications at different points of time. Five engagement indicators were also calculated which would reflect self-regulation and engagement. Students who scored below 5% over the passing mark were considered to be potentially at risk of under-achieving. Results: At the end of the course, we were able to predict the final grade with 63.5% accuracy, and identify 53.9% of at-risk students. Using a binary logistic model improved prediction to 80.8%. Using data recorded until the mid-course, prediction accuracy was 42.3%. The most important predictors were factors reflecting engagement of the students and the consistency of using the online resources. Conclusions: The analysis of students’ online activities in a blended medical education course by means of LA techniques can help early predict underachieving students, and can be used as an early warning sign for timely intervention. © 2017 Informa UK Limited, trading as Taylor & Francis Group.

Before 2017

Amin O., Saqr M. (2011). Blended learning in orthopedics course: an evaluation study. Int J Health Sci.

The results from our survey indicate that the students’ performance - although not reported due to problem of multiplicity of variables - and the tutor’s experience markedly enhanced due to E-learning experience. E-learning is helpful to students in terms of better communication between peers and tutors, easier access to information and better chance for self-assessment throughout the course. Findings from our study are consistent with similar studies in other settings. It should be emphasized that the nature of course, the available resources, and the technical skills of tutors and students (in terms of use of computer and internet) play a significant role in the process. We conclude that implementation of blended learning in orthopedics helped students get easy access to information, better interact with tutors and improve their understanding of the subject.
blended-learningorthopedicsmedical education
Saqr M. (2008). Shall Migraine be Considered a Simple Benign Headache Disorder?. Int J Health Sci. https://pubmed.ncbi.nlm.nih.gov/21475481/.

Migraine is a primary headache disorder which has received little attention from health care policies and physicians. This has led to ineffective management and more suffering to the patients and society. Migraine per se is a disabling disease which has its impact on the patient, family and work. It is associated with high incidence of psychiatric co-morbidities, especially depression and anxiety as well as other mental disorders. Depression affects around 80% of chronic migraineurs, an association that adds to the suffering. It has been confirmed as risk factors for developing radiographic and clinically evident ischemic cerebrovascular infarctions. Lately, it was associated with angina, myocardial infarction and intracerebral hemorrhage. Migraine plays a central role in the pathogenesis of these diseases, not just a simple association. These comorbidities and the disabilities migraine makes should change our views of migraine as a simple headache disorder, and directs our efforts to a better recognition and an effective management for the prevention of the disease associated morbidity.
migraineheadacheconsequences
Talaat F.M., Alboraey M.F., Hamdy M.M., Saqr M. (2004). Transcranial Doppler study in patients with migraine. The Egyptian Journal of Neurology Psychiatry and Neurosurgery.

Transcranial Doppler (TCD) is an ultrasound technology that measures physiological parameters of blood flow in the major intracranial arteries. It is used in assessment of the haemodynamics of the cerebral circulation in patients with different neurological disorders, its use in migraine is still limited. This study was conducted on 90 persons divided into 2 groups, the 1st one was 49 migrainous patients and the 2nd one was 41 healthy persons matching group 1 in age and sex. The 2 groups were subjected to transcranial insonation of the middle cerebral arteries bilaterally before and after adequate hyperventilation for 5 minutes. The baseline flow velocities in the middle cerebral arteries did not differ significantly between the 2 groups. However, the cerebrovascular reactivity showed statistically significant difference between the 2 groups. So, the cerebrovascular reactivity may be used as an additional tool for the diagnosis of migraine. In addition, it may characterize some migrainous persons that may help in the management. (Egypt J. Neurol. Psychiat. Neurosurg., 2004, 41(2): 479-487).
migrainetranscranial doppler headache

Conference Papers

2024

Huhta K., López-Pernas S., Saqr M. (2024). Mapping the topics, trends, and themes of education technology in legal education with topic modeling and network analysis. Proceedings TEEM 2023: Eleventh International Conference on Technological Ecosystems for Enhancing Multiculturality. TEEM 2023. Lecture Notes in Education Technology, pp. 894-903. doi: 10.1007/978-981-97-1814-6_87.

The digital transformation varies markedly across disciplines in the way technologies are used, how much, and for what purposes to foster educational innovation. In legal education, they are used for various purposes to respond to the capacities and competencies that are required from contemporary lawyers and legal professionals. This article addresses a gap in existing research by approximating digital technologies and digital transformation in legal education research. It uses topic modeling and network analysis to explore the digital transformation in legal education research and to demonstrate how digital technologies used for pedagogical purposes are reflected in legal education research. It finds that while digitalization is a clear recent trend in legal education research, the role of digital technologies in legal education research is not as strong as in other fields of higher education and that practical skills and the practice of law continue to have a central role in legal education irrespective of the education technologies used.
legal educationeducation technologybibliometricstopic modeling
López-Pernas S., Gordillo A., Barra E., Saqr M. (2024). The dynamics of students’ playing profiles in a programming educational escape room (Best paper award). Proceedings TEEM 2023: Eleventh International Conference on Technological Ecosystems for Enhancing Multiculturality. TEEM 2023. Lecture Notes in Education Technology, pp. 21-31. doi: 10.1007/978-981-97-1814-6_2.

Educational escape rooms have gained recognition as immersive and engaging learning activities. While existing research has primarily focused on students' perceptions and learning outcomes, little attention has been given to their performance, behavior, and interactions during these activities. This study aims to fill this gap by employing person-centered methods to analyze students' gameplay data from a computer-based educational escape room. Using Gaussian mixture models, we identified four distinct profiles of players using their gameplay data: efficient players who complete the game smoothly with little help, supported players who make good progress with the help of hints, relentless players who devote their time to seeking help rather than working on the escape room puzzles, and laggers who make little progress and fail to obtain the help they need. We further investigate the relationships between puzzle completion times and hint-requesting behavior using Bayesian Gaussian graphical models. The findings point to hints as the key to support profiles of players that lack the necessary skills to complete the activity on their own. Lastly, we analyze the relationship between the identified profiles and knowledge acquisition during the escape room. We found that students’ initial and final knowledge differed by profile but learning gains were comparable except for the laggers who make little progress in the activity.
educational escape roomslearning analyticsgaussian mixture modelsgame-based learning
Saqr M., López-Pernas S. (2024). Why Learning and Teaching Learning Analytics is Hard: An Experience from a Real-Life LA Course Using LA Methods. Proceedings TEEM 2023: Eleventh International Conference on Technological Ecosystems for Enhancing Multiculturality. TEEM 2023. Lecture Notes in Education Technology, pp. 781-789. doi: 10.1007/978-981-97-1814-6_76.

Learning analytics emerged more than a decade ago to harness the power of data to understand and optimize learning, learners’ behavior, and learning environments. Ever since, the field has grown to encompass a diverse range of methods, research strands and traditions. Recent literature reviews tell us that most common applications of learning analytics include predictive analytics, social network analysis, sequence and process analysis, visualizations, and dashboards to mention a few. In the same vein, the research field has attracted several interdisciplinary researchers and practitioners from computer science, education, data science, engineering, administration, and from the education technology industry. Whereas such diverse backgrounds and perspectives bring a wealth of different perspectives to the field, it makes teaching and learning analytics hard to narrow down in a single course. This study reports on the analysis of students' approach to learning learning analytics, reflects on the insights that learning analytics offers, and makes recommendations for future researchers who are teaching or investigating similar courses.
learning analyticssequence analysisdata miningpsychological networkscomputer science education
Akçapınar G., López-Pernas S., Er E., Saqr M. (2024). How a learning analytics dashboard intervention influences the dynamics of students’ learning behavior (Best paper award)More data is not always better data: A learning analytics case study in early prediction. Proceedings TEEM 2023: Eleventh International Conference on Technological Ecosystems for Enhancing Multiculturality. TEEM 2023. Lecture Notes in Education Technology, pp. 810-819. doi: 10.1007/978-981-97-1814-6_79.

Interventions play a crucial role in completing the learning analytics cycle. However, there is limited research available on how students utilize these interventions or whether there is any change in their learning behaviors following the intervention. Existing studies primarily rely on students' self-report perceptions, while neglecting the temporal aspect of the data in data-driven studies. This study examines the impact of a learning analytics intervention in the form of a learning analytics dashboard provided to students in a remote programming course on their learning behaviors. To achieve this goal, learning sessions before and after the introduction of the dashboard were identified using students' learning traces in the learning management system. Subsequently, these learning sessions were analyzed using sequence analysis, process mining, and Bayesian Gaussian graphical models. Assignment submissions, formative quizzes, forum interactions, interactions with video materials, and participation in live classes were considered to determine students' learning behaviors. The findings of the study indicate that there were changes in students' learning behaviors after the introduction of the dashboard. Specifically, before the dashboard, learning sessions were mainly focused on graded activities such as assignments and quizzes, whereas after the dashboard, there was an increase in interactions with non-graded activities such as video materials. The results are also supported by the process mining analysis.
learning analyticsdashboard interventionsequence analysisprocess miningbayesian gaussian graphical models
Rai P., López-Pernas S., Saqr M. (2024). More data is not always better data: A learning analytics case study in early prediction. Proceedings TEEM 2023: Eleventh International Conference on Technological Ecosystems for Enhancing Multiculturality. TEEM 2023. Lecture Notes in Education Technology, pp. 830-838. doi: 10.1007/978-981-97-1814-6_81.

This study aimed to explore the early prediction and forecasting of students' performance using learning analytics methods. We do so by examining a large number of students at four different time points and employing four machine learning algorithms. We seek to fill a gap in the literature regarding the effect of data volume across time. The results revealed several important findings. To begin with, the final course data did not consistently outperform earlier time points across all performance indicators. Surprisingly, forecasts based on data from the first week had reasonable accuracy, implying that preemptive interventions can be implemented early on. Furthermore, predictions made on second-week data performed the best, probably due to students' initial motivation and early differentiation among those who were actively involved. Furthermore, decision trees (DT) emerged as the most effective early prediction method, consistently displaying acceptable performance across all time points. This study has far-reaching ramifications. It indicates that employing learning analytics for early prediction is not only viable, but also dependable, with decent accuracy. Although further research is needed to corroborate these findings in diverse circumstances, the second week of the course looks to be a vital stage for generating correct predictions. Furthermore, when compared to other algorithms, DT stands out as a superior early prediction algorithm.
learning analyticsearly predictionspredicting performancedata mining
Elmoazen R., Saqr M., Tedre M., Hirsto L. (2024). Capturing the Sequential Pattern of Students’ Interactions in Computer-Supported Collaborative Learning. Proceedings TEEM 2023: Eleventh International Conference on Technological Ecosystems for Enhancing Multiculturality. TEEM 2023. Lecture Notes in Education Technology, pp. 800-809. doi: 10.1007/978-981-97-1814-6_78.

Interventions play a crucial role in completing the learning analytics cycle. However, there is limited research available on how students utilize these interventions or whether there is any change in their learning behaviors following the intervention. Existing studies primarily rely on students' self-report perceptions, while neglecting the temporal aspect of the data in data-driven studies. This study examines the impact of a learning analytics intervention in the form of a learning analytics dashboard provided to students in a remote programming course on their learning behaviors. To achieve this goal, learning sessions before and after the introduction of the dashboard were identified using students' learning traces in the learning management system. Subsequently, these learning sessions were analyzed using sequence analysis, process mining, and Bayesian Gaussian graphical models. Assignment submissions, formative quizzes, forum interactions, interactions with video materials, and participation in live classes were considered to determine students' learning behaviors. The findings of the study indicate that there were changes in students' learning behaviors after the introduction of the dashboard. Specifically, before the dashboard, learning sessions were mainly focused on graded activities such as assignments and quizzes, whereas after the dashboard, there was an increase in interactions with non-graded activities such as video materials. The results are also supported by the process mining analysis.
learning analyticsdashboard interventionsequence analysisprocess miningbayesian gaussian graphical models
Kaliisa R., Misiejuk K., López-Pernas S., Khalil M., Saqr M. (2024). Have Learning Analytics Dashboards Lived Up to the Hype? A Systematic Review of Impact on Students’ Achievement, Motivation, Participation and Attitude.. The 14th Learning Analytics and Knowledge Conference (LAK ’24) (in-press).
Conde J, López-Pernas S., Barra E., Saqr M. (2024). The Temporal Dynamics of Procrastination and its Impact on Aca- demic Performance: The Case of a Task-oriented Programming Course:. SAC '24: Proceedings of the 39th ACM/SIGAPP Symposium on Applied Computing, pp. 48-55. doi: 10.1145/3605098.3636072.
Kaliisa R., Misiejuk K., López-Pernas S., Khalil M., Saqr M. (2024). Have Learning Analytics Dashboards Lived Up to the Hype? A Systematic Review of Impact on Students' Achievement, Motivation, Participation and Attitude. Proceedings of the Fourteenth International Conference on Learning Analytics & Knowledge (LAK'24). doi: 10.1145/3636555.3636884.

While learning analytics dashboards (LADs) are the most common form of LA intervention, there is limited evidence regarding their impact on students learning outcomes. This systematic review synthesizes the findings of 38 research studies to investigate the impact of LADs on students' learning outcomes, encompassing achievement, participation, motivation, and attitudes. As we currently stand, there is no evidence to support the conclusion that LADs have lived up to the promise of improving academic achievement. Most studies reported negligible or small effects, with limited evidence from well-powered controlled experiments. Many studies merely compared users and non-users of LADs, confounding the dashboard effect with student engagement levels. Similarly, the impact of LADs on motivation and attitudes appeared modest, with only a few exceptions demonstrating significant effects. Small sample sizes in these studies highlight the need for larger-scale investigations to validate these findings. Notably, LADs showed a relatively substantial impact on student participation. Several studies reported medium to large effect sizes, suggesting that LADs can promote engagement and interaction in online learning environments. However, methodological shortcomings, such as reliance on traditional evaluation methods, self-selection bias, the assumption that access equates to usage, and a lack of standardized assessment tools, emerged as recurring issues. To advance the research line for LADs, researchers should use rigorous assessment methods and establish clear standards for evaluating learning constructs. Such efforts will advance our understanding of the potential of LADs to enhance learning outcomes and provide valuable insights for educators and researchers alike.
learning analytics dashboards (lads)systematic reviewimpactlearning outcomes
Ito H., López-Pernas S., Saqr M. (2024). A Review of Idiographic Research in Education: Too Little, But Not Too Late. Proceedings - IEEE 24st International Conference on Advanced Learning Technologies, ICALT 2024, pp. 10-12. doi: 10.1109/ICALT61570.2024.00010.

It stands to reason that if we want to offer “personalized” education, our methods should be designed to capture the person and the intraindividual processes. However, an idiographic approach that investigates within-person processes and provides insights on the person has been so far lagging. We conducted a scoping review to explore how the idiographic approach has been applied in educational research (i.e., what methods, topics, data, and statistical approaches). We found that person-specific analysis has mostly been used to investigate education psychology constructs. In addition, many idiographic studies have employed basic statistical techniques, whereas advanced statistical methods have been applied only recently. Therefore, considering the recent development of educational data science, the potential of idiographic methodology needs to be explored further.
idiographicperson-specificwithin-personlearning analyticsscoping review
Saqr M., López-Pernas S. (2024). Momentary emotions emerge and evolve differently across students and time, yet are surprisingly stable. Proceedings - IEEE 24st International Conference on Advanced Learning Technologies, ICALT 2024, pp. 86-90. doi: 10.1109/ICALT61570.2024.00031.

Research on academic emotions has explored different granularities that range from a full program to a single task. Yet, most of the existing research stems from cross-sectional studies. While immensely useful, lacking a temporal depth obfuscates the process of emotions into a flat process. To fill this gap, this study takes a process-oriented approach to study the momentary changes in academic emotions as they unfold in time into phases, changes, and successions of sequences during two lectures. We use intensive longitudinal data in the form of ecological momentary surveys. We rely on mixture models to cluster the data into states, use sequence analysis to map the longitudinal unfolding and mixture hidden Markov models to answer why certain longitudinal patterns emerge. Our findings point to differences among students in their reactions to contextual variables, yet, such reactions are relatively stable within the short time window that we studied.
learning analyticsacademic emotionsaffectgaussian mixture modelssequence analysismixture hidden markov modelspanavavasstra
Saqr M., López-Pernas S. (2024). The Features Learning Analytics Students Want the Most: Help Them Learn Over All Else. Proceedings of the 14th International Conference on eLearning (eLearning-2023) (CEUR Workshop Proceedings) (in-press).

To be effective, support based on learning analytics (LA) necessitates that students’ attitudes, needs, and expectations are taken into account. Recently, research exploring students’ needs and expectations has attracted the attention of LA researchers and practitioners driven by increasing focus on personalized learning and focus on the delivery of effective LA insights. Yet, most of such research comes from students who have a faint idea of LA, who do not firmly understand the potentials and the possible drawbacks inherent in LA. This current study aimed to fill this gap by surveying well-informed students —who completed an advanced course on LA— about the features they need from LA themselves. We also complemented our analysis with a network approach to understand the association and interplay between different needs. Our findings have shown that most of the students want LA features that help them perform their academic tasks: recommendations, feedback and reminders of deadlines. Students were most skeptical about comparing them with other students and suggesting other students as partners in academic work. The network analysis has confirmed such features and pointed out that resources and recommendations are the most central features that make students interested in LA. In a nutshell, students want LA to help them learn and support their learning journey over all else.
learning analyticsexpectationssurveystudents
Kaliisa R., López-Pernas S., Misiejuk K., Saqr M., Damsa C. (2024). Exploring the Dynamics and Trends of Knowledge Exchange: A Structured Topic Modeling Approach of the CSCL Conference Proceedings. Proceedings of the International Society of the Learning Sciences (in-press).

This paper presents the topical trends of Computer-Supported Collaborative Learning (CSCL) through a structured topic modelling of CSCL conference proceedings. The study highlights the multidisciplinary nature of CSCL, revealing theoretical, methodological, and epistemological diversity. Noteworthy findings include a decline in interest in scripting and concept maps, reflecting an evolving emphasis on learner autonomy and the study of collaboration based on various artifacts. The impact of technological advances, particularly the focus on multimodal collaboration analytics, indicates a dynamic interplay between technology and CSCL discourse. As the field stands on the precipice of the artificial intelligence (AI) era, there is anticipation that AI will significantly influence CSCL methodologies, offering opportunities for enhanced collaboration analytics and adaptive learning environments.
López-Pernas S., Gordillo A., Barra E., Saqr M. (2024). Tracking students' progress in educational escape rooms through a sequence analysis inspired dashboard. Inclusive and equitable quality education for all. EC-TEL 2024. Lecture Notes in Computer Science, art. no. 15160, pp. 119-124. doi: 10.1007/978-3-031-72312-4_15.

Learning analytics dashboards are the main vehicle for providing educators with a visual representation of data and insights related to teach-ing and learning. Recent research has found that the data visualizations pro-vided by dashboards are often very basic and do not take advantage of the latest research advances to analyze and depict the learning process. In this article, we present a success story of how we adapted a visualization used for research purposes for its integration in a dashboard for its use by teach-ers in daily practice. Specifically, we described the process of transforming and integrating a static sequence analysis visualization into an interactive web visualization in a learning analytics dashboard for monitoring stu-dents’ temporal trajectories in educational escape rooms in real time. We in-terviewed teachers to find out how they made use of the dashboard and pre-sent a qualitative content analysis of their responses.
learning analyticssequence analysisgame-based learningeducational escape roomsdashboards
Saqr M., López-Pernas S. (2024). The Features Learning Analytics Students Want the Most: Help Them Learn Over All Else. Proceedings of the 14th International Conference on eLearning (eLearning-2023) (CEUR Workshop Proceedings), vol. 3696, pp. 15-23. https://ceur-ws.org/Vol-3696/article_2.pdf.

To be effective, support based on learning analytics (LA) necessitates that students’ attitudes, needs, and expectations are taken into account. Recently, research exploring students’ needs and expectations has attracted the attention of LA researchers and practitioners driven by increasing focus on personalized learning and focus on the delivery of effective LA insights. Yet, most of such research comes from students who have a faint idea of LA, who do not firmly understand the potentials and the possible drawbacks inherent in LA. This current study aimed to fill this gap by surveying well-informed students —who completed an advanced course on LA— about the features they need from LA themselves. We also complemented our analysis with a network approach to understand the association and interplay between different needs. Our findings have shown that most of the students want LA features that help them perform their academic tasks: recommendations, feedback and reminders of deadlines. Students were most skeptical about comparing them with other students and suggesting other students as partners in academic work. The network analysis has confirmed such features and pointed out that resources and recommendations are the most central features that make students interested in LA. In a nutshell, students want LA to help them learn and support their learning journey over all else.
learning analyticsexpectationssurveystudent
Misiejuk K., López-Pernas S., Kaliisa R., Saqr M. (2024). Learning together: Modeling the process of student-AI interactions when generating learning resources. Proceedings TEEM 2025: Twelveth International Conference on Technological Ecosystems for Enhancing Multiculturality. TEEM 2025. Lecture Notes in Computer Science..

Introducing Large Language Models (LLMs) into the classroom in an effective way implies that students must acquire the necessary AI literacy skills, such as prompt engineering. This proves challenging, given that the knowledge base around effective prompting is still developing. This paper reports on a case study in which students make use of an LLM to generate personalized learning materials in a social network analysis course. We analyze students’ qualitatively coded interactions with the LLM and their evolution throughout the course using Epistemic Network Analysis (ENA). Our findings show that students evolved their prompting strategies throughout the course, providing more contextual information in their initial prompts by the final assignment. Specifically, high achievers were more likely to add contextual information in initial prompts and focused on refining the output rather than engaging in conversational exchanges with the LLM. They were also more likely to use polite language in their interactions. Our results highlight the need for further research and training in effective prompting to take full advantage of the potential of LLMs in education and to improve student-AI collaboration in academic settings.
synthetic datasocial network analysisprompt engineeringepistemic network analysislarge language models
López-Pernas S., Conde M.Á., Raspopović Milić M., Saqr M. (2024). Frequencies and averages miss the point of SRL evolution: A complex dynamic systems approach. Proceedings TEEM 2025: Twelveth International Conference on Technological Ecosystems for Enhancing Multiculturality. TEEM 2025. Lecture Notes in Computer Science..

Self-regulated learning (SRL) is a complex phenomenon with many interacting components and phases that evolve over time. As such, it requires an analytical lens that takes into account those characteristics. In this study, we investigate how the dynamics between the different SRL components evolve throughout four blended-learning courses. We use psychological networks —a computational method commonly used to model complex systems— to map the interaction between SRL components at the beginning and at the end of the courses. Our results show significant differences in students’ SRL processes at the two time points. Our findings have implications that can inform interventions that target students’ SRL.
learning analyticsself-regulated learningcomplex dynamic systemspsychological networks
Heikkinen S., Saqr M., Malmberg J., Tedre M. (2024). The interplay of engagement and learning regulation in online learning. Proceedings TEEM 2025: Twelveth International Conference on Technological Ecosystems for Enhancing Multiculturality. TEEM 2025. Lecture Notes in Computer Science..

This study is of significant importance as it examines the interplay between student engagement and self-regulated learning (SRL) in online learning environments. We analyse longitudinal data comprising student survey responses and log files from a learning management system (LMS). The participants were 165 students enrolled in a Finnish higher education institution course. Data collection included trace data from the LMS, detailing each student's actions with timestamps, and survey data collected at multiple points throughout the course. Students were classified based on their engagement patterns and self-regulation abilities. We utilised clustering methods to group students into engagement and SRL clusters. The findings, which are of great relevance, indicate a positive association between student engagement and SRL, suggesting that enhancing SRL skills can lead to improved student engagement and academic performance. The study, therefore, underscores the need to develop effective learning strategies and interventions that support students' self-regulation and engagement in online courses.
learning analyticsprocess miningengagementself-regulated learning
Deriba F., Saqr M., Tukiainen M. (2024). Exploring the Role of AI Tools in Enhancing Accessibility and Design Efficiency in User Interface Design: A Qualitative Study of Student Experiences. Proceedings TEEM 2025: Twelveth International Conference on Technological Ecosystems for Enhancing Multiculturality. TEEM 2025. Lecture Notes in Computer Science..

As the importance of accessibility in design grows, integrating AI tools into HumanComputer Interaction (HCI) education holds promise for equipping students with the skills to create inclusive user interfaces (UI). This study investigated the effectiveness of AI tool assistants in supporting student learning within an HCI course. Forty-two students participated in a design project divided into nine teams. The study employed a pre-design and post-design of the class project. During the pre-design phase, students worked without AI tools. Subsequently, students were introduced to the AI design assistants and used them to improve their UI designs based on the tools' suggestions. The thematic analysis explored the student perceptions of using AI tools in their HCI course project. Our findings revealed that students valued the AI tools as reminders and suggestion providers for their UI design; as a source of quick and clear information, emphasizing how AI assisted them in creating accessible UI. However, the study also identified challenges, including student concerns about the accuracy of AI responses and variability in AI tool suggestions; students' knowledge gap regarding AI tools, and difficulty in selecting implementable design suggestions from AI. Our study highlights the potential of AI tools to enhance the accessibility of UI design and promote a more user-centered design while suggesting the future research direction on the use of AI tools in software design and development.
accessibility in designai toolsuniversal designuser-centered designlearning analytics
Pope N., Kahila J., Vartiainen H., Saqr M., López-Pernas S., Roos T., Laru J., Tedre M. (2024). An XAI Social Media Platform for Teaching K-12 Students AI-Driven Profiling, Clustering, and Engagement-Based Recommending. Proceedings of EAAI: The Symposium on Educational Advances in Artificial Intelligence (in-press).

2023

Vartiainen H., López-Pernas S., Saqr M., Kahila J., Parkki T., Tedre M., Valtonen T. (2023). Mapping students’ temporal pathways in a computational thinking escape room. Proceedings of the 1st Finnish Learning Analytics and Artificial Intelligence in Education Conference (FLAIEC 2022) (CEUR Workshop Proceedings), vol. 3383, pp. 77-88. https://ceur-ws.org/Vol-3383/FLAIEC22_paper_9625.pdf.

This case study explored the applicability of sequence mining and process mining methods on qualitative video data of a group-based problem-solving situation. For the case study, audio and video data were collected from a pilot experience of an educational escape room, which was designed to practice the application of computational thinking (CT) skills. The escape room combined digital and physical affordances into CT puzzles and challenges. To examine processes and patterns of collaborative learning and problem-solving in the context of the CT escape room, video data from pre-service teachers’ game activities were collected. A unique contribution of this case study is that it demonstrates how sequence and process mining methods can be applied to a type of qualitative content analysis often found in research on collaborative learning.
computer science educationeducational escape roomsteacher educationcollaborative learning
Elmoazen R., Saqr M., Tedre M., Hirsto L. (2023). How social interactions kindle productive online problem-based learning: An exploratory study of the temporal dynamics. Proceedings of the 1st Finnish Learning Analytics and Artificial Intelligence in Education Conference (FLAIEC 2022) (CEUR Workshop Proceedings), vol. 3383, pp. 68-76. https://ceur-ws.org/Vol-3383/FLAIEC22_paper_2919.pdf.

Online computer-supported collaborative learning (CSCL) has risen in popularity in knowledge sharing and problem-solving. This research explored students' online activity in online problem-based learning (PBL) using process and sequence mining approaches. Process mining modeled students’ time-stamped activities and links between them. Sequence mining provided an overview of the flow and frequencies of students’ activities through sequential process maps. Our finding showed that the most frequent students’ activities were nonargument discussions followed by sharing knowledge and social interactions. The process model of the students’ discussion started with sharing knowledge most of the time and then students either evaluate or argue others’ messages to end discussions through social interactions. The sequence mining model showed that social interaction and non-argument discussion are the most common starting activities by students. It is concluded that process and sequence mining allowed us to identify different stages of online forum discussion in PBL .
learning analyticsprocess miningsequence miningcomputer-supportive collaborative learningcsclonline discussionproblem-based learningpbl
Hirsto L., Saqr M., López-Pernas S., Valtonen T. (2023). A systematic narrative review of learning analytics research in K-12 and schools. Proceedings of the 1st Finnish Learning Analytics and Artificial Intelligence in Education Conference (FLAIEC 2022) (CEUR Workshop Proceedings), vol. 3383, pp. 60-67. https://ceur-ws.org/Vol-3383/FLAIEC22_paper_9536.pdf.

The field of learning analytics emerged in the last decade to take advantage of the increasing availability of data about learners that digital systems generate. Existing research in learning analytics has focused on higher education, as this context often relies heavily on digital platforms such as online learning management systems, making data collection easier. In this paper, we focus on LA research in the context of elementary level teaching. We provide a systematic narrative review in which we analyze the articles that had the most impact in the field. Our results show the existence of some recurring themes such as gamification and multimodal methods. We make a distinction between papers in which learning analytics is the target of the study (e.g., dashboards) and papers in which learning analytics methods were used as a means to study a given behavior/skill/phenomenon (e.g., problem-solving skills). Lastly, we found that most studies lack a strong theoretical foundation on education science and, thus, there is a need to develop more elaborated theoretical perspectives in future research on school-level learning analytics, as well as papers that deliver a real impact on learning and teaching.
learning analyticsk-12elementary schoolliterature revieweducational data mining
Heikkinen S., López-Pernas S., Malmberg J., Tedre M., Saqr M. (2023). How do business students self-regulate their project management learning? A sequence mining study. Proceedings of the 1st Finnish Learning Analytics and Artificial Intelligence in Education Conference (FLAIEC 2022) (CEUR Workshop Proceedings), vol. 3383, pp. 51-59. https://ceur-ws.org/Vol-3383/FLAIEC22_paper_2583.pdf.

The relation between learning strategies and academic achievement has been proven to be strong in multiple studies. Still, the connection between micro-level SRL processes and the academic achievement of business students in learning project management remains unstudied. The current study aims to find how sequence mining can identify students using different learning tactics and strategies in terms of micro-level SRL processes. Our findings show that there are differences in the use of tactics and strategies between low and high performing students. Understanding the differences in how low and high performing students apply different micro-level SRL processes can help practitioners identify students in need of support for SRL.
sequence miningmicro-level srl processeslearning tacticsacademic achievementlearning analyticsproject management
Nissinen M., Silvennoinen E., Saqr M. (2023). How assessment analytics can help to improve reliability, efficiency, and fairness of entrance examinations. Proceedings of the 1st Finnish Learning Analytics and Artificial Intelligence in Education Conference (FLAIEC 2022) (CEUR Workshop Proceedings), vol. 3383, pp. 21-29. https://ceur-ws.org/Vol-3383/.

The study examined the level of difficulty, discrimination, and reliability of multiple-choice questions [MCQs] in Finnish national Law entrance examinations. The purpose was to assess whether MCQs could be used to rank applicants in a sufficiently reliable manner. The data set consists of anonymized scores from three exams (years 2018, 2019, 2021) containing 11,201 applicants all together. The study found that the MCQs provide a reliable, adequate, and high-quality discrimination. MCQ scores were also shown to correlate with essays and total scores
assessment analyticsentrance examinemultiple choice question
López-Pernas S., Saqr M. (2023). From Variables to States to Trajectories (VaSSTra): A Method for Modelling the Longitudinal Dynamics of Learning and Behaviour. Proceedings TEEM 2022: Tenth International Conference on Technological Ecosystems for Enhancing Multiculturality. TEEM 2022. Lecture Notes in Computer Science., pp. 1169-1178. doi: 10.1007/978-981-99-0942-1_123.

Research in learning analytics needs longitudinal studies that explore the learner’s behaviour, disposition, and learning practices across time, a gap this article aims to bridge. We present VaSSTra: an innovative method for the longitudinal analysis of educational data that can be applied at different time scales (e.g., days, weeks, or courses), and allows the study of different aspects of learning as well as the factors that explain how such aspects evolve over time. Our method combines life-events methods with sequence analysis and consists of three steps: (1) converting variables to states (where variables are grouped into homogenous states); (2) from states to sequences (where the states are used to construct sequences across time), and (3) from sequences to trajectories (where similar sequences are grouped in trajectories). VaSSTra enables us to map the longitudinal unfolding of events while taking advantage of the wealth of life-events methods to visualize, model and describe the temporal dynamics of longitudinal activities. We demonstrate the method with a practical case study example.
longitudinal methodslearning analyticslife-events methodssequence analysisclustering
Conde M.Á., López-Pernas S., Peltekova E., Pancheva K., Raspopovic Milic M., Saqr M. (2023). Multi-stakeholder Perspective on the Gap Between Existing Realities and New Requirements for Online and Blended Learning: An Exploratory Study. Proceedings TEEM 2022: Tenth International Conference on Technological Ecosystems for Enhancing Multiculturality. TEEM 2022. Lecture Notes in Computer Science., pp. 1109-1116. doi: 10.1007/978-981-99-0942-1_117.

Online and blended learning are teaching modalities that have become very popular and widespread all over the world. Applying these modalities requires specific knowledge as well as an appropriate technological infrastructure. The COVID-19 pandemic caused an important online migration in most educational institutions. In this regard, existing literature covers issues such as the impact, the challenges, the tools, the problems, etc. What could also be interesting is to understand students’, teachers’ and administrative staff’s perspectives about how blended and online learning were developed and how it is going to be applied in the future. With this in mind, the ILEDA project team has carried out an exploratory study, which takes into account these three collectives in four different European universities. From the study, it is possible to see that the institutions and their lecturers and staff were probably not prepared for the online migration and the possibilities they had were quite different from students’ expectations.
higher educatione-learningblended learningcovid-19 periodlearning analytics
Mairinoja L., López-Pernas S., Elmoazen R., Niskanen E.A., Kuningas T., Wärri A., Saqr M., Strauss L. (2023). International Online Team-Based Learning in Higher Education of Biomedicine - Evaluation by Learning Analytics. Proceedings of the Technology-Enhanced Learning in Laboratories Workshop (TELL 2023) (CEUR Workshop Proceedings), vol. 3393, pp. 49-60. https://ceur-ws.org/Vol-3393/TELL23_paper_1668_5.pdf.

Teamwork skills are important to practice during higher educational studies to prepare students for the future working life. Since online learning has established itself as a relevant part of higher education, we present here an approach to online team-based learning and show the performance of students during the teamwork, proven by learning analysis data. In addition, results from a feedback survey of students´ opinions on teamwork are presented. Online teamwork was implemented for master level biomedicine students from four different Universities in Nordic countries, and student interaction was evaluated. Learning analytics data were collected from Discord, which was the communication platform for students and teachers during the teamwork. The Community of Inquiry (CoI) framework was used as guidance, and indicators of CoI's social, cognitive, and teaching presences were used as a scheme for coding the interaction. To recognize the process of collaboration, the data were first analyzed by using process mining. Further, to understand the multidimensional property of collaboration, we developed a network analysis and visualized the results using Gephi and the Fruchterman-Reingold layout algorithm. The quantitative results of the feedback survey were analyzed by using descriptive statistics and visualized using the R package likert. The learning analytics data included 316 posts divided to 686 annotations, which were categorized to codes. Our results indicate that the most frequent codes were the ones related to the social dimension of CoI, determined with attributes such as ‘interactive’ (173), ‘cohesion’ (119) and ‘affective’ (116). The remaining most frequent codes alternated between ‘facilitation’ and ‘cognition’. Thus, social presence, in the context of CoI was considerable in our online team-based learning approach. However, to enhance students' cognitive presence, and thereby their ability to construct and confirm meaning of what they are learning, students' work should be facilitated by increasing teaching presence through teacher’s contribution online. In line with the learning analytics data, the results of the survey pointed out the need of more in-depth instructions on how to carry out the team exercises, which belongs to the teaching presence category in the frame of CoI. Based on the results of this study and the existing literature, we aim to improve our teambased learning approach and outcomes in the future by increasing students’ contribution through regular feedback assignments during the work and encouraging learners to reflect on their own work, contribution and thinking.
online teamworklearning analyticsdiscordvirtual collaborative learningonline teambased learningcommunity of inquiry
Deriba F., Saqr M., Tukiainen M. (2023). Exploring Barriers and Challenges to Accessibility in Virtual Laboratories: A Preliminary Review. Proceedings of the Technology-Enhanced Learning in Laboratories Workshop (TELL 2023) (CEUR Workshop Proceedings), vol. 3393, pp. 68-77. https://ceur-ws.org/Vol-3393/TELL23_paper_789_7.pdf.

Virtual laboratories (VL) have become an essential tool for educational sectors, allowing students to develop practical skills in a remote environment. However, the accessibility of VL remains a significant challenge for learners. This research paper aimed to investigate the accessibility barrier in VL and explores potential solutions to overcome them. To achieve this, we conducted a comprehensive literature review spanning from 1997 to 2023, focusing on the accessibility of VL. Our search was conducted solely on the Scopus database, resulting in 164 papers, from which we carefully selected 21 primary studies for detailed analysis. The result indicates still there is high barrier to accessing VL. Based on the analysis, we identified four major barriers: technological, infrastructural, pedagogical, and cultural. To address the issues, a range of solutions have been proposed. These findings highlight the critical need to tackle accessibility barriers in VL, thereby enabling all students to have equal opportunities to develop their practical skills.
virtual laboratorieslearning analyticsaccessibility barriersremote laboratorieseducational technologyonline learningaccessible learning
Conde M.Á., Georgiev A., López-Pernas S., Jovic J., Crespo-Martínez I., Raspopovic Milic M., Saqr M., Pancheva K. (2023). Definition of a Learning Analytics Ecosystem for the ILEDA Project Piloting. Learning and Collaboration Technologies. HCII 2023. Lecture Notes in Computer Science., vol. 14040, pp. 444-453. doi: 10.1007/978-3-031-34411-4_30.

Understanding how students progress in their learning is an important step towards achieving the success of the educational process. One way of understanding student progress is by using learning analytics methods on different student data. The ILEDA project aims to improve online and blended learning by using educational data analytics. For this purpose, the project involves four universities from four different countries and develops several activities. One of these activities. That aims to facilitate the analysis of student progress, is the definition of a Learning Analytics Ecosystem. The aim of defining the ecosystem is to generate solutions that will benefit all institutions and that will allow to look for possible patterns and common issues needing addressing. This paper describes the development of such an ecosystem and its future implementations.
learning analyticsinteroperabilityecosystemsevidencesdashboards

2022

Saqr M., López-Pernas S., García Á.H., Conde M.Á., Poquet O. (2022). Networks and Learning Analytics: Addressing Educational Challenges. Proceedings of the NetSciLA22 workshop (CEUR Workshop Proceedings), vol. 3258, pp. 1-3. https://ceur-ws.org/Vol-3258/xpreface.pdf.

Network Analysis is an established method in learning analytics research. Network Analysis has been used to analyze learners' interactions, to inform learning design, and to model students' performance. The workshop entitled "Using Network Science in Learning Analytics: Building Bridges towards a Common Agenda", carried out within the LAK2021 conference, resulted in valuable insights and outcomes: guidelines for better reporting, methodological improvements, and discussions of several novel research threads. Traditionally, the focus of the conversation has been on methodological issues of network analysis. This year, we would like to extend the conversation by slightly shifting the focus to what network analysis can do to improve learning and educational opportunities. As such, this new edition of the workshop aims to build on the fruitful achievements of the previous iteration to address new themes, which we refer to as “challenges and opportunities” in relation to practice. This edition of the workshop sought contributions around examples of applications and impact, including those that can help address societal challenges embedded within educational practices and those that foster an open conversation about privacy and ethical implications of network data. © 2022 Copyright for this paper by its authors.
learning analyticsnetwork analysisnetwork sciencesocial network analysis
Saqr M., López-Pernas S. (2022). The Why, the What and the How to Model a Dynamic Relational Learning Process with Temporal Networks. Proceedings of the NetSciLA22 workshop (CEUR Workshop Proceedings), vol. 3258, pp. 33-40. https://ceur-ws.org/Vol-3258/article_4.pdf.

Research on online learning has benefited from intensive data collection to understand students' online behavior and performance. Several learning analytics techniques have been operationalized to examine the temporal nature of learning that includes changes, phases, and sequences of students' online actions. Moreover, to account for the relational nature of learning, researchers have harnessed the power of network analysis to model the relational dimensions of data, mapping connections between learners and resources, and discovering interacting communities. However, prior research has rarely combined the two aspects (temporal and relational), but rather most researchers rely on aggregate networks where the time dimension has been ignored. To combine both these aspects, temporal networks provide a rich framework of statistical and visualization techniques that allow to fully understand, for instance, the evolution and building up of learning communities, the sequence of co-construction of knowledge, the flow of information, and the building of social capital, to name a few examples. Since temporal networks have been rarely used in educational research, with this study, we aim to provide an introduction to this method, with an emphasis on the differences with conventional static networks. We explain the basics of temporal networks, the different subtypes thereof, and the measures that can be taken, as well as examples from the few existing prior works. © 2022 Copyright for this paper by its authors.
learning analyticssocial network analysistemporal network analysis
Saqr M., López-Pernas S., Hernández-García A., Conde M.A., Poquet O. (2022). Concluding remarks of the NetSciLA22 Workshop. Proceedings of the NetSciLA22 workshop (CEUR Workshop Proceedings), vol. 3258, pp. 41-46. https://ceur-ws.org/Vol-3258/article_5.pdf.

The NetScila22 workshop builds on the previous iterations of network analysis workshops. The current year themes addressed educational challenges as well as opportunities for future research and for strengthening the community. The workshop included valuable discussions and interactions with both experts and emerging researchers. Such discussions were augmented by a survey that gathered insights form workshop attendees. The discussants recommended improving methodological rigor, leveraging methods that positively impact learning, address data issues, e.g., collection, privacy and reporting as well as better alignment with theory. Other recommendations proposed human-centred artificial intelligence approaches grounded on cognitive science, better communication with stakeholders, sharing ideas within the community and organizing hands-on seminar. The workshop also included presentations that address methodological advances and future opportunities, e.g., temporal networks, semantic networks and attention network. © 2022 Copyright for this paper by its authors.
learning analyticsnetwork analysisnetwork sciencesemantic networkstemporal network analysis
Saqr M., López-Pernas S. (2022). Instant or Distant: A Temporal Network Tale of Two Interaction Platforms and Their Influence on Collaboration. Educating for a New Future: Making Sense of Technology-Enhanced Learning Adoption. EC-TEL 2022. Lecture Notes in Computer Science, vol. 13450 LNCS, pp. 594-600. doi: 10.1007/978-3-031-16290-9_55.

This study compared two iterations of the same course where students had the same assignments. In the first iteration, the students had to use the typical discussion forums offered by the popular Moodle learning management system. In the second iteration, students had to use Discord, the popular gaming chat application. Students’ interactions were retrieved from both platforms and cleaned. Two social networks were constructed using the same methods to evaluate the differences in patterns of interaction between the two platforms, the group interactivity, the reciprocity, and the quality of interactions. The aim is to study how far an instant messenger facilitates or otherwise constrains collaboration. We use temporal network methods to further understand the pace, rhythm, and temporality of interactions. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.
cscllearning analyticssocial network analysistemporal network analysis
Sointu E., Saqr M., Valtonen T., Hallberg S., Väisänen S., Kankaanpää J., Tuominen V., Hirsto L. (2022). Emotional behavior in quantitative research methods course for preservice teachers. Learning analytics approach.. Proceedings of the Society for Information Technology & Teacher Education International Conference, pp. 1880-1889. https://www.learntechlib.org/primary/p/220997/.

Preservice teacher training is research intensive in Finland. Additionally, teaching as profession is highly valued among young people. However, quantitative methods courses are challenging for teacher students from many reasons. Particularly, this is due to previous negative experiences and emotions (among other things). Thus, new approaches for teaching quantitative methods are warranted. In this research we used Flipped Learning, online teaching and learning analytics to support the content learning. The aim of this research was to investigate teacher students’ (N = 40) emotional profiles (i.e., cluster) based on their emotional level (anxiety, boredom and enjoyment) towards quantitative research methods studies and online behavior. For creating profiles, we used questionnaire data. These profiles were then further analyzed with learning analytics data, more precisely, time-ordered data of teacher students’ interactions (i.e., frequencies). Based on the results, three distinct profiles were found: “medium”, “pro quantitative”, and “scared” teacher students towards quantitative research methods. Further investigation revealed that scared students demonstrated statistically significant transitions in majority of the learning management systems compared to other profiles. Interestingly, pro quantitative had the lowest and medium teacher students had no difference in these results. The results are discussed further in the conclusions.
preservice teacherresearch methodsquantitative methodsemotionsonline learninglearning analytics
Hirsto L., Valtonen T., Saqr M., Hallberg S., Sointu E., Kankaanpää J., Väisänen S. (2022). Pupils’ experiences of utilizing learning analytics to support self-regulated learning in two phenomenon-based study modules. Proceedings of the Society for Information Technology & Teacher Education International Conference, pp. 1682-1688. https://www.learntechlib.org/p/220967/.

The aim of this study is to understand primary level pupils’ experiences and perspectives of learning analytics (LA) and self-regulated learning after two phenomenon-based learning modules in a blended learning environment. For this, a specifically designed module was built into a learning management system (LMS) following the principles and key phases of self-regulated learning process. Using this web-based learning environment with their personal tablet computers, pupils were able to set goals for their studies, evaluate the extent to which they were achieving these goals, and monitor their progress and level of success in the learning process. According to the results, pupils’ understanding of LA remained on quite low level, but their perspectives on utilizing LA was quite positive. Three groups of pupils were identified in terms of pupils’ experiences on the utility of LA and how it seemed to support the development of pupils learning skills and self-efficacy. These groups also differed significantly on their more general experiences of the teaching-learning environment.
elementary schoolpupilsphenomenon-based learningself-regulated learninglearning analytics

2021

Saqr M., Lopez-Pernas S. (2021). Idiographic learning analytics: A definition and a case study. Proceedings of the IEEE 21st International Conference on Advanced Learning Technologies (ICALT 2021), pp. 163-165. doi: 10.1109/ICALT52272.2021.00056.

Idiographic methods have emerged as a way to examine individual behavior by using several data points from each subject to create person-specific insights. In the field of learning analytics, such methods could overcome the limitations of cross-sectional group-level data that may fail to capture the dynamic processes that unfold within each individual learner and less likely to offer relevant personalized learning or support. In this study, we provide a definition of idiographic learning analytics and we explore the possible potentials of this method to zoom in on the fine-grained dynamics of a single student. Specifically, we make use of Gaussian Graphical Models - an emerging trend in network science - to analyze a single student's dispositions and devise insights specific to him/her. Our findings offer a proof of concept of the potential of this novel method in revealing personalized valuable insights about students' self-regulation. While our specific findings apply to a single student, our method applies to every student regardless of context. © 2021 IEEE.
idiographic learning analyticslearning analyticsnetwork sciencepsychological networksraphical gaussian models
Apiola M., Tedre M., Lopez-Pernas S., Saqr M., Daniels M., Pears A. (2021). A Scientometric Journey Through the FIE Bookshelf: 1982-2020. Proceedings of the Frontiers in Education Conference (FIE). doi: 10.1109/FIE49875.2021.9637209.

IEEE/ASEE Frontiers in Education turned 50 at the 2020 virtual conference in Uppsala, Sweden. This paper presents an historical retrospective on the first 50 years of the conference from a scientometric perspective. That is to say, we explore the evolution of the conference in terms of prolific authors, communities of co-authorship, clusters of topics, and internationalization, as the conference transcended its largely provincial US roots to become a truly international forum through which to explore the frontiers of educational research and practice. The paper demonstrates the significance of FIE for a core of 30% repeat authors, many of whom have been members of the community and regular contributors for more than 20 years. It also demonstrates that internal citation rates are low, and that the co-authoring networks remain strongly dominated by clusters around highly prolific authors from a few well known US institutions. We conclude that FIE has truly come of age as an international venue for publishing high quality research and practice papers, while at the same time urging members of the community to be aware of prior work published at FIE, and to consider using it more actively as a foundation for future advances in the field. © 2021 IEEE.
Saqr M., López-Pernas S. (2021). The Dire Cost of Early Disengagement: A Four-Year Learning Analytics Study over a Full Program. Technology-Enhanced Learning for a Free, Safe, and Sustainable World. EC-TEL 2021. Lecture Notes in Computer Science, vol. 12884 LNCS, pp. 122-136. doi: 10.1007/978-3-030-86436-1_10.

Research on online engagement is abundant. However, most of the available studies have focused on a single course. Therefore, little is known about how students’ online engagement evolves over time. Previous research in face-to-face settings has shown that early disengagement has negative consequences on students’ academic achievement and graduation rates. This study examines the longitudinal trajectory of students’ online engagement throughout a complete college degree. The study followed 99 students over 4 years of college education including all their course data (15 courses and 1383 course enrollments). Students’ engagement states for each course enrollment were identified through Latent Class Analysis (LCA). Students who were not engaged at least one course in the first term was labeled as “Early Disengagement”, whereas the remaining students were labeled as “Early Engagement”. The two groups of students were analyzed using sequence pattern mining methods. The stability (persistence of the engagement state), transition (ascending to a higher engagement state or descending to a lower state), and typology of each group trajectory of engagement are described in this study. Our results show that early disengagement is linked to higher rates of dropout, lower scores, and lower graduation rates whereas early engagement is relatively stable. Our findings indicate that it is critical to proactively address early disengagement during a program, watch the alarming signs such as presence of disengagement during the first courses, declining engagement along the program, or history of frequent disengagement states. © 2021, Springer Nature Switzerland AG.
early disengagementlearning analyticstrajectories of engagement
Lopez-Pernas S., Saqr M. (2021). Idiographic learning analytics: A within-person ethical perspective. Companion Proceedings 11th International Conference on Learning Analytics & Knowledge (LAK21), pp. 369-374. https://www.solaresearch.org/core/lak21-companion-proceedings/.

One of the main obstacles impeding the widespread use and adoption of learning analytics is the threat that it poses to students’ data privacy. In this article, we present a proposal for generating person-centered insights for learners by combining data from multiple sources while preserving students' privacy. The basis of our approach is idiographic learning analytics, in which data are collected and insights are generated for each student individually. On the one hand, all the data collection and processing are performed locally on the student’s device, thus preserving student privacy. On the other hand, being based on person-based methods, the idiographic approach helps deliver personalized insights.
ethicslearning analyticsidiographicprivacy
Saqr M., López-Pernas S. (2021). Idiographic learning analytics: A single student (N=1) approach using psychological networks. Proceedings of the NetSciLA21 workshop (CEUR Workshop Proceedings), vol. 2868, pp. 16-22. https://ceur-ws.org/Vol-2868/article_4.pdf.

Recent findings in the field of learning analytics have brought to our attention that conclusions drawn from cross-sectional group-level data may not capture the dynamic processes that unfold within each individual learner. In this light, idiographic methods have started to gain grounds in many fields as a possible solution to examine students' behavior at the individual level by using several data points from each learner to create person-specific insights. In this study, we introduce such novel methods to the learning analytics field by exploring the possible potentials that one can gain from zooming in on the fine-grained dynamics of a single student. Specifically, we make use of Gaussian Graphical Models -an emerging trend in network science- to analyze a single student's dispositions and devise insights specific to him/her. The results of our study revealed that the student under examination may be in need to learn better self-regulation techniques regarding reflection and planning. © 2021 Copyright for this paper by its authors.
graphical gaussian modelsidiographic learning analyticsnetwork sciencepsychological networks
Poquet O., Saqr M., Chen B. (2021). Recommendations for network research in learning analytics: To open a conversation. Proceedings of the NetSciLA21 workshop (CEUR Workshop Proceedings), vol. 2868, pp. 34-41. https://ceur-ws.org/Vol-2868/article_7.pdf.

Network science methods are widely adopted in learning analytics, an applied research area that focuses on the analysis of learning data to understand and improve learning. The workshop, taking place at the 11th International Learning Analytics and Knowledge conference, focused on the applications of network science in learning analytics. The workshop attracted over twenty researchers and practitioners working with network analysis and educational data. The workshop included work-in-progress and group-wide conversations about enhancing the quality of network research in learning analytics. The conversations were driven by concerns around reproducibility and interpretability currently discussed across research communities. This paper presents a snapshot of the workshop discussions beyond its work-in-progress papers. To this end, we summarize a literature review presented to the workshop participants, with the focus on the elements related to the reproducibility and interpretability of network research in education settings. We also provide a summary of the workshop discussions and conclude with suggested guidelines for the reporting of network methods to improve generalizability and reproducibility. © 2021 Copyright for this paper by its authors.
educationlearning analyticslearning sciencesnetwork sciencerecommendations
Poquet O., Chen B., Saqr M., Hecking T. (2021). Using network science in learning analytics: Building bridges towards a common agenda. Proceedings of the NetSciLA21 workshop (CEUR Workshop Proceedings), vol. 2868, pp. 1-2. https://ceur-ws.org/Vol-2868/article_1.pdf.

Interest in using networks in the analysis of digital data has long existed in learning analytics (LA). Applications of network science in our field are diverse. Some researchers analyze social settings in online discussions, knowledge building software, and group formation tools. Others use networked techniques to capture epistemic and cognitive processes. Networked approaches have been pioneered for psychometrics, for the analysis of time-series data, and for various types of clustering of relational observations. Finally, modelling of variables where networks are used as representations of causal relations is also gaining traction. Given the diversity of the thematic foci that researchers engage in when applying network science to learning analytics, this workshop aims to identify common challenges experienced through the use of network science methodologies. The workshop will invite researchers working in the area to share their work and reflect on common challenges. We envision themes of causality, linkage between micro- and macro-processes, use of time and space, elements of generalizability and validity to surface in the group discussions. The workshop aims to gather LA scholars to collectively build a solid foundation of advanced network modeling of learning data and shape strategies of future work in this important sub-field of LA. © 2021 Copyright for this paper by its authors.
common challengeslearning analyticsnetwork science
Saqr M., Viberg O., Peteers W. (2021). Using psychological networks to reveal the interplay between foreign language students' self-regulated learning tactics. Proceedings of the 2020 STELLA Symposium (CEUR Workshop Proceedings), vol. 2828. https://ceur-ws.org/Vol-2828/article_2.pdf.

Students' ability to self-regulate their individual and collaborative learning activities while performing challenging academic writing tasks is instrumental for their academic success. Presently, the majority of such learning activities often occur in computer-supported collaborative learning (CSCL) settings, in which students generate digital learner data. Examining this data may provide valuable insights into their self-regulated learning (SRL) behaviours. Such an understanding is important for educators to provide adequate support. Recent advances in the fields of learning analytics (LA) and SRL offer new ways to analyse such data and understand students' dynamic SRL processes. This study uses a novel psychological network method, i.e., Gaussian Graphical Models, to model the interactions between the students' SRL tactics and how they influence language learning in a CSCL setting for academic writing. The data for this study was generated by first-year foreign language students (n=119) who used a Facebook group as a collaborative space for peer review in an academic writing course. The theoretical lens of strategic self-regulated language learning was applied. The findings show a strong connection between the following tactics: writing text, social bonding and acknowledging. Strong connections between students' reflective activities and their application of feedback, as well as between acculturating, organising and using resources were also identified. Centrality measures showed that acculturating is most strongly connected to all other tactics, followed by acknowledging and social bonding. Expected influence centrality measures showed acculturating and social interactions to be strong influencers. Students' academic performance and their use of tactics showed little correlation. © 2020 Copyright for this paper by its authors.
computer-supported collaborative learningforeign language learninggaussian graphical modelspsychological networksself-regulated learning
Saqr M., Nouri J., Fors U., Viberg O., Alsuhaibani M., Alharbi A., Alharbi M., Alamer A. (2021). How Networking and Social Capital Influence Performance: The Role of Long-Term Ties. Lecture Notes in Networks and Systems, vol. 181, pp. 335-346. doi: 10.1007/978-3-030-64877-0_22.

Recently, students have become networked in many ways, and evidence is mounting that networking plays a significant role in how students learn, interact, and transfer information. Relationships could translate to opportunities; resources and support that help achieve the pursued goals and objectives. Although students exist, interact, and play different roles within social and information networks, networks have not received the due attention. This research aimed to study medical student’s friendship- and information exchange networks as well as assess the correlation between social capital and network position variables and the cumulative Grade Point Average (GPA) which is the average grade obtained over all the years. The relationships considered in our study are the long-term face-to-face and online ties that developed over the full duration of the study in the medical college. More specifically, we have studied face-to-face and information networks. Analysis of student’s networks included a combination of visual and social network analysis. The correlation with the performance was performed using resampling permutation correlation coefficient, linear regression, and 10-fold cross-validation of binary logistic regression. The results of correlation and linear regression tests demonstrated that student’s social capital was correlated with performance. The most significant factors were the power of close friends regarding connectedness and achievement scores. These findings were evident in the close friends’ network and the information network. The results of this study highlight the importance of social capital and networking ties in medical schools and the need to consider peer dynamics in class assignment and support services. © 2021, The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG.
learning analyticspeer relationsperformancesocial capitalsocial network analysis

2020

Peeters W., Saqr M., Viberg O. (2020). Applying learning analytics to map students' self-regulated learning tactics in an academic writing course. Proceedings of the 28th International Conference on Computers in Education (ICCE 2020), pp. 245-254. https://apsce.net/icce/icce2020/proceedings/paper_143.pdf.

Academic writing is a complex and challenging language learning activity, in which self-regulation is a key critical factor for learner success. Today, a large number of academic writing activities occur in digital learning spaces, including computer-supported collaborative learning settings. Recent advances in the fields of learning analytics (LA) and computer-assisted language learning have provided new opportunities, in part because of the accessibility to new digital learner data, to better understand and ultimately support students' self-regulated learning (SRL) processes. Even though some related efforts have been made, there is yet a paucity of research targeting foreign language students' interactive SRL behaviour in online environments. This study aims to shed more light on this issue, and on the possible ways to fill this gap. We used LA methods (frequency analysis, network analysis, statistical analysis and process mining) to analyse and visualise students' SRL tactics when collaborating with their peers on academic writing tasks on social networking sites. The dataset was obtained from a case study performed at the University of Antwerp (Belgium). In this study, a private Facebook group was integrated in an academic writing course for first-year foreign language majors of English (n=124) and served as an online collaborative space for peer review. The results show, firstly, that foreign language learners use a range of SRL tactics to manage their academic writing process and, secondly, that the strategic SRL task phases (i.e., plan, perform and evaluate) are strongly interconnected. Learners exhibit a readiness and willingness to activate knowledge, monitor progress, interact to adjust to the socio-cultural context and form an identity as a learner. There is a significant positive correlation between students' use of SRL tactics and their learning performance, which provides novel ground for designing and providing relevant SRL support mechanisms in computer-supported collaborative learning. © ICCE 2020 - 28th International Conference on Computers in Education, Proceedings. All rights reserved.
academic writinghigher educationlearning analyticsself-regulated learningsocial networking sites
Saqr M., Montero C.S. (2020). Learning and social networks-similarities, differences and impact. Proceedings of the IEEE 20th International Conference on Advanced Learning Technologies (ICALT 2020), art. no. 9155624, pp. 135-139. doi: 10.1109/ICALT49669.2020.00047.

Previous work in learning analytics have been fruitful in shedding lights on collaborative learning environments, such work has provided insights and recommendations that helped improve the collaborative process in computer-mediated learning environments. Given the importance of social interactions and their influence on learning (e.g., in determining academic growth, perseverance in the course and persistence). In this study, we look at both learning and social networks, what factors they share, how they impact or influence learning, and what influences the formation of these networks. Our results show similarities and differences between both networks such as: interactions in the social network predict those in the learning network, however, only centrality measures in the learning network correlate with performance, probably due to the selective nature of replies and interactions in the learning network. © 2020 IEEE.
collaborative learningcscllearning analyticsmedical educationsocial network analysissocial networks
Saqr M., Nouri J. (2020). High resolution temporal network analysis to understand and improve collaborative learning. ACM International Conference Proceeding Series, pp. 314-319. doi: 10.1145/3375462.3375501.

There has been significant efforts in studying collaborative and social learning using aggregate networks. Such efforts have demonstrated the worth of the approach by providing insights about the interactions, student and teacher roles, and predictability of performance. However, using an aggregated network discounts the fine resolution of temporal interactions. By doing so, we might overlook the regularities/irregularities of students' interactions, the process of learning regulation, and how and when different actors influence each other. Thus, compressing a complex temporal process such as learning may be oversimplifying and reductionist. Through a temporal network analysis of 54 students interactions (in total 3134 interactions) in an online medical education course, this study contributes with a methodological approach to building, visualizing and quantitatively analyzing temporal networks, that could help educational practitioners understand important temporal aspects of collaborative learning that might need attention and action. Furthermore, the analysis conducted emphasize the importance of considering the time characteristics of the data that should be used when attempting to, for instance, implement early predictions of performance and early detection of students and groups that need support and attention. © 2020 Copyright is held by the owner/author(s).
collaborative learninglearning analyticsmedical educationproblem-based learningsocial network analysistemporal networkstemporarily
Saqr M., Viberg O. (2020). Using diffusion network analytics to examine and support knowledge construction in cscl settings. Addressing Global Challenges and Quality Education. EC-TEL 2020. Lecture Notes in Computer Science, vol. 12315 LNCS, pp. 158-172. doi: 10.1007/978-3-030-57717-9_12.

The analysis of CSCL needs to offer actionable insights about how knowledge construction between learners is built, facilitated and/or constrained, with the overall aim to help support knowledge (co-)construction. To address this, the present study demonstrates how network analysis - in a form of diffusion-based visual and quantitative information exchange metrics - can be effectively employed to: 1. visually map the learner networks of information exchange, 2. identify and define student roles in the collaborative process, and 3. test the association between information exchange metrics and performance. The analysis is based on a dataset of a course with a CSCL module (n = 129 students). For each student, we calculated the centrality indices that reflect the roles played in information exchange, range of influence, and connectivity. Students’ roles were analysed employing unsupervised clustering techniques to identify groups that share similar characteristics in regard to their emerging roles in the information exchange process. The results of this study have proved that diffusion-based visual and quantitative metrics can be effectively employed and are valuable methods to visually map the student networks of information exchange as well as to detect and define students’ roles in the collaborative learning process. Furthermore, the results demonstrated a positive and statistically significant association between diffusion metrics and academic performance. © Springer Nature Switzerland AG 2020.
csclhigher educationinformation exchange metricsknowledge exchangestudent performancestudents’ roles
Misfeldt M., Spikol D., Bruun J., Saqr M., Kaliisa R., Ruis A., Eagan B. (2020). Quantitative ethnography as a framework for network analysis–a discussion of the foundations for network approaches to leaning analysis. Companion Proceedings 10th International Conference on Learning Analytics & Knowledge (LAK20), pp. 600-603. https://www.solaresearch.org/core/lak20-companion-proceedings/.

This workshop explores quantitative ethnography as a framework for discussing network approaches to learning analysis. In many learning contexts, we increasingly have access to large amounts of rich process data. To make meaning of this evidence, our goal is to develop a qualitatively “thick” description of the data, and thus of learning. However, the more data we have, the more difficult this process becomes: qualitative analysis becomes less feasible, and quantitative analysis becomes less reliable. Quantitative ethnography addresses this problem by using statistical techniques to warrant claims about the quality of thick description. The result is a more unified mixed-methods approach that uniquely links the evidence we collect to learning processes and outcomes. This workshop focuses on different techniques that address this challenge, including epistemic network analysis, social network analysis, and Social Learning Analytics. The aim of the workshop is to examine these techniques of network analysisthrough a quantitative ethnography frame in order to generate a more unified methodology for modeling learning processes and providing actionable in insights for research and teaching practices.
quantitative ethnographynetwork analysisenasnaslamixed-methods research
Saqr M., Viberg O., Nouri J., Oyelere S. (2020). Multimodal temporal network analysis to improve learner support and teaching. Proceedings of CrossMMLA in practice (CEUR Workshop Proceedings), vol. 2610, pp. 30-33. https://ceur-ws.org/Vol-2610/paper6.pdf.

A learning process involves interactions between learners, teachers, machines and formal and/or informal learning environments. These interactions are relational, interdependent and temporal. The emergence of rich multimodal learner data suggests the development of methods that can capture time-stamped data from multiple sources (e.g., heart rate data and eye tracking data), thus allowing researchers to examine learning as a continuous process rather than a static one. This leads us to propose a new methodological approach, the Multimodal Temporal Network Analysis to: i) measure temporal learner data deriving from the relevant interactions and ii) ultimately support learners and their teachers in learning and/or teaching activities. Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International 1 (CC BY 4.0).
multimodal learning analyticssocial network analysistemporal networks
Wedberg M., Viberg O., Saqr M. (2020). Facilitating Disciplinary-Specific Knowledge Sharing: A Usability Study of a Dementia Library. World Conference on Mobile, Blended and Seamless Learning, pp. 51-58. https://www.learntechlib.org/p/218888/.

There is a lack of technology that facilitates disciplinary-specific knowledge dissemination in the medical sector specialties. In several countries there is a shortage of medical staff with the proper education to take care of patients. However, modern mobile- and web technology pave the way for new online knowledge sharing platforms which could help remedy this problem. This study investigates how an interface of a mobile e-library, aimed at sharing disciplinary-specific (dementia in our case) knowledge, could be designed. It also examines how care workers perceive it and if they could be willing to adopt the technology in the future. This study was carried out at Svenskt Demenscentrum, a non-profit Swedish organization with the purpose of disseminating and collecting knowledge concerning dementia. The prototype was designed using the double diamond design process. This included an initial literature study and state-of-the-art analysis, which was followed by two co-creation workshops with professional care workers. The final design was created iteratively with feedback from a focus group. A total of four sessions with the focus group were organized. The findings of the usability testing suggest that the study participants perceived the interface as both useful and easy to use. This result also indicates that the users, according to the Technology Acceptance Model model, would be willing to adopt the technology if fully developed.
knowledge sharingdementia careusabilitymultimedia librarymobile learningprototypinge-library

2019

Saqr M., Nouri J., Jormanainen I. (2019). A Learning Analytics Study of the Effect of Group Size on Social Dynamics and Performance in Online Collaborative Learning. Transforming Learning with Meaningful Technologies. EC-TEL 2019. Lecture Notes in Computer Science, vol. 11722 LNCS, pp. 466-479. doi: 10.1007/978-3-030-29736-7_35.

Effective collaborative learning is rarely a spontaneous phenomenon. In fact, it requires that a set of conditions are met. Among these central conditions are group formation, size and interaction dynamics. While previous research has demonstrated that size might have detrimental effects on collaborative learning, few have examined how social dynamics develop depending on group size. This learning analytics paper reports on a study that asks: How is group size affecting social dynamics and performance of collaborating students? In contrast to previous research that was mainly qualitative and assessed a limited sample size, our study included 23,979 interactions from 20 courses, 114 groups and 974 students and the group size ranged from 7 to 15 in the context of online problem-based learning. To capture the social dynamics, we applied social network analysis for the study of how group size affects collaborative learning. In general, we conclude that larger groups are associated with decreased performance of individual students, poorer and less diverse social interactions. A high group size led to a less cohesive group, with less efficient communication and less information exchange among members. Large groups may facilitate isolation and inactivity of some students, which is contrary to what collaborative learning is about. © 2019, The Author(s).
collaborative learningcomplexitygroup sizeinteraction analysislearning analyticsmedical educationproblem based learningsocial network analysis
Nouri J., Larsson K., Saqr M. (2019). Identifying Factors for Master Thesis Completion and Non-completion Through Learning Analytics and Machine Learning. Proceedings TEEM 2022: Tenth International Conference on Technological Ecosystems for Enhancing Multiculturality. TEEM 2022. Lecture Notes in Educational Technology, pp. 28-39. doi: 10.1007/978-3-030-29736-7_3.

The master thesis is the last formal step in most universities around the world. However, all students do not finish their master thesis. Thus, it is reasonable to assume that the non-completion of the master thesis should be viewed as a substantial problem that requires serious attention and proactive planning. This learning analytics study aims to understand better factors that influence completion and non-completion of master thesis projects. More specifically, we ask: which student and supervisor factors influence completion and non-completion of master thesis? Can we predict completion and non-completion of master thesis using such variables in order to optimise the matching of supervisors and students? To answer the research questions, we extracted data about supervisors and students from two thesis management systems which record large amounts of data related to the thesis process. The sample used was 755 master thesis projects supervised by 109 teachers. By applying traditional statistical methods (descriptive statistics, correlation tests and independent sample t-tests), as well as machine learning algorithms, we identify five central factors that can accurately predict master thesis completion and non-completion. Besides the identified predictors that explain master thesis completion and non-completion, this study contributes to demonstrating how educational data and learning analytics can produce actionable data-driven insights. In this case, insights that can be utilised to inform and optimise how supervisors and students are matched and to stimulate targeted training and capacity building of supervisors. © 2019, The Author(s).
completiondropoutlearning analyticsmachine learningmasterretentionthesis
Saqr M., Jalal N., Santolini M. (2019). Towards group-aware learning analytics: Using social network analysis and machine learning to monitor and predict performance in collaborative learning. Proceedings of the 13th International Technology, Education and Development Conference (INTED), pp. 7652-7659. doi: 10.21125/inted.2019.1881.

We know that employing collaborative learning strategies does not lead to productive collaborative learning per se. In fact, some groups are dysfunctional and might have a detrimental influence on group members. This issue has methodologically been studied through self-reported surveys, transcripts coding, and observational methods. Although these methods are informative, they are also time intensive, exhausting and not practical to be applied in real practice. Social network analysis (SNA) and learning analytics, on the other hand, open the door for using automatic and non-intrusive methods that can help us monitor group interactions. Here we study if SNA combined with machine learning techniques can be employed in order to better understand and predict how different type and quantities of collaborative interactions in online environments affect individual and group performance. More specifically, we study the correlation between group interaction parameters as measured by SNA and groups and individual’s performance. Using interaction parameters and machine learning methods, we identify the indicators that best predict groups that will perform and gain and groups that will not, as well as individuals who gain in performance and those who do not. The article provides support for the idea that learning analytics can help automatically monitor group performance and offer an opportunity for educators and learners to support productive collaborative learning.
learning analyticscollaborative learningsmall groupsinteraction analysissocial network analysis

2018

Saqr M., Nouri J., Fors U. (2018). Temporality matters: A learning analytics study of the patterns of interactions and its relation to performance. Proceedings of the 10th International Conference on Education and New Learning Technologies (EDULEARN), pp. 5386-5393. doi: 10.21125/edulearn.2018.1305.

Although temporality is embodied in instructional design, implicitly present in several learning theories and central to the self-regulation of learning and awarding of credits, it has not received the due attention in the field education. This learning analytics study included four higher education courses in dental education over a full year duration. Temporality in terms of when students engage in learning was studied on daily, weekly, course, and year basis. The patterns of low and high achiever groups in each period were visually plotted and compared. Correlation with the performance was evaluated using the non-parametric Spearman correlation test using re-sampling permutation technique. The findings of this study highlight some important points; temporality is a defining factor of how students regulate their learning and should be taken into account when designing a possible monitoring system. High achievers were always active early in the year, in the course, and on assignments. Low achievers, on the other hand, tend to be significantly more active close to examination times. Using only temporality predictors, we were able to predict high achievers with 100% precision and low achievers with 82.3% to 93.3% class precision. Since early participation was the predictor, it means that an early alert indicator can be achieved that enables timely intervention.
learning analyticstemporalitycollaborative learningtimeprocrastination.
Saqr M., Nouri J., Fors U. (2018). What shapes the communities of learners in a medical school. Proceedings of the 10th International Conference on Education and New Learning Technologies (EDULEARN), pp. 7709-7716. doi: 10.21125/edulearn.2018.1792.

A positive association between social ties has been reported between social relationships or peer interactions and better performance. However, these findings were reported using traditional descriptive methods that suffered endogeneity, positing a serious threat to the inferences made. Moreover, little is known about how the networks of friendships in a medical school form and what factors derive the community structure. This study was done to evaluate the factors that shaped the social structure of medical students’ communities with particular emphasis on the role of academic performance and gender differences. The results of the correlation test between the social popularity measures and performance were statistically significant in the male group and insignificant in the female group. This variance might point out to a different mechanism of community building and social ties that differs among genders. To investigate the factors that affect community building, we used exponential-family random graph models to model the networks and identify the factors that best predict the emergence of ties. The male network included 69 nodes and 365 edges. Besides reciprocity, triangle closure, the city of residence, out-degree and in-degree popularity; the academic performance was a significant factor in terms of both the GPA and the difference between grades of both nodes. In the female network, (50 nodes and 176 edges), academic performance was a not significant factor, both the GPA and the difference between grades of both nodes; while reciprocity, triangle closure, the city of residence, out-degree and in-degree popularity were. The final model in male and female network showed a high degree of goodness-of-fitness statistics. These results highlight the issue of homophily on performance, as a significant factor in how males in this study build their friendship network in contrast to females. It also emphasizes the need for better inferential models that genuinely capture the network effect on performance before jumping to conclusions using traditional descriptive models that suffer the risk of endogeneity.
social network analysisacademic performancecommunity buildinginteractionsnetwork modelling

Book Chapters

2024

Helske J., Helske S., Saqr M., López-Pernas S., Murphy K. (2024). A Modern Approach to Transition Analysis and Process Mining with Markov Models in Education. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_12.

This chapter presents an introduction to Markovian modelling for the analysis of sequence data. Contrary to the deterministic approach seen in the previous sequence analysis chapters, Markovian models are probabilistic models, focusing on the transitions between states instead of studying sequences as a whole. The chapter provides an introduction to this method and differentiates between its most common variations: first-order Markov models, hidden Markov models, mixture Markov models, and mixture hidden Markov models. In addition to a thorough explanation and contextualisation within the existing literature, the chapter provides a step-by-step tutorial on how to implement each type of Markovian model using the R package seqHMM. The chapter also provides a complete guide to performing stochastic process mining with Markovian models as well as plotting, comparing and clustering different process models.
markov modelslearning analyticssequence analysistransition analysis
Hernández-García Á., Cuenca-Enrique C., Traxler A., López-Pernas S., Conde M. Á., Saqr M. (2024). Community Detection in Learning Networks Using R. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_16.

In the field of social network analysis, the quest for understanding interactions and group structures takes a center stage. This chapter focuses on finding such groups, constellations or ensembles of actors who can be grouped together, a process often referred to as community detection, particularly in the context of educational research. Community detection aims to uncover tightly knit subgroups of nodes who share strong connectivity within a network or have connectivity patterns that demarcates them from the others. This chapter explores various algorithms and techniques that unveil these groups or cohesive clusters. Using well-known R packages, the chapter primarily delves into the core approach of identifying and visualizing densely connected subgroups, offering practical insights into its application within educational contexts. Ultimately, the chapter aims to serve as a guide to unraveling learning communities, providing educators and researchers with valuable tools to discern and harness the power of interconnectedness in learning networks.
community detectionsocial network analysislearning analyticseducational data mining
Jongerling J., López-Pernas S., Saqr M., Vogelsmeier L. (2024). Structural Equation Modeling with R for Education Scientists. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_21.

Structural Equation Modeling (SEM) is a method for modeling the multitude of relationships between latent variables and the observable indicators, as well as the relationship between the latent variables themselves to test theories. In its most common form, SEM combines confirmatory factor analysis (CFA with another method named path analysis. Just like CFA, SEM relates observed variables to latent variables that are measured by those observed variables and, as path analysis does, SEM allows for a wide range of regression-type relations between sets of variables (both latent and observed). This chapter presents an introduction to SEM, an integrated strategy for conducting SEM analysis that is well-suited for educational sciences, and a tutorial on how to carry out an SEM analysis in R.
structural equation modelinglearning analyticssem
Jovanovic J., López-Pernas S., Saqr M. (2024). Predictive Modelling in Learning Analytics using R. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_7.

Prediction of learners’ course performance has been a central theme in learning analytics (LA) since the inception of the field. The main motivation for such predictions has been to identify learners who are at risk of low achievement so that they could be offered timely support based on intervention strategies derived from analysis of learners’ data. To predict student success, numerous indicators, from varying data sources, have been examined and reported in the literature. Likewise, a variety of predictive algorithms have been used. The objective of this chapter is to introduce the reader to predictive modelling in LA, through a review of the main objectives, indicators, and algorithms that have been operationalized in previous works as well as a step-by-step tutorial of how to perform predictive modelling in LA using R. The tutorial demonstrates how to predict student success using learning traces originating from a learning management system, guiding the reader through all the required steps from the data preparation all to the evaluation of the built models.
learning analyticspredictive modellingr programmingtutorialmachine learning
Kopra J., Tikka S., Heinäniemi M., López-Pernas S., Saqr M. (2024). An R Approach to Data Cleaning and Wrangling for Education Research. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_4.

Data wrangling, also known as data cleaning and preprocessing, is a critical step in the data analysis process, particularly in the context of learning analytics. This chapter provides an introduction to data wrangling using R and covers topics such as data importing, cleaning, manipulation, and reshaping with a focus on tidy data. Specifically, readers will learn how to read data from different file formats (e.g. CSV, Excel), how to manipulate data using the dplyr package, and how to reshape data using the tidyr package. Additionally, the chapter covers techniques for combining multiple data sources. By the end of the chapter, readers should have a solid understanding of how to perform data wrangling tasks in R.
data wranglingdata cleaningtidyverser programminglearning analytics
López-Pernas S., Misiejuk K., Kaliisa R., Conde M. Á., Saqr M. (2024). Capturing the Wealth and Diversity of Learning Processes with Learning Analytics Methods. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_1.

The unique position of learning analytics at the intersection of education and computer science while reaching out to several other disciplines such as statistics, psychometrics, econometrics, mathematics, and linguistics has accelerated the growth and expansion of the field. Therefore, it is a crucial endeavor for learning analytics researchers to stay abreast of the latest methodological and computational advances to drive their research forward. The diversity and complexity of the existing methods can make this task overwhelming both for newcomers to the learning analytics field and for experienced researchers. With the motivation to accompany researchers in this challenging journey, the book “Learning Analytics Methods and Tutorials - A Practical Guide Using R” aims to provide a methodological guide for researchers to study, consult, and embark upon the first steps toward innovation in the learning analytics field. Thanks to the unique wealth of authors’ backgrounds and expertise, which include authors of R packages and experts in methods and applications, the book offers a comprehensive array of methods that are described thoroughly with a primer on their usage in prior research in education. These methods include sequence analysis, Markov models, factor analysis, process mining, network analysis, predictive modeling, and cluster analysis among others. A step-by-step tutorial using the R programming language with real-life datasets and case studies is presented for each method. In addition, the initial chapters are devoted to getting novice researchers up to speed with the R programming learners and the basics of data analysis. The present chapter serves as an introduction to the book describing its main aim and intended audience. It describes the structure of the book and the methods covered by each chapter. It also points the readers to the companion code and data repositories to facilitate following the tutorials present in the book chapter.
learning analyticseducational data miningtutorialmethodslearning analyticscurriculum
López-Pernas S., Misiejuk K., Tikka S., Saqr M., Kopra J., Heinäniemi M. (2024). Visualizing and Reporting Educational Data with R. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_6.

Visualizing data is central in learning analytics research, underpins learning dashboards, and is a prime method for reporting results and insights to stakeholders. In this chapter, the reader will be guided through the process of generating meaningful and aesthetically pleasing visualizations of different types of student data using well-known R packages. The main visualization types will be demonstrated with an explanation of their usage and use cases. Furthermore, learning-related examples will be discussed in detail. For instance, readers will learn how to visualize learners’ logs extracted from learning management systems to show how trace data can be used to track students’ learning activities. In addition to creating compelling plots, readers will also be able to generate professional-looking tables with summary statistics.
learning analyticsdata visualizationggplot2visual analytics
López-Pernas S., Saqr M. (2024). Modelling the Dynamics of Longitudinal Processes in Education: The VaSSTra Method. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_11.

Modeling a longitudinal process in educational research brings a lot of variability over time. The modeling procedure becomes even harder when using multivariate continuous variables, e.g., clicks on learning resources, time spent online, and interactions with peers. In fact, most human behavioral constructs are an amalgam of interrelated features with complex fluctuations over time. Modeling such processes requires a method that takes into account the multidimensional nature of the examined construct as well as the temporal evolution. In this chapter we describe the VaSSTra method, which combines person-based methods, sequence analysis and life-events methods. Throughout the chapter, we discuss how to derive states from different variables related to students, how to construct sequences from students’ longitudinal progression of states, and how to identify and study distinct trajectories of sequences that undergo a similar evolution. We also cover some advanced properties of sequences that can help us analyze and compare trajectories. We illustrate the method through a tutorial using the R programming language.
learning analyticssequence analysislife-events methodsperson-based methodslongitudinal methods
López-Pernas S., Saqr M. (2024). The Why, the How, and the When of Educational Process Mining in R. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_14.

This chapter presents the topic of process mining applied to learning analytics data. The chapter begins by introducing the fundamental concepts of the method, with a focus on event log construction and visual representation using directly-follows graphs. A review of the existing literature on educational process mining is also presented to introduce the reader to the state of the art of the field. The chapter follows with a guided tutorial in R on how to apply process mining to trace log data extracted from an online learning management system. The tutorial uses the bupaverse framework for data handling and visualization. We finish the chapter with a reflection on the method and its reliability and applicability.
process miningbupaverselearning analyticseducational data mining
López-Pernas S., Saqr M., Conde J., Del-Río-Carazo L. (2024). A Broad Collection of Datasets for Educational Research Training and Application. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_2.

In this chapter, we present the main types of data that are commonly used in learning analytics research. Learning analytics has grown to encompass the digital trails left by online learning technologies —clicks, events, and interactions—, sensor data and self-reports among others. We present a collection of curated real-life open datasets that represent the most common types of educational data. The datasets have been collected from diverse sources such as learning management systems, online forums, and surveys. These datasets are used throughout the book to illustrate methods of analysis such as sequence analysis, social network analysis, Markov models, predictive analytics and structure equation modeling, to mention a few. Each data set in the chapter is presented with its context, main properties, links to the original source, as well as a brief exploratory data analysis.
learning analyticsdatasetsopen dataeducational data mining
López-Pernas S., Saqr M., Helske S., Murphy K. (2024). Multichannel Sequence Analysis in Educational Research Using R. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_13.

This chapter introduces multi-channel sequence analysis, a novel method that examines two or more synchronised sequences. While this approach is relatively new in social sciences, its relevance to educational research is growing as researchers gain access to diverse multimodal temporal data. Throughout this chapter, we describe multi-channel sequence analysis in detail, with an emphasis on how to detect patterns within the sequences, i.e., clusters —or trajectories— of multi-channel sequences that share similar temporal evolutions (or similar trajectories). To illustrate this method we present a step-by-step tutorial in R that analyses students’ sequences of online engagement and academic achievement, exploring their longitudinal association. We cover two approaches for clustering multi-channel sequences: one based on using distance-based algorithms, and the other employing mixture hidden Markov models inspired by recent research.
learning analyticsmultichannel sequence analysismixture hidden markov modelsmultimodal data
Murphy K., López-Pernas S., Saqr M. (2024). Dissimilarity-based Cluster Analysis of Educational Data: A Comparative Tutorial using R. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_8.

Clustering is a collective term which refers to a broad range of techniques aimed at uncovering patterns and subgroups within data. Interest lies in partitioning heterogeneous data into homogeneous groups, whereby cases within a group are more similar to each other than cases assigned to other groups, without foreknowledge of the group labels. Clustering is also an important component of several exploratory methods, analytical techniques, and modelling approaches and therefore has been practiced for decades in education research. In this context, finding patterns or differences among students enables teachers and researchers to improve their understanding of the diversity of students —and their learning processes— and tailor their supports to different needs. This chapter introduces the theory underpinning dissimilarity-based clustering methods. Then, we focus on some of the most widely-used heuristic dissimilarity-based clustering algorithms; namely, K-Means, K-Medoids, and agglomerative hierarchical clustering. The K-Means clustering algorithm is described including the outline of the arguments of the relevant R functions and the main limitations and practical concerns to be aware of in order to obtain the best performance. We also discuss the related K-Medoids algorithm and its own associated concerns and function arguments. We later introduce agglomerative hierarchical clustering and the related R functions while outlining various choices available to practitioners and their implications. Methods for choosing the optimal number of clusters are provided, especially criteria that can guide the choice of clustering solution among multiple competing methodologies —with a particular focus on evaluating solutions obtained using different dissimilarity measures— and not only the choice of the number of clusters for a given method. All of these issues are demonstrated in detail with a tutorial in R using a real-life educational data set.
agglomerative hierarchical clusteringaverage silhouette widthdissimilarity-based clusteringk-meansk-medoidsleaning analytics
Saqr M. (2024). Temporal Network Analysis: Introduction, Methods, and Analysis with R. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_17.

Learning involves relations, interactions and connections between learners, teachers and the world at large. Such interactions are essentially temporal and unfold in time. Yet, researchers have rarely combined the two aspects (the temporal and relational aspects) in an analytics framework. Temporal networks allow modeling of the temporal learning processes i.e., the emergence and flow of activities, communities, and social processes through fine-grained dynamic analysis. This can provide insights into phenomena like knowledge co-construction, information flow, and relationship building. This chapter introduces the basic concepts of temporal networks, their types and techniques. A detailed guide of temporal network analysis is introduced in this chapter, that starts with building the network, visualization, mathematical analysis on the node and graph level. The analysis is performed with a real-world dataset. The discussion chapter offers some extra resources for interested users who want to expand their knowledge of the technique.
temporal networkstemporal analysissocial network analysislearning analytics
Saqr M., Beck E., López-Pernas S. (2024). Psychological Networks: A Modern Approach to Analysis of Learning and Complex Learning Processes. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_19.

In the examination of psychological phenomena within educational environments, a multitude of variables come into play, and these variables have the potential to interact with, trigger, and exert influence on one another. To grasp the intricate dependencies among these variables, investigating the linear associations between each variable pair is not enough. Instead, this complexity demands the application of more advanced techniques that capture the full spectrum of interactions between these variables. One of such techniques is psychological networks. In contrast to social networks, where nodes typically represent individuals and edges signify their interactions or relationships, psychological networks differ in that the nodes represent observed psychological variables, and the edges denote the statistical relationships between them. This chapter serves as an introduction to psychological networks within educational research, offering a tutorial on their estimation, visualization, and interpretation using the R programming language.
psychological networkspartial correlation networkscomplex systemslearning analytics
Saqr M., López-Pernas S., Conde M. Á., Hernández-García Á. (2024). Social Network Analysis: A Primer, a Guide and a Tutorial in R. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_15.

This chapter introduces the concept and methods of social network analysis (SNA) with a detailed guide to analysis with real world data using the R programming language. The chapter first introduces the basic concepts and types of networks. Then the chapter goes through a detailed step by step analysis of networks, computation of graph level measures as well as centralities with a concise interpretation in a collaborative environment. The chapter concludes with a discussion of network analysis, next steps as well as a list of further readings.
learning analyticssocial network analysiscentrality measurestutorial
Saqr M., López-Pernas S., Helske S., Durand M., Murphy K., Studer M., Ritschard G. (2024). Sequence Analysis in Education: Principles, Technique, and Tutorial with R. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_10.

Sequence analysis is a data mining technique that is increasingly gaining ground in learning analytics. Sequence analysis enables researchers to extract meaningful insights from sequential data, i.e., to summarize the sequential patterns of learning data and classify those patterns into homogeneous groups. In this chapter, readers will become familiar with sequence analysis techniques and tools through real-life step-by-step examples of sequential trace log data of students’ online activities. Readers will be guided on how to visualize the common sequence plots and interpret such visualizations. An essential part of sequence analysis is the discovery of patterns within sequences through clustering techniques. Therefore, this chapter will demonstrate the various sequence clustering methods, calculator of cluster indices, and evaluation of clustering results.
sequence analysissequence mininglearning analytics
Saqr M., Schreuder M. J., López-Pernas S. (2024). Why educational research needs a complex system revolution that embraces individual differences, heterogeneity, and uncertainty. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_22.

Whereas the field of learning analytics has matured, several methodological and theoretical issues remain unresolved. In this chapter, we discuss the potentials of complex systems as an overarching paradigm for understanding the learning process, learners and the learning environments and how they influence learning. We show how using complex system methodologies open doors for new possibilities that may contribute new knowledge and solve some of the unresolved problems in learning analytics. Furthermore, we unpack the importance of individual differences in advancing the field bringing a much-needed theoretical perspective that could help offer answers to some of our pressing issues.
learning analyticscomplex systemsindividual differences
Scrucca L., Saqr M., López-Pernas S., Murphy K. (2024). An Introduction and R Tutorial to Model-based Clustering in Education via Latent Profile Analysis. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_9.

Heterogeneity has been a hot topic in recent educational literature. Several calls have been voiced to adopt methods that capture different patterns or subgroups within students’ behavior or functioning. Assuming that there is “an average” pattern that represents the entirety of student populations requires the measured construct to have the same causal mechanism, same development pattern, and affect students in exactly the same way. Using a person-centered method (finite Gaussian mixture model or latent profile analysis), the present tutorial shows how to uncover the heterogeneity within engagement data by identifying three latent or unobserved clusters. This chapter offers an introduction to the model-based clustering that includes the principles of the methods, a guide to choice of number of clusters, evaluation of clustering results and a detailed guide with code and a real-life dataset. The discussion elaborates on the interpretation of the results, the advantages of model-based clustering as well as how it compares with other methods.
gaussian mixture modellatent profile analysismodel-based clusteringlearning analytics
Tikka S., Kopra J., Heinäniemi M., López-Pernas S., Saqr M. (2024). Getting started with R for Education Research. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_3.

The R programming language has become a popular tool for conducting data analysis in the field of learning analytics. This chapter provides an introduction to the basics of R programming, with a focus on the Rstudio integrated development environment and the tidyverse programming paradigm. The chapter covers topics such as data types and structures, control structures, pipes, functions, loops, and input/output operations. By the end of the chapter, readers should have a solid understanding of the basics of R programming and have the tools necessary to learn more in-depth topics such as data wrangling and basic statistics using R.
r programmingr studiolearning analytics
Tikka S., Kopra J., Heinäniemi M., López-Pernas S., Saqr M. (2024). Introductory Statistics with R for Educational Researchers. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_5.

Statistics play a fundamental role in learning analytics, providing a means to analyze and make sense of the vast amounts of data generated by learning environments. This chapter provides an introduction to basic statistical concepts using R and covers topics such as measures of central tendency, variability, correlation, and regression analysis. Specifically, readers will learn how to compute descriptive statistics, conduct hypothesis tests, and perform simple linear regression analysis. The chapter also includes practical examples using realistic data sets from the field of learning analytics. By the end of the chapter, readers should have a solid understanding of the basic statistical concepts and methods commonly used in learning analytics, as well as a practical understanding of how to use R to conduct statistical analysis of learning data.
statisticslearning analyticshypothesis testing
Vogelsmeier L. V. D. E., Saqr M., López-Pernas S., Jongerling J. (2024). Factor Analysis in Education Research using R. Learning analytics methods and tutorials: A practical guide using R (in-press). doi: 10.1007/978-3-031-54464-4_20.

Factor analysis is a method commonly employed to reduce a large number of variables into fewer numbers of factors. The method is often used to identify which observable indicators are representative of latent, not directly-observed constructs. This is a key step in developing valid instruments to assess latent constructs in educational research (e.g., student engagement or motivaion). The chapter describes the two main approaches for conducting factor analysis in detail and provides a tutorial on how to implement both techniques with the R programming language. The first is confirmatory factor analysis (CFA), a more theory-driven approach, in which a researcher actively specifies the number of underlying constructs as well as the pattern of relations between these dimensions and observed variables. The second is exploratory factor analysis (EFA), a more data-driven approach, in which the number of underlying constructs is inferred from the data, and all underlying constructs are assumed to influence all observed variables (at least to some degree).
factor analysisexploratory factor analysisconfirmatory factor analysislearning analytics
Sointu E., Väisänen S., Hirsto L., Paavilainen T., Saqr M., López-Pernas S., Valtonen T. (2024). Creatively Opening the Constraints of Learning Analytics in Inclusive, Elementary School-Level STEAM Education. Creativity Theory and Action in Education, vol. 8(in-press).

Learning analytics has been a topic of great interest to researchers and practitioners over the past decade. The challenge, however, is translating rich data into practice. Moreover, learning analytics in inclusive elementary school settings has been little studied. This article discusses using learning analytics in elementary school STEAM education, focusing on outer space content to address this gap. We adopted a collaborative approach in preservice teacher training with in-service teachers and researchers in the university practice school setting. Fifty-two 11–13-year-old students from the Finnish school context participated in this study. We used a process-oriented approach with sequence and process mining for data analysis. The results showed little to no difference between students with and without pedagogical support. Technologies such as learning analytics in various learning management systems can present both opportunities and challenges for students with support needs. However, this study’s results challenge the assumption that students with support needs in inclusive settings are less able to work independently and find coherent learning strategies in digital learning environments.
learning analyticselementary schoolinclusive educationsteamsupport

2023

Saqr M., López-Pernas S., Apiola M. (2023). Capturing the Impact and the Chatter Around Computing Education Research Beyond Academia in Social Media, Patents, and Blogs. Past, Present and Future of Computing Education Research: A Global Perspective. Springer, pp. 171-191. doi: 10.1007/978-3-031-25336-2_9.

Research impact goes beyond academia and exists in the multiplicity of digital platforms that we use to read, share, and discuss knowledge. Computing education research (CER) is no exception: it is created in academia and typical research institutions but is talked about widely on social media, blogs, and news websites. The aim of this study is to have a comprehensive analysis of how research in CER has been received, talked about in social media, discussed on blogs, and spread to the news and media. In addition to common analysis of trends of growth, we analyze trends of usage of social media and quantitative analysis of platforms, articles, and venues. The analysis also includes which articles and in which subfields had a wide impact, and for whom (i.e., which platforms had more impact). The results show that Altmetrics adoption is weak, yet increasingly growing fast. Gender and diversity issues made it to popular news sites, e.g., Scientific American, Los Angeles Times, and Christian Science Monitor, while articles about ethics, programming education, introductory courses as well as computational thinking and inclusion have captured the attention of social media users. There was weak—or no—correlation between article, author or topic impact and the traditional impact measures, e.g., citation count.
theoretical computer sciencemedia studiescomputing educationcomputer science educationaltmetricbibliometricsscientometricssocial mediatwitter
López-Pernas S., Saqr M., Apiola M. (2023). Scientometrics: A Concise Introduction and a Detailed Methodology for Mapping the Scientific Field of Computing Education Research. Past, Present and Future of Computing Education Research: A Global Perspective. Springer, pp. 79-99. doi: 10.1007/978-3-031-25336-2_5.

Scientometrics has emerged as a research field for the evaluation and mapping of scientific fields, exploring research themes, collaboration clusters and identifying gaps and future trends. While early implementations have focused on quantitative metrics, recent directions emphasize a more nuanced approach that combines qualitative methods with quantitative analysis that triangulates several aspects, e.g., temporal trends, network, and semantic analysis. This chapter reviews scientometrics as a research methodology and discusses the strengths and weaknesses and how such weaknesses can be amended. The chapter also discusses the main methodological approach, and its theoretical underpinnings, used in some of the book chapters that make use of scientometrics as a means to map the field of CER.
scientometricscomputing education researchbibliometrics
Apiola M., Saqr M., López-Pernas S. (2023). The Hands that Made Computing Education Research: Top Authors, Networks, Collaboration and Newcomers. Past, Present and Future of Computing Education Research: A Global Perspective, Springer, pp. 101-119. doi: 10.1007/978-3-031-25336-2_6.

Computing Education Research (CER), like any other discipline of science, is fundamentally driven by people and their networks of collaboration. Previous research has analysed authorship patterns and collaboration in distinct well known dissemination outlets of CER. In this chapter, we add to that approach by analysing a comprehensive set of metadata of CER publications. We analyse author productivity including newcomer patterns, clusters of co-authorship and international collaboration, and authors who build bridges between communities. Our results reveals top authors and their production before and after 2000, clusters of collaborators and their areas of topics as revealed by top keywords, a healthy evolution of newcomer-patterns, and a set of authors who build bridges between communities. In all, our macro-level analysis adds a significant contribution to our understanding of the role of authors in the evolution of CER.
computing education researchauthorsscientometrics
Apiola M., Saqr M., López-Pernas S. (2023). The Evolving Themes of Computing Education Research: Trends, Topic Models, and Emerging Research. Past, Present and Future of Computing Education Research: A Global Perspective, Springer, pp. 151-169. doi: 10.1007/978-3-031-25336-2_8.

A combined body of reviews, meta-research and other analyses demonstrates the evolution of computing education research (CER) through the decades with experience reports evolving to empirical research, increased attention paid to educational research, methods and reporting rigor. Previous analyses of CER publications show the sustained focus of CER on programming education, which has, by far, been the all-time most popular topic in CER. In the recent decade, other top researched areas include K-12 computing education and computational thinking. In this chapter, we add new insights to the top research areas of CER. We followed the PRISMA-S (Preferred Reporting Items for Systematic reviews and Meta-Analyses) literature search extension to capture the relevant literature on CER. The process of data retrieval, screening, and pre-processing resulted in a total of 16,863 articles included in the dataset. We use a combination of keyword analysis and structural topic modeling, and introduce a model of 29 topics. We also introduce emerging topics in recent years through an analysis of emerging common words in abstracts and titles during recent years. The results paint a unique picture about the dominating and trending research areas of CER, and of how common research topics are connected with each other. The analysis also reveals under-researched areas of CER.
computing education researchscientometricskeywords
López-Pernas S., Apiola M., Saqr M., Pears A., Tedre M. (2023). A Scientometric Perspective on the Evolution of the SIGCSE Technical Symposium: 1970–2021. Past, Present and Future of Computing Education Research: A Global Perspective, Springer, pp. 193-212. doi: 10.1007/978-3-031-25336-2_10.

ACM’s Special Interest Group on Computer Science Education’s (SIGCSE) Technical Symposium is considered by many to be ACM’s flagship conference for computing educators. SIGCSE Technical Symposium has been held annually since 1970 as an in-person conference, with the exception of 2020, when it was cancelled (with some papers presented in 2021). The conference attracts many computing educators, numbering in the thousands in recent times and is by far the top publication outlet in computer science education with regards to number of published articles. This chapter explores the evolution of the first 51 years of SIGCSE from its inception in 1970 to the present day, using primarily scientometric data. We explore the evolution of the SIGCSE conference with regards to shifts in research themes, influential authors, author networks and clusters of keywords. We also explore the potential for internationalization of the conference. Participation in the SIGCSE symposium has strong US roots, and we examine the impact on participation as ACM SIGCSE membership expanded to Europe and Australasia, and new conferences such as ACE, ICER and Koli Calling established themselves.
computing education researchscientometrics
Apiola M., López-Pernas S., Saqr M., Malmi L., Daniels M. (2023). Exploring the Past, Present and Future of Computing Education Research: An Introduction. Past, Present and Future of Computing Education Research: A Global Perspective, Springer, pp. 1-7. doi: 10.1007/978-3-031-25336-2_1.

This chapter is an introduction to the book “Past, Present and Future of Computing Education Research: A Global Perspective.” This book uses a mixture of scientometrics, meta-research and case studies to offer a new view about the evolution and current state of computing education research (CER) as a field of science. In its 21 chapters, this book presents new insights of authors, author communities, publication venues, topics of research, and of regional initiatives and topical communities of CER. This chapter presents an overview of the contents of this book.
Dagienė V., Gülbahar Y., Grgurina N., López-Pernas S., Saqr M., Apiola M., Stupurienė G. (2023). Computing Education Research in Schools. Past, Present and Future of Computing Education Research: A Global Perspective, Springer, pp. 481-520. doi: 10.1007/978-3-031-25336-2_20.

One of the most researched domains of computing education research (CER) that attracts attention is computing education in schools, starting from pre-primary level up to upper secondary level (K-12). A high number of initiatives and related research contributions have appeared over half a century of computing history in schools. This chapter presents an overview of CER in the K-12 domain, including globally influential movements such as that of Logo pedagogy, constructionism, inquiry based learning or computational thinking (CT). Development of CT in education, based on a number of previous reviews on CT and K-12, paints a diverse picture of the approaches, educational technologies, pedagogical innovations, and related challenges such as lack of teacher training or shortage of learning resources. This article presents also a scientometric overview of CER research in the K-12 domain. The analysis identifies the top topics of research, and foundational articles. While much of the research is centered around the US, key research from other parts of the globe is also highlighted. Emergence of new trends such as teaching artificial intelligence and machine learning in schools are also discussed.
Becker B.A., Bradley S., Maguire J., Black M., Crick T., Saqr M., Sentance S. & Quille K. (2023). Computing Education Research in the UK & Ireland. Past, Present and Future of Computing Education Research: A Global Perspective, Springer, pp. 421-479. doi: 10.1007/978-3-031-25336-2_19.

In 1970, SIGCSE had five members from England, Scotland and Wales. By 1983 Ireland had a member, followed by Northern Ireland in 1995. Well before then researchers from these countries had contributed to the growing CER community. In 1998 the 3rd ACM ITiCSE conference was held in Ireland. Since then researchers in these countries have contributed to advancing CER regionally and globally, hosting numerous ITiCSE and ICER conferences and spawning several influential research projects and groups. In the last decade two ACM SIGCSE Chapters (UK, and Ireland) were established along with two annual conferences: Computing Education Practice (CEP), and the UK & Ireland Computing Education Research conference (UKICER) now in their 7th and 4th years. In 2022 ITiCSE returned to Ireland. This chapter describes more than 50 years of history and growth of CER within, and to come out of, the UK and Ireland, along with a scientometric study of research outputs.
Agbo F.J., Ntinda M., López-Pernas S., Saqr M., Apiola M. (2023). Computing Education Research in the Global South. Past, Present and Future of Computing Education Research: A Global Perspective, Springer, pp. 311-333. doi: 10.1007/978-3-031-25336-2_15.

Scientometric analyses of publication data from all major computing education research (CER) outlets show that many countries and whole continents are greatly underrepresented on the global map of contributions to CER. For example, only a minor portion of CER has originated from countries in the Global South (GS) or has addressed challenges of computing education in the GS. In this chapter, we shift the focus to scientometrically analyse CER papers that originate from countries in the GS. From the metadata of all CER publications in central publication outlets of CER, we have selected a subset of articles with authors affiliated to an institution in a GS Country, as defined by the United Nations (UN). The analysis shows publication trends, prolific authors, and country collaboration patterns. A number of crucial and interesting avenues for future research and collaboration are presented.
Apiola M., López-Pernas S., Saqr M. (2023). The Venues that Shaped Computing Education Research: Dissemination Under the Lens. Past, Present and Future of Computing Education Research: A Global Perspective, Springer, pp. 121-150. doi: 10.1007/978-3-031-25336-2_7.

A fundamental part of science is dissemination. Previous research has analysed dissemination practices in few of the central publication outlets. In this chapter, we add into that research by analysing dissemination practices of CER in a range of 1523 journals and conference series that are (a) exclusively dedicated to CER, and (b) in outlets that publish CER together with other topics, such as general education, engineering or computer science. Our results show that a small and highly dedicated core of venues publishes a remarkable share of CER, that CER has a conference-oriented publication tendency, significant variations in citation rates, and differences in the diversity of topics within and between the dedicated and non-dedicated publication outlets. In all, our macro-level analysis makes a significant contribution into the research of dissemination in CER.

2022

Malmberg J., Saqr M., Järvenoja H., Haataja E., Pijeira-Díaz H.J., Järvelä S. (2022). Modeling the complex interplay between monitoring events for regulated learning with psychological networks. The Multimodal Learning Analytics Handbook, Springer, pp. 79-104. doi: 10.1007/978-3-031-08076-0_4.

This chapter builds on the notion that multimodal learning analytics (MMLA) has potential to offer new innovations for the field of education and can be gradually implemented to provide constitutive explanations to improve individual student regulation. We will ground the chapter on the theoretical framework of self-regulated learning (SRL), addressing especially the pivotal role of metacognitive monitoring as a part of regulated learning cycle. Hence, we will introduce the relationship between physiological arousal and monitoring. In particular, we will demonstrate via empirical examples how network analysis methods can provide new insights on the complex interrelations of how monitoring manifests in interactions and evolves across phases of SRL at the a) group level, b) temporal level, and c) individual student level. We conclude the chapter by discussing the prospects, possibilities and challenges of the network analysis as a methodological approach for SRL research.
self-regulated learningmultimodal learning analyticspsychological networks

Editorials

2024

Saqr M., López-Pernas S., Conde M.Á., Pavlović O., Raspopović Milić M. (2024). Preface: The Critical Challenges of Artificial Intelligence in Education. Proceedings of the 14th International Conference on eLearning (eLearning-2023) (CEUR Workshop Proceedings) (in-press).

2023

Hirsto L., López-Pernas S., Saqr M., Valtonen T., Sointu E., Väisänen S. (2023). Bridging Education Learning Analytics and AI: Challenges of the Present and Thoughts for the Future. Proceedings of the 1st Finnish Learning Analytics and Artificial Intelligence in Education Conference (FLAIEC 2022) (CEUR Workshop Proceedings), vol. 3383, pp. 1-6. https://ceur-ws.org/Vol-3383/FLAIEC22_preface.pdf.
Elmoazen R., López-Pernas S., Misiejuk K., Khalil M., Wasson B. , Saqr M. (2023). Reflections on Technology-enhanced Learning in Laboratories: Barriers and Opportunities. Proceedings of the Technology-Enhanced Learning in Laboratories Workshop (TELL 2023) (CEUR Workshop Proceedings), vol. 3393, pp. 1-4. https://ceur-ws.org/Vol-3393/TELL23_preface.pdf.

This preface discusses the potential of virtual labs (VLs) as a flexible and immersive alternative to traditional physical labs on the light of Technology-Enhanced Learning in Laboratories workshop (TELL 2023), which features seven papers showing various methodologies and perspectives on using VLs. The papers cover topics such as VLs in biomedicine, students' perception of VLs, and collaborative learning in VLs, and examine challenges and barriers to VL accessibility. We discuss some of the main findings of the papers, such as the potential of digital applications and online materials to enhance digital teaching and the importance of developing strategies to enhance team-based learning through encouraging students to reflect on their own work. Overall, the preface demonstrates the potential of VLs to enhance students' practical skills, and learning outcomes, with insights into the challenges and barriers to VLs accessibility.
virtual labsdigital labslearning analyticsonline learningcollaborative learning
Conde M.Á., López-Pernas S., Saqr M., Raspopovic-Milic M. (2023). Addressing the complexity of online education: A learning analytics and big data perspective. Proceedings of the 13th International Conference on eLearning (eLearning-2022), vol. 3454(in-press), pp. 1-2. https://ceur-ws.org/Vol-3454/preface.pdf.

2022

Saqr M. (2022). Is GDPR failing? a tale of the many challenges in interpretations, applications, and enforcement. International Journal of Health Sciences, vol. 16(5), pp. 1-2. https://pubmed.ncbi.nlm.nih.gov/36101846.

2021

Saqr M., López-Pernas S. (2021). Toward self big data. International Journal of Health Sciences, vol. 15(5). https://pubmed.ncbi.nlm.nih.gov/34548857.

2020

Saqr M., Wasson B. (2020). COVID-19: Lost Opportunities and Lessons for the Future. International Journal of Health Sciences, vol. 14(3), pp. 4-6. https://pubmed.ncbi.nlm.nih.gov/32536841.

2019

Saqr M., Tedre M. (2019). Should we teach computational thinking and big data principles to medical students?. International Journal of Health Sciences, vol. 13(4), pp. 1-2. https://pubmed.ncbi.nlm.nih.gov/31341448.

2018

Saqr M. (2018). Beyond the sophomoric promises of artificial intelligence in medicine. International Journal of Health Sciences, vol. 12(2), pp. 1-2.

2017

Saqr M. (2017). Assessment analytics: The missing step. International Journal of Health Sciences, vol. 11(1), pp. 1-2. https://pubmed.ncbi.nlm.nih.gov/28293161/.
Saqr M. (2017). Big data and the emerging ethical challenges. International Journal of Health Sciences, vol. 11(4), pp. 1-2. https://pubmed.ncbi.nlm.nih.gov/29085259.

Before 2017

Saqr M. (2015). Learning Analytic and Medical Education. International Journal of Health Sciences, vol. 9(4), pp. 4-5. doi: 10.12816/0031225.

Other publications

Saqr M., López-Pernas S. (2023). The idiographic paradigm shift needed: Bringing the person back into research and practice. Proceedings of the 1st Finnish Learning Analytics and Artificial Intelligence in Education Conference (FLAIEC 2022) (CEUR Workshop Proceedings), vol. 3383, pp. 116. https://ceur-ws.org/Vol-3383/FLAIEC22_paper_6676.pdf.

Self-efficacy, self-directedness, self-regulation, autonomy, and self-control –inter alia– have all been around for decades. While such concepts may differ, they share a quintessential element which is the “self” component. As we currently stand, we are not short of theories around the importance of the “self” or the student who has become the center of all initiatives for improving education over the last decades. Furthermore, we have a vast count of empirical studies, systematic reviews, and meta-analyses of intervention that promise meaningful intervention. Yet, the quest to bring real changes on the ground has so far fell short of promise. The reasons pertain to the discord between how research is conducted, assessed, or applied. While we –theoretically– embrace and emphasize the value of the “self” or “student” as a central point of departure from existing methods or theories, research is conducted by using data from a “group” of many others. That is, data are collected from a sample of students to explore their inter-individual differences and their average behavior to derive generalizable laws or norms. Such norms are expected to apply to everyone, and the lessons learnt from studying others are expected to be generalizable. Nonetheless, such group data, are barely –if at all– represent any single person, ergo a paradigm shift is needed to bring the very person into our approach to research and practice. We show how data can be collected to model the within-person behavior and learning process. Such analysis is more representative of the “self”, offer more valid inferences about the personal processes and a better potential for personalizing and adapting education.
learning analyticsidiographicwithin-person
López-Pernas S., Gordillo A., Barra E., Saqr M. (2023). Game learning analytics: The case of online educational escape rooms. Proceedings of the 1st Finnish Learning Analytics and Artificial Intelligence in Education Conference (FLAIEC 2022) (CEUR Workshop Proceedings), vol. 3383, pp. 121-122. https://ceur-ws.org/Vol-3383/FLAIEC22_paper_7043.pdf.

Educational escape rooms are team-based learning activities in which students solve puzzles related to a certain subject to accomplish a final goal (usually escaping from a room) [1]. These activities have proven capable of increasing students’ knowledge in a variety of subjects and contexts while improving motivation and engagement. A key characteristic of educational escape rooms is that they are time-constrained. Therefore, if students do not complete all the puzzles in time they will not gain exposure to part of the learning materials in the activity. As such, it is crucial to provide timely support to students to prevent them from getting stuck and frustrated, and ensuring they progress through the activity. However, providing such support can be challenging for instructors since they often have to monitor several students at the same time, which becomes even harder in online teaching environments [2]. The Escapp platform [3] provides a solution for this challenge. Escapp is a web platform that allows to conduct online educational escape rooms. Besides providing all the features needed for instructors to set up their escape rooms both online or face-to-face, Escapp provides a learning analytics dashboard that allows to closely monitor students while they play, enabling the detection of lagging players and the provision of hints to help them advance through the escape room. The Escapp platform has been used to conduct several escape rooms at Universidad Politécnica de Madrid [4]–[8] where the learning analytics dashboard has been used to detect lagging students and to optimize the game design. In this presentation, we will show an example of one of these educational escape rooms and how the learning analytics dashboard has played a crucial role in the correct development of the activity. We will discuss current and potential uses of the dashboard and of the data collected from the students. Our goal is to offer an innovative perspective on learning analytics and how they can be adapted to the specific learning scenario of educational escape rooms.
learning analyticsgame learning analyticseducational escape roomsdashboardgame-based learning

Tedre M., Saqr M., Kliukina S., Ofir Z. (2021). End of Programme Evaluation of Sida’s Support to The World Academy of Science (TWAS), 2017–2021. SIDA. https://cdn.sida.se/app/uploads/2021/09/01125941/DE2021_28_62426en.pdf.

Saqr M., Joshi S., Perfumi S., Casu O., Sotirchos D., Ekenberg L., Komendantova N., Kyza E., Panos D., Karapanos E. (2018). Co-Inform: Co-Creating Misinformation-Resilient Societies: Pilot Requirements and Service Design. International Institute for Applied Systems Analysis (IIASA). https://pure.iiasa.ac.at/15660.
Joshi S., Koulolias V., Sjösberg C., Perfumi S., Saqr M., Petritsopoulou M., Casu O., Sotirchos D., Kyza E., Shah S. (2018). Co-creation framework–building a sustainable ecosystem. International Institute for Applied Systems Analysis (IIASA). https://pure.iiasa.ac.at/15590.

Thesis

Saqr M. (2018). Using Learning Analytics to Understand and Support Collaborative Learning. Stockholm University. http://www.diva-portal.org/smash/record.jsf?pid=diva2%3A1245435&dswid=3586.

Learning analytics (LA) is a rapidly evolving research discipline that uses insights generated from data analysis to support learners and optimize both the learning process and learning environment. LA is driven by the availability of massive data records regarding learners, the revolutionary development of big data methods, cheaper and faster hardware, and the successful implementation of analytics in other domains. The prime objective of this thesis is to investigate the potential of learning analytics in understanding learning patterns and learners’ behavior in collaborative learning environments with the premise of improving teaching and learning. More specifically, the research questions comprise: How can learning analytics and social network analysis (SNA) reliably predict students’ performance using contextual, theory-based indicators, and how can social network analysis be used to analyze online collaborative learning, guide a data-driven intervention, and evaluate it. The research methods followed a structured process of data collection, preparation, exploration, and analysis. Students’ data were collected from the online learning management system using custom plugins and database queries. Data from different sources were assembled and verified, and corrupted records were eliminated. Descriptive statistics and visualizations were performed to summarize the data, plot variables’ distributions, and detect interesting patterns. Exploratory statistical analysis was conducted to explore trends and potential predictors, and to guide the selection of analysis methods. Using insights from these steps, different statistical and machine learning methods were applied to analyze the data. The results indicate that a reasonable number of underachieving students could be predicted early using self-regulation, engagement, and collaborative learning indicators. Visualizing collaborative learning interactions using SNA offered an easy-to-interpret overview of the status of collaboration, and mapped the roles played by teachers and students. SNA-based monitoring helped improve collaborative learning through a data-driven intervention. The combination of SNA visualization and mathematical analysis of students’ position, connectedness, and role in collaboration was found to help predict students’ performance with reasonable accuracy. The early prediction of performance offers a clear opportunity for the implementation of effective remedial strategies and facilitates improvements in learning. Furthermore, using SNA to monitor and improve collaborative learning could contribute to better learning and teaching.
learning analyticssocial network analysiscollaborative learningmedical educationinteraction analysismachine learning