Multi-Agent System Observer: Intelligent Support for Engaged E-Learning

Abstract

Moodle is a widely deployed distance learning platform that provides numerous opportunities to enhance the learning process. Moodle’s importance in maintaining the continuity of education in states of emergency and other circumstances has been particularly demonstrated in the context of the COVID-19 virus’ rapid spread. However, there is a problem with personalizing the learning and monitoring of students’ work. There is room for upgrading the system by applying data mining and different machine-learning methods. The multi-agent Observer system proposed in our paper supports students engaged in learning by monitoring their work and making suggestions based on the prediction of their final course success, using indicators of engagement and machine-learning algorithms. A novelty is that Observer collects data independently of the Moodle database, autonomously creates a training set, and learns from gathered data. Since the data are anonymized, researchers and lecturers can freely use them for purposes broader than that specified for Observer. The paper shows how the methodology, technologies, and techniques used in Observer provide an autonomous system of personalized assistance for students within Moodle platforms.

1. Introduction

With the implementation of new technologies in educational systems, the goal is to increase the quality of teaching, that is, to improve the process of students’ acquisition of knowledge [1]. Most educational institutions today use the internet in the learning process [2], and virtual teaching is becoming part of the program, even of institutions known for a long, campus-based tradition [3]. On the other hand, the current example of the rapid spread of the COVID-19 virus puts into focus the significant advantage of the application of distance learning, which is the continuity of the teaching process in different emergencies.

Contemporary distance learning has, above all, enabled students to access teaching content on their plan, independently deciding on the schedule and how much time they will spend learning [4]. The most common way that teaching is provided to students via the internet and other computer networks is learning management systems (LMSs) [2]. LMSs offer lecturers the opportunity to distribute information to students, produce teaching materials, prepare assignments and tests, initiate discussions, manage distance learning classes, and facilitate learning through collaboration through forums [5].

Modular Object-Oriented Dynamic Learning Environment, abbreviated as Moodle, is the most used and widespread LMS solution [4]. Although LMS Moodle has many features that help lecturers create and manage courses available through a computer network, it does not meet students’ individual needs [6]. Artificial intelligence support for lecturers and students could provide what Moodle lacks [7]. Learning is a cognitive activity that varies from student to student, and tailoring e-learning to students requires modeling their cognitive characteristics, especially learning styles, as the most explored cognitive traits [8]. Data learning techniques can be applied in distance learning systems [9], aiming to study user behavior, perform behavioral assessment and improve the system to support the user [5].

The Observer system presented in this paper is a multi-agent system that enhances the Moodle platform into intelligent tutoring systems (ITS). Such systems try to understand the process of individual learning. They are tasked with building an appropriate data structure to represent the student’s cognitive characteristics as realistically as possible [1]. The Observer system is somewhat different because it focuses on a single learning style, and it is an engaged learning style. It monitors engaged learning indicators continuously, drawing students’ attention to whether they are on track to complete the course successfully or need additional effort.

Engaged learning can be implemented without the use of technology, but technology facilitates engagement in a way that is difficult to achieve otherwise [10]. Research related to engaging learning is focused on the student’s activities, his/her involvement, and his/her efforts in achieving academic success [3]. A specific study presented in [2], which compared 17 blended courses using Moodle LMS, showed that the number of clicks, time spent in interaction with distributed teaching material, and an overall number of visited pages significantly increased the positive effect to the final grade in the exam. Having that in mind, the idea behind the implementation of the Observer system was to provide information to students about their progress and current level of engagement by comparing previous measurements with current ones, as well as with the indicators of measurement of other students in the same course.

What sets Observer apart from most LMS learning support systems is the data source. Although Moodle keeps detailed logs regarding the use of systems to track what materials a student has accessed [5], the Observer system uses data that it collects, anonymizes, and stores on its own. In this way, a large amount of information is provided that can extend the system’s purpose shown in this paper. Additionally, Observer enables the creation of a centralized system that can connect an arbitrary number of Moodle platforms of different organizations, thereby enhancing its data learning capabilities as well as third-party analytic capabilities.

The rest of the paper is organized as follows: A brief overview of related work is presented in Section 2, emphasizing the difference between them and our approach. In Section 3, we discuss engaged learning from the perspective of the basis for ITS development. Section 4 presents the proposed multiagent system Observer, while in Section 5, we describe the technologies used to develop it. Finally, in Section 6 and Section 7, we summarize our findings, draw some conclusions, and state directions for future work.

2. Related Work

MIMLE is a learning system presented in [1] that uses one of the diagnostic techniques of an ITS system called model tracing. It offers students theorems and definitions through a help window to help solve a given problem. The applicable agent uses the Markov process of deciding whether or not a help window should appear to minimize disruption to the student. It is a reflex agent that makes decisions based on counting correct and incorrect answers and measuring a student’s reaction time. The Observer system tries to be less disruptive to the student but with a different approach than MIMLE, choosing the particular Moodle interface item called a block.

One of the solutions implemented using the Moodle system is presented in [7]; it is a multi-agent system based on the Java Agent Development Environment (JADE). Based on the information collected from the Moodle databases, the system provides information to students and lecturers about student activity. For example, the number of posts on each student’s forum, whether there are students who have not posted yet, whether there are any disagreements and similar activities. The Observer system’s most significant difference is the data source for analysis since Observer itself collects them and is not connected to the Moodle database.

Predicta presented in [4] is a Java desktop tool for tracking students based on Moodle database records. It selects tables according to user needs and prepares data for analysis in WEKA data learning software, displaying results in the dedicated module. The second component of the system predicts student success, that is, whether there is a risk of failure. Predicta uses the Moodle database, but concerning the LMS, it is the external solution itself as well as Observer. The difference is that with Observer, the filtering of data is done by lecturers or course creators who choose teaching material or Moodle pages from which information is gathered.

In [6], a nonagent-based example, such as Predicta, is presented, but it supports a specific learning style, as is the case with the Observer system. It is a solution that supports self-regulated learning across all three stages. The planning phase is supported here by choosing a predefined learning profile after which the system offers links to access the learning recommendations in the main menu of the Moodle plug-in. The learning phase provides a service that recommends learning materials (various content, exercises, and tests) that are neither too easy nor too difficult for a particular student. At this stage, self-regulation is based on the student’s free choice among the learning material offered. The reflection process system supports two different tools: the progress rating tool and the visualization tool. Although always present, or to the extent determined by the course creator, the Observer system in the student learning process, using the LMS, participates only in measuring its engagement. Observer predicts the course’s outcome by processing the measuring and gives suggestions while trying to stay unobtrusive.

In order to achieve as little disruption as possible, the Observer system uses a novel approach of agents interacting with students, using the Moodle block. The application of the block itself is innovative since it is used to implement the agent. The application of the block allows the data to be filtered by the creators of the course themselves, choosing positions in the learning material where the block will be placed. The Observer system improves the individual work of students, offering them the opportunity to be informed of, based on indicators of this engaged learning style, whether their activities are within the group of students with a chance for success or they need to invest additional effort. As an additional contribution, the Observer system provides an anonymized dataset for further analysis with a potential connection of an external entity via a web service or creating an analytical solution within the system itself.

3. Theoretical Background

Engaged learning theory represents a fundamental basis for the development of Observer. It is an active student’s cognitive process and involves creating, reasoning, problem-solving, decision making, and evaluating what has been achieved. The theory behind this learning style is based on the idea of creating successful, collaborative teams engaged in ambitious projects whose significance goes beyond classroom frameworks [10]. Before broader acceptance of LMS, student engagement was most often measured by attendance, which is a raw measure of participation because it does not measure quality but has proven to be an essential variable in determining student success [3]. The time that students spend on a particular task, called engagement time, is of great importance for our research also.

Student engagement refers to the degree of attention, interest, and commitment when learning or in class. Students committed to improving their skills and knowledge will undoubtedly be active within the virtual learning environment, such as Moodle, so engagement can be estimated using such systems logs [11]. The Observer system measures learning engagement in a virtual environment based on the number of clicks, time spent on the teaching material, and the number of reviews, all of which have proved to be an important indicator of success [2]. After measurement completion, the system provides suggestions to the student regarding his/her engagement by comparing it with their previous engagement and the engagement of other students.

Concerning engagement indicators, one should note the research [3] conducted on a large sample of students—91,284 who used Blackboard LMS and 1515 who used Moodle—which is considered to be statistically significant. The study found a general correlation between student engagement indicated by clicks and academic performance indicated by grades. However, there is a limitation related to the number of clicks as an indicator of engagement. Although a link to a student’s final success in a particular course exists, its importance to success is not determined.

In applying the indicator, portability is essential, which is related to the question of whether there is a universal set of variables for predicting student success. Studies have shown that the portability of prediction models is not high between different courses. However, a prediction is successful when it comes to a particular course, while grouping data at the student level makes it easier to find portability between different courses [2]. For this reason, during indicators processing, Observer considers all data related to a particular student, i.e., the system user, and the data of other students who work with the same course’s teaching material.

4. Observer ITS System

The Observer system is a multi-agent system that assists students in engaging in learning by using LMS Moodle. The system independently collects student activity data, anonymizes and stores it in its database, prepares to extract engagement learning indicators, creates a training set and data-learning set, and implements two different machine learning algorithms for passing on suggestions to students. It can be used with a specific Moodle platform by educational or other institutions, or it can be the focal point of multiple Moodle platforms.

Software agents can be defined as programs that act on behalf of human beings, resolving inconsistencies, finding and integrating information from different sources, filtering out irrelevant or unwanted information, and adapting them to human needs [7]. Pedagogical agents are autonomous software entities that support the learning process through interaction with students, lecturers, and other participants and in collaboration with other similar agents [12]. The Observer system uses the following: tutoring agents; an agent who collects data, stores them in the system’s internal database, and informs tutoring agents about student suggestions from the system; a data preparation agent; and an agent who learns from the data and forwards the results (as shown in Figure 1).

Agents communicate through a WebSocket protocol that enables two-way communication and has already been used to develop multi-agent environments for this purpose [13]. Different from the usual purpose of communication channels between agents, in the Observer system, instead of conversation using some of the agent languages, only a limited set of data are exchanged through messages, due to the potentially high dynamics in communication.

4.1. Tutoring Agents

The Observer ITS tutoring agent is dedicated to monitor the student’s work, record his/her interaction with the teaching material, and inform students how the system values their efforts. For that purpose, the LMS Moodle elements named blocks are used to implement the agent. The blocks are graphical interface elements added to the left, right, or center column of the Moodle Page. Blocks are placed by the Moodle administrator or course creator on each specific page of teaching material where they deem it necessary or binds the presentation to the block setting’s context, allowing blocks to be placed on multiple pages at once. For an example of the Observer block placed on the LMS Moodle page, see Figure 2.

It should be noticed that some activity modules (e.g., quiz modules) do not allow blocks to be displayed by default. Administrator or course creator could solve the problem in the setup of the quiz module.

Block Observer performs three essential functions from the perspective of the system. The first function is to retrieve the username and email address from the Moodle database and generate hash values, using the MD5 algorithm to obtain the user (student) identification number. This number is linked to all subsequently collected data, which makes them anonymous. Figure 3 shows a section of the block’s PHP and JavaScript program code that provides an identification number and creates a hidden component of an HTML document whose value is student identification. Figure 4 contains the JavaScript code for the module.js file that creates and runs an agent.

The space in the created block, shown in Figure 2, is used by an agent to display the Observer system suggestion. Offering suggestions to the student is another crucial role of the Observer block for the system’s functioning. The third function is to implement agent code and include the agent in the Moodle environment. Opening each new Moodle page that houses the Observer block creates an agent class object.

After creation, the agent starts to collect information, including identification number, the course title, and HTML document title, as well as the time that the agent is created, and sends a message to the agent for data collection and information. In response, the tutoring agent receives a current Observer system suggestion, which then displays via the block. On the creation, the tutoring agent also starts to register students clicks on the Moodle page. After each click, the agent sends a message to the data collection and information agent in the form of a JSON object.

Algorithm 1 shows the process of data collection by a tutoring agent.

Algorithm 1 Data collection by a Tutoring Agent.

Input: The text content of the clicked HTML element

Output: Collected data in JSON format

function createTutoringAgent(t)

type←“open”

uID←getUserId(“observer user”)

pTitle←getTitleFromPage()

sendToDataCollect&InfoAgent(type,t,uID,pTitle)

addClickListener()

end function

function onClick(tText,t)

type ←“click”

uID←getUserId(“observer user”)

pageTitle←getTitleFromPage()

sendMessageToDataColl&InfoAgent(type,t,uID,pTitle,tText)

end function

if Course page is open then

createTutoringAgent()

end if

if Click registered then

onClick(targetText, currentTime)

end if

4.2. Data Collection and Information Agent

There are three agents on the server-side of the system and they have the highest workload in processing the collected information. The first is a data collection and information agent who communicates with the tutoring agents, receives new information from them, and sends suggestions from the system. The message received from the tutoring agent is parsed, after which the JSON object is created.

Agents use objects when storing the collected data in the database. Among the data obtained from the JSON object is time. The timestamp accuracy depends on the settings on the client device itself, which the data collection and information agent could unify by downloading time from the server. The purpose of the time information is not to determine when something happened but decide how long the student stays on a particular page; therefore, accuracy is not necessary.

In addition to storing data, the agent checks in the database for system suggestions intended for the student for whom the particular tutoring agent works. The agent translates the results of machine learning from a numerical value into a human-readable message. The agent also adds information about how old the suggestion is, and both pieces of information are sent back to the tutoring agent.

4.3. Data Preparation Agent

Preparing data for mining involves extracting student engagement indicators based on the information gathered by the tutoring agent. The preparation agent is activated every 10 min and prepares, up to that point, unprocessed data. Figure 5 and Figure 6 show the raw data from the Observer system repository and the data after processing and extracting the engaged learning indicator from the raw data.

For the data preparation process, the relevant information is a raw data record type. The type ”open” suggests opening the page whereby the tutoring agent is created and counting this kind of record system, extracts the first indicator—the number of views. The “click” type marks the data collected after registering the student click and gives another indicator. For the third indicator, the agent uses recorded times. Except for engagement indicators, the agent includes the username, name of the course, and process start time in a prepared set. The information added to the indicators is later used in the data mining process and as references for gaining insight into the results, i.e., informing.

Upon completing the data preparation, the agent forwards a message to the data mining agent containing information about the processing’s start time. The process of data preparation is shown in Algorithm 2.

Algorithm 2 Extraction of student learning indicators by the Data Prep. Agent.

Input: list ← (uID1, t1, type1), (uID2, t2, type2), … (uIDn, tn, typen)

- List of collected data filtered by student uID

Output: Indicators of engaged learning - clicks, time, views

clicks ← 0, views ← 0

for (i ← 0; i < list.length) do

if I = 0 then

tstart ← list [0][2]

user ← list [i][0]

end if

if list[i][3] = “open” then

views++

end if

if list[i][3] = “click” then

clicks++

end if

tend ← list [i][2]

end for

time ← (tend − tstart)

saveIndicatorsForStudent(user, clicks, time, view)

4.4. Data Mining Agent

The work of a data mining agent starts with the message that it receives from the data processing agent. Based on the information about the start of the preparation process contained in the message, the agent recognizes new data and creates a query to the database. The result of the query is a multidimensional array with prepared data and references. Due to the disproportionate value of the indicators in the prepared set (the number of views and clicks can be significantly less than the number of seconds spent using the course material), Observer normalizes the data by the minimum and maximum values method. The agent uses the k-nearest neighbors (k-NN) algorithm to classify the data by calculating the approximate distance between different points on the input vectors and then assigns to the unmarked point the class of its k nearest neighbors. The distance D(i,j) between the samples we determined by calculating the Euclidean distance.

The last activity of a data mining agent is to determine the direction of regression. The purpose of the regression direction, which is determined using the least-squares method, is to include the time-flow component as a vital factor of engaged learning, especially in the case of making too many multi-day breaks between the use of teaching materials distributed through the LMS Moodle. The system uses the implemented regression method’s results to supplement the suggestion with information about the student’s overall engagement. Figure 7 shows the result of the data mining agent’s work, where the current suggestion is obtained with the k-NN algorithm and overall with linear regression.

Algorithm 3 presents the process of forming a suggestion intended for a specific student.

Algorithm 3 Forming a suggestion for a student by Data Mining Agent.

Input: list ← (uID1, clicks1, time1, views1), …(uIDn, clicksn, timen, viewsn)

- obtaining learning indicators of all users in the last ten minutes

Output: A suggestion for a student

allClicks←getClicksSortByNum()

allTimes←getTimesSortByDuration()

allViews←getViewsSortByNum()

(Cnorm, Tnorm, Vnorm) ←minMaxNormalization(allClicks, allTimes, allViews)

trainingSet←getKnnTrainingSet(Cnorm, Tnorm, Vnorm)

for (i← 0; i < list.length) do

currentSugestion←applyKnnlAgorithm(list[i], trainingSet)

prevResults←getPreviousStudentSugestions(list[i][0])

overallSugestion←applyLinearRegression(currentSugestion, prevResults)

createMessageForStudent(currentSugestion, overallSugestion)

end for

5. Implementation of Observer System

For developing the multi-agent Observer, we use JavaScript programming language. The principal reason for choosing JavaScript is related to the concept of monitoring students’ work, which is based on their interaction with the teaching material. Since everything relevant for the Observer system is happening on the Moodle platform’s client-side, JavaScript is the logical solution. JavaScript is the leading scripting language for internet browsers, a de facto standard when it comes to the client-side, and essential for the development of modern web applications [13]. The implementation of the Node.js software environment allows JavaScript to be used on the server-side as well. It is an open-source software based on Google’s V8 core and event-driven architecture that enables asynchronous I/O, as well as creating extensible server applications without using threads [14]. It is efficient and allows the development of web applications that work intensively with real-time data [15]. Unlike traditional, multi-threaded software models, where the thread has to wait for the demanding I/O operation to finish, Node.js can delegate such activities as asynchronous operations, thus not degrading performance [16]. However, the Observer works inside environments with large numbers of students or connecting multiple Moodle platforms. In that case, there could be a blockage of the only thread used by Node.js (looper thread), primarily during the execution of a data preparation agent and a mining agent code.

Agents need to be run as child processes to avoid blocking due to processing large amounts of data [17]. The multi-agent approach in itself has the potential to expand the system by adding new agent centers, which, through distributed processing, ensures load balancing but also avoiding the existence of a single point of failure of the entire system, enabling the execution of computer-intensive tasks [17].

The NoSQL ArangoDB database is the core of the system in which agents store all collected data, whether raw, prepared, or data representing learning results. What distinguishes NoSQL databases is their horizontal extensibility, storing different data structures with less demanding hardware [18]. ArangoDB is a multi-model database supporting various data models, storing key-value pairs, documents, and graphs; all data can be accessed in the same ArangoDB Query Language (AQL). Compared to other similar databases that specialize in graphing, Arangodb leads in performance, but its main advantage is its multi-model architecture [19].

6. Results and Discussion

Agents in the Observer system are designed to meet a minimum set of fundamental traits that characterize software agents, such as being autonomous, capable of operating as a standalone process and performing actions without user intervention [20]. We create agents according to the reactive or executive agents’ model, representing the type of agents that can only directly follow the basic program (task-oriented model of agents versus goal-oriented model). They are considered primitive, not applying reasoning for the causes or effects of their action. We use this model in the research because of its main characteristic: efficiency based on simplicity [21]. As they are executed within browsers, tutoring agents must not significantly affect resources and interfere with the work of the Moodle platform. On the other hand, agents that accept and process data are oriented toward reactivity due to the projected large amount of data.

To evaluate the system’s operation, we experimented with actual university Moodle LMSs during distance learning due to COVID-19. The experiment with the Observer system was performed on two different courses, undergraduate and master studies, which were attended by a total of 36 participants.

Over three months, the system collected a total of 4913 records from tutoring agents.

Table 1. Learning indicators extracted and normalized.

Student ID	Visits	Clicks	Time
04a7aa6645b28f848253ec5132290ae1	0.04348	0.01117	0.07731
268046f034b2e014dabf73ba8e780cf9	0.02174	0.00559	0.00000
34c01e64351a8f1e0176d90d32b0a99a	0.39130	0.39665	0.55339
3acb0baab491ea28de32247da3e9a992	0.08696	0.05028	0.14701
5d781e010054aeeebf26d029664d6618	0.34783	0.46369	0.26440
...	...	...	...

shows a preview of the results of the work of the data preparation agent. The table contains the learning indicators collected by Observer after normalization as part of the preparation for the application of data mining algorithms and performed by the data mining agent.

Figure 8 shows the result of applying the final kNN classifications to the data collected at the end of the course, shown in Table 1, on which the predicted exam outcomes are performed.

After the end of the course, 31 students took the exam. Based on students’ data known to the creator of the courses, we created hash values and compared the results of the system prediction with the actual results on the exam. The system achieved an accuracy of 0.742, or related to the confusion matrix, we observed eight true positives, fifteen true negatives, two false positives, and six false negatives.

At this point, we have to address some limitations we had regarding objective evaluation of proposed system effectiveness. Although it was implemented in the university Moodle LMS for distance learning due to COVID-19, we were not able to communicate directly with students in order to get their feedback regarding the Observer utilization. At this point, we do not have information about their experience with the Observer system and whether the suggestions they received from it affected, in any way, their further engagement. We could measure and evaluate only the parameters of their engagement and consequently, produced predictions of their success on exams, for which the system achieved a reasonable accuracy score (0.742). Our plans are to address this issue in the experiment we are going to perform in the next semester in autumn.

7. Conclusions

In this paper, we introduced Observer—the multi-agent system for intelligent support for engaged students. It combines two different machine learning methods that, based on the student engagement indicators, predict success at the end of the course and then make suggestions based on that prediction. The primary measures of engagement, teaching attendance and additional indicators previously found to be related to the student’s final success in the course, are utilized in the Observer system. They include the number of clicks, number of views that are, access to various pages, and the time that the student spends using the Moodle Course’s teaching content. Although there are limitations to the use of indicators, such as the fact that click counting cannot determine the learning process’s quality, it should be noted that neither can the grade itself. It only indicates that the assessment criteria are met, which may or may not be a measure of learning effectiveness [3].

The goal of the proposed system is to support the student but not to interfere with his/her work. The element of the Moodle interface called block is selected for communication with students because it occupies a small portion of the Moodle page. Observer block contains only a brief suggestion and information on how old the suggestion is. In addition, the Observer system is devoid of redundant functionality, designed to be mostly autonomous, which is why we choose the agent architecture. Other used technologies provide a reasonable basis for upgrading the role of the Observer system in distance learning. Node.js is designed to build a real-time system, and the multi-model ArangoDB database can handle a wide variety of datasets, most notably enabling the use of graphs that allow new dimensions to analyze and learn from data.

This paper has shown that through the application of data mining, an intelligent and mostly autonomous system can be developed to support students in engaged learning, while not neglecting the critical role of lecturers and course creators. Lecturers who provide teaching material distributed through the Moodle platform by selecting the appropriate Observer block location give significance to any data collected and analyzed. The database containing anonymized data allows their use much more widely than Observer’s primary purpose. Analysis can be directed toward evaluating existing courses, comparing them, and other research since free access to data without compromising student privacy is provided.

Further research will focus on considering the possibility of supporting other learning styles, especially for self-regulating students.