Deep Learning Recommendations of E-Education Based on Clustering and Sequence

Abstract

Commercial e-learning platforms have to overcome the challenge of resource overload and find the most suitable material for educators using a recommendation system (RS) when an exponential increase occurs in the amount of available online educational resources. Therefore, we propose a novel DNN method that combines synchronous sequences and heterogeneous features to more accurately generate candidates in e-learning platforms that face an exponential increase in the number of available online educational courses and learners. Mitigating the learners’ cold-start problem was also taken into consideration during the modeling. Grouping learners in the first phase, and combining sequence and heterogeneous data as embeddings into recommendations using deep neural networks, are the main concepts of the proposed approach. Empirical results confirmed the proposed solution’s potential. In particular, the precision rates were equal to 0.626 and 0.492 in the cases of Top-1 and Top-5 courses, respectively. Learners’ cold-start errors were 0.618 and 0.697 for 25 and 50 new learners.

1. Introduction

In the last decade, effective recommendation systems (RSs) have become an essential tool in modern online education applications, such as Coursera and Udemy. The RS suggests popular and most-enrolled courses to new learners of these applications, and also helps distance learning websites to suggest e-learning resources, such as books, lectures, and educational links, along with new and leading courses to existing learners. However, with the exponential increase in the amount of available online educational resources, the RSs have to overcome the challenge of resource overload and find the most suitable material for educators. Commercial e-learning platforms have been using RSs that assess and manage learner–item interactions to boost learner satisfaction, customize suggestions, and raise revenue. The e-learning platforms have established a few models for building RSs based on explicit and implicit features of learners and educational materials. For instance, the initial RS models were based on similarity [1]. More precise and quicker latent-factor methods were introduced immediately after the similarity-based paradigm; these use ratings to analyze customer input, and then categorize customers and products based on their features. Latent-factor methods are typically realized via matrix factorization [2] using a vector of variables based on a rating matrix to identify product and customer features. Due to the nature of the collected data, the aforementioned methods for designing recommendation models are categorized as content-based filtering (CBF) and collaborative filtering (CF) techniques. The accessibility to learner–item interaction input is evaluated using CBF, which involves the management of many explicit features [3,4]. In contrast, when generating suggestions, a CF-based RS [5] employs a customer’s previous rating scores to anticipate unrated products and subsequently gathers customer requirements from a specified cohort to process predictions. In recent times, deep learning (DL) has radically changed recommendation algorithms and improved their effectiveness. In addition to enabling the modeling of increasingly complicated conceptions as data representations in the top layers, DL successfully records non-linear and complex customer-product associations [6]. Various RSs using DL algorithms [7,8,9] have recently attracted significant attention due to their unique potential. Neurons in neural models can be differentiated from one end to another and offer appropriate logical biases tailored to the data format. When a model with a specific form can be applied to a given data format, it may also be applied to other data that contain related structures. Furthermore, a differentiable network comprising many deep neural networks (DNNs) can be trained from start to completion. Consequently, hybrid RSs are easier to handle. However, a DL recommender mechanism may be unsuitable if both text metadata (tweets, tags, reviews, etc.) and graphic data (predicted photos) are used together, for instance. A combined (end-to-end) representation training is not possible in such cases, and hence conventional RS algorithms frequently fail. As an example, processing comments requires expensive preprocessing (such as extractive summarization), while content data (text) may be immediately processed by DL-based algorithms [10]. DNN characteristics are extensively and widely applied to assist RSs that use heterogeneous data [11,12]. However, DL and matrix factorization approaches [13,14] exhibit a continuous increase in prediction time when a system contains an enormous number of customers and products. To solve this issue, algorithms that generate top-K candidates have been suggested [15,16]. Unfortunately, these algorithms are often unable to simultaneously use both consecutive and heterogeneous inputs (i.e., semantic features). Therefore, we propose a novel DNN method that combines synchronous sequences and heterogeneous features to more accurately generate candidates in e-learning platforms that face an exponential increase in the number of available online educational courses and learners. Mitigating the learners’ cold-start problem was also taken into consideration during the modeling. To the best of our knowledge, both features have not been previously used together, despite the substantial amount of research conducted to independently apply sequences and homogenous features in building RSs. Grouping learners in the first phase, and combining sequences and heterogeneous data as embeddings into recommendations using deep neural networks, are the main concepts of the proposed approach. The primary highlights of our study are as follows:

➢

Initialize learners as a homologous category via clustering;

➢

Combine synchronous sequences and heterogeneous data;

➢

Make time-aware course recommendations based on the sequence of learners’ history;

➢

Overcome the learners’ cold-start problem by concatenating additional features;

➢

Improve overall candidate generation performance when utilizing a large dataset.

The remainder of this paper is organized as follows. Section 2 provides an overview of the latest DL recommendation methods that use sequences and heterogeneous features to generate the top-N candidates, solve learner cold-start problems, and increase overall recommendation efficiency. Section 3 and Section 4 explain the proposed method in detail and demonstrate its accuracy by comparing it with alternative approaches through experiments and evaluations. Section 5 summarizes the findings and scope of the study. In general, most of the relevant sources included are comparatively newer publications.

2. Literature Review

2.1. Sequence-Oriented Recommendation Methods

To tackle the ongoing challenges around course recommendations and course cold starts, several recent investigations [17,18,19,20,21,22] have used heterogeneous and sequence-oriented metadata as auxiliary features. Specifically, Zhao et al. [23] established a book RS based on customer history that considers the length of time between consecutive purchases. To mitigate the item cold-start problem, a novel methodology was created by Riedl and Yu [21], who built customized tales utilizing the greatest series of customer-observed events. In addition, a CF technique to compose dual probabilistic viewpoint approaches using pure customer data was designed by Lam et al. [18]. Likewise, extensive research [4,24,25] has concurrently incorporated multiple types of implicit and explicit data features in the prediction phase. Most candidate generation techniques and top-N RSs are relatively identical in terms of objectives. Therefore, similar to candidate generation, top-N recommendation algorithms exploit the prediction strength to select perspective products over large product spaces. For example, product-based carts [15] were suggested to consumers based on an analysis of product commonality. The basic autoencoder-based AutoRec [26] mechanism expressed by Formula (1) is another innovative top-N recommendation model:

$$h\left(r;\theta \right)=f\left(W·g\left(Vr+\mu \right)+\beta \right)$$

(1)

where β and µ are the biases of each layer; g and f represent the model’s activation functions; and r is the input’s reconstruction. In this model, all customer ratings are processed as a training dataset by the input layer, which further reduces them in the encoder section. The decoder component of the model was based on the bottleneck at the midsection, where the layers of the model were widened. The original dimension in the input layer was reinstated in the output layer.

Moreover, the overfitting problem was tackled using L2 regularization to train the model, while also reducing the sum of the MSE between the input and output ratings. The parameters W and V in Formula (2) represent the first- and second-layer weights, respectively:

$$\underset{\theta }{\min }{{\displaystyle \sum _{i=1}^n\Vert r^i-h(r^i;\theta )\Vert }}^2+\frac{\lambda }{2}(\Vert W{\Vert }_F^2+\Vert V{\Vert }_F^2)$$

(2)

where λ is the regularization parameter, and r represents each learner’s rating history.

Despite their flexible nature and high prediction accuracy, issues are encountered by methods using autoencoders when processing learner content to generate recommendation data because of the learner’s history. To prevent this, a ground-breaking top-N RS was proposed in [27], which is based on a neural probabilistic language model. The model reduces the sparse-valued input into a fixed authentic-valued embedding vector given by Formula (3):

$$V^i=\left[v_1^i,\dots ,v_j^i\dots ,v_p^i\right]\in {\mathrm{W}}^{n\times p}$$

(3)

where p is the embedding vector size, i is the customer id, and W is the embedding matrix. The embedding vector is used to train the probabilistic component of the sentence sequence and a distributed description of every single word.

A further illustration of candidate selection is the deep recommendation algorithm of YouTube [16]. Candidate selection and product ratings are the two key components. By eliminating irrelevant films, the candidate selection phase reduces thousands of available films to a few hundred. The initial layer of the candidate-selection neural network mimics the factorization technique, using no deep layers and relying solely on customer sequence history; hence, the candidate-selection phase can be considered a nonlinear variation of the factorization technique. The approach proposed in this study closely resembles the neural network model of YouTube. Nonetheless, most recommendation engines do not usually consider the product order. In particular, calculating the mean of the embedded customer history causes the YouTube neural network algorithm to suffer from sequence features. In addition, the customer’s past experience is transformed into a customized Bayesian rating [28], so that the ratings never store the customer’s past order data.

2.2. Clustering-Based DL Recommendations

The accessibility to vast online educational content has made it more complex for learners to quickly choose the most pertinent content. To overcome this obstacle, a considerable number of DL recommendation techniques [29,30,31] have been developed using a combination of clustering methods as an initial step to guide learners overloaded with data. For example, a context-aware DL-based RS [32] was first proposed to derive contextual features utilizing a word-embedding technique. Then, a clustering method was applied to create positive, neutral, and negative sentiment clusters via the derived features. Finally, a deep recurrent neural network (RNN) was used to determine the most desirable customer from each cluster with the help of information entropy and biclustering [33], which offers a composite sorting approach to handle data diversity and sparseness. Biclustering can precisely identify the density components of a rating matrix, and an information entropy measure determines the similarity of a potential client to the density components. Moreover, to facilitate customer CF, Binbusayyis [34] developed a deep-clustering method that leverages the strengths of both fuzzy clustering and DNNs. The developed method first uses a deep autoencoder to extract the customers’ underlying feature representation from the initial data matrix, and then performs fuzzy clustering operations on that information to create customer groups. The authors of [31] suggested using improved hierarchical reinforcement learning via a clustering approach that combines a hierarchical reinforcement neural network and a pretrained network to lessen the sparsity of book-renting information. Prior to recommending courses, clustering can be employed as a preparatory step to boost recommendation accuracy, as indicated earlier. This promotes the customization of suggestions by enabling computers to produce additional content representations from the learners’ input. In summary, the proposed methodology first leverages clustering to initialize learners as a homologous category and decrease data sparsity, and then applies synchronous sequencing and heterogeneous features to generate top-N prospective courses and overcome the course cold-start problem, thus enhancing the overall candidate generation performance on big datasets. The possibility that the learners’ tastes change over time sets the envisioned paradigm, with the exception of previous efforts. To the best of our knowledge, there is currently no other methodology that applies clustering to create homologous groups, and simultaneously integrates heterogeneous and sequence features within those groups.

3. Proposed Approach

This section explains the proposed approach, which combines sequences and heterogeneous features to more accurately generate candidates in e-learning platforms that face an exponential increase in the number of available online educational courses and learners. The following subsections describe the details of the recommendation model developed in this study.

3.1. Initializing Learners via K-Means

Because the numbers of available learners and courses on e-learning platforms have significantly grown, conventional RSs are unable to fulfil the online demand due to the time-consuming requests of potential learners in the entire e-learning platform. Furthermore, the efficiency in generating potential courses decreases as the number of cells augment. The initial sparseness of the input is the primary cause of poor efficiency. To mitigate the sparseness of the initial input while developing the candidate generation system, our study proposes a personalized course recommendation strategy that first applies a clustering technique based on the k-means algorithm to initialize learners as a homologous category, and then aggregates DNNs in each group of learners, utilizing synchronous sequences and heterogeneous features to generate the top-N prospective courses. Learners are categorized as homologous groups based on the input matrix. Depending on the resemblances between the potential learner and group center, the peers near the potential learner can be found and utilized to improve course generation. After the homologous groups have been established, course generations for a potential learner are produced by summing the responses from the remaining learners within the same homologous group. As illustrated in Figure 1, the idea is to use a learner clustering technique to divide the learners of a current e-learning platform into similar categories. The clustering process might result in a varying number of divisions of a particular size, or a defined set of divisions of varying sizes according to their similarity.

Algorithm 1 is a widely investigated procedure for initializing learners using the k-means algorithm:

Algorithm 1: Learners’ clustering algorithm

Input:learner–course interactions data, k-number of clusters

Output:dense learner clusters

Begin:

Define learner set l$=\left\{l_1, l_2,l_3\dots ,l_m\right\}$;

Define course set c$o=\left\{co, c{o}_2,c{o}_3\dots ,c{o}_n\right\}$;

Choose primary r rating learners acting as the clustering $\mathrm{cl}=\left\{c{l}_1,cl,c{l}_3\dots ,c{l}_m\right\}$;

The clustering kernel is null as c $=\left\{c_1,c_2,c_3\dots ,c_k\right\}$;

do

for each learner $l_i\in l$

for each cluster kernel c$l_i\in cl$

calculate similarity sim($l_i$, c$l_i$);

end for

sim($l_i$, c$l$) = $\max \left\{\mathrm{sim}(l_i,\mathrm{c}{l}_1),\mathrm{sim}(l_i,\mathrm{c}{l}_2) \dots ,\mathrm{sim}(l_i,\mathrm{c}{l}_k)\right\}$;

$c{l}_m=c{l}_m\cup l_i$

end for

for each cluster $c{l}_i\in cl$

for each learner $l_j\in l$

c$l_i=average\left(c{l}_i,l_j\right)$;

end for

end for

while (c is not change)

Finish

The sparseness of the input data is probably the most crucial challenge in building RSs. We specifically initialize learners as a homologous group from the input data, and then apply the sequence-oriented course generation approach within each group separately. We determine dense learner interactions with certain courses by initializing the learner’s homologous group. Eventually, the initial input data transforms into a dense learner–course data matrix for each homologous group.

3.2. Sequence-Oriented Embedding

A discrete categorical variable is mapped to a vector of continuous integers during embedding. Embeddings are low-dimensional, continuously learned vector representations of discrete variables used in neural networks. Neural network embeddings are helpful because they can reduce the dimension of categorical variables and accurately reflect categories in the converted space. Neural networks’ weights that receive one-hot encoding of categorical data as their input, as shown in Figure 2, are called embeddings.

Here, h is the hidden layer node, p is the hidden layer embedding vector size, V is the embedding vector for course three, v is the embedding vector weight, and N is the number of courses. These weights are stored as a matrix of variables to reduce the computational cost.

As part of the training phase, the network receives categorical values from the relevant row of the matrix, and the variables in this row and all other weights in the network are then trained. Pretrained or randomly initialized weights can be utilized as embedding vectors depending on the circumstances. Utilizing trained weights is advised when time is limited. Vectors obtained from techniques such as the Bayesian personalized ranking (BPR) [28] may be used to build a trained embedding. To validate the proposed method, we used the Ubob.com dataset (accessed on 24 December 2022), which contains categorical data in every field. Because every field has a different length, it is challenging to analyze the data using conventional techniques. In our approach, the number of courses unique to each learner constitutes the embedding layer. To address this issue, we used the length of the learner history sequence to generate a mask, which was then used in combination with the embedding vector to eliminate superfluous embedding. The mean vector of courses was initially determined through a conventional deep RS using Equation (4):

$$\frac{1}{n}{\displaystyle \sum }_{i=0}^n{V}_i$$

(4)

where n denotes the size of the learner history, and Vi is the embedding of the course that the learner has chosen.

The forecasts are identical across all combinations of member histories because this embedding strategy assumes that learner preferences are constant over time. This assumption may not be accurate in every circumstance. Initially, we create a learner–course connection matrix $R\in {ℝ}^{m\times n}$, where m and n represent the number of learners and courses, respectively, $r^{\left(u\right)}=\left(R_{u1}, \dots R_{um}\right)$ is the sequenced history of each learner, $u\in U=\left\{1,2,\dots m\right\}$, and $r^{\left(i\right)}=\left(R_{1i}, \dots R_{mi}\right)$ describes the sequenced history of every course $i\in I=\left\{1,2,\dots m\right\}$. Furthermore, assumptions are made as follows: if learner u_k studied “management” last year and is currently taking “economy”, while learner u_l had already taken “economy” and is now learning “management”, then:

• ▪ u_k: {management, economy}
• ▪ u_l: {economy, management}

Compared with u_l, who studied economy and management, in that order, u_k has a noticeably different preference. The first customer could have previously completed management, and eventually may change their mind and choose economy, whereas the second learner would want to enroll in the management course after completing economy. This demonstrates that the sequence should not be disregarded and suggests that proposing the same products may not render a reliable forecast. We provide a novel embedding vector that comprises sequence information to exploit the sequence data (learner history). The learner’s category history should be input into the model without modifying its order (for example, (i₁, i₂, i_3…i_N)). Software was used to create the model. The history duration should be equalized by adding a certain constant number that will be subtracted in the following layer because N changes for various learners and neural networks can employ a constant matrix form. This integer number (the history of learners) is passed onto the subsequent embedding layer, which substitutes integers with the appropriate float dense vector. Large datasets typically use vectors with a length of 128; however, this may be altered. The importance of items that were consumed in the past diminishes over time, as previously indicated, and products that consumers have recently taken may be given a higher weight when forecasting the next items. To describe the order of the item in the learner’s history, the values are multiplied by each embedding vector of the learner’s history:

$$V^{\prime }=\frac{V}{\mathit{N-order}+1}$$

(5)

In this case, ‘order’ is the number of courses in the learner’s history, N is the number of courses, and V is the embedding vector of the current course. Equation (6) is used to obtain the average of these vectors following consecutive embedding of objects:

$$V_{sequential}=\frac{1}{N}{\displaystyle \sum }_{i=1}^N{V^{\prime }}_i.$$

(6)

The input layer is then concatenated with V-sequential and actual embedding, as shown in Figure 3, to determine the layer about the learner’s average and sequential preferences.

3.3. Content Data

Generally, the customer content data comprise continuous and categorical (small and large) information. The continuous data are received by neural networks as inputs and large amounts of category data are used to function as embedding vectors. However, it is possible to enter categorical data into a network serving as a sparse vector (one-hot encoding) when the datasets are small. For instance, if the learner data include a field such as gender, it may include three categorical features: male, female, and missing information. In this situation, the data should be encoded using the one-hot method described below:

• Female:      [1, 0, 1]
• Male:        [0, 1, 0]
• Missing data:   [0, 0, 1]

Our dataset comprised the learner’s “jobcode” information, which utilized keywords to describe the learner’s occupation. We calculate the mean vector and concatenate deep networks using these data as an embedding vector input because there are more than 100 job codes, and a single learner could have multiple job codes.

3.4. Deep Neural Network

As shown in Figure 4, the network begins by concatenating each feature vector into a single layer. Two or three completely linked layers may be added, depending on the length of the training data (three throughout all of our tests). Each hidden layer included 64, 32, and 16 units. While the customers’ “jobcode” and “company information” were encoded employing 16 units each, the courses’ embedding dimension was encoded employing 32 units. Subsequently, ReLU, Leaky ReLU, and sigmoid activation functions were implemented. It was determined that a ReLU as given by Equation (7) was optimal for DNNs:

$$\mathrm{ReLU}=\{\begin{array}{c}x: if(x>0)\\ o: if (x<0)\end{array}$$

(7)

Finally, with an identical number of active courses in the dataset, the SoftMax function expressed by Equation (8) was deployed in the output layer of the proposed DNN:

$$\mathrm{Softmax}=\sigma {\left(z\right)}_i=\frac{e^{z_i}}{{\sum }_{j=1}^K{e}^{z_j}}$$

(8)

It should be noted that the output value corresponds to the number of courses in the dataset.

4. Experimental Results

4.1. Dataset

The Korean e-learning platform “www.ubob.com” (accessed on 24 December 2022) provided a dataset to design and implement the recommendation approach. The dataset was primarily used to assess and compare the recommendation approach with other related methods. The dataset structure comprises four tables that contain sequence and heterogeneous information, as illustrated in Figure 5.

The dataset comprises variables with varying sizes, each of which included categorical features. For example, the table “student” stores information about learners. Therefore, it is difficult to establish approaches to examine these features. Moreover, several features, including sex and age, lack sufficient data to be employed in the recommendation engines. Consequently, the table no longer contained these features. There were approximately 222,000 learners on www.ubob.com (accessed on 24 December 2022); however, only 82,000 learners were associated with the learner–course interactions that were implemented in our model. Additionally, learners provide “jobcode” data, which summarizes their occupation with a single term. Every learner’s “jobcode” feature in the platform has a distinct length. There were 528 eligible “jobcode” features serving 82,000 learners. The network’s final layer, known as “SoftMax”, could not learn all the classifications when the dataset was trained only using the features that were already available. Therefore, to mitigate this problem, the dataset was expanded using a data-augmentation process based on “the sliding-window technique”. The data augmentation process is presented in

Table 1. Data augmentation process.

Learner History Sequence	Following Course
L1 = [C1, C2, C3]	C4
L2 = [C1, C2]	C3
L3 = [C1]	C2

.

For instance, one learner may have a C_1, C_2, C₃, and C₄ course history sequence, meaning that after taking courses C_1, C_2, and C_3, the learner has chosen course C₄. Moreover, if there is a C₁ and C₂ sequence history, the following course is C₃. Based on this strategy, data augmentation was implemented to increase the dataset size.

After implementing data augmentation, 320,000 examples were obtained, which impact the efficiency of the proposed network model. To evaluate the performance of the proposed model, 80% of the dataset was used for training and the remaining 20% was used for testing.

4.2. Implementation Settings

The proposed approach was implemented using software, hardware configurations, and model parameters, as illustrated in

Table 2. Configurations.

Software	Programming tools	Python, Pandas, Keras-TensorFlow,
	OS	Windows 10
Hardware	CPU	AMD Ryzen Threadripper 1900X 8-Core Processor 3.80 GHz
	GPU	Titan Xp 16 GB
	RAM	128 GB
Parameters	Epochs	20
	Learning rate	0.001
	Optimal Clusters	8

.

4.3. Model’s Results

4.3.1. Top-N Predication Performance

First, the proposed sequence-oriented deep network approach initializes learners to make similar groups. Then, the whole candidate generation process synchronously integrates the learners’ history sequence data and content data accomplished in each initialized group. In the implementation process, precision was utilized for the top-N performance results, and the mean absolute error was used to evaluate the cold-start case. In addition, to compare the performance of the proposed approach with its competitors, we contrasted it with the following benchmarks, and the prediction results are illustrated in Table 3.

• AutoRec [26]: To deliver individualized suggestions, a method based on the autoencoder approach attempts to use customer preference data for various products.
• YouTube model [16]: The authors utilized categorical features and continuous data to make recommendations, whereas the sequence of customer history was disregarded.
• LightGCN [35]: This model learns customer and product embeddings by linearly distributing them on a customer-product interaction graph. The weighted sum of the embeddings was then utilized as the last embedding layer.
• FISM [36]: The authors offer a technique that creates two low-latent factorized matrices by learning the product–product matrices to represent and retain the relationships between products.
• E-LCRS [37]: This recommendation model was built based on the history and preferences using a collaborative filtering mechanism.
• Cluster-IHRS [38]: The recommender evaluates and learns the styles and features of the learners automatically. Split and conquer strategy-based clustering is used to process the various learning styles. The algorithm then makes intelligent suggestions based on the ratings of frequently occurring sequences.

Table 3. Top-N performance.

Model	Precision@1	Precision@5	Precision@10
AutoRec [26]	0.309	0.213	0.175
YouTube model [16]	0.599	0.461	0.294
FISM [36]	0.553	0.3238	0.2358
LightGCN [35]	0.589	0.4702	0.3768
E-LCRS [37]	0.611	0.453	0.209
Cluster-IHRS	0.46	0.352	0.216
SODNN (proposed)	0.626	0.498	0.385

Table 3. Top-N performance.

Model	Precision@1	Precision@5	Precision@10
AutoRec [26]	0.309	0.213	0.175
YouTube model [16]	0.599	0.461	0.294
FISM [36]	0.553	0.3238	0.2358
LightGCN [35]	0.589	0.4702	0.3768
E-LCRS [37]	0.611	0.453	0.209
Cluster-IHRS	0.46	0.352	0.216
SODNN (proposed)	0.626	0.498	0.385

Furthermore, using a trained model, we attempted to illustrate the similarity in the predicted courses (

Table 4. Predicted course similarity.

Courses	Similarity
Real Estate Disclosure Act	0.801
Real Estate Finance	0.775
Real Estate Investing	0.698
Real Estate Course Introduction	0.623

) related to the course called “Real Estate Broker_Introduction 2017 (1)”.

Moreover, the network was trained without overfitting, as evidenced by the visual representation of the loss functions of the training and validation sets. Furthermore, we may conclude that the training model performed the best in the 25th epoch (Figure 6).

4.3.2. Cold-Start Case

The cold-start issue may be minimized because the Ubob.com (accessed on 24 December 2022) dataset will have sufficient interaction data on learners and courses after clustering. Additionally, because courses were permanent and never altered, adding new content is a challenge for this online educational platform. Moreover, generating recommendations only for new subscribers could be challenging. To address the issue of new subscribers, the combined default embedding vector, with the help of new learners’ “company_id” and “jobcode” features, was utilized to predict courses where the system already had those features for the new learners.

Table 5. Learners’ cold-start performance.

Cold-Start Cases		Model
		AutoRec [26]	YouTube Model [16]	FISM [36]	Light-GCN [35]	SODNN (Proposed)
MAE	New 25 learners	0.771	0.664	0.792	0.784	0.618
	New 50 learners	0.835	0.753	0.843	0.840	0.697

demonstrates that the proposed approach performed better than the compared benchmarks. Thus, simultaneously utilizing the learners’ sequence history and content data generated good recommendation results in a cold-start scenario.

5. Conclusions and Future Work

In this research, we attempted to build a novel deep neural network that combines synchronous sequences and heterogeneous features to help e-learning platforms generate candidates when there is an exponential increase in the number of learners and available online educational courses. Specifically, we first focused on applying a clustering technique based on the k-means algorithm to initialize learners as a homologous group. Then, based on aggregating deep neural networks to each group of learners, synchronous sequences and heterogeneous features are utilized to generate the top-N prospective courses. The goal of our research was to initialize learners as a homologous category by clustering and combining synchronous sequences and heterogeneous data. Moreover, the proposed methodology overcomes the learners’ cold-start issue by concatenating additional features and enhancing the overall performance of candidate generation on big datasets. The empirical results show that the proposed approach outperforms baseline methods. However, some limitations need to be addressed to further its development:

• Combine the approach with other more advanced and accurate clustering techniques [39,40];
• Address the need to offer the simplest possible dynamic candidate generation;
• Address the “gray sheep” problem, in which a learner cannot be associated with any homologous cluster and the online platform is incapable of suggesting relevant courses.

Moreover, future studies should investigate more advanced models to calculate the significance of the hidden user and object features and mitigate the above-mentioned problems. In addition, it should be possible to create an emotion-based candidate generation model that involves heterogeneous speech data [41] and learners’ history sequence data, and to develop a recommendation model for visually impaired people [42] based on their characteristics.