Recommender Systems Applications: Data Sources, Features, and Challenges

Abstract

In recent years, there has been growing interest in recommendation systems, which is matched by their widespread adoption across various sectors. This can be attributed to their effectiveness in reducing an avalanche of data into individualized information that is meaningful, relevant, and can easily be absorbed by a single person. Several studies have recently navigated the landscape of recommendation systems, attending to their approaches, challenges, and applications, as well as the evaluation metrics necessary for effective implementation. This systematic review investigates the understudied aspects of recommendation systems, including the data input into the systems and their features or outputs. The data in (input) and data out (features) are both diverse and vary significantly from not just one application domain to another, but also from one application use case to another, which is a distinction that has not been thoroughly addressed in the past. In addition, this study explores several application domains, providing a comprehensive breakdown of the categorical data consumed by these systems and the features, or outputs, of these systems. Without focusing on any particular journals or their rankings, this study collects and reviews articles on recommendation systems published from 2018 to April 2024, in four top-tier research repositories, including IEEE Xplore Digital Library, Springer Link, ACM Digital Library, and Google Scholar.

1. Introduction

There is a plethora of definitions and interpretations of recommendation or recommender systems (RSs) in information science literature. According to [1], RSs are web applications capable of predicting the future opinion of a user about a set of items. Similarly, in [2], RSs are defined as systems that help users to decide on appropriate items and find preferred items within a collection. Additionally, in [3], they are defined as web-based tools that tailor vendor’s offerings to consumers according to their preferences [4]. However, one widely accepted definition of RSs is that they are systems that suggest items to users based on their preferences, interests, or behaviors. Based on this definition, the main components of RSs include the user (either individuals or businesses), the item (products or services), and the user’s preferences or interests.

The term “recommendation system” initially appeared as a branch of data or knowledge retrieval [5] and filtering before becoming an independent science about four decades ago [6,7]. In recent years, research on RSs has significantly increased. Several factors are responsible for this increased interest, including the rapid and exponential growth of data (particularly big data); the ubiquitous nature of information on the internet or web, which has inevitably caused information overload; and, last but not least, global demand for digital devices including computers and mobile phones. While each of these factors has played a huge role in the unprecedented excitement around RSs, the most compelling factor for researchers to study and investigate RSs has undoubtedly been the internet environment itself. Because the internet is crowded with information accessible through multiple applications, filtering out what is relevant for an everyday user is a challenging task. Thus, researchers have proposed RSs as viable solutions to information overload. Furthermore, the internet provides a platform for collecting data from users, which is subsequently utilized by RSs to suggest relevant items, making RSs particularly adept for this task.

RSs aim to not only retrieve the most relevant information for the user from a vast amount of data but also to predict the most appropriate elements for the user based on the characteristics depicted in their preferences and behavior. Besides recommending previously seen items, RSs also suggest new and unexpected items to users. This diversity and randomness have been deemed useful and effective in RSs as of late, as they ensure that the user does not get bored of repetitive recommendations [8].

A multitude of real-world applications benefit from RSs. From e-commerce, social media, and messaging, to audio and video content streaming, RSs are widely adopted to enhance the customer experience. Amazon (https://www.amazon.com, accessed on 4 March 2024), eBay (https://www.ebay.com, accessed on 4 March 2024), and Alibaba (https://www.alibaba.com, accessed on 4 March 2024) are some of the most popular business-to-consumer (B2C) sites with powerful RSs engines. They have adopted the collaborative filtering (CF) technique, which recommends products to a customer based on the correlation between the customer and other consumers who have purchased similar products [9]. CF aggregates user-provided ratings, typically assessed on a scale of 1 to 5 (with increasing level indicating a preference or likeability), in order to generate new recommendations for customers.

Motivated by the breadth of underexplored aspects in RSs, this study conducts a comprehensive survey of existing literature on RSs to uncover details on the input and outputs of RSs. Upon a preliminary review of the recent literature from 2018 to April 2024, this study identifies, and focuses on, six carefully chosen application domains, studying three fundamental aspects pertaining to the input and outputs of RSs. These three fundamental aspects of rapidly evolving RSs are: (1) the data (the input into RSs) that is provisioned to RSs to enable them to function, (2) the features or functions (the output from RSs), and (3) the challenges faced in the research, development, and implementation of the aforementioned systems. This study organizes recent and current knowledge pertaining to these aspects into the different application domains or fields.

Based on the above, the main contributions of this work include the following: (1) classifying RSs into several domains according to their applications in business and (2) studying the features, challenges, and data sources for each application of RSs.

The remainder of this paper is organized as follows. First, Section 2 reviews critical studies and articles. This is followed by Section 3, which describes the data collection methods used in this study. This paper utilized four stages (Identification, Scanning, Eligibility, and Inclusion) adopted from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (see Supplementary Materials). The next section (Section 4) breaks down the various RSs application domains, providing an independent and distinct review for each, concentrating on the data (inputs), features (outputs), and challenges faced by RSs applications in their respective domains. Finally, the paper summarizes the literature review’s findings in the discussion Section 5 followed by the conclusion in Section 6.

2. Related Work

The number of literature reviews or surveys focused on RSs has increased significantly over the past few years. Many of these surveys explore one or more of the following topics: (1) Approaches or techniques of RSs; (2) Challenges or problems facing RSs; (3) Applications of RSs; and (4) Methods for evaluating RSs [10,11,12,13,14,15,16,17].

Several approaches [10,13,15,18], models [11], or techniques [17,19], as they are sometimes called, play an essential role in building RSs. The aforementioned approaches, as they are referred to in this study, are critically examined in almost every literature review of RSs [10,11,12,13,14,16,18,19,20]. The approaches are usually divided into three main categories: content-based filtering (CBF), collaborative filtering (CF), and hybrid filtering (HF) [10,11,13,14,16,17,18,19,20]. CBF is further divided into two approaches, which are model-based and memory-based [10,11,12,14,17], whereas HF comprises several classifications such as weighted, switching, and mixed hybrid [11,16,18]. In addition, these literature reviews also cover other approaches, such as knowledge-based filtering [13,14,15,17].

Within the most important topics examined in the literature review of RSs, the problems [10,11] encountered—commonly termed as “challenges” [14,15,17,18,19,20]—are considered to be the second most important subject, after the approaches of RSs. Cold start, data sparsity, and scalability [10,11,13,15,17,18,19] are the most common challenges presented in these studies. Challenges such as gray sheep [11,17], running time [10], and privacy issues [19] are also addressed in these reviews, albeit to a lesser extent. These challenges have been studied in order to reduce their impact on recommendation systems; for example, the cold start challenge has been managed by proposing a combined collaborative filtering technique that integrates a rating-oriented approach with a pairwise ranking-oriented approach [21]. Data sparsity has been addressed by providing a customized recommendation with hybrid feedback to improve implicit data [22]. It is important to know that, although there are general challenges to RSs as a field, some RS applications face challenges that others do not [23]. This study focuses on the challenges specific to each RS application type without attempting to address the proposed solutions for them, as that is beyond the scope of this work.

As mentioned earlier in Section 1, this study focuses on three aspects of RSs applications: the inputs, outputs, and challenges faced by these systems. To better understand them, RS applications were studied and analyzed. These applications are examined extensively in five literature reviews [11,13,15,17,20]. One study [11] classified the applications of RSs into seven service categories as follows: streaming, social network, tourism, e-commerce, healthcare, education, and academic information services. The second [13] classified RSs applications into a myriad of categories, including tourism, movies, consumer electronic products, education, research papers, medical treatments, music, electronic books, and job opportunities. A third study [15] created five classifications, which were e-commerce, transportation, agriculture, healthcare, and media. A fourth study [20] sorted RSs applications into four main groups, mobile, social, cloud, and traditional (such as e-commerce, e-shopping, e-learning, and e-library). Finally, in [17], RSs applications were defined as one of four types: e-commerce/e-shopping, entertainment, content, and service oriented.

Table 1. Various works with a classification list of their respective RSs application domains. ‘NA’ indicates that the number of categories for RSs is not available or mentioned explicitly.

Existing Studies	No. of Categories	Categories
[11]	Seven	Streaming, social network, tourism, e-commerce, healthcare, education, and academic information services
[13]	NA	Tourists, movies, consumer electronics products, education, research papers, medical treatments, music, electronic books, and job opportunities
[15]	Five	E-commerce, transportation, agriculture, healthcare, and media
[20]	Four	Mobile, social, cloud, and traditional (e.g., e-commerce, e-shopping, e-learning)
[17]	Four	E-commerce/E-shopping, entertainment, content, and service oriented

categorizes this literature on RSs applications. Of note is that the above five studies [11,13,15,17,20] examined multiple applications of RSs without reviewing their data sources, features, and challenges. In other words, the data sources, features, and challenges of RSs have not been reviewed before, leaving a critical gap in the literature.

Table 2. A summary of previous literature reviews on RSs and the subjects they cover. The symbol ‘✓’ indicates that the subjects are covered in the study, while ‘NA’ means the subjects were not covered in the study.

Existing Studies (References)	Approaches	Challenges	Evaluation Metrics	Applications
[10]	✓	✓	✓	NA
[11]	✓	✓	✓	✓
[12]	✓	NA	NA	NA
[13]	✓	✓	✓	✓
[14]	✓	✓	NA	NA
[15]	✓	✓	✓	✓
[18]	✓	✓	NA	NA
[20]	✓	✓	NA	✓
[16]	✓	✓	NA	NA
[17]	✓	✓	✓	✓
[19]	✓	✓	✓	NA

summarizes the literature reviews or surveys on RSs, indicating the particular subject on which they focused their investigations. Figure 1 further shows the percentage of topics treated in these studies. As seen in these visuals, approaches are the most widely studied RSs subject, followed by challenges, then evaluation, and finally applications.

After an extended review of the literature, this study discovered that most of the focus has been on the methods and techniques of RSs, while only a small amount of attention has been paid to the inputs and outputs of RSs. This leaves a gap, which this study investigates.

3. Methodology

The field of RSs is evolving rapidly and has garnered much attention in recent years. As result, the number of recommendation systems has increased due to the exponential growth of data, especially on the Internet, and the need for systems to manage this vast amount of information. Previous relevant studies discussed recommendation systems over different periods. For example, study in [11] examined recommendation systems from 2003 to 2018, while study in [13] covered recommendation systems from 2010 to 2021. This study focuses on recommendation systems in the last approximately six years, specifically from 2018 to April 2024.

3.1. Search Criteria

This study limits its investigation to four popular digital libraries that serve as repositories for top-tier journals and conference papers in information science. These libraries include IEEE Xplore Digital Library (https://ieeexplore.ieee.org/Xplore/home.jsp, accessed on 2 April 2024), Springer Link (https://link.springer.com/, accessed on 7 April 2024), ACM (https://dl.acm.org/, accessed on 11 April 2024), and Google Scholar (https://scholar.google.com/, accessed on 15 April 2024) (up to the first 20 pages of search results). Despite the rich history of RSs dating back over four decades [24], this paper only considers studies from 2018 to April 2024. This was done with the intention of capturing recent trends and benefitting from the recent uptick in RS adoption. Key search phrases were used to search and retrieve articles. These included “recommendation systems”, “recommender system”, “recent trends of recommendation systems”, “recommendation system applications”, and “data mining technology/technique”. These key phrases are predominantly used in RS articles, and including them thereby ensures the retrieval of recent and relevant studies.

3.2. Article Selection and Exclusion

This paper adopted a structured selection process, only choosing articles that fulfilled at least one the following conditions: (1) The article’s title included the phrase “recommendation system”, “recommender system”, or “data mining”; (2) The article’s title or abstract indicated the proposal or development of am RS for a specific domain; (3) The article’s title or abstract depicted the proposal or development of an RS framework for a specific domain; or (4) The article’s title or abstract addressed one of the following: challenges, data sources, and/or features for a specific type of RS application.

Subsequently, this study excluded the following articles: (1) non-English articles; (2) pre-prints (in any language) awaiting publication and other unpublished articles that were not associated with any mainstream journal or conference; (3) partial studies; and (4) publications dated before 2018.

Data Analysis and Findings

Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement [25], this study used four stages to select articles for inclusion. In the first stage (identification), 1878 papers were identified based on the keywords mentioned in preceding section. In the second stage (scanning), 1620 papers were retained after duplicate articles were removed. Additionally, at this stage, a preliminary review of titles and abstracts was conducted based on the inclusion and exclusion criteria noted above, which resulted in 622 articles requiring full-text reading. During the third stage (eligibility), 345 articles were identified as irrelevant, citing the following reasons: they did not discuss the applications of recommendation systems (for example). In the final stage (inclusion), 139 studies were retained for the full analysis conducted in this research. Figure 2 illustrates this search strategy.

The 139 studies included in this research were meticulously reviewed, with particular attention given to discussions on RS applications, their inputs (or data sources), features (or functions), and challenges. Additionally, these articles were categorized into their respective application domains (Section 5).

4. Recommendation System Applications

The popularity of RSs has transcended e-commerce sites that aim to sell various items to consumers. RSs have been adopted in a wide range of established business-to-business (B2B) and B2C websites and applications [26]. Current research trends reveal that RSs are being explored for several additional scenarios, some of which include drug recommendations [27,28,29], as well as disease and diagnosis predictions in healthcare [30,31]. Likewise, education stakeholders have begun using them for recommending personalized learning pathways and courses [32,33], to students using ontology [34,35]; for example, farmers are also using them to recommend agricultural items that would lead to optimal production in different seasons [36].

This section conducts a comprehensive survey of RS applications from 2018 to April 2024. It extensively discusses various RS-oriented aspects in the application domain. These include data sources available in the domain, its features, and the challenges faced in the application domain.

Key terms in the following subsections include the following:

Data sources: A recommendation or suggestion provided to an RS end-user premised on existing data, which may be user-generated. Examples of such include reviews, feedback, preferences, and social media data [37]; content-based data such as product attributes [38]; and context and demographic data such as gender, income, age, location, and time [39].

Features: This denotes the functionalities and/or services of RSs, which vary from one application domain to another. Each of these RS can have practical applications in, and a great impact on, human life [17].

Challenges: These are the unique obstacles faced by RS applications. These challenges could be generic or domain specific.

This investigation focuses on six application domains, because there is a plethora of prior work on them within the literature on RSs. These domains include e-learning, e-commerce, e-health, tourism, entertainment, and jobs.

4.1. E-Learning

E-learning RSs provide personalized suggestions in the form of learning material such as courses, learning pathways, and solutions to tasks for online learners [40,41,42].

Data: E-learning RSs leverage user data categorized under the following:

• Learner personal data: This includes personal demographic data (name, date of birth, and addresses) [43,44].
• Learner profile data: This includes a user’s background qualifications, preferred learning style (e.g., course work or group learning) and level of progress (beginner, intermediate or experienced) [45].
• Learning objectives: This indicates the user’s preferred learning outcomes [45,46].
• Log files: This includes a history of the user’s queries [40].

Features: Features in e-learning are learning objects, which, conceptually, are items or services that an e-learner expects to be recommended by the RS. Formally, a learning object has been defined as an entity, digital or non-digital, that may be used for learning, education, or training [47], or any digital resource that can be reused to support learning [48]. A feature of an e-learning system may entail one or more learning objects. Below is a classification of the main features discussed in prior literature:

• Learning paths or pathways: This feature aims to retrieve personalized learning paths for e-learners based on their current profile. A learning path is an optimized sequence of modules and tasks that a learner can follow to achieve a learning objective or outcome [49,50,51].
• Course or modules: This feature suggests relevant courses or learning programs to users (e.g., learners) based on their profile (e.g., preferences and requirements) [43,44]. In addition, this feature aims to analyze a learner’s skills and knowledge gaps based on their performance and assessments and then subsequently recommend specific modules, courses, or learning resources to address those gaps and facilitate skill development [45,52].
• Material: This feature recommends content such as topics, articles, books, and lessons to an e-learner [53].
• Activity: This feature suggests learning tasks such as a class, group discussion, or conference to a learner [42,54].

Challenges: Despite the similarities between this and other RS application domains, recommendations in e-learning are quite ambiguous and complex [55]. Often, an e-learner may have inadequate knowledge on what kind of item they need despite knowing what qualification they want. Thus, they are left unsatisfied with the recommendations presented. For instance, Figure 3 shows a scenario where an e-learner interested in a data analyst qualification is recommended a list of courses that are related to his preferred qualification; however, they are not necessarily relevant or suitable in actually obtaining the qualification, as depicted by the Venn diagram on the right. In contrast, all seven courses recommended are related to the data analyst qualification, they are not necessarily relevant for the e-learner to attain the qualification, i.e., only three of them are prerequisite courses (C1, C5, and C7). According to [55], this conundrum of narrowing down a large pool of related learning objects to a finite set of objects that are suitable and relevant is a challenge for tutors or teachers preparing learning material.

4.2. E-Commerce Recommendation System

Retailers, wholesalers, traders, and businesses that target the e-market have been the biggest beneficiaries of RSs since their inception [56,57]. They leverage RSs to achieve two main goals: to firstly suggest products to their customers and, secondly, to provide information to help their customers in making decisions on which product to purchase [57,58]. To achieve this, these RSs analyze customer data such as browsing behavior, purchase history, demographic information, and user preferences [26,39].

Data: Unlike RSs in other application domains such as e-learning, where the customer’s initial profile can effectively inform the RS of his or her preferences [49,51], as well as the most relevant entity required to generate recommendations in e-commerce is the customer’s behavior, which is hinged on the customer’s interaction with the e-commerce site [59]. Customer behavior is obtained after customers perform tasks on the site such as liking, rating, viewing, or/and purchasing a product.The authors in [26,59] discussed how likes, ratings, views, and purchases can subsequently be used to create a customer preference matrix and a product feature matrix, both of which are significant inputs for the e-commerce RSs.

Table 3. Customer preference matrix.

Computer	Likes	Dislikes	Ratings (0–10)	Purchases
Dan	Macbook, Dell	Acer	Macbook, Dell, HP	Macbook
Alec	Dell, Acer, HP	-	Macbook, Dell, HP, Acer	Dell, HP
Ed	Macbook	Acer, HP	Macbook	Macbook
Greg	Dell, Acer, HP	-	Macbook, Dell, HP, Acer	HP

includes examples of a product feature matrix and a preference matrix. Whether a user has an existing profile (expecting a personalized recommendation) or not (expecting a non-personalized recommendation) [26,59], the product feature matrix would suffice in generating a recommendation solely based on the feature statistics drawn from the product matrix. From

Table 4. A product feature matrix used in generating recommendations. Numbers in bold indicate the best score for that particular feature.

Computer	Likes	Dislikes	Ratings (0–10)	Views	Purchases
MacBook	4	1	8	100	5
Dell	4	2	9	98	6
HP	3	1	6	110	6
Acer	1	3	3	20	2

, Dell (Dell Inc., Round Rock, TX, USA) computers will be recommended, because they have the highest scores across most features, achieving 3 out of 5. Conversely, Acer (Acer Inc., Taipei, Taiwan) products would not be recommended, because it has no best scores at all.

In addition to customer behavior, product characteristics are another commonly discussed variable that influences customers in e-commerce. The study conducted by [60] categorized attributes into two categories: hedonic and utilitarian. The latter refers to cognitively driven, instrumental, and goal-oriented attributes that demonstrate what a product accomplishes. Hedonic products, on the other hand, are effective and sensory, being more concerned with aesthetics or sensual pleasure, fantasy, and fun. Broadly, the hedonic–utilitarian attribute [61] has been shown to influence consumer product search behavior, purchase decisions, and even consumers’ value of products [62].

Features: Traditionally, e-commerce RSs only output products as their recommendations to the user; however, this has recently changed, with RSs recommending lists of products ranked in order to relevance [57,63]. The most common RSs in this domain are Amazon (https://www.amazon.com, accessed on 4 March 2024), Netflix (https://www.netflix.com/sa-en/, accessed on 6 March 2024), and eBay (https://www.ebay.com, accessed on 4 March 2024).

Challenges: The burden of RSs in the e-commerce application domain is incomparable because of its huge client-base, which is rapidly expanding every day. This vast, exponential demand inevitably creates a need to constantly scale the RSs, as well as their resources, such as the data stores available in order to keep clients satisfied. Because of this, scalability is a largely documented challenge for e-commerce RSs [17,64,65,66]. For instance, Amazon has to provide recommendations to over 20 million clients from their pool of over 18 million products [64]. In addition to scalability, another very pertinent challenge in e-commerce is data sparsity, which results from some items or products lacking a rating, score, or any metric value that is informative of the product’s likeability. Data sparsity is largely connected to the cold start challenge, which implies that there is insufficient information about an item or a user for a RS to make a recommendation.

4.3. E-Health Recommendation Systems

E-health RSs are designed to provide personalized recommendations to users, which can include patients, healthcare providers, or medical researchers. The primary goal of e-health RSs is to improve various aspects of healthcare delivery and decision-making processes through the generation of personalized suggestions tailored to the specific needs and preferences of the users [67].

Data: E-health RSs consume and analyze data from various sources to provide personalized medical and health recommendations. Some of the most commonly used sources are outlined below:

• Electronic Health Records (EHRs): EHRs are a primary source of patient data recorded at various patient care levels by different healthcare workers, including physicians, nurses, consultants, clinical researchers, etc. Data in EHRs includes medical history or progress notes, diagnostic information, treatment outcomes, and more [68,69].
• Commercial and free health apps and systems: Patient-generated inputs through apps or health portals include subjective data regarding well-being, treatment outcomes, and health status [70].
• Wearable devices: These include smartwatches, fitness trackers, and other health-monitoring devices that offer up-to-the-minute information on a person’s physical activity [71], heart rate, sleep patterns, and other relevant data [72,73,74].
• Clinical research repositories: These are databanks that store publicly accessible biomedical literature published after clinical tasks and activities such as clinical trials, genomics, etc. For example, see PubMed (https://pubmed.ncbi.nlm.nih.gov/, accessed on 8 March 2024) [67].
• Social media: Social media platforms generate vast amounts of user-generated content, including posts, comments, likes, shares, and connections. In recent years, adopting social media data for healthcare research has become common practice [75].
• Internet of Things (IoT) devices: Beyond wearables, other connected devices in home or healthcare settings can provide continuous data streams about patient behavior and their environment [68,76,77].
• Pharmacy data: These are records of pharmaceutical purchases and dispensations, which help in monitoring drug adherence and interactions [31].

Features: The recommendation items provided by e-health recommendation systems are categorized into five main categories below:

• Treatment-based recommendations: These focus on suggesting information that relieves, alleviates, or controls unwanted escalations of symptoms and other health related challenges, e.g., drug recommendation systems [27,28,78].
• Healthcare management-based recommendations: These focus on providing strategies or methods to manage, handle, respond, and live with or without certain conditions. These may range from personalized diet and nutrition recommendations [79,80,81,82] to physical activity and exercise recommendations [71,83,84], health hazard alerts, and tips [85].
• Lifestyle and behavioral management-based recommendations: These focus on providing recommendations for daily habits, behaviors [86], and lifestyle as a whole [87].
• Condition-specific recommendations: These are tailored to meet distinctive needs for specific conditions. For example, mental health [88,89], emotional health (e.g., depression) [90], and well-being [91,92] RSs provide therapy to mental health patients and those suffering from chronic diseases [93] such as diabetes [94] and heart disease [95].
• Specialist care recommendations: These focus on suggesting the most suitable clinicians or trained personnel to provide medical care to patients [96,97,98].

Challenges: Several challenges have hampered research on e-health RSs. This is due, in part, to the fact that there are ethical issues revolving around the dissemination and use of health–related data, especially data from EHR systems [99,100,101]. EHR data are not readily available, and even when they are made available, researchers have to comply to a multitude of data protection and impact assessment policies. Besides the challenge of ensuring the privacy and security of sensitive health data, researchers are confronted by the absence of standardized specifications or classifications that are universally recognized and used. The study conducted by [102] refers to this as the proliferation of healthcare standards, essentially implying that different entities or organizations will use different specifications and classifications for healthcare attributes such as drugs, diseases, and examinations.

Accuracy is another pertinent challenge for e-health RSs because of the tragic consequences that can result from inaccurate recommendations. This is applicable not only to recommendation systems, but to healthcare information technology as a whole [99]. As a result, establishing trust and providing transparent explanations for the recommendations generated by e-health systems becomes crucial for user acceptance and adoption. Furthermore, handling large-scale health data, real-time recommendations, and ensuring efficient and scalable algorithms all present essential challenges in e-health recommendation systems. Needed too is the need to develop appropriate evaluation metrics and methodologies to assess the effectiveness and impact of e-health recommendation systems, which is considered to be a significant challenge for researchers and practitioners [103].

4.4. Tourism Recommendation Systems

E-tourism RSs aim to provide personalized and relevant recommendations to users in the tourism domain [104]. They seek to assist users in making informed decisions about their travel plans [105], destinations or locations [106,107], attractions [108,109], accommodations, hotels [110], restaurants, and activities [104,111].

E-tourism recommendations are is built on the concept of personalization. On several occasions, this personalization has been credited for significantly contributing to the success of the tourism industry [112,113]. The European IST project CRUMPET (Creation of User-friendly Mobile Services Personalized for Tourism) [114] describes the following as features of personalization:

• Adaptation to user’s tourism-related interests and other preferences [115].
• Adaptivity, i.e., the automatic update of the user model based on the user’s interaction history.
• Location-awareness, i.e., the awareness of a user’s current spatial context.

The technologically oriented upheaval in the tourism industry has allowed for the collection of vast amounts of data through internet devices (such as mobile phones) from websites, social media applications, and other web portals [109].

Stakeholders—including researchers, hospitality and catering providers, and tourism service providers—are leveraging this rich source of data to discover more about the interests and preferences of tourists [109]. Subsequently, in consuming this data, researchers have explored RSs and proposed a variety of RS techniques aimed at enhancing tourism services for destinations worldwide [116,117,118].

Data sources: RSs acquire data from various sources, which can be summarized as follows:

• User preferences: These consist of the specific or implied choices users have in relation to their travel selections, encompassing favored destinations, types of attractions, and accommodations. These preferences can be gathered through surveys, and questionnaires, or by studying user interactions and behavior within the recommendation system [109,119].
• Tourist reviews and ratings: User-generated content such as reviews and ratings provided by users are pivotal in tourism recommendation systems. They offer valuable insights into the viewpoints and experiences of past travelers, aiding in the assessment and recommendation of destinations, accommodations, attractions, and activities. Review data can be obtained from various platforms like online travel agencies, review websites, and social media [109].
• Geographical information: This type of data plays a vital role in tourism recommendation systems by furnishing details about the geographical locations of destinations, attractions, and accommodations [120].
• Social media: This is a valuable resource for tourism recommendation systems as social platforms contain copious amounts of user-generated content related to travel experiences. This data can be utilized to grasp user preferences, interests, and connections in order to derive insights into trending destinations, attractions, and influencer-based suggestions [119,120].
• Contextual and environmental data: This type of acquired data comprises details such as weather conditions, seasonal variations, local events, transportation choices, and other environmental aspects that might influence travel decisions. Integrating this data enables the system to offer contextually aware and pertinent recommendations [104,119].

The primary features of tourism RSs can be broadly classified into three main categories:

• Restaurant recommendation systems: These are designed to provide personalized suggestions for restaurants based on user preferences, location, and other relevant factors [115].
• Tourism destination recommender systems: These provide personalized recommendations for travel destinations based on user preferences, interests, and other relevant factors [121].
• Tourist activities recommendation systems: The system provides personalized suggestions for leisure activities, experiences, or events based on user preferences, interests, and other relevant factors [119].

Challenges: Tourism recommendation systems face several challenges. Firstly, data sparsity is a hurdle due to the difficulty of obtaining comprehensive and diverse data on user preferences, reviews, and geographical information. Sparse data can impact the quality of recommendations. Secondly, scalability becomes a challenge as the number of users and tourism-related entities both increase. Efficiently processing recommendations for a large userbase while maintaining real-time performance is demanding. Similarly, ensuring diversity and serendipity in recommendations is another challenge. While personalization is important, introducing novelty and unexpected suggestions to enhance user experiences is necessary. Additionally, subjectivity and trust pose challenges as recommendations are subjective and dependent on individual preferences. Building trust in the system is challenging due to differing expectations and perceptions of relevance and quality. Incorporating contextual information, such as weather and local events, adds further complexity to recommendations. Thus, these dynamic factors require sophisticated modeling techniques and up-to-date data sources. Lastly, privacy and data protection are concerns, as these systems rely on user data. Safeguarding user information while delivering personalized recommendations requires robust data protection mechanisms and compliance with privacy regulations [104,111].

4.5. Entertainment Recommendation Systems

This type of RS suggests a variety of entertainment content to users based on factors like interests, preferences, user ratings, and viewing history [122]. Recommended items may include music, movies [122,123], games, books [124], and other items used for entertainment purposes, typically including genres like romances, mysteries, or novels. However, this type of RS excludes scientific books, publications, and scientific journals, which are often classified under e-learning.

This kind of system aims to help users find content that is suitable and compatible with their preferences and interests, including discovering the right music or movie. User behavior analysis, user ratings, and other factors are employed to offer suitable recommendations [125,126].

Data sources: Entertainment RSs acquire data from various sources, which can be summarized as follows:

• User preferences and behavior: When users interact with the recommendation system, their views, ratings, reviews, and likes are logged in their profile. These data assist the system in anticipating the user’s interests and preferences. Some entertainment recommendation systems can anticipate user behavior, enhancing the user profile and delivering recommendations based on behavior [127,128].
• Content metadata: Here, information about movies, music, and books is gathered. This data source involves collecting source information such as details about movies (e.g., director and actors), music (e.g., singers and keywords), and books (e.g., author(s) and titles) [123].
• Third-party application programming interfaces (APIs): Entertainment recommendation systems are distinguished by their capacity to acquire data from external sources, including social media data, which are commonly known as third-party APIs. Third-party APIs function as a unique supplement to the algorithms of entertainment RSs, improving the precision of their recommendations.

The primary features of entertainment RSs can be broadly classified into the four main categories below:

• Movie RS: This type of recommendation system suggests TV shows, movies, and documentaries to users [129]. These systems not only provide support and assistance to users, but they also benefit content creators by boosting and expanding movie viewership. A prominent example of a movie recommendation system is the Netflix RS, which offers personalized movie suggestions tailored to viewers’ preferences [122,130,131].
• Music RS: Music recommendation systems suggest music objects such as songs that align with user preferences, listening history, and ratings [132]. These systems help listeners by recommending music that resonates with their preferences, benefiting content creators by boosting the listenership of their content [133,134]. Some of the websites that perform music recommendations include Soundcloud (https://soundcloud.com/, accessed on 2 May 2024), lastfm.com (https://www.last.fm/, accessed on 2 May 2024), and AllMusic (https://www.allmusic.com/, accessed on 2 May 2024).
• Game or video game RS: A game recommendation system suggests games that are tailored for the user based on factors like preferences, interests, and gaming history. These systems help users discover games that match their preferences while also supporting game developers by boosting game usage or downloads [135,136,137].
• Book RS: A book recommendation system suggests books to readers based on factors like their preferences and reading history [126]. One notable book recommendation system is Goodreads (www.goodreads.com, accessed on 3 May 2024), a subsidiary of Amazon. The Goodreads website recommends books using various factors, including reader ratings [124].

Challenges: Entertainment RSs comprise such an enormous collection of items available for recommendation [138]. However, a significant portion of this collection is less known or unpopular [139]. These unpopular or new items belong to the long tail of item distribution [140]. Recommending these less-popular items presents a challenge for entertainment RSs, given that there is virtually no prior knowledge of user preference or interest in the items. Striking a balance between suggesting less popular and more popular items is essential to offer users a set of diverse recommendations.

Unless an RS is domain-specific, such as a movie RS, a generic entertainment RS will often have a pool of diverse items to recommend to users. This poses the challenge of accurately recommending a collection of distinct items. Thus, accuracy is crucial for delivering relevant advice, but the lack of diversity restricts the exploration of new user preferences that could be utilized as future recommendations [141].

Obtaining an adequate number of ratings and comments from users is particularly challenging, especially in video game or book RSs. This is particularly evident when compared to other domains within entertainment RSs (such as music). This scarcity of data leads to incomplete insights, resulting in a lack of understanding of user preferences and choices, thereby leading to inaccurate recommendations [141].

4.6. Job Recommendation Systems

Internet-based online recruiting platforms [142], e-recruitment platforms [143], or job RSs are designed to provide guidance not only to job seekers based on their academic qualifications, job experience, preferences, and interests, but they also aid employers based on their hiring needs. These systems analyze job applicants’ profiles, including CVs, and compare them with job requirements to offer relevant advice for job seekers. Similarly, job RSs can provide guidance to employers based on their conditions and compare them with job applicant profiles to suggest suitable candidates for their job requirements [143]. Online recruitment using RS is considered to be one of the most successful business platforms, because it has brought about a change not only in the recruitment of candidates by employers but also in matching job seekers with jobs that suit their abilities and qualifications [142].

Job RSs are required to align or match two different types of stakeholders: job seekers and employers. This approach is referred to, in some studies, as Two-Sided Engagement (TSE). This is in stark contrast to other RSs, whose success is often measured by unilateral user actions (such as recommending a movie to a viewer on the Netflix platform based on their previous preferences or viewings) [142]. This type of RS therefore faces challenges that generally differ from those of other recommendation systems.

Data sources: Job RSs acquire data from various sources, which can be summarized as follows:

• Job seeker applications: This includes qualifications, skills, experience, previous jobs, location, and working hours. This information is typically available in the job applicant’s CV [144,145].
• Employer requirements: This encompasses the specifications of the job offered or available by the employer, detailing the job nature, required skills, qualifications, experience, and additional information [146] such as workplace, working hours, and salary.
• Job postings: Job recommendation systems utilize job postings or advertisements published or submitted by employers as a data source to inform job recommendations. These advertisements outline the requirements and qualifications needed for the job [147]. Platforms like LinkedIn can be used to capture and identify available job postings.
• Feedback and ratings: The advice provided by the system is rated by either the job seeker regarding their satisfaction with the recommended job, or by the employer regarding their satisfaction with the candidate. This, in turn, leads to improved advice in the future [148].

The primary features of job RSs can be broadly classified into two main categories:

• Job seeker RS: This type focuses on providing advice to job seekers. It compares professional skills, qualifications, and experiences with job requirements to suggest suitable job opportunities for the job seeker [144,145,149].
• Job employer RS: This type functions conversely, as it provides advice to the employer rather than the job seeker. It compares job requirements, including qualifications, experience, skills, and required salary, with job applicants to offer appropriate recommendations to the employer [144,146].

Challenges: One of the challenges facing job RSs is finding relevant information [143,150]. As a result, inaccurate or unhelpful advice or recommendations may be provided to the stakeholder, whether it be the job seeker or the employer. This is attributed to numerous factors, with the most significant being the need to align job seekers’ preferences with the job requirements, as previously mentioned. One more challenge that job recommendation systems deal with sensitive data, especially for job seekers. These data are linked to the job seeker’s CV (such as their date of birth, gender, age, and address). Thus, ensuring data privacy and the non-violation of data is one of the challenges facing job RSs [148]. Additionally, recommender systems, in general, and job RSs, in particular, suffer from well-known biases. Providing a fair and transparent recommendation in job RSs is more important than in other types of RSs because it is a high-risk area, as job seeking has a long-term impact on people’s careers and lives. In addition, work plays an important role in shaping and formatting the competitive advantage of institutions and companies in the market. E-recruitment has been classified as a high-risk area according to the EU’s Artificial Intelligence Act (proposal) [143]. Considering fairness, merit, and racial bias are among the most prevalent challenges faced by job RSs compared to other domains of RSs [150].

By the end of this section, the main goal of this paper has been achieved, which is to study the data sources, features, and challenges of different applications of recommendation systems (RSs). A full list of the references reviewed for all six categories of the RSs applications studied above can be found in

Table 5. RS Application Studies by Use and Category.

Category	Application	Reference
E-learning	Activities (e.g., tests) recommendations	[42]
	Courses or/and modules recommendations	[43,44,46]
	Material content suggestions	[53]
	Path recommendations for learners	[49,50,51,54]
E-commerce	Fashion (e.g., retail) recommendation	[57,63,151,152,153,154]
	Product recommendations	[62,155,156,157]
	Taxi recommendations	[158,159,160]
E-health	Drug recommendations	[27,28,78]
	Diet, nutrition, or food recommendations	[79,80,81,161,162,163]
	Physical activities or exercises recommendations	[71,83,84]
	Behavior recommendations	[86,164]
	Lifestyle recommendations	[87]
	Behavior recommendations	[86]
	Mental health recommendations	[88,89]
	Emotion (e.g., depression) recommendations	[90]
	Chronic diseases recommendations (e.g., diabetes and heart disease)	[93,95,165]
E-tourism	Hotels recommendation	[110,166,167,168]
	Restaurant recommendations	[169,170]
	Destination, location, attraction or travel recommendations	[105,106,107,108,109]
E-Entertainment	Movie recommendations	[123,130,171,172,173,174,175,176]
	Music recommendations	[133,177,178,179,180,181,182]
	Game recommendations	[135,137,183]
	Book recommendations	[184,185]
E-Job	Job seeker recommendations	[144,145,146,149,186]
	Job employer recommendations	[144,146]

.

5. Discussion

RSs are central to how organizations and business operate and deliver services to their clients. Previously, organizations relied on traditional data gathering techniques such as questionnaires, surveys, interviews, and focus groups to gain intel on user preferences. However, all of that can be digitally accomplished today using RSs. Many businesses boast large customer bases as a result of leveraging RSs, including Amazon, Netflix, Google, LinkedIn, etc.

RSs have attracted a lot of attention in computer science research in recent years. Discussions on improving the quality of recommendations provided by RSs have been further intensified by prediction-based algorithms based on machine learning and artificial intelligence. The breadth of application use-cases for RSs has been widening in recent years.

This study systematically selected research articles on RSs that were published from 2018 to April 2024, and it surveyed them with a keen focus on three fundamental aspects of RSs: data sources, features, and challenges. While investigating several application domains, this study observed that although user-generated data obtained through end-users (such as personalized and demographic data, reviews, ratings, and preferences) are the main source of information for RSs, other sources have been adopted for the same purpose. Public social media networks were identified data sources for RSs such as tourism and entertainment, particularly because the nature of recommendations can benefit from discourse on public forums. To present a simple example, if there are many people praising a tourist destination on Facebook, it benefits an RS to recommend that destination to users.

In terms of features, it was noted that, as expected, features or functionalities of RSs are domain-specific. In other words, the type of recommendation varies from one domain to another. E-commerce recommends retail or wholesale products, e-learning recommends learning objects or learning materials, and e-health recommends therapies that could be informed about of drugs, behaviors, etc. Moreover, this study found that the products, items, and/or services recommended can be categorized into different groups for their respective RSs. Example of such include e-health sharing treatment-based recommendations and lifestyle and behavioral managed-based recommendations, etc. Similarly, e-learning has learning-pathways-based recommendations, skills-based recommendations, courses, module-based recommendations, etc.

This study also analyzed the challenges faced in the development and implementation of RSs. Unsurprisingly, data sparsity or the cold-start problem, where there is little to no information available for a RS to make a recommendation, is the most widely documented challenge among RSs. Other significant issues included the multiplicity of demands from RSs and their inaccuracies, which can be damaging depending on the nature of recommendation. It was also revealed that there is a consensus among researchers on the necessity of more robust systems. These enhanced systems are needed not only to meet the escalating demand of users but also to provide high-quality and reliable recommendations.

As displayed in Table 5, it is evident that e-health and e-entertainment RS applications are deemed to be the most prevalent, ranking first and second, respectively. They are followed by e-tourism and e-economic RS applications. In contrast, e-learning and e-job RS applications are the least common. Figure 4 illustrates the relative proportions of these applications, thus providing a visual representation of their distribution.

The limitations of this study can be summarized as follows. One of the most important is the lack of studies related to all three aspects of this study (data sources, features, and challenges), especially since, in some cases, there was no explicit and clear statement of these aspects in applications’ studies of recommendation systems; as such, this study had to infer/assume these three aspects from context. In addition, there is an overlap in RS system applications, such as e-commerce RSs with e-tourism (e.g., restaurant RSs) and e-commerce and e-entertainment (e.g., game RSs). It can be difficult to distinguish between categories when there is extensive overlap. Moreover, some features of RS applications are complex, which makes it further difficult to distinguish between them; take, for example, behavior and lifestyle applications in e-health RSs. These limitations are managed and addressed by referring to many studies in the field to obtain a breadth of information that provides a full picture of the knowledge of that field.

6. Conclusions and Future Work

This review explored three understudied aspects of RSs applications: data sources, features, and challenges. To fulfill the primary objective of this study, RS applications were reviewed and classified into six distinct categories: e-learning, e-commerce, e-health, e-tourism, e-entertainment, and e-job recommendation systems. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement was used to select relevant papers by implementing four different stages (identification, filtering, eligibility, and inclusion). This resulted in the review of 139 papers included in this study from four (top-tier) research repositories: the IEEE Xplore Digital Library, Springer Link, ACM Digital Library, and Google Scholar (first 20 pages of search results). Additionally, this study included a review of the RS literature (Section 2), which in turn helped identify weaknesses in the field of RSs and thus define the research parameters. To this extent, the analysis in Section 4 revealed that researchers and RSs developers are adopting other data sources such as social media platforms to supplement the existing array of user-generated data sources for RSs. The various functionalities or features for RSs were deemed domain-specific, since they varied from one application domain to another. This study further reported that data sparsity or the cold-start problem as the most widely documented challenge of RSs. In light of the prevailing challenges encountered in research and development of RSs, the need for more robust RSs is indisputable.

In future work, other RS applications, such as social media platforms (e.g., Facebook and Instagram), should be studied as well, instead of focusing only on the six most common applications of RSs. Such an expansion would be beneficial to a broader audience, as it would address the critical aspects investigated in this study: data sources, features, and functionalities, as well as the challenges of RSs. Additionally, the research could be expanded to include other digital libraries, such as Elsevier and SpringerLink platforms. In addition, possible solutions to these challenges of RSs applications should be analyzed and studied. Furthermore, future work should provide accurate explanations to the user about why a particular item of advice was provided, which is an area that still requires research. This is especially true for recommendations derived from the user’s behavior and feelings, which undoubtedly represents a major challenge for those interested and researchers in the RSs. Providing these explanations, which requires deep learning algorithms, enhances user trust and satisfaction in RSs and increases the chance of users using and relying on them.