Sustainable Methodology for Operational and Formal Digital Skills Acquisition: A Case Study of e-Health Inclusion

Abstract

This study explains the rationale of a methodology developed by the Universidad Politécnica of Madrid (UPM) group TechPeopleCare as applied to the e-Health Inclusion through ICT Training project partly funded by the European Institute of Technology EIT-Health in 2019. An initial sample of 168 participants with different lifestyles and migrant backgrounds, with high female participation, were recruited in three different countries by three different organisations following strict ethical protocols that limit the data that can be shared. The learning materials were aimed at people lacking the operational and formal skills to use digital media, for example, using a mouse, a keyboard, and navigating the Internet. This learning would enable these cohorts to become beneficiaries of e-Health interventions, such as making a doctor’s appointment, accessing a health record, finding the location of a health centre or the nearest open pharmacy. By the end of the training programme, we found that the motivation to learn was high. The possibility of reviewing learning content at the individual’s pace and without the need of an instructor was appreciated, especially by younger cohorts with migrant backgrounds. A majority reported being satisfied with their learning of the health systems, unique to each country, and willing to learn more regardless of the training method. However, allowing for individual and independent learning “by doing” appears more accessible to suit different lifestyles and more sustainable than traditional computer classes. Since social and digital inequality are intertwined, sustainable and innovative learning programmes in developing countries within communities specifically addressing the acquisition of operational and formal skills are a pre-condition to move forward and bridge the gap of being on the wrong side of the digital divide.

1. Introduction

Ninety-three per cent of the entire world population lives within reach of a mobile network. Nevertheless, Internet access is higher than computer ownership worldwide. At one extreme, 87% of individuals in developed countries are online, in contrast with 19% in the least developed countries. These figures indicate that more people are accessing the Internet using other electronic devices [1]. Nevertheless, access to Information and Communication Technologies (ICTs) is not enough [2] to acquire digital media skills. To put it differently, it appears that the main reason why people in the European Union (EU) do not have internet at home is that they lack digital media skills [3] rather than access to the Internet. The Digital Economy and Society Index (DESI) 2020 [4] reports that 42% of people in Europe lack basic digital media skills, for example, sending an e-mail with an attachment. Apart from the 20% of the population that are 65 and over [5], there are also individuals with a migrant background, refugees, adults with disabilities, those living in rural areas, and adults with low prior formal education that are finding it difficult to embrace today’s technological demands.

At the same time, it is argued that the way information is presented on the Internet is not cognitively accessible to all. Equally important are the operational skills needed to use the mouse or touchpads, the keyboard or the formal skills needed to navigate the Internet without getting lost. Without these initial skills, it is challenging for anyone to make progress in the digital skills learning process. However, traditional learning strategies, such as computer labs used to run computer classes led by a computer instructor, do not always work for older generations and other groups. Scholars claim that resources aimed to teach and learn digital media are deficient [2]. Generally speaking, decision-makers use an old education paradigm to approach digital media skills acquisition that does not address the younger or older population’s needs. Therefore, other solutions are needed [2,6].

At the present time, it is important to interpret what the pandemic situation had highlighted when communication and learning were done mainly online. Emergent research shows the deficiencies concerning educational systems and how it affects those from a disadvantaged socioeconomic and educational background. At the same time finding new formulas to remove issues related to digital literacy is an immediate priority for at least five of the Sustainable Development Goals (SDG 1, SDG 5, SDG 8, and SDG 10) to be reached by 2030 [7]. Therefore, we argue that delegating this critical task only to cooperation projects funded by donations and performed by volunteers is not a sustainable solution when addressing the digital divide. For instance, we consider that traditional group-based digital media training is not a practical option for accessibility, scalability, and sustainability requirements [6,8,9] neither in developing or developed countries based on our investigations since 2006 to date.

Meanwhile, according to the 2020 DESI’s report [4], Internet use peaked during the pandemic, although it has been a trend with 49% of people using video calls in 2018 compared to 60% in 2019. In contrast, only 11% completed a course online. While data from the International Telecommunication Union (ITU) [1] indicates that younger age groups have more skills than older groups, we can see in this study that owning and using a smartphone does not reflect on basic digital media skills. Moreover, e-learning is far from accessible to cohorts from traditionally socio-demographic excluded groups. It is not only the lack of access to the hardware that creates the digital divide. Nevertheless, access to Information and Communication Technologies (ICTs) defined as the first level of the digital divide is not enough [2,10] to acquire digital media skills.

On the other hand, while older people can improve their quality of life [4] with online activities, the pandemic has highlighted how “disconnected” they are compared to younger generations. However, research suggests that e-learning is not accessible to people with low digital media skills [11] regardless of their age. New research based on online learning problems experienced by young families during the lockdowns appears to point out that learning online is not an option without digital literacy basics. However, this is not covered in our review.

Not surprisingly, and despite all digital skills programmes in the European Union and other developed countries over the years, many cohorts still need specific support due to ageing, technological advances, poorly designed e-services, among other factors. Therefore, it remains a challenge for all stakeholders to make public e-Health related information cognitively accessible to all. The project in which this study is based supported three key priorities through digital inclusion: promoting healthy living, supporting active ageing, and improving healthcare. Our role as project leaders was to develop the training material, coordinate the project, and present results reports and conclusions. The purpose of this study is to examine the effectiveness of the TechPeopleCare (TPC) methodology in the acquisition of operational and formal digital skills of adult learners in Europe that would enable them to access public e-Health services or health information. Learners in this project lacked the operational and formal skills to use a computer and navigate the Internet even if they owned the tools to do so. Thus, we address the following research question: “How effective is the TPC methodology to acquire operational and formal digital skills in the case of adult learners in developed countries?”

In this study, we examine the sample demographics and how effectively the TPC methodology was applied through the ICT Training project (eHi-ICTT project, also called connect@health project) [12]. It was completed at the end of 2019, and it aimed to develop and test learning materials aimed at groups of people that lacked operational and formal digital skills to access public e-Health services or health information from which they could benefit. To answer the research question, we use a case study to give a more holistic view of the methodology’s effectiveness [13]. Three participating organisations hosted the training pilots: eSeniors (France), SERMAS (Spain), and Leyden Academy on Vitality and Ageing (The Netherlands). They recruited a sample of 168 adults living in the three European countries, respectively. The experiences we captured are in real-world context within the existing natural settings typical of training or health care and appear suitable for a case study [14,15]. In contrast with experiments, manipulating circumstances to see different outcomes and establish cause and effect [14] is not a case study’s remit.

The results are descriptive and were obtained using both quantitative and qualitative tools such as attendance, pre and post questionnaires, semi-structured interviews and one-to-one unstructured interviews, focus groups, and observations. We examined the reports provided by each organisation against the research question for each pilot. The results presented in this study are generic due to data protection conflicting issues. Empirical data are treated using the statistical weighted average method.

By the end of the training programme, we found that (i) the motivation to learn was high, (ii) the possibility of reviewing learning content at the individual’s pace and without an instructor’s presence was appreciated, especially by cohorts with migration backgrounds, (iii) a majority of the participants reported being satisfied with the learning of their health systems, and (iv) participants were willing to learn more in the use of technologies regardless of the training method. However, allowing for individual and independent learning “by doing” without a teacher, appears to be effective to acquiring operational and formal digital skills. It also seems more accessible and sustainable to suit different lifestyles as discussed further in Section 4.

Since the research was conducted before the pandemic, an investigation on how learning these particular skills in the context of e-health services remains very relevant to our ongoing research and objectives. Ultimately, knowing the impact that this training may have had for those who took it, will further add empirical evidence to validating TPC methodology.

The paper is organised as follows. The next section reviews the background underpinning the TPC digital learning methodology’s rationale and current thinking regarding the acquisition of digital media skills. Section 3 explains the TPC methodology and how it was applied in the connect@health project. Section 4 summarises and discusses the results. Finally, in Section 5, we conclude and discuss the limitations of the TechPeopleCare methodology in this project, future research, and policy implications.

2. Literature Review

This section reviews literature underpinning the rationale of the TPC methodology in more depth than in previous publications. However, while addressing the digital divide phenomena, our focus remains on having sustainable solutions that work with excluded cohorts in developed countries. We find it necessary to include a section on e-accessibility and the relationship between quality of life, digital inclusion, and the accessibility of e-health services. We define the skills addressed in this project within an existing framework that identifies different categories that aid the empirical measurement of digital media skills. With this in mind, we distinctly identify the skills addressed in this study. We find it necessary to dedicate a section to deal with learning in four different dimensions by (i) exploring learning metaphors relevant in the context of basic computer instruction and the Internet and the typical challenges that this presents in a traditional computer class, (ii) the status quo of education paradigms that do not seem to address digital inclusion, (iii) exploring what are relevant materials and the importance of co-creation, (iv) why to use instructional videos innovatively to promote independent learning and learning by doing. This background, although not exhaustive, aims to share essential concepts to be considered when creating content and solutions for older adult learners and other excluded groups. In our review of previous and current methods of teaching operational and formal digital skills, we did not come across other methodologies similar to TPC for comparison, only the traditional computer classes for beginners.

2.1. Digital Divide

The digital divide research has accumulated a considerable body of literature over the past 25 years [2], identifying three levels. The first level is about motivation and access to ICTs. The second level refers to digital skills and usage of digital media or ICTs. The third level represents the positive and negative outcomes of using digital media or not. With the Internet reaching almost all parts of the world, “The Internet has become a basic utility for social inclusion” [16] (pp. 354–355). Van Dijk says that social and digital inequality are inseparable and that the consequences of being digitally excluded exacerbate the inequality experienced by disadvantaged cohorts. He explains that “Categorical inequalities in society produce an unequal distribution of resources that, in turn, causes unequal access to digital technologies, also depending on the characteristics of these technologies. Unequal access to digital technologies brings about unequal outcomes of participation in society which reinforces categorical inequalities and unequal distribution of resources”. Van Dijk concludes that, “the digital divide cannot be closed but only mitigated” [2] (p. 132).

2.2. e-Accessibility

There are two intertwined areas of concern in e-Accessibility. One area is related to Assistive Technologies (AT), and the second is the area of general cognitive accessibility to the content and software on the Internet. In this paper, our focus lies in the second area. Many stakeholders agree that everyone will benefit from suitable products accessible to all. General cognitive accessibility or “one size fits it all” should no longer be the standard [17] (p. 525). The International Federation of Library Associations and Institutions (IFLA) [18] published a professional report with the guidelines for easy-to-read (E2R) materials claiming that “Providing easy to read materials is a matter of democracy and accessibility”. Missenberger et al. [17] (p. 527) posited that “The majority of users acts as a consumer and at the same time provider/author of information, independently from a possible (cognitive) disability. This makes it necessary that the rules and guidelines to be followed are presented and implemented in a way that can be understood and used by everyone”.

The United Nations Convention on the Rights of Persons with Disabilities (CRDP), World Wide Web Consortium (W3C), IFLA, the European Union (EU), and other stakeholders are working to address discrimination and exclusion from access to digital content [19]. For this reason, the EU introduced laws that will address e-Accessibility and digital equality issues that Waddington [19] discusses in detail. Among them are the Public Sector Web Accessibility Directive and European Accessibility Act. In addition, the standard “Accessibility requirements for ICT products and services” is a highly technical document that serves both instruments to assess websites [19].

Article 4 of the directive of Public Sector Web Accessibility as cited in Waddington’s paper [19] (p. 10), says that “Member States shall ensure that public sector bodies take the necessary measures to make their website and mobile applications more accessible by making them perceivable, operable, understandable and robust”. These four concepts were produced by W3C [17,19]. Waddington explains that “perceivable” means that Web content can be perceived by the user’s brain regardless of the senses they can use; “operable” means that the web content can be accessed and navigated regardless of the user’s devices; “understandable means that web content can be understood as easily as possible through simple language and contextual information; and “robust” means that web content can be accessed regardless of the user’s operating system, browser, and browser version, including with the use of assistive technologies” [19] (p. 10). These are also the concepts framing the Accessibility Requirements for ICT Products and Services standard.

There is much more to add to this discussion and how designers, developers, publishers, providers, content creators, engineers, and the involvement of peer-researchers [20], who are the ultimate beneficiaries of these outcomes, can contribute to what could be a solution for most users with or without a disability. We consider that this topic is directly related to e-Health interventions and the quality of life of its users.

2.3. e-Health and QoL (Quality of Life)

The e-Health services industry is a growing area of concern for all stakeholders. e-Health refers to health services that use electronic processes via the Internet. Many of the current e-services are designed under certain assumptions, for example, that people have specific knowledge or access to relevant technology and, therefore, can benefit from it. These assumptions do not reflect the reality of many people that are digital excluded as we witnessed at the beginning of the pandemic. For instance, accessing a private e-health record, or making a doctor’s appointment online can present several challenges to any user. Without support, for example, family’s support, digitally excluded people may depend on friends and other organisations that can be under-resourced to attend to technical needs and issues. Moreover, these informal supports can raise data privacy concerns. The fact that younger generations in the family can operate older family’s member devices often makes older people feel dependent [21,22,23]. It does not align with active aging and this support may not always be the solution to solve digital inclusion for this cohort. Ali et al. [10] (p. 1) list factors directly related to QoL such as “lifestyle, access to digital technologies, social and community environment, physical and mental health conditions and social inclusion and demographic factors”. Their study was carried out in Australia and investigated how simultaneous the relationship between QoL and digital inclusion is using broadband access data. They recognise that although this is not how digital inclusion should be defined, even if access to the Internet is considered an essential utility [16], it showed a strong relationship between QoL and digital inclusion.

In our review, we did not find specific information about the skills necessary to access a digital health record using the Internet. Therefore, we use the following example to illustrate possible assumptions made by decision-makers about the beneficiaries’skills of e-Health online services. Generally speaking, some assumptions are (1) a person or user can log in and resolve issues, such as forgetting password, often resolved via e-mail regardless of the device they use; (2) they also know how to either upload or attach a file (find it on the device) or download a file (find it on the device) in the event that this is a requirement of some process; (3) users own and know how to use a computer (or related device) without help to do (1) and (2); (4) users own a relevant device and have all the skills in place to do (1) and (2) on their own or with help; (5) that someone will be able to show them how to acquire the skills to do the necessary processes; (6) that someone will show users how the process of accessing e-health records works or that they will figure it out using guides. These assumptions do not need to be in any particular order, neither are they exhaustive. The skills related to these generic assumptions are the concern of many basic digital literacy skills acquisition programmes such as the one in this study. There is a disparity between the assumptions and what many users can actually do. Therefore, while advances in technology make possible solutions that will improve the quality of life of those who need it, many people still lack essential digital skills that will enable them to take advantage of technological progress. These skills are very much related to QoL, and they need attention. Thirty years of online access and these skills remain a challenge to teach and master for the foreseeable future if public sector online services, such as making a doctor’s appointment online, are to replace fully offline services.

2.4. Digital Media Skills Framework

Different frameworks can throw light on new phenomena such as the digital divide. However, it is challenging to work with different frameworks when the conceptual classification is new and varied. Policymakers’ digital skills programmes seem to have a different approach than scholars. They tend to look at the digital divide from a labour market and financial perspective [2]. Van Dijk and van Deursen created an empirical framework [24] taking into consideration social and cultural factors including income, time available, needs, and cognitive and technical ability [2]. The scholars use those concepts to understand and measure the skills needed to use the Internet. The framework is not normative but sets six medium and content-related digital skills to explain the different levels of knowledge. This framework recognises operational and formal skills as a precondition to learning more advanced digital media skills and can be described as the action required to operate a digital medium or “button knowledge”.

On the one hand, operational skills refer to the required skills to use the Internet. On the other hand, formal skills refer to the abilities that allow browsing the Internet efficiently, for example, not getting disoriented. According to this framework, the combination of both is called medium-related digital skills. They argued that without these skills, progress in acquiring higher skills will not happen [25]. Innovations that made the medium itself accessible to all may, in the future, eliminate the problem [2,17,19]. However, this will take time. According to Van Dijk and van Deursen [24], people learn digital media skills informally with self-study, learning-by-doing and others’ help. In their classification is also included learning at work, learning through formal education (primary to third-level education), and informal education (for example public learning centres, libraries). Digital skills can also be acquired informally from family and friends. Van Dijk and van Deursen’s framework also focus on information, communication, content creation, and strategic skills, all of which are content-related and not the topic of this study.

2.5. A Brief Overview on Learning

Learning is a complex and abstract subject that cannot be thoroughly covered in this article. We have chosen the topics that follow for these reasons: (1) to illustrate the level of learning we address with TCP methodology in this study using learning metaphors, (2) to show the status quo of adult learning solutions considering digital inclusion, (3) to illustrate what relevant materials mean to adult learners, the creators of content and software, and why co-creation is essential, and (4) to explore the role of instructional videos, a critical component in the TCP methodology.

2.5.1. Learning Metaphors

Different theories of learning explain the abstract concept of how we learn. Educators and researchers use these metaphors to formulate teaching and learning strategies and learning frameworks. Vygotsky’s theory [23] helps us to illustrate in this paper the type of help adult learners need to make progress when they lack basic digital media skills. Vygotsky’s [26] Zone of Proximal Development (ZPD) concept essentially explains that there are things that a person cannot do without the help of others regardless of their age but also that all learning happens in a social context. Albeit this is a widely accepted view in many technology-enhanced learning approaches, it has a caveat. When helping someone that cannot make progress autonomously, the individual’s needs must be clearly understood and addressed [27] by the “more capable adult”. To put this abstract concept in the context of how medium digital skills are taught in current learning programmes remains a challenge. For example, a beginners computer class’ syllabus trying to cover many topics briefly. When adults are involved in informal learning using digital devices, they often seek support from the instructor to help them “learn by doing” a specific process. Unfortunately, this is not always possible due to lack of resources such as time. Moreover, this insufficient support, confusing instructions, a cluttered screen, and a non-customised device, can create many stressful situations for the adult learner, particularly older learners, and especially at the beginning of the learning process.

In traditional computer tuition, one teacher may need to help a group of eight to ten people within a typical time constraint of between one to two hours. People who find digital media technologies alien often need one-on-one help, particularly with dexterity challenges [28]. For instance, learning to use the mouse (touchpad or touchscreen) may take longer for older learners or learners that have never use it before. First, the learner needs to get used to positioning the hand over the device to notice the shape and learn its functionality. Second, the learner needs to practice with it and with relevant tasks to see what happens. Typically, a computer tutor cannot afford this time and may lack the resources to scaffold this process. Scaffolding is another learning metaphor mentioned first by Bruner [29], who linked it to the ZPD [27] and is used in instructional design. Effective scaffolding is critical for adults that are lacking basic digital media skills regardless of the training methodology.

2.5.2. Adult Learning

Sustainable Development Goal (SDG) four is part of the 2030 Agenda for Sustainable Development [7] to highlight the need for inclusive and equitable quality education among the groups facing the most significant barriers to participation in Adult Learning and Education (ALE) [30]. In these groups, there are people with migrant backgrounds and refugees, older adults, adults with disabilities, those living in rural areas, and adults with low prior educational attainment, which is approximately 42% of the European population [4]. Adult learning theories explain why adults learn differently from children [23,24]. Some of these theories emphasise using different teaching and learning strategies for different ages, for example, younger versus older adults, arguing that each cohort’s needs, for example cognitive abilities, are different. It seems that involving the learner in material and content development processes [23] would be a good standard. However, national and international policies regarding Adult Learning and Education lack a vision of how to develop and integrate holistic solutions to today’s technological challenges [23,30].

Until recently, policymakers believed that having access to a computer and little instruction will suffice to acquire digital media skills. However, there has been a shift in more developed countries and they are exploring now more holistic solutions to reduce digital exclusion [2], but there is still a lack of research. At the same time, there is another narrative that does not find the teaching of basic digital skills a priority as this need will eventually disappear. They reason that most people old enough will not exist, and younger people will acquire them naturally, for example, at work or through formal or informal education [2]. However, this view has been weakened by the pandemic putting on the spot formal and informal learning organisations experiencing an unprecedented challenge regarding the digital skills of many children, parents, teachers, and adult learners.

In addition, research shows that these views are inaccurate, and it is because of these views that the methodology TPC was developed. We investigate innovative ways to make it easier for adult learners that otherwise are facing exclusion from learning lifelong digital media skills due to their age, cognitive ability, socio-economic status, education, or lifestyle.

2.5.3. Relevant Materials

In the learning process and according to Van Dijk [2] (p. 154), solutions have to be “relevant and adapted to the particular needs, social relationships, lifestyles, and cultures of the potential users”, such as special hardware and software with particular cohorts such as functionally illiterate and people with specific disabilities. He also explains that “special content relevant for cultural minorities and socially deprived groups” can benefit these cohorts if adapting local information to the minority group’s language to provide relevance [6,8,9].

In a recent study by Martinez-Alcala et al. [21], researchers identified the importance of collaborating with older seniors to consider their experiences and interaction with the software and the content. They compared face-to-face teaching with blended digital literacy workshops using a content management system that older adults have to learn how to navigate. In another study, testing two different online banking websites [31], participants requested changes that designers and developers did not consider challenging, such as menus.

In a European project involving four countries, the researchers [28] did not find any difference in older learners’ attitudes towards learning how to use a tablet for the first time. Most people who had never used a tablet before were willing to learn, but not all. In this experiment, a control group used a tablet to play games before taking a traditional computer class on how to use the smartphone. The purpose was to test how this prior experience, instead of having the instructor dealing with dexterity issues, would improve these skills before taking the smartphone class. They found that people who used the tablet in the experiment could do tasks faster than those that did not play games with a tablet. The group that played games on the tablet showed more familiarity with the smartphone’s touchscreen, hence transferring the previous learning. However, some games required complicated dexterity gestures for all users, highlighting that some hardware is not accessible to all. Moreover, we argue that this innovative approach involving games may not be initially perceived as relevant by older learners. The researchers mentioned that some participants were reluctant to participate in the experiment and had to be “tricked”.

In order to learn operational and formal digital media skills, individuals must interact with the media. The relevance of the learning content [25], such as materials designed for children or younger people, is as essential as the device they will operate. This content has to have value for the adult learner. Adult literacy literature has extensive research on the importance of the learning content to engage learners with the learning process, which seems to be missing, in our view, when developing digital literacy programmes aimed at adult learners. However, exploring these areas in depth is beyond the scope of this paper.

2.5.4. Instructional Videos

Instructional videos are available online and offline, providing solutions for different needs. Many argue that e-learning and offline learning materials are costly to produce. There is no standard to measure their usability and purpose, and the content can become outdated, be too easy, too complex, have a poor design or may not be fit for purpose. Although the typical problem seems to be the wrong audience accessing the learning content, these assumptions are all valid. For example, learning content for younger adults do not necessarily work with older adults [32]. Moreover, perceptions towards newer technologies and learning are different from a younger person to an older person [22,33].

In general terms, cognitive theories inform instructional design methods where learning is considered a mental process [23]. The aim of the design process of instructional videos is to scaffold [29] the learning at hand. This happens by breaking down the information into components or learning blocks. The main purpose of instructional design should be to allow the users to learn independently without requiring extra help, for example, without needing a teacher or the “more capable other” [26], assuming that it targets the correct audience. In this design process, ideally, the developers should involve the beneficiaries [17]. Learners can have a clearer idea of their needs instead of the needs addressed as perceived by developers, designers, and researchers, as the study of Gatsou et al. [31] discussed in the online banking study. In many cases, simplicity is the key, and when this is accomplished, more users may benefit from the same content.

3. Methodology

The previous section dealt with why this methodology seems to be aligned with current thinking and why we think it is sustainable and effective in acquiring basic digital skills. This section explains how we apply TPC methodology to reach those assumptions by means of the connect@health project [12] using a case study that helps us blend a description of events with their analysis [13]. This study attempts to evaluate the effectiveness of an innovative methodology with four distinctive groups, learning informally in various learning environments across three countries, adding validity to the innovation tested. The variety of measurement tools makes the case study an appropriate method not to generalise [14] but to highlight the differences among the groups in real-world contexts within the natural settings [14,15] typical of training, healthcare and well-being. The expectation is to see similar outcomes from the different groups, i.e. they have learned basic skills that will enable them to improve further as long as they are motivated to continue learning.

The following items explain each of the subsections that illustrate the approach:

• Training needs analysis
• Material development
• Training experience
• Feedback compilation
• Evaluation

3.1. Training Needs Analysis

The TPC methodology creates a specific course for target groups with particular learning needs following some concepts of adult learning such as specific help mentioned briefly in the literature review. This strategy aims to match the overall learning goals with the group’s characteristics. The learning needs analysis informs about the programme’s specific learning objectives and environments, access to equipment, characteristics of the target audience, for example, language, age, gender, previous education. In the context of acquiring operational and formal digital skills, understanding the group’s learning needs is paramount to developing the learning components. This is in addition to the needs identified to help the facilitators with the software environment. In this methodology, facilitators are the staff that assist with the equipment whenever is necessary, mainly at the initial session only. It is essential to clarify that, facilitators are not teachers and do not need any specific technical or pedagogical training. However, they need to have the necessary skills for these learning scenarios, for example a librarian.

There are different possibilities to address the training need analysis. One is defining the overall needs of a specific group by talking to the institutions or companies that want to apply the methodology. Then, once participants have been recruited, for Training Experience (Section 3.3) refine the training needs analysis when necessary. A second approach is first to recruit the participants and analyse their actual needs. The advantage of the second approach is that the concrete needs are known from the beginning. However, the advantage of the first approach is that it can be done in a shorter period since only representatives from the interested parties take part in this initial analysis. They have a good knowledge of the target group typically and have the means to recruit participants that fit the criteria.

In this project, given the one-year time frame of the whole project, the first approach was taken. Initially, the institutions involved in the project, SERMAS, eSeniors, and Leyden Academy on Vitality and Ageing, identified the main characteristics of their target groups. SERMAS or Madrilenian Health Service also proposed items in the course contents (Appendix A). However, the rest of the participating organisations agreed that the contents were relevant for their groups too. Each country has a different health system and SERMAS requested specific e-health topics for Group 2 such using a paediatric dose calculator or learning how to search for food allergies among others. This will explain the discrepancy on videos in Appendix A. Another organisation requested content related to use e-mail. However, the skills related to e-mail are more advanced and according to the digital skills framework discussed in Section 2.4 belong to communication skills rather than operational and formal skills [24].

Table 1. Group’s Classification.

Group	1 (SERMAS)	2 (SERMAS)	3 (eSeniors)	4 (Leyden)
Language	Spanish	Arabic	French	Arabic
Country	Spain	Spain	France	The Netherlands
Age	Senior citizen	Senior or Non-senior	Senior citizen	Senior citizen
Gender	All	Female	All	All
Lifestyle	Citizen	Migrant background	Citizen	Migrant background

displays the information gathered in the initial training needs analysis regarding the classification of the four groups. The specific learning needs that all stakeholders involved agreed upon were accessing eHealth related web content using the Internet, a PC, a mouse, and a keyboard. Smartphone and tablets have advantages over laptops or PCs, but they cannot be considered a substitute [16] for various reasons, such as smaller screen sizes and reduced functionality when typing, that may add more cognitive overload according to Bao et al., Murphy et al., and Napoli and Obar as cited by [16] (p. 357).

3.2. Material Development

In the previous section, we have described the training need analysis that provides the overall setting for the development of the training material used in the training experience. The key ingredients of this component are a two-screen approach, easy to understand videos, and a controlled software environment.

In this project, we developed a new software training application that provided a controlled environment adapted explicitly to users without basic digital knowledge with the purpose of acquiring medium digital media skills. The use of this software is critical for the training’s success allowing participants to use it without assistance due to its simplicity.

In this study, a controlled software environment helped with the challenge of “cluttered or busy” PCs or tablets. This backdrop facilitates the progress over the audio-visual content flawlessly, as the potential distractions from the screen are eliminated. As mentioned earlier in the literature [26], there are learning situations where people cannot progress further without others’ help. In order to operate a computer, there are certain things that everyone needs to know how to do independently, for example, knowing where to click on the screen. In this case, the controlled software environment has anticipated many potential interferences avoiding cognitive overload to have the learner focused of the task.

When comparing traditional computer classes’ set-ups with this controlled software environment set-up, the former has significant disadvantages when teaching operational and formal skills. Naturally, the equipment is not exclusively dedicated to people challenged by new technologies. This issue with equipment sharing could probably be resolved in other ways (profile accounts), but generally speaking, this is not done in many public computer labs. Other individuals have to use the same computers, and they can be full of applications for all users. Furthermore, interactive icons can make the desktop screen look “busy or cluttered” for the untrained eye. Other artefacts, such as notifications, add to the confusion if they are interacted with accidentally. Typically, the computer instructor has to rectify the confusion in the training session, first by identifying what happened and then bringing back the right screen. This process takes time, interrupts the class flow, and in many instances, the adult learner feels that it has done something wrong, which is true. However, since they do not understand what they did, a simple task appears more complicated, impacting the learner’s sense of achievement but also reducing the instructor’s time. Hence, the relevance of the controlled software environment when acquiring formal and medium digital media skills.

Starting earlier in our projects, we observed that the use of two screens for training reduces cognitive overload with younger cohorts [6,20,34]. We found that similar tasks were performed faster if they could be carried out on separate screens: one that carried the instructions and another to execute them, for example, a tablet. A tablet is efficient and affordable equipment that does not require the use of a mouse or keyboard to interact with. However, it may present a problem to users with dexterity issues that have not used a device with a touchscreen before. The facilitator’s role is important in the initial stages of the training. All things considered, a second screen that is easy to navigate offers clarity between the teaching, that is, the tablet or instruction screen, and the practising environment or the screen used to execute the tasks. As a result, the process of “learning by doing” becomes easier through real-time imitation that the inexperienced learner can perform individually with minimum assistance. This dual-screen set-up also helps the learner to stay focused on the task at hand [29,34,35].

The audio-visual materials job is to replace the instructor in the sense of a traditional computer class, allowing adults to learn at their own pace watching the videos as many times as needed independently. It could be argued that this is what e-learning does, but e-learning refers to learning online. This learning of formal and operational skills is a pre-condition to be online. The learners that take this training cannot avail of courses online without mastering specific skills first. Many practitioners teaching older adults would agree that it is very frustrating not to reach everyone at the early stages. For instance, learning dexterity with the mouse or keyboard is challenging for most beginners without someone’s help [26]. For older people, this is one of the initial biggest challenges.

This critical learning goal has negative implications if perceived as too challenging, complicated, or tedious, or if it makes the learner think they are wasting people’s time, regardless of the role of the person teaching, for example, a family member, a tutor or a librarian. The pedagogy behind the development of the audio-visual materials considers the cognitive processes for each task, for example, memory [29]. We mentioned in Section 3.1 that the request of including e-mail in the contents was not accepted and had to be discussed at length. Learning how to use e-mail requires a different learning process related to communication skills. However, we agree that is a necessary skill to have but it needs to be addressed when other skills are in place i.e. using the keyboard. The cognitive aspect of the methodology has been observed through different projects, fine-tuned and improved, for example, by slowing down the video’s pace for particular individuals or groups or by eliminating interfering contents.

In this project, each group’s syllabus consisted of approximately 80 videos. Each video unit undertakes an iteration of design, review, development. A second review guarantees the audio-visual training material’s pedagogical, linguistic, and academic quality. In this project we observed the number of videos that were watched several times by more than one user with groups 1 and 2. As mentioned in the literature review, creating content that is relevant to the target audience is critical [2] (p. 154) [23,24]. It is also crucial in the case of software development [17,21,31,32]. We also use the concept of co-creation in the development of the material. For instance, in this particular project, the materials were preliminarily tested with groups 1 and 2. This helped us to find technical glitches or “bugs” and to include additional functionality overlooked by the developers in the initial design. Their feedback was used to inform the creation of content for the remaining groups classified in Table 1. Other adjustments were required for every country’s e-Health system environment since they are very different, so the co-creation was with the organisations in order to choose what was more relevant on their Health systems that learners could benefit from. In Appendix A, we present the generic content specifically created for this project.

3.3. Training Experience

The training experience is the component of our methodology, where the developed training material is provided to the learners. In a non-pandemic situation, the ideal learning environment is to have the equipment and materials available through the open hours of the learning centre so that it is accessible to people throughout the day to maximise the use of computer equipment. This is a key feature of the sustainability of the methodology. Nevertheless, any logistics is possible depending on the characteristics of the training experience’s host. For example, establishing a fixed time slide for training, booking the use of the digital equipment, sending the equipment to the home of each participant, or even downloading the software and training materials in the user’s technical equipment as long as this environment can be controlled initially for best results following this methodology’s core principles.

As it has been explained earlier, our methodology improves the learning experience by using a double screen to facilitate the “learning by doing” [23], allowing an imitating process in real-time. Therefore, one device is needed to practice, for example, a laptop or a PC with a screen, keyboard, and a mouse. Furthermore, an auxiliary device, typically a tablet, is needed to play the audio-visual training material complemented with headphones. Headphones are needed when the learning occurs in a noisy environment, especially if there is more than one learner in the room, therefore, allowing people to follow their own learning pace with less distractions. Typically, introductory videos are devoted to explaining how to use the devices and how to manipulate the tablet to advance during the course. Furthermore, a help document for the facilitators provides the framework for the teaching and learning environment where the learning will take place.

In this project, it was essential to use an external mouse because it helped with some seniors’ motricity, and it is easier to use than touchpads on laptops in general [28]. It was also a specific requirement by one of the organisations. The learning environments where the learning took place were different. They are classified in

Table 2. Characteristics of each group.

Group	1	2	3	4
Language	Spanish	Arabic	French	Arabic
Country	Spain	Spain	France	Netherlands
Age/lifestyle	Senior citizen	Non-senior migrant background	Senior citizen	Senior Migrant citizen
Initial Sample	63	42	26	37
Women	36	42	15	27
Men	27	0	11	10
Participated in the training	29	28	26	37
Completed the training	26	25	26	0
Lifestyle	Citizen	Migrant background	Citizen	Migrant background
Learning Environment	Informal Computer learning centre	Informal learning centres	Informal Computer learning centre for seniors	Informal meet and greet centres for migrants

as informal [24]. Table 2 displays the information about each group’s characteristics. In this case, the learning took place simultaneously with groups of people learning with the organisation’s equipment. However, other experiences could be organised as explained before to be received in the home environment, for example, for people confined at home regardless of the pandemic.

In this project, participants were selected by key health agents such as General Practitioners, nurses, teachers, and community representatives for the different groups and locations such as meet and greet centres, cultural centres, health centres, health training schools, day-care facilities for older adults and alike, or computer training centres for older people. There were strict ethics and protocols that each organisation needed to adhere to before the training.

More than half of the 37 participants from Group 4 have issues with the hosting country’s language and the Arabic language. They were learning to read and write through Dutch and had low literacy skills in their dialect. The audio-visual instructions were not fit for purpose due to the lack of knowledge of generic Arabic. A different type of intervention was needed here.

3.4. Feedback

The fourth component of our methodology involves the feedback received by all stakeholders in applying the methodology. Timely data collection is a critical feature in any project to demonstrate cohesion and fitness for purpose [13]. It helps to establish what causes an issue at different stages. This could be with the training needs analysis, with the learning content, with the software environment, with the content development process or with the training experience. Early feedback from the training recruitment process is used to refine the training needs analysis. As explained earlier, the methodology follows a collaborative co-creation development. For example, in this project, groups 1 and 2 took part in early testing the final version of the videos, helping to address some design and technical issues. They voluntarily provide their comments via unstructured interviews with the developers and facilitators to improve the audio-visual materials’ content while also obtaining user experience feedback. After the training, more feedback is obtained regarding satisfaction with the learning experience, motivation and the training methodology.

This feedback is obtained throughout the project using different tools to determine the quality of the learning materials, the methodology, and the contents. Each project has different requirements but data sets are triangulated to evaluate the effectiveness of the methodology. The following feedback was obtained:

3.4.1. Target Group Definition

Aims of this methodology are design, development, and implementation of courses suitable for people with a history of migration; therefore, language is an essential aspect in the process. The translations have to be precise and in the same common language as the one spoken by the group or the hosting country but using easy to read language. However, in this study, Group 2 and 4 showed a preference to learn more about the host country’s language itself. Group 3 reported inaccuracies with the translation of some terms Furthermore, in Group 4, there were participants learning how to read and write for the first time through the hosting country’s language and this was not effectively addressed.

3.4.2. Motivation

Are the participants motivated to learn? Do they keep interested until the course completion? In this study, most participants were very motivated initially, and they kept interested until the course completion. Most of the participants who could not complete the course had problems understanding the contents because they had low literacy levels. In other instances, life events such as jobs or illness were the reasons to stop the training. An important aspect of TPC methodology is to have the materials available so people can have access to this learning when it is convenient to them. Having fixed dates and times defeats one of the potential strengths that can be measured for sustainability and accessibility expected from the methodology. The same number of participants, such as Group 3, or less such as Group 1 and 2 were able to finish the training.

3.4.3. Satisfaction with the Methodology

Do participants appreciate the characteristics of the methodology? Do they consider it an advantage to get training that suits their learning pace and their availability? In this study, participants appreciated the characteristics of the innovative methodology, especially for individual training. Many people commented positively on their experience with the audio-visual materials as the instruction source rather than learning from a teacher directly because it was vital for them to repeat the explanations and have time to assimilate the new information. This alone does not provide evidence of the effectiviness of the methodology.

3.4.4. Satisfaction with the Training

We also checked whether participants were satisfied with their learning. In this study, the participants were highly satisfied and reported interest in continuing the training with further contents. However, this alone does not provide evidence of learning. To provide more understanding that the learning took place other data tools were used. For example, in order to reach the end of the training the participants must complete all the activities following the sequential order without assistance. There were also one-to-one unstructured interviews with groups 1, 2, 3, 4 with the participant researchers and organisation representatives. Although we present Group 4 results as zero, individuals with less language difficulties were observed to have completed the training.

3.5. Evaluation

The fifth component of the TPC methodology is the evaluation of the participants’ skills acquisition. There are different methods to carry out such an evaluation. For instance, asking participants to perform some tasks they have been learning during the training. According to Van Deursen and van Dijk [16], this method shows more tangible results regarding the skills acquisition than asking participants to assess their perceived level of performance. Another approach is to use questionnaires before and after the training experience as well as focus groups and observations. This second approach was the one chosen in the connect@health project [12] due to the project time constrains. For example, many people who had never used the Internet, a mouse, or a keyboard before were observed performing several search tasks individually without assistance after the training. This indicates that one of the objectives of the training regarding the acquisition of medium skills was achieved.

4. Results and Discussion

In the previous section, the main characteristics of the TPC methodology have been described and illustrated by means of the connect@health project [12]. The goal was to train adult learners who lacked operational and formal digital skills that would enable them to access e-Health services or health information using the Internet. In this section, we discuss the feedback (Section 3.4) component through narrower research questions to answer: “How effective is the TPC methodology to acquire operational and formal digital skills in the case of adult learners in developed countries”?

4.1. How Do We Know That Participants Were Motivated? Do They Keep Interested until the Course Completion?

We gather data through attendance sheets, by the number of videos they completed, and by one-to-one unstructured interviews where they expressed their reasons for attending or not attending. Not only the participants that completed the training were motivated. People had to drop out of the course for other reasons. The exact number of participants from Group 3 remained until the end. Three people in Group 1 and 2 did not complete the training out of 29 and 28 participants, respectively. Fourteen people from Group 2 were not able to do the training due to family or work commitments. If the training course had been available to their convenience, there is reason to speculate that they would have continued.

4.2. How Do We Know That They Were Satisfied with the Methodology?

There are limitations to the validity of the responses. The initial response from a majority was that they preferred to learn with a teacher. This response alone does not provide evidence of the effectiveness of the methodology. On the one hand, people were observed repeating challenging videos and reported that it was vital for them to watch the explanations again and have the time to assimilate the new information. On the other hand, more abstract reflections are needed: (1) Are questionnaires adequate tools for measurement this aspect of the methodology? (2) Should focus groups or semi-structured interviews have been used instead? The question was initially written in English and translated to Arabic, Spanish and French. It has only a yes or a no response. Data collected using one-to-one unstructured interviews in the co-creation process with groups 1 and 2, and later with Group 4 showed contradictions. Older learners also wanted to take notes or have the instructions in print. This is not effective or pragmatic, regardless of the methodology used. However, Group 3 are used to attend traditional computer classes with a teacher. They liked these classes. It could be the case that respondents were trying to please the researchers or representatives of each organisation [13]. This could mean that they were biased and worried that they would disappoint the teacher or that their answers would end being in groups (where they socialise). Since many participants have low education levels, we think that questionnaires are not a tool fit for where written language can be an issue.

4.3. How Do We Know That the Participants Were Satisfied with Their Training? Do Participants Appreciate the Characteristics of the Methodology? Do They Consider It an Advantage to Get Training That Suits Their Learning Pace and Their Availability?

Based on the responses we gathered from the initial reports, the expectation would have been that participants would be reluctant to learn through this methodology or to follow the training materials. However, this alone does not provide evidence of the effectiveness of the methodology. Not surprisingly, a majority of participants that finished the training were highly satisfied and reported interest in continuing the training with other contents.

4.4. How Did We Evaluate the Participant’s Skills in This Project? How Do We Evaluate That the Desired Learning Took Place?

Due to the project time constraints, we used questionnaires so participants could rate themselves. Therefore, this is not enough to provide evidence of learning. However, as mentioned in Section 3.5, it is better to ask them to perform tasks and measure the results. We have seen evidence of this self-efficacy in other projects in developing countries.

Nevertheless, the organisation’s representatives, facilitators or the participant researchers with Group 1 and Group 2 reported that many people who had never used the Internet, a mouse, or a keyboard before the training, were observed performing several search tasks individually without assistance during the training. In particular, the tasks related to the web browser category and map and addresses are listed in Appendix A. These skills are one of the main objectives, which shows a strong indication that they were achieved.

Following are the descriptive results we can share from the reports provided by each organisation before, during and after the training. Of 168 initial recruited participants, 77 completed the training as shown in Table 2. By the end of the training programme we found that (i) the motivation to learn was high, (ii) the possibility of reviewing learning content at the individual’s pace and without an instructor’s presence was appreciated, especially by cohorts with migration backgrounds, (iii) a majority of the participants reported being satisfied with their learning of the health systems, unique to each country, and (iv) participants were willing to learn more in the use of technologies regardless of the training method. However, allowing for individual and independent learning “by doing” appears more accessible and sustainable to suit different lifestyles.

Forty-one per cent of respondents had a migrant background compared to 59% citizens (French and Spanish). Female participation was positive with 75%. Eighty-two per cent were 60+ years old. However, there was low male participation in the mixed groups. There could be several reasons to explain this. For example, one reason may have been that males are more advanced or think they are, which could be a case of self-efficacy [16]. Another reason may be that there are underlying cultural issues regarding learning perceptions at a late age. In Group 4, male participants had more language and computer skills, with a majority owning a smartphone.

In the following, we analyse and discuss some feedback obtained according to Section 3.4. The results from questionnaires show the knowledge and ability to access the health system in each country are as follows. Before the learning experience, 70% of groups 1 and 2 found it difficult to complete tasks such as making an appointment with health centres or searching for the closest pharmacy open, searching for healthcare resources or searching for information about diseases or ailments. At the end of the training, 90% thought that their knowledge of the health system improved. In Group 3, 56% knew some aspects of the health system, 40% wanted to know more, and 4% were not interested. At the end of the training, 100% said they knew more about their health system than before they did the course to locate open pharmacies, hospitals, and make a medical appointment using the Internet. In Group 4, 65% knew their health system. However, tasks that needed to be done online concerning insurance issues were difficult to access and understand because they are written in the country’s hosting language, presenting a barrier. This mixed group also commented on the feelings of dependency from their family and their willingness to learn to eliminate this dependency [10,21,22] by learning digital skills.

Participants were also asked about the methodology regarding self-study and using videos to learn without a teacher. There was a majority that, given the opportunity, preferred a teacher, particularly those from groups 1 and 3. Our interpretation of this answer is threefold. In the first place, those who prefer the teacher feel the need to have someone nearby to ask if something does not work for them, some sort of support, because typically, they are afraid of breaking the equipment. The second one is probably cultural and age-related, where the traditional learning system is all they had ever experienced, if they had at all, and they are expecting to be taught that way rather than using technology that they are just starting to understand. Learning happens at an individual level but mainly in a social context, and socialising to senior people is as important as the learning goals. In contrast, Group 4 find a class group to learn computers distracting. They needed to concentrate rather than engage with other social activities and found the individual training with headphones appropriate.

Participants were asked if they owned a smartphone and, if they did, what did they use it for and how often. The same question was asked about PCs and laptops. Regarding smartphones, 75% of 114 respondents owned one, although nearly 13% only use it to make ordinary phone calls. Many women in group 4 will use the smartphone only for calls and other digital tasks with their grandchildren’s help.

We will add to the results with the following discussion. Learning is embedded in everything we do. ICTs have a major role in what we need to learn, how, when, and why. The term lifelong learning is there to remind us that technology adoption is a long process. We upskill, learn again, or learn something new as our lifestyles change overtime. ICTs are ubiquitous, especially in developing countries and it is hard to believe that some people struggle with doing basic tasks like sending e-mails, but the evidence tells another story.

In Section 2.3, we mentioned the assumptions taken by decision-makers when designing and developing e-Health services. It is worth to mention that the content for acquiring basic digital skills with a desktop computer and browsing the Internet was not decided by the learners for obvious reasons. Therefore, the choices were based on the specific contents covered in traditional computer classes and considered relevant for beginners by the organisations in this project.

In this methodology, the accessibility of the learning materials refers to the time allocated to access the learning contents in the physical location and without an instructor, and sustainability refers to the number of people, with similar characteristics who can avail of the same relevant materials over a period of time to acquire a particular set of skills. We compare these two aspects with the following: (i) accessibility to traditional computer classes teaching similar skills where the time with the computer instructor is generally limited to one or two hours per day or per week; (ii) the number of learners per classroom that can avail of the learning at any given time. The availability of the learning equipment typically dictates this; (iii) the number of times that each learner may need to learn the same materials. However, in this project, participants accessed the materials at times convenient to the organisations instead of the learners’ convenience. Within the project’s timeframe most learners completed the programme, seemingly, having the time to learn at their own pace. Compared to previous projects, we could only measure the number of learners that took advantage of these learning materials during the project’s timeframe and from the study’s sample. The project also had a follow up planned to measure the impact of the training after six months, provided that funding was available. Unfortunately, this was not the case, and with this in mind, our research question can only be answered based on the data obtained during the one-year time frame of the project, limiting the validity of this research.

5. Conclusions

In this paper, we have explained the TPC methodology aimed to acquire operational and formal digital skills within an eHealth context. We have illustrated the methodology by means of the connect@health project, where more than 77 participants completed the training. The methodology relies on concepts like self-study, learning by doing without a teacher, and co-creation. In this project, we were able to expand the TPC methodology with targeted groups living in Europe. After the training, these skills allowed most participants to use the Internet and access eHealth services using a computer, a mouse, and a keyboard, one of the project’s main goals. Therefore, we consider that the methodology was effective.

In previous publications, we focused on a more in-depth analysis of the sustainability of this methodology initially developed to teach various computer skills to younger adults in developing countries. Our arguments from previous publications also apply to developed countries. We find that our reflection can open other research paths in this area considering the lack of investigations regarding adult learners’ acquisition of operational and formal digital media skills. Therefore, we think that this study can contribute to the knowledge in this field.

Since the research was conducted before the pandemic, an investigation on how learning these particular skills in the context of e-health services remains very relevant to our ongoing research and objectives. Ultimately, knowing the impact this training may have had on those who took it, will add additional empirical evidence to validating TPC methodology.

If member states of the EU have public sector websites that are incomprehensible for many people, it means that the online sites fail to be accessible for all. Implementing the e-Accessibility laws in the EU will probably set a positive precedent. We think it is essential to look for solutions for digital inclusion that work at a grassroots level. They need to be adaptable to lifestyles in communities and be ubiquitous, scalable, sustainable, and improved with the beneficiaries’ participation.