The Development of Assistive Systems to Support Older People: Issues that Affect Success in Practice ^†

Abstract

Due to an aging population in Europe, the development of Ambient Assisted Living technologies (AAL) is increasingly the target of research financing. These technologies promise to enable older people to remain in their own homes longer, something many people report wanting and which may also reduce the costs of care. To date however there are few systems on the market. Other studies have tried to understand this by looking at user acceptance. However, by looking only at the user acceptance, we may miss important aspects to explain why systems went wrong in the first place, since decisions made during the development may affect the success later. To address this, we report on a study in which we applied qualitative methods, including workshops and an interactive poster, to consult with people who have experience in the development of these technologies. We present the common issues reported across various AAL development projects done in Europe that the participants thought were important to the success or failure of projects. We also describe some ideas from participants for addressing some of these problems. The results demonstrate the importance of issues related to financing with grants for projects but also the need for improved user-centered practice.

1. Introduction

Europe is aging. It has long been discussed that the age pyramid is shifting—People are living longer and fewer are being born [1]. Through this there will be not only fewer people who can care for the older population but also fewer people working to cover the costs [2].

Some see technology as a possible solution. The term “Ambient Assisted Living” (AAL) was coined in Europe to describe these systems. As the term implies, many of these technologies depend on pervasive technologies to assist people. These include systems to monitor people and generate alarms, for example, if someone has fallen or their blood sugar levels are abnormal. These systems can save money, because not only can carers check on older people less often but they can also enable older people to live independently in their own homes longer.

To support the development of such systems, there has been substantial funding including that of the AAL Joint Program (AAL-JP) of the European Union (EU), now called the “Active and Assisted Living Program.” Despite the fact this funding program started over eight years ago and aimed to get systems on the market in just two to three years [3] (p. 7) and there were also other sources of funding, there are not many systems on the market today. This is demonstrated by the experiences of people looking for systems for family members but also by the fact that this funding program was extended in 2014—This time with a stronger focus on getting products to markets.

Our research tries to understand issues that projects developing AAL technology experience. Most projects include users in the analysis and/or testing phases [3] (p. 10), something, that though not sufficient on its own, is thought to contribute to success in software development more generally [4,5]. Thus, since an effort is being made to understand the needs of users, it is also worthwhile to try to understand the issues developers working in projects face in practice, which they think are most important and whether teams have control over these. To gather information from a diverse set of people with experience developing AAL technologies, two workshops and an interactive poster were conducted in the European context.

2. Developing Technology for Aging in Place

2.1. Assistive Systems for Older People

Following on the EU report on aging in 2006, some companies became aware of the opportunities this expanding market could provide. To encourage investment, a number of funding schemes were introduced. This includes the AAL-JP mentioned previously, in addition to national programs in various countries, such as, the British “Preventative Technology” Grant for telecare services, the German bmbf (German Federal Ministry of Education and Research) focus on “Altersgerechte Assistenzsysteme” (i.e., senior friendly assistance systems) and the Austrian bmvit (Austrian Ministry for Transport, Innovation and Technology) program “benefit”. Many of these were aimed at small and medium size enterprises (SME) that have less capital to invest in research and development projects. The AAL-JP alone supported 130 projects with over 600 million Euros in the time frame 2008 to 2013 [3].

As the name implies, many AAL technologies are assistive. There is a wide range of systems conceived to support older people and the people who care for them, both medical professionals and informal carers, such as family members. These systems include pure software systems, systems with sensors and actuators and also robotic systems. There are many examples of systems intended to assist older people in their everyday lives [6], for example, systems that help people to order groceries, or support them communicating with friends and family. In contrast to standard “apps” and web-based systems, systems aimed at older people may have a simpler user interface or may be operated using remote control and screen from a television. There are also smart home type systems that help people with physical limitations e.g., to turn on the lights without getting up or open and close shades by just pressing a button. Furthermore, there are also robotic systems that can fetch things for people e.g., [7]. Systems that assist older people can improve the quality of daily living by reducing the effort to do things, especially for those with physical disabilities. This is important, as many older people have some sort of physical, sensory or cognitive disability [8]. Due to this and also their life experience, there is a great variability that can pose challenges for developers of systems aimed at older users [8].

There are also many systems designed to support carers working with older people. Some of these systems also have a component for the older people. In this category there are pill boxes that remind people to take their medicine and also ensure they do not take too much e.g., [9]. There are also sensor-based systems that increase security in the home [6], for example, wearable systems to detect falls, ambient systems to detect if someone with dementia leaves a designated area or systems to monitor whether there has been activity in the home. These systems may even include video systems to allow carers to check the situation before calling emergency services, or a button to allow the older person being monitored cancel the alarm. The goal is to increase safety and reduce effort, e.g., carers do not have to come by as often to check if the older person is okay, though evaluations do not show a clear benefit [6,10]. This type of pervasive system can also be combined with pattern analysis of Activities of Daily Living (ADLs) that may identify changes in behavior and support informed decisions about health care [11]. Systems like these may enable people to live in their own homes longer before moving into a care facility, something many older people express a desire to do.

Despite the investments and potential benefits, uptake of such technologies has been slow [12]. This is also supported by the fact that as recently as 2014 new funding programs were launched to continue to encourage investment by companies. Uptake has been slow, even with systems that make it to market. In the Whole System Demonstrator in the UK, the largest randomized trial for these types of telecare and telehealth systems, it was difficult to find people to participate and even among those willing to discuss the possible installation, a high percentage did not actually participate (36%) [12]. Thus, it is interesting to understand why this is happening.

2.2. Previous Research

There has been some research into the user acceptance of these types of systems. Some barriers to adoption of assistive technologies for older people concern privacy, costs and usability; uncertainty about benefits and reliability; suitability for everyday use; lack of perceived need; stigma of using technologies specifically aimed at older people; and fear of dependence and loss of autonomy [13]. Thus, although these systems allow older people to live in their own homes longer, people may also feel less independent or worry that carers will no longer come by [12]. In fact, it has been found that particularly with monitoring systems, there is a tendency to add functions that give control to carers and so may actually reduce independence [14]. At the same time, these monitoring systems are valuable as they may be most acceptable to people who are not open to “care” per se [10].

However, by looking only at the user acceptance of these systems, we may miss important aspects that explain why systems went wrong in the first place. During the development decisions are made about the functionality and design of the systems. These decisions can affect user acceptance later, for example with respect to usefulness or usability that have been identified as barriers to adoption and hence also affect the success on the market. Design decisions have been found to be a decisive factor in deciding whether products are a commercial success, even with systems from the same company like Windows XP^TM or Microsoft BOB^TM [15]. Furthermore, developers may be facing issues that affect whether products are completed successfully, for example technical difficulties with pattern recognition. To examine these issues, we need to consult another important stakeholder—The people involved in the development. Others have investigated the perceived risks in software development more generally to understand why projects fail, in an effort to manage these issues in future projects [16].

Thus, it is valuable to look into the risks, problems or issues that teams face during the development of AAL systems specifically. For example, is it difficult to communicate with older people about these systems, or is it difficult to find people to involve? There are indications from the literature about some problems projects face. An evaluation of projects financed by the AAL-JP of the EU identified problems with integration and scalability [3] (p. 16). An evaluation of AAL projects with Austrian involvement indicates some of the biggest problems related to having unclear project goals, insufficient coordination but also technical problems [17] (p. 30). Others have identified the need to tailor these systems to a narrower group of users [18]. Although the AAL-JP evaluation report that the majority of projects included users [3] (p. 10), an evaluation study done in Austria, covering also projects with broader European involvement, found that people in projects felt this inclusion of users was in many cases insufficient [17] (p. 30). The problems could also relate to challenges working with older people that have been reported elsewhere [19].

Contributing to this understanding, the authors have previously conducted two case studies of projects developing AAL systems in order understand the problems teams face during the development in detail. One was a system developed to run on Android tablets [20]; the other was a more comprehensive smart home system [21]. Both included monitoring features to check the activities of older people and generate an alarm if a problem was detected. Neither was successful in their own goal of establishing a new product on the market. Through these case studies we identified issues in applying user-centered methods and also issues with funding. In the first, although the older people were considered in detail, due to a variety of factors including technical difficulties related to reliability, as the project progressed they were reduced to mere signal generators and the needs of the family carers who received the alarms gained more focus. In the second, many of the difficulties became apparent during the long-term pilot, such as unreliability of the newly installed technical infrastructure, problems with usability, people not using the features they requested at the outset, lots of user support being needed and even small design aspects, such as an LED light, being an issue with users. Moreover, the development was complicated by including multiple partners in order to meet the requirements of the funding scheme used. Some of the issues identified were experienced by both: the diversity of older people, reliability receiving more attention than usability and the needs of carers taking precedence.

This research builds on these case studies. Although some of the issues from the case studies have been reported elsewhere e.g., [22,23], it is not clear whether these problems have been faced more widely. Since it does not make sense for future researchers to focus on finding solutions to unimportant problems or those experienced only in single project, here a focus is put on problems reported by multiple people. Thus, the goal here was to get input from a broad range of people with experience developing AAL systems in Europe, to investigate which issues they have faced, whether these issues have been faced by multiple people, which ones are thought to be most important to the success of projects and how the issues are linked in order to better understand them. In a final step, we asked about what ideas people with experience in the area had for proposing solutions for some of these issues.

This version extends the conference paper [24] by providing more information about the methods and participants, presenting additional results and augmenting the discussion section. Regarding the results, whereas the previous paper presented only an overview of the most important issues, this paper presents all issues identified and also provides more description of the issues, categories and solutions identified by participants.

3. Materials and Methods

The research included two workshops, followed by an interactive poster (see Figure 1). The interactive poster was a poster displayed at a conference, which attendees were invited to add to and annotate. The workshops focused on getting more in-depth information from people who have ample experience and expertise in AAL from either a development or research perspective; the aim of the interactive poster was to get input about issues from a broader and more international group of people.

Both the workshops and interactive poster served to confirm known issues and collect new issues. Since the focus was on issues faced by multiple projects, a list of issues found in previous studies conducted by the authors was provided to participants during these studies.

Established methods were applied. Using workshops to brainstorm issues and find solutions is an established method, e.g., the futures workshop methods [25]. Using an interactive poster at a conference to get input from a broad group of people with experience in a specific area has been used successfully previously [26].

The categorization, labelling, rating of issues, identifying duplicates and finding possible solutions were done by the participants. For both the workshop and the poster, the rating was similar to that used for project risks [16] and was based on two dimensions: (a) how important these were thought to be and (b) whether these are in or outside the control of teams. Literature supports that it is key to consider those issues over which project leaders have some control and hence could possibly prevent in the future [16]. In addition, in the workshop the participants also grouped or categorized the issues they identified and provided some potential solutions. In the analysis, a focus was put on issues faced by multiple projects. Since a novel format was used to identify the key issues in a short amount of time, the individual steps to gather information from the experts are described in more detail in the following.

3.1. The Workshops

Two workshops were held, referred to as workshop 1 and workshop 2. Both had the same format and consisted of a number of steps (see Figure 2). To ensure a broad range of participants, workshop 1 was held by invitation and in conjunction with an AAL meeting in Austria to make it easy for people to attend. Workshop 2 was held at the International Conference on Computers Helping People with Special Needs (ICCHP) [27], which has a track on AAL. This workshop was open to all participants. It was advertised through the conference, workshop organizers and the AAL Austria.

Step 1 of the workshop entailed having participants identify issues. At the beginning, a quick introduction was given to provide people with a common starting point. The participants were asked to brainstorm issues in pairs, based on their personal experience in AAL projects. Each issue was written on a sticky note.

In step 2, the participants were asked to categorize the issues they found to make sense of these issues. To promote discussion, pairs were now asked to join in bigger groups of four or five people. The participants were then asked to go through their issues, mark any duplicates, group the sticky notes with issues into categories inductively and then label the categories. The method for this categorization was not specified—Thematic clustering is common to many methods, e.g., thematic analysis [28] but not always labelled the same way; it also exhibits similarities to affinity mapping. At this point participants were also asked to consider a list of issues identified through the case studies. The complete list is provided in

Table A2. Issues given to workshop participants to gain support for them.

Issue	Supported
Shouldn’t be too obvious, e.g., LED light	✓
Reliability–In all situations, over time	✓
Problems with needed technical infrastructure
Adding partners adds complexity	✓
Diversity of user group	✓
Developers not older—Fall back on stereotypes
Reliability takes over—But usability important to acceptance	✓
What people say (e.g., show home), isn’t necessarily what they do
Special needs for maintenance (esp. with private homes)	✓
Finding people for long-term tests	✓
Changing needs of older people	✓
Needs of carers take precedence	✓
Lots of support needed at start	✓
Change in care makes difference to functions needed
Technologies not financially supported—Carers are	✓

in Appendix A. It was stressed that the participants “can” add issues from the case studies but that it was not necessary to add any of these.

Step 3 of the workshop entailed rating the issues. The participants were asked to work together for this. The rating was set by the position of the sticky note with the issue on the poster and based on the dimensions described above: perceived importance to “success” and the degree to which the issue was in their control (shown in Figure 3). Note that the term “success” was not defined. Other studies indicate that the perceptions of success may vary but in many areas relate to having a working system and satisfied customer and not just being on time and in budget [5].

Finally, in step 4, the participants considered solutions. They were asked to select an issue or cluster of issues of interest to them and explore possible solutions in groups of two to four people. As part of this, they were asked first to describe the problem in more detail and then add ideas about solutions for these. These were then presented to the other participants, at which time additional comments were added.

During the workshops, the researcher took notes on what was said by the participants. Photographs were taken after each step to record the intermediate results. In addition, audio recordings of the workshops were made with separate recordings of each group.

Thus, from the workshops we gained information about issues faced by projects, how these were categorized by the participants, a rating for each issue and also proposed solutions to a few of the issues identified. Including people with a broad range of experience helped to ensure the quality of the results.

3.2. The Interactive Poster

The interactive poster was displayed at the European AAL Forum conference [29] to ensure participation by a wider group of people with experience in AAL projects. The poster was pre-populated with some issues identified in the case studies and/or workshops. The complete list of these issues is provided in

Table A4. Issues included on interactive poster to gain support for them.

Issue	Supported
Communication between stakeholders	✓
Partners chosen to get funding not expertise	✓
Project goals set in stone if funded	✓
Don’t really understand needs of the user group	✓
Project ends with a prototype	✓
Very diverse user group	✓
Privacy concerns	✓
Needs of the users take precedence	✓
People not willing to pay/technology not funded	✓
Benefits hard to prove	✓
Initial support needs high
Hard to find people for long-term tests	✓
Development lifecycles differ: computer vs. buildings
Technical partners want easy funding	✓

in Appendix B.

People could interact with the poster in two ways: people could add a sticky dot next to an issue to indicate they had also experienced this, or they could add new issues by adding sticky notes (see Figure 4). Instructions were provided on the poster and the needed materials were attached to the base of the pin board, so that people could participate at any time during the two days of the conference.

Thus, issues with dots next to them were ones that had been experienced by other people. Putting this at a conference about AAL helped to ensure that people with experience in the area were included. However, since there was no control over who participated, the number of dots is an indication only—Having a high number of “support” dots does not necessarily mean that the problem is most common overall; no dots next to an issue only means that the people who participated with the interactive poster had not experienced the problem and does not necessarily mean no one else has this problem. The dimensions people used to rate the issues were the same as in step 3 of the workshop.

One researcher tried to be close to the poster to encourage people to participate, also because none of the other posters were interactive. Through this it was possible to gather information about some of the people who participated and also gain additional information about the issues people added. The notes taken were used during the analysis.

3.3. Analysis

The two methods of gathering information complemented each other. Through the workshops it was possible to gather information about the experience of the participants and thus ensure a broad range had been included. The workshops went in more detail; the interactive poster reached a larger number of participants from a number of different European countries. Including people involved in AAL projects ensured it examined what happens in the “real world,” i.e., in real settings and not just ideas of what problems could arise.

For the analysis, the issues were checked to see which were supported or duplicated by another participant. For this, all issues and the ratings were entered into Excel^TM. First the items were ordered by source (i.e., workshop group or poster) and grouped by the categories. For the rating, the lines from the poster were used (see Figure 3 above). For the dimension importance, the three color bands were distinguished: the most important (1) being those in the dark band on the right and the least important (3) in the paler area at the left. For control, during the analysis it was only distinguished if the issue was in (1) or outside (0) our control, i.e., above or below the line. First items identified as duplicates by the participants were marked using coloring and notes. The issues were read and re-read in different orders to search for further duplicates between the different columns representing the different sources. Keeping each source in a separate column made it possible to go back to the notes, photos and audio recordings. Including the categories in the worksheet gave additional information about what was meant by the respective issue.

At the end, a list of issues that had been “supported” was made. The final rating was then determined. If the issue had different ratings in different columns, the ratings were averaged and then rounded up.

The categories from the workshops were also compared to look for similarities. This was done both by looking at the names of the categories and also the issues the participants had placed in each of the categories. The issues from the interactive poster were also analyzed to see if there were some that did not seem to fit in any of the categories from the workshops.

The solutions developed by workshop participants were re-drawn and summarized.

Note that two people who were in the case studies took part in one of the workshops. Thus, issues that these people confirmed from the case studies were not considered as confirmed, unless they were confirmed by another source, as well.

Any texts in German were translated by the authors.

3.4. Participants

In workshop 1 five people participated, of which four were invited and one asked to join in on the day. All participants were based in Austria. However, the attendees worked at a diverse set of organizations: two universities, a research organization, a large international company and a start-up. All people had worked in at least one AAL project for a couple of years—One participant had been involved in a number of different projects over a period of eight years. These projects had been funded by various sources, including financing for product development through the company itself, as well as projects including partners from different countries that were financed by the AAL-JP mentioned previously. The backgrounds of the participants included technical disciplines, sociology and business.

Table A1. Information about workshop participants. The column WS indicates in which workshop the person participated. Under Type of funding, benefit is a national funding scheme in Austria for AAL. A Fachhochschule is a “University of Applied Sciences”, a type of university that cannot grant doctorates.

WS	Type of Organization	Focus or Discipline	Type of Funding
1	large company	electrical engineering, management	product
	research organization	computer science, HCI	AAL-JP
	university	HCI, sociology	AAL-JP
	university	electrical engineering, assistive technology	AA-JP, benefit
	start-up company	economics, strategy	product
2	university	computer science, HCI	AAL-JP
	university	embedded systems, assistive technology
	Fachhochschule	sociology, families	AAL-JP, Interreg
	Fachhochschule	computer science, health care
	Fachhochschule	computer science, HCI, health care	benefit
	companies (various)	computer science, pervasive systems	product, AAL-JP, benefit
	Fachhochschule	sociology, telehealth, older people	Interreg, benefit
	research organization	computer science, older people	AAL-JP
	Fachhochschule	human service management, older people	product, benefit

in Appendix A provides more details about the participants. In all, workshop 1 identified 23 issues and developed two solutions.

Workshop 2 had nine participants; even though it was an international conference, all but one of the workshop participants were based in Austria, the other was from Italy. It still included a certain diversity, including people working in higher education, one person from a research organization and one working at a company. Again, all had been involved in at least one AAL project and had diverse backgrounds, including technical disciplines, healthcare and the humanities (see Table A1 in Appendix A for more details). In all, workshop 2 identified 48 issues and developed three solutions.

The interactive poster was displayed during the entire conference and attracted a lot of attention over the two days. Participants included at least sixteen people from nine European countries (see

Table A3. Information about people who contributed to the interactive poster.

Country
Austria
Belgium
Germany
Ireland
Italy
Luxembourg
Romania
Switzerland
United Kingdom

in Appendix B for a list of countries that could be confirmed). In addition to the backgrounds from the workshops, there were also people from funding agencies and gerontology. The projects that were mentioned were also very diverse, including projects with monitoring features like those studied in the case studies, apps, platforms to integrate services, robotic systems and solutions to support older people working longer. In all 28 new issues and 42 dots supporting other issues were added.

3.5. Ethical Aspects

The university does not have a formal ethics approval process. Instead, general ethical principles and processes of working with human subjects for studying software development were applied. According to Singer and Vinson [30], aspects that need to be considered when human subjects are involved in these type of non-invasive studies include being informed, consent, the ability to withdraw, coercion and anonymity.

Participation in all studies was voluntary. Signed consent forms were obtained from all workshop participants in order to record the sessions and also to publish the results. This ensured people were informed about the purpose of the research, that they consented, could restrict the type of data collected and knew they could withdraw. All people were under little pressure to participate: none of these people were under obligation to the researchers or would have had a disadvantage if they had not participated. Regarding anonymity, the data used for the analysis did not include the real names in the files. The data was stored in locked cupboards or on devices accessible only with a password known solely to the researcher.

People who contributed to the interactive poster could choose to participate or not but did not explicitly give their consent. Through the text on the poster, they were informed about the purpose of the research. Their contributions were anonymous, as no information was recorded linking the contributions to the poster to the participants observed.

4. Results

The results include the issues that were confirmed by multiple sources, how these are categorized and the solutions developed at the workshops.

4.1. Issues Identified

Since issues from the workshops were included on the interactive poster, the results from the workshops are presented first.

4.1.1. Workshops

In total 71 issues were identified by the participants. Of these, sixteen were confirmed from multiple sources and hence are included in the final results (Section 4.3)—Nine issues from the list from the case studies and seven issues from other participants. In workshop 1 most issues were rated either most important or of medium importance. In workshop 2 the issues were more distributed (see Figure 3 with the methods). Both workshops identified issues that were in and outside of the control of teams, with the majority of issues rated outside their control.

4.1.2. Interactive Poster

The interactive portion of the poster included an area similar to the poster used for rating issues in the workshops, in addition to information about the research and how to interact with the poster. An excerpt of some new issues and dots supporting issues are shown in Figure 4 with the methods.

In all 28 new issues were added. Thirteen of the fifteen issues printed on the poster and fifteen of the new issues added by other participants were confirmed or supported with red dots.

4.2. Categories

In step 2 of the workshops, in which participants were asked to group the issues into categories (see Figure 5) and to give these a name, each group came up with between three and five categories, each containing several issues (see

Table 1. Categories from the workshops. The number of issues for each category is displayed in parentheses afterwards. Note that in workshop 1, one issue was in two different categories, thus the 1/2.

Workshop 1	Workshop 2, Group a	Workshop 2, Group b
Funding perspective (8)	Users (9)	End-user/development (9)
Commercialization—Marketing (7 1/2)	Money (7)	Financing (5)
User perspective (5)	Product (5)	Solutions (5)
Product perspective—Usability (2 1/2)	Project/Organization (5)
	Other Stakeholders (3)

listing the categories identified by participants and number of issues in each). Within each column, the list is ordered with the categories containing the most issues at the top.

Looking closer, we see that all groups identified categories related to users, financing and the product (see Figure 6 visualizing the categories from each workshop/group), although the wording used by participants was slightly different, e.g., “funding perspective” or “money” or “financing.” Furthermore, some groups had categories that were considered as multiple categories by others, e.g., “End-user/development.” Looking at the issues in the categories supported developing a unified set of categories.

Based on the categories used by participants, the final categories used were:

• Users and other stakeholders: This included aspects both to understanding the needs about the diverse group of older users but also difficulties related to including older people in projects. There was also one issue related to other stakeholders, i.e., the carers.
• Financing: These issues related to using funding programs to support projects. Many of these related to requirements for including additional partners, thus linking it to the category Project.
• Marketing: These issues related to getting products to market (i.e., commercialization), both issues at the start of projects regarding understanding the competition but also later when actually going to market.
• Product: This category included issues related to the features needed in AAL products, such as usability and reliability. Since these relate to the user needs, this is connected to the category Users.
• Project: This included aspects during the development itself, including both more technical aspects, such as testing and aspects related to project management.

The categories related to Financing and Users received the highest number of issues in all groups.

4.3. Issues Identified by Multiple Participants

Not counting issues confirmed by both the workshops and the interactive poster, 33 different issues were identified by participants that were confirmed from multiple sources. These are shown in

Table 2. Issues confirmed from multiple sources, grouped by categories.

Category	Issue	Importance	Control
Users (and other stakeholders)	Don’t really understand needs of the user group	high	in
	Very diverse user group	med	out
	Lots of support needed at the start	med	in
	Finding people for long-term tests	med	in
	Privacy concerns	med	in/out
	Older people don’t want to pay for solutions	med	out
	Changing needs of older people	low	out
	Needs of carers take precedence	low	in
Financing	Problems finding funding/financial partners for new projects	high	out
	Partners chosen to get funding, not expertise	high	out
	Adding partners adds complexity	high	out
	Project ends with only a prototype	high	out
	Companies not interested in investing own resources	low	out
	Technical partners want easy funding	low	in
Marketing	Lack of overview of the players/which projects have been developed	high	in
	Technologies not financially supported—Carers are	high	out
	Low project impact—Business development too late	med	in
	Project goals do not include go-to market rationale	med	in
Product	Technical prototypes too complex for field tests	med	in
	Unproven benefits	med	in
	Benefits hard to prove	med	out
	High reliability needed—In all situations, over time	med	in
	Shouldn’t be too obvious, e.g., LED light	med	out
	Maintenance requires special requirements	low	out
	Reliability takes over—But usability important to acceptance	low	in
	Development lifecycles differ—Computer vs. buildings	low	out
Project	Communication problems between project partners/stakeholders	high	in
	Product definition driven by research topics and tech push (rather than user input)	high	out
	Still little inclusion of users beyond testing	med	in
	Different interests in consortium	med	in
	Different language of different experts—Hard to find agreement	low	in
	Structure & culture of companies differ in consortium	low	out
	Time needed to prove results longer than duration of project	low	out

grouped by the categories identified in the workshops. Of these, nine were rated by participants to be most important to success (i.e., were placed in the dark ring). These are labelled with “high” in the column “Importance” in the table.

Since the issues were written on sticky notes, the issues provided by participants (Table 2) are very described only briefly. Thus, some may be difficult to understand without additional explanation. For example, “Lots of support needed at the start” referred to the fact that older people found the systems difficult to use and required a lot of support from the company when the system was first installed. “High reliability needed—In all situations, over time” pointed to the difficulty of ensuring systems are reliable in real situations and not just in lab environments. In practice, in some projects unexpected occurrences, such as a broken muffler or shifting sun, caused repeated false alarms when both those people being monitored and those receiving the alarms were sleeping and could ultimately lead to the systems being uninstalled. “Shouldn’t be too obvious, e.g., LED light” referred to the fact that for the older users that the systems should not be too intrusive and that in practice even an LED light disturbed some. “Adding partners adds complexity” referred to the fact that many grants financing projects require that partners meeting specific criteria be included, for example partners from different countries or domains, which added complexity to projects. “Project ends with only a prototype” referred to the fact that in part due to the time limitations, the result from projects funded by these programs was actually only a prototype and still far from a commercially viable product.

The categories Users (and other stakeholders), Product and Project have the most supported issues. The category Financing has the most issues rated with a high importance; the category Product, though mentioned by all groups (i.e., in center of Figure 6), included no issues rated with the highest importance.

4.4. Proposed Solutions

In each workshop, the participants were asked to choose one or more issues and consider solutions for these in smaller groups. A description of the issues chosen and results follow.

Workshop 1:

• Financing: Several different issues (all highest level of importance and outside the control of teams): Here the participants focused on funding programs and suggested increased flexibility is needed, including thematic flexibility during the initial phases and more flexibility with deadlines during development to still allow sufficient time for tests despite delays. Here thematic flexibility referred to the possibility to adjust the goals or functionality for a project after a user needs analysis with permission of the financing agency, before the actual development is started. Established test areas or living labs were also thought to have potential to support sufficient long-term testing. Furthermore, additional support for getting to market and coaching for partners was recommended. See Figure 7, showing a redrawn version of the solution from the participants.
• Marketing: “Lack of overview of the players” & “Lack of overview of which projects have been developed” (highest level of importance but in our control), considered to be a single issue by the group. Here the participants developed a list of the players in Austria, including end users, sources of financing and also organizations involved in the development (Figure 8).

Workshop 2:

• Financing: Several different issues (all highest level of importance and outside our control): Here the participants focused on the market perspective and concluded that rather than specific AAL solutions, smaller add-ons to existing products, such as robotic vacuums with added AAL features, may be more successful, at least for established companies.
• Project: “Too little value—AAL technologies” & “Benefit difficult to prove” (medium level of importance but in our control), considered to be a single issue by the group. Here the solution was two-fold: on one hand, developers need to be less technology focused and look at the true needs; on the other, more data is needed from researchers to support the value in order to encourage investment, also from the end-users. During the presentation, the participants discussed that this came about in part due to the fact that AAL emerged from funding.
• Users: “Developers are young → older people stereotyped” (lowest level of importance but in our control). Note this issue was one on the list from the case studies provided by the workshop organizers. Here the group concluded that there is a lot of information about older users available. Better methods and project management are needed to ensure this information is better integrated into projects and that developers talk to the older users. During the presentation, several participants felt that as educators they had failed, as developers were not sufficiently aware of the need to gather information about the users and their needs, despite the fact that older users are clearly so different from themselves.

5. Discussion

Our goal was to understand issues from the perspective of people involved in AAL-related projects that are thought important to success. The most important issues identified were related to the categories Financing, Users, Marketing and Project. Those that were rated to be in the control of teams indeed seem to be things that can be fixed: by creating a list of the players (something that was done for Austria in workshop 1), ensuring there is regular communication between project partners/stakeholder and finding out the needs of the users. Problems with communication in projects may seem surprising coming from experts, as it points to poor project management. However, others have found that the lack of formal project management skills is more generally a risk for software development projects [16].

Since the categories Financing and Users were identified by all workshop groups and both included some of the issues rated as most important, the discussion below centers on these, though also some issues from the other categories are touched upon. Finally, some methodological aspects will be considered.

5.1. Money is Always Important

In both workshops, the issues related to Financing were given a high level of importance. The level of importance may also help to explain why at both workshops someone chose to look at solutions for issues related to financing, even though many were thought to be outside their control. Examining the issues more closely, participants are not just placing the blame on funding agencies—Some issues also show reflection, such as the need to make smaller steps or “Unrealistic project expectations/goals” by project partners.

Particularly the issues in the category Marketing may point to unrealistic expectations on the part of project teams with regard to commercialization—The funding agencies aim only to “foster the emergence” also through “localization” [31]. Issues from the poster indicate that in practice, going to market is in some cases an add-on and not considered from the start. For example, some items from the interactive poster: “Taking into account commercial aspects from the very first moment of concept” or “Product definition driven by research topics and tech push (should be user demand driven).” Both of these were rated in the control of projects. It is interesting to note, that the person who added the issue about commercial aspects was from a funding agency, so developers may not be sufficiently aware of this issue.

If we want to get AAL systems to the market, funding issues are of primary concern. In workshop 1, all issues in the Financing category were rated of highest importance and most issues given highest importance (8 of 13) were in this category (with 3 more about Marketing). In workshop 2, more than half of the issues related to Financing were given the highest rating (7 of 12) and all issues given the highest rating were from this category. Funding problems can result in entire projects being stopped, which was demonstrated by the case studies done previously [20,21]. But they can also lead to people changing the focus. One of the case studies that led to this study being done found that due to costs associated with installation and maintenance a company switched from focusing on people in their own homes to those in care homes; this in turn affected the needs to be addressed in subtle ways and ultimately may have affected the success [21]. Thus, funding agencies may want to reconsider funding guidelines to better support this phase, for example, by providing more flexibility, as suggested by the participants of workshop 1 (see Figure 7), even if this just means delaying the time at which results are submitted with no additional funding, or providing new funding schemes aimed at this.

Once systems are on the market, to be successful they obviously also need to be purchased. The issue “Benefits unproven” indicates there is currently little impetus to invest in these systems. Also other recent research supports the need for more evidence about their effectiveness [32]. Thus, more research is needed with regard to this, however, this is difficult as “Benefits [are] hard to prove.” Another complication is that the aspects that need to be investigated or proven vary depending on who will pay for these systems, whether it is individuals, care agencies or social systems. This determines what is deemed as more important: an extension the time people can live independently, a reduction in the number of visits from professional carers or a reduction in the number of accidents requiring emergency care, just to name a few possibilities. Also others report that there is no agreement about whether savings or effectiveness are most important [33] (p. 73). While the case studies conducted and some of the systems described by workshop participants were aimed at private customers, i.e., older people and their families, others have indicated that these systems could amortize for social insurances in just a few months [34].

Participants in one workshop felt that the lack in value related to the fact that the technology emerged from the funding, e.g., aimed at keeping people in their homes longer [31], rather than emerging from the needs of older people. This is an important issue. If these systems are primarily targeted at saving money for social insurances and features for older people are only added to induce people to agree to having them installed, perhaps we should not be surprised that they are not popular with older people. Older people not only are the ones who are faced with these systems on a day-to-day basis, they are concerned about other aspects, including how useful the systems are for themselves and also whether they will lose existing care structures [23]. As identified by the participants in these studies, from the point of view of older people small aspects about the product, even just an LED that is too obvious, can be problematic, as the appearance is an important factor, even if the primary goals of the system are related to health [35]. If financing programs are framed to save money on care, an LED is a good feature, as it gives feedback about whether the system is still working and so developers may not give sufficient consideration to the fact that it may make the home of the person feel more like a hospital and so affect acceptance. Thus, user needs and preferences must be given more weight—Something that may be impeded by current grant structures that require project goals be set before a user needs analysis has been done. Still there is hope for future projects—Participants believe there are things that can be done both by funding agencies (workshop 1) and companies (workshop 2).

5.2. Users and User-Centered Design

Since the authors come from the area of Human-Computer Interaction (HCI), they expected to see issues related to user-centered design (UCD) and participation. In fact, several issues relating the Users were confirmed, though only one was rated most important. The relatively low importance given to these issues may be due to the fact that older people were not consulted. It may also be due to the fact that many of these issues were considered to be in the control of teams. Furthermore, it has been found that most AAL-JP projects do include users in some form [3]. Since people are including users, they may think there are fewer problems than there actually are.

In contrast to the perceived importance given to these issues, two of the five “solutions” specifically mention that teams need to be more user-centered. This fits with the findings of the evaluation of the EU-wide funding program AAL-JP: “Although users are well represented within projects, they are not sufficiently integrated and overall effective user involvement is still sub-critical” [3] (p. iv). The solution developed in workshop 2 for the issue “Developers are young → older people stereotyped” indicates that participants felt software engineering education has failed to make developers really look at the needs of the users, further evidenced by the issues “Don’t really understand needs of the user group” and “Still little inclusion of users beyond testing.” Furthermore, the solution developed in workshop 1 (Figure 8) demonstrates the large number of stakeholders involved, many of who have different goals. If users are included only for testing, the design and features may be based on the perceived needs described by others.

Given the problems understanding the needs of older people, it is perhaps surprising that the diversity of older people mentioned elsewhere [19,22] was given only medium importance by participants. Although this issue was confirmed by multiple sources, it was not one of the most important issues, perhaps because it is not something people really reflect on, because it is widely known and something that just has to be dealt with. However, one of the solutions developed provides a potential way of dealing with this diversity: developing smaller add-ons rather than large comprehensive products. This could support matching the more specific needs of people by tailoring the features incorporated, i.e., only those desired and needed. This would also address the barrier identified elsewhere, that people are concerned about the suitability of these systems for everyday use [13]. Allowing flexibility with regard to the functionality may, however, create new challenges, as it can compromise reliability and requires all possible configurations be tested if the systems relate to security [36]. Moreover, some sort of support or advising may be needed to support people selecting appropriate features.

Overall, the issues in the category Users indicate available resources and methods are not being tapped by developers in order to understand older users and their diversity. Many developers have access to people with more experience with older users and family members who are older. Furthermore, funding programs have financed the development resources specifically for AAL. This includes a set of personas developed to help raise understanding of and empathy with older people [37] and a toolbox of UCD methods appropriate to AAL projects [38]. Despite their experience in AAL, even the participants of workshop 2 did not seem to be aware of these resources. In fact, a study conducted in Austria found projects developing AAL technology tend to use more general methods like interviews [39], rather than methods specific to user-centered design, such as shadowing, that are recommended by this toolbox of methods for AAL [38]. This has been found also by other studies on software development done elsewhere in Europe [40]. When selecting UCD methods the features of a project need to be considered, e.g., the amount of time pressure, the skills of the people and also the amount of access to users available [41]. Furthermore, special methods may be required for working with older people [42,43].

Even if developers do know and apply appropriate methods to gain understanding about the needs of users, there may be difficulties applying these methods. A recent survey of projects developing these types of technologies found that at least in Austria older users are often not included in the early phases of AAL projects and only included in the final evaluation [39]. This is especially a problem since one issue identified was “Finding people for long-term tests,” something that has been reported in a publication about projects including partners from different European countries [44]. Instead participants reported that other stakeholders are included in the development instead and that “Needs of carers take precedence”. This is critical, as others have found the priorities of older people may differ substantially to those of carers, for example, by putting more importance on privacy and aesthetics than on safety [22,36,45]—Also the participants reported that privacy is an issue when developing AAL technologies. Not giving older people sufficient voice in products and using them only to check if things work as expected, may impact the success of the systems.

The problems may also arise later during the development, after needs analysis. This same study about projects in Austria indicates there are problems communicating with the developers [39], meaning that the user researchers and even designers may know the needs of the users but the information does not reach the developers, who may need to make design decisions during the project. Instead, it may just be that teams are distracted from user issues as technical issues take over, as has been found in one of the case studies leading up to the workshops [20]. The issues from the workshops, e.g., “High reliability needed in diverse situations,” that “Reliability takes over—But usability important to acceptance,” support this may indeed be happening. Reliability is certainly an important aspect for these types of systems [13,46]. To help ensure that the users and their needs do not get forgotten, techniques like those used to ensure personas are kept alive may help, for example, using the user stories to evaluate functions considered, or regular e-mails with new facts relating to the users [47]. Good project management and frequent team meetings could also help remedy these problems in the development by reminding the team of the project goals. At the same time, these measures could address the communication issues that were identified in the category Project, which were thought to be important to success.

Finally, there are challenges related to the evaluation of the systems with users. On one hand, it is advantageous if evaluations are done early in the development while changes are still possible, however, in practice “Technical prototypes (are) too complex for field tests” making it difficult to get feedback. As documented elsewhere it may also be difficult to explain intangible aspects, such as those related to sensors and activity monitoring, to older people [19]. As reported by the participants, people involved in long-term tests may need a lot of support initially. This should not deter from doing evaluations with older people as they are particularly important with AAL systems, also because participants reported “Changing needs of older people,” which could mean that the needs of the people involved may change even in the course of a single project. Having longer periods of test may help uncover a realistic range of changes people experience and give a more accurate view of systems to ensure the resulting systems meet the real needs of older people longer term. Also others have found that the needs of older people change with regard to these types of systems even after six months, as people may experience physiological changes and also gain confidence with the technology [48].

The issues relating to the users indicate the need on the one hand to understand more precisely what is happening in projects, for example through further case studies and on the other to spread information about UCD methods in the AAL community but also to gather more evidence regarding their suitability.

5.3. Methodological Aspects

The methods aimed to explore issues projects have faced. It included only a limited number of projects and participants. As such, the results do not include all issues experienced by projects, or even necessarily the most important ones. Furthermore, although the issues presented here were reported by multiple people involved in projects, there are other projects that have not experienced some or even all of the issues identified.

Input was gained from at least 30 participants with experience in the development of AAL systems. Although this is not a large number in comparison to more general studies investigating risks in projects [16], we are dealing with only a small field. The specific invitations to workshop 1 ensured that a wide variety of participants and people with a lot of experience were included.

The participants may not be representative of the community as a whole. Thus, the results only say whether another participant had the problem, not that no one else did or how frequent it is. The focus was placed on the European context. However, since most participants in the workshops were from Austria, it may mean that some of the issues from the workshops and solutions are specific to this context. But this may have less importance than it first seems, certainly with respect to the AAL-JP, Austria has been one of the countries involved in the largest number of projects [17]. Furthermore, over half (at least 8 of 15) of those workshop participants from Austria had been involved in projects that included partners from several European countries (e.g., those funded by the EU funding programs AAL-JP or Interreg). Conducting the interactive poster at a European conference in another country ensured views also from researchers based in other European countries were included, so that in all approximately half of those participants contributing to the issues were based in other countries.

In workshop 1, all issues were assigned either the highest or medium level of importance—None were included in the lowest “band.” In the second workshop, both groups had issues in all bands of importance. Thus, the participants in the workshop 1 may have simply given things a higher priority but they may also have focused on those things that were most important, because they were invited specifically to share their knowledge.

From the text on a sticky note alone, it can be hard to know what participants meant with some issues, particularly with the interactive poster, where no categories were available to help interpret. If an author was nearby, in some cases additional information could be gained, which was also useful when for checking for duplicates. For the workshops, the audio recordings and categories supported understanding what was meant.

6. Conclusions

This research gained evidence about the issues faced in practice in projects developing AAL technologies in the European context. When asked, people with experience in the development of AAL systems identified a wide variety of issues about users, financing, commercialization, projects and products that affect whether the development of AAL systems is successful. Many of the issues rated most important related to monetary aspects—Both funding/grants and marketing. Many of these issues were felt to be outside of the control of the teams. Although some of the issues have been mentioned previously, together with the categories they give a more complete picture and contribute to understanding the difficulties teams face when developing these technologies.

One area of concern is that despite users being included in the development, experts feel that developers are not user-centered enough. Workshop participants suggested teams need to focus on the real needs of older people and that both better methods and better project management are needed to remedy this. As a community, we also need to spread the word about information about older people and appropriate methods that is available, so every team does not start from scratch. The issues identified indicate that the situation is complex, so it is not just an issue of trying to be more user-centered. Current funding structures also introduce challenges for projects developing these types of technologies by having specific requirements for partners and setting both goals and deadlines in stone. Additional flexibility in grants, such as flexibility regarding the goals and functionality in the early phases and more flexible deadlines during development may help support more success in the future.

Since these studies were done in the European context, with a majority of participants based in Austria, it would be interesting to extend this research through similar studies in other contexts.