The Co-Design of an Embodied Conversational Agent to Help Stroke Survivors Manage Their Recovery

Abstract

Whilst the use of digital interventions to assist patients with self-management involving embodied conversational agents (ECA) is emerging, the use of such agents to support stroke rehabilitation and recovery is rare. This iTakeCharge project takes inspiration from the evidence-based narrative style self-management intervention for stroke recovery, the ‘Take Charge’ intervention, which has been shown to contribute to significant improvements in disability and quality of life after stroke. We worked with the developers and deliverers of the ‘Take Charge’ intervention tool, clinical stroke researchers and stroke survivors, to adapt the ‘Take Charge’ intervention tool to be delivered by an ECA (i.e., the Taking Charge Intelligent Agent (TaCIA)). TaCIA was co-designed using a three-phased approach: Stage 1: Phase I with the developers and Phase II with people who delivered the original Take Charge intervention to stroke survivors (i.e., facilitators); and Stage 2: Phase III with stroke survivors. This paper reports the results from each of these phases including an evaluation of the resulting ECA. Stage 1: Phase I, where TaCIA V.1 was evaluated by the Take Charge developers, did not build a good working alliance, provide adequate options, or deliver the intended Take Charge outcomes. In particular, the use of answer options and the coaching aspects of TaCIA V.1 were felt to conflict with the intention that Take Charge facilitators would not influence the responses of the patient. In response, in Stage 1: Phase II, TaCIA V.2 incorporated an experiment to determine the value of providing answer options versus free text responses. Take Charge facilitators agreed that allowing an open response concurrently with providing answer options was optimal and determined that working alliance and usability were satisfactory. Finally, in Stage 2: Phase III, TaCIA V.3 was evaluated with eight stroke survivors and was generally well accepted and considered useful. Increased user control, clarification of TaCIA’s role, and other improvements to improve accessibility were suggested. The article concludes with limitations and recommendations for future changes based on stroke survivor feedback.

1. Introduction

One-third of the people who survive a stroke are left with residual deficits and problems which render them dependent on others to engage in activities or daily living and participation in the community [1]. Self-management intervention studies demonstrate benefit for stroke survivors’ recovery [2], highlighting the benefits in enabling stroke survivors to take an active role in managing their stroke recovery, physical and mental health, and lifestyle behaviors which contribute to secondary stroke risk. The self-management intervention for stroke recovery, the ‘Take Charge’ intervention, has been shown to contribute to significant improvements in disability and quality of life after stroke [3,4].

The ‘Take Charge’ intervention is a narrative-based tool that can be delivered in person by a non-health professional who has skills in listening. This person, referred to as a facilitator by the developers of the tool, does not provide solutions for the person with stroke; rather, they encourage them to identify their stroke-related problems, nominate their stroke recovery goals, and then describe what they perceive they need to do to achieve these goals. The conversation between the stroke survivor and the facilitator seeks to explore what is important to the stroke survivor by working through the 11 modules: Who I Really Am; Hopes and Aspirations; Main Fear; My Best Day; Physical Needs; Communication; Emotional Issues; Information Needs; Financial Issues; My Support Network; and Stroke Prevention [5]. The first four modules comprise the ‘Looking at the Big Picture’ component of the program, aiming to motivate the stroke survivor to take charge (or control) of their recovery. The remaining seven modules of the eleven-module session are goal- and task-focused. The ‘Take Charge’ intervention tool includes a participant workbook and facilitator training resources.

Generally, the delivery of face-to-face interventions like the ‘Take Charge’ intervention poses some challenges both in delivery and participation by the stroke survivor—for example, facilitators reaching remote areas, financial costs involved in delivery by and travel to a facilitator, time availability, limited trained facilitators, and flexibility and/or preference of the stroke survivor to disclose sensitive information and feelings after stroke. Artificially Intelligent technology known as Embodied Conversational Agents (ECA) has been used in healthcare to address these challenges. ECAs can be defined as computer-generated humanlike characters that can exhibit similar properties as humans in a face-to-face conversation, including the ability to produce and respond to communication in a verbal and non-verbal way [6]. As a conversational agent, ECAs are like chatbots, but chatbots do not have an embodiment. Some chatbots also do not have a voice, only textual dialogue. As well as a humanlike face and partial or full body representation, ECAs tend to have animations such as facial expressions and body gestures and also tend to have spoken dialogue. Thus, ECAs include more humanlike qualities than chatbots. ECAs are also known as Intelligent Virtual Agents, highlighting that their embodiment is virtual, rather than physical. Therefore, they are better able to address the aforementioned challenges than assistive or social robots that have a physical presence requiring specialized hardware. By existing in virtual environments, ECAs enhance virtual worlds by providing a social experience [7]. However, Cassell, Bickmore, Campbell, Vilhjalmsson and Yan [8] reinforce that when designing an interactive system, ECAs should be specifically conversational in demonstrating their humanlike behaviors.

ECAs provide an opportunity to engage patients in humanlike dialogues to deliver tailored conversations, address healthcare problems, such as mental health issues and rehabilitation goals, and support medical treatment adherence [2]. Due to the many advantages of using ECAs in patient-centered computer-based interventions for behavior change [9], such as increasing accessibility, confidentiality, tailored information, and expressing empathy, ECAs have been utilized to interact with patients and address several problems within the healthcare space [10]. ECAs have also shown evidence of treatment adherence effectiveness through the development of a therapeutic/working alliance [11]. Finally, due to their use of everyday conversational language, ECAs can overcome health literacy barriers [12], a problem that often impedes the effectiveness of eHealth interventions [13,14].

Given that ECAs have been found to build trust, provide flexibility, and have the potential to be widely computer-accessible via desktops, mobile phones, and/or tablets [15], we believe that an ECA could be an effective means by which to deliver some or all of the modules within the ‘Take Charge’ intervention. It is not our goal to replace the human Take Charge Facilitators but to see whether ECA technology can address gaps or opportunities that might exist, recognizing that these facilitators are a finite resource who require appropriate skills and training and have limited availability. Lisetti [9] identifies 10 reasons why an agent may be preferable to a human in a health behavior change context. In addition to the benefits described above relating to accessibility, empathy, working alliances, and addressing poor health literacy, she adds increased patient disclosure, patient tailoring, patient–physician matching, less physician variability or bias, the avoidance of righting reflex, and endless patience on the part of the virtual physician. However, given the nature of the content and target users, the feasibility of using an ECA to encourage stroke survivors to gain new perspectives and increased motivation to self-manage their stroke recovery as delivered by human facilitators within the ‘Take Charge’ intervention remains unknown.

A scoping review of conversational agents for the rehabilitation of adults with brain-related neurological conditions [16] identified one paper involving stroke survivors. This one study [17] concerns the use of a virtual trainer to provide guidance and feedback on performing physical exercises, similar to the use of socially assistive robots [18], virtual reality [19], and Kinect technology [17] to encourage patients to practice physical or even cognitive [20] exercises. It is a common practice for stroke survivors to perform rehabilitative exercises not only with professional support but also regularly on their own; thus, providing virtual coaches to encourage these activities is clearly desirable. In contrast, the overarching aim of this research project, iTakeCharge, was to determine whether the application of ECA technology for the self-management of stroke recovery is effective in supporting better motivation, self-awareness, and behavior change in stroke survivors for the purposes of improving quality of life after stroke. The overarching aim of the iTakeCharge project is to determine whether use of an ECA, which incorporates modules of the ‘Take Charge’ intervention (including the workbook activities), can contribute to significant improvements in disability and the quality of life of stroke survivors, as shown when delivered by a human facilitator [4]. The specific research question for this study was:

Is it feasible to deliver the evidence-based ‘Take Charge’ intervention via an ECA–Patient dialogue?

This paper reports the co-design process and evaluation results of TaCIA (TakeCharge Intelligent Agent), an ECA supporting the self-management of stroke recovery, inspired by the evidence-based stroke recovery ‘Take Charge’ intervention. We report the findings from co-designing TaCIA with: (i) the ‘Take Charge’ intervention researchers/developers; (ii) human facilitators who have delivered the ‘Take Charge’ intervention to stroke survivors in person; and (iii) stroke survivors.

2. Background

Self-management intervention studies indicate a positive effect on patients’ outcomes [2]. Self-management emphasizes the enablement of individuals in taking an active role in managing their health condition. The principles of self-management are often applied in the field of chronic disease health management and include three main elements: the management of the medical aspects of one’s condition, the management of role and or behavior changes, and the management of emotional changes accompanying a health condition [21]. Self-management in the format of a ‘talking therapy’ has demonstrated effectiveness in improving quality of life and independence for stroke survivors [3,4].

The next subsection briefly discusses ECAs’ applications in healthcare and stroke recovery (Section 2.1), followed by an introduction to the ‘Take Charge’ intervention tool (Section 2.2).

2.1. ECAs in Healthcare and Stroke

ECAs have been used in numerous applications in the medical field. For example, ECAs have been used to address mental disorder problems, detecting verbal and non-verbal symptoms of depression and anxiety to help people with psychological distress [22]. In other work, ECAs have been used for coaching behavioral changes and lifestyles, such as alcohol addiction [23], or for coaching older adults in health styles through exercising [24]. ECAs have been designed for disease and chronic condition management by promoting empowerment and treatment adherence in a virtual session environment [25] or have even been combined with an external device in complex heart diseases, such as atrial fibrillation, to provide education and counseling to patients [26].

Emotional support has also been delivered by ECAs, with a service provided to meet peoples’ needs across different phases of life, such as counseling terminally ill patients and helping them address unmet spiritual needs [27] or assisting patients with low health literacy at the time of hospital discharge [12]. ECAs have also shown evidence of treatment adherence effectiveness through developing a therapeutic alliance [11].

In the stroke rehabilitation field, many studies have been conducted in the fields of artificial intelligence and machine learning to assist rehabilitation therapy in addressing the problems caused by strokes. Johnson, Loureiro and Harwin [28] tested a collaborative telerehabilitation and robot-mediated therapy for stroke rehabilitation at home. Cameirão, Oller and Verschure [29] presented a virtual reality-based rehabilitation gaming system, delivering personalized and automated training to stroke survivors. Aljaroodi, Adam, Chiong, Cornforth and Minichiello [30] recognized the importance of cognitive, physical, environmental, and motivational aspects in encouraging stroke survivors to follow their exercises and have created an empathic avatar in the form of a digital twin that helps the stroke survivor to see themselves in an empathic and positive way. Their approach focuses on the completion of rehabilitative exercises. The Take Charge intervention, however, is a narrative-based approach, better suited for a conversational agent rather than an avatar demonstrating physical tasks. Griol and Callejas [20] developed a conversational interface to deliver cognitive exercises and cognitive behavioral therapy to stroke survivors. Vasco, Willemse, Chevalier, De Tommaso, Gower, Gramatica, Tikhanoff, Pattacini, Metta and Wykowska [31] compared the use of a social robot with a virtual agent and found that the social robot achieved higher engagement and movement performance than the virtual agent. This raises the question of whether a social robot or virtual agent is a better choice for self-managed therapy.

Numerous studies have been conducted that examine the role of embodiment, the claimed benefit of a robot over a virtual agent [32]. However, it is important to differentiate between the physical embodiment of the virtual agent or robot and whether the agent/robot is collocated with the human [33]. A study by Powers, Kiesler, Fussell and Torrey [34] with 113 participants evaluated a collocated robot, remote robot, and virtual agent playing the role of a nurse by answering the user’s questions about healthy habits. They found that participants socially engaged more with the robots that they found to be more lifelike, more helpful, and better communicators. While the collocated robot outperformed the remote robot in terms of these measures, there was no difference in terms of influence on health behaviors. However, the participants disclosed more to the virtual agent and also remembered more content compared to the robots. The authors suggest that the robot interaction was more shallow than the interaction with the agent.

In a meta-analysis of studies investigating the impact of embodiment, Li [33] reviewed 38 studies that looked at physically present robots, telepresent robots, and/or virtual agents. Differences in terms of players’ attitudes and behaviors were measured. No significant differences were found between the telepresent robots and virtual agents. However, the physically present robots were more likely to keep the user’s attention and be perceived as more enjoyable, trustworthy, useful, and persuasive. The performance and response speed were sometimes better with a virtual agent. Of note, the participants were more likely to follow or accept recommended physical tasks from a physically present robot. Another key theoretical implication from the study was the conclusion that a sense of presence and the sharing of space, as in a face-to-face context, with the robot or agent are more important than realism or embodiment. This is important, as face-to-face communication has been identified as an important factor in encouraging treatment adherence, which is supported by the embodied nature of ECAs [35]. Furthermore, a sense of co-presence can be achieved with virtual agents [36,37].

Previous studies, such as the aforementioned one, that sought to assist with stroke recovery were focused on the physical or cognitive rehabilitation of stroke survivors, and ECAs, where used, acted more like a coach encouraging following an exercise regime. Due to the physical nature of those exercises and the findings reported above [33], it is not surprising that having a physically co-present agent would produce better results. This is similar to having a patient follow exercises with the support of a physiotherapist while visiting their office. However, a strong motivation of our work was to provide a solution to the limited access to and cost of human facilitators delivering the Take Charge intervention. Due to the greater willingness of humans to disclose to a virtual agent [34] and studies commonly not showing any improvement in terms of behavior change with a co-present robot [33], ECAs seem to be a viable option that is also easier to adapt and change as a software-based solution.

No studies have utilized an ECA to promote an attitude of self-rehabilitation among stroke survivors, leaving a gap that could contribute to reducing the burden caused by the stroke not only on the survivor patient but also on their family, their caregivers, and the community [1]. This gap opens future directions of research in the stroke field and motivates the research reported in this paper.

2.2. The Taking Charge after Stroke (the ‘Take Charge’ Intervention)

The first months after having a stroke can be overwhelming for many people. The ‘Take Charge’ intervention facilitates a process where the stroke survivor ‘takes charge’ of their life after a stroke. The objective is for the stroke survivor to take on the responsibility of their rehabilitation and the process of re-establishing themselves as a person independent of the effects of the stroke (Fu, Weatherall, and McNaughton, 2017; Fu et al., 2020) [3,4,38].

As presented in the ‘Take Charge’ intervention training material, the first step is asking big life questions and then moves to supporting the stroke survivors in setting achievable but challenging goals [5]. A unique aspect of the ‘Take Charge’ intervention is the encouragement of self-reflection by the stroke survivor prior to the identification of rehabilitation and recovery goals.

The ‘Take Charge’ intervention is a conversation tool where the facilitator guides the conversation but the stroke survivor remains in control. A booklet is available to enable the person with stroke to document their thoughts, but its use is not essential. The session is presented in a structured way, where the facilitator is trained to deliver three stages that contribute to developing the ‘taking charge’ mindset:

Stage 1

Baseline Assessment—nine different assessments are performed with the stroke survivor, giving to that person a full picture of their current state, both physical and emotional. These assessments aim to trigger the stroke survivor to start reflecting on where they are now to prepare them to start thinking about the progress they want to see in the future.

Stage 2

Looking at the big picture—this stage guides the stroke survivor to describe who they really are as a person, which is beyond the stroke condition. The expected outcome is that the stroke survivor would change from seeing themselves as a stroke person to a person who happens to have had a stroke. Questions such as ‘Who am I?’, ‘Who are the important people in my life?’, ‘What really matters to me?’, ‘Who is by my side?’, and ‘What does your ideal day look like?’ are examples of reflections performed in this step.

Stage 3

Breaking it down into do-able pieces—this stage leads the stroke survivor to formulate specific actions to work on in the areas they want to change: physical, communication, emotional, information, financial, support network, and prevention of a second stroke. This stage is the empowering step for taking charge of their recovery as a result of the self-awareness developed during the two previous stages of the session.

The ‘dose’ shown to deliver the greatest benefit is two separate sessions, six weeks apart, within the first six months of a stroke [5]. However, the delivery of the program by human facilitators has resource costs to the health system as well as patient access issues (timing, location, cost of travel). Also, some individuals may be reluctant to talk face-to-face with someone.

3. Methodology

Co-design is the process of designing with people that will use or deliver a product or service [39]. Co-design typically involves discussions with stakeholders prior to development to understand the nature of the problem to be addressed and possible solutions. However, because the Take Charge intervention and associated training materials already existed, we commenced the co-design process with discussions with the people who had developed, delivered, and used the evidence-based Take Charge tool for the purposes of adapting it into an ECA. As our goal was to develop an ECA and evaluate if it could successfully deliver the Take Charge intervention, we took a design science research approach where the prototype would be iteratively evaluated by different groups of stakeholders and the design features and enhancements from one group would lead to the next version based on the feedback received. We commenced our co-design and evaluation process with safe participants (researchers and facilitators) in Stage 1, only moving to stroke survivors in Stage 2.

To determine if an ECA can be used to deliver elements of the ‘Take Charge’ intervention, we adapted materials used in the program, including patient workbooks and facilitator manuals, and began a process of co-design with key stakeholders and experts in the field. We gained Human Ethics approval to conduct a study to evaluate TaCIA. As shown in

Table 1. The Two Stages of the iTakeCharge Project.

Project Phase	Phase Group	Phase Description
Stage 1—Co-design with former deliverers	Phase I	Experts in the field—researchers and designers of the ‘Take Charge’ intervention
	Phase II	Former ‘Take Charge’ intervention facilitators who have experience with delivery
Stage 2—Co-design with end users	Phase III	Feasibility testing of the agent-based system with stroke survivors

, we first performed evaluation with the researchers who designed and tested the ‘Take Charge’ intervention program (Phase I—TaCIA V.1.0) and with facilitators (Phase II—TaCIA V.2.0) who have had experience in delivering the Take Charge intervention and who acted as stroke survivors to perform the initial testing of the ECA. Their feedback was collected, and changes to improve the ECA were applied accordingly. Finally, the feasibility of an ECA to deliver the iTakeCharge intervention was evaluated with stroke survivors in one-on-one sessions with the Researcher (author HJ) in Australia (Phase III—TaCIA V.3.0).

The overall aims of the iTakeCharge project are to:

(1)

confirm components of the ECA through co-design with former deliverers (Stage 1),

(2)

determine the feasibility of the proposed intervention through evaluation with stroke survivors (Stage 2), and

(3)

inform the future design and development of an intervention to be tested in a future clinical trial.

To achieve these aims, we refined our general research question into the following specific research questions:

• RQ1.1 What are the essential and desirable elements as perceived by stakeholders of an ECA that aims to deliver the ‘Looking at the big picture’ component of The Take Charge session?
• RQ1.2 Has the stroke survivor engaged positively with the materials and the ECA in terms of the therapeutic alliance?
• RQ1.3 Have the dialogues and multiple-choice menu utterances available for each patient’s interaction been suitable and representative of options in real life?
• RQ1.4 Do the data collected via the questionnaires indicate self-awareness and motivation to change and take charge among stroke survivors that have taken the virtual ECA session?
• RQ1.5 How does TaCIA compare with The Take Charge session?

3.1. Materials

To design the ECA’s appearance, virtual environment, dialogue content and flow, and appropriate adaptation from the Taking Charge tool material, we adopted the method of collaborative design (co-design). The project team is composed of domain experts from the medical field, physiotherapy and rehabilitation, and computer science and stroke survivors working together using agile processes involving iteration and feedback. There are many benefits of applying co-design methods in experiments, such as the enhancement of mutual learning, integrating different people and expertise, innovation, and user adherence to products [40].

The contribution of stakeholders from the medical field, former ‘Take Charge’ intervention facilitators, and stroke survivors was considered fundamental in achieving the goal of designing and implementing an ECA that engages stroke survivors in feeling empowered to take charge of their health condition. All stakeholders were directly involved or consulted across each step of the project design, and their feedback was used to enhance the system with the goal of meeting the needs of stroke survivors.

3.1.1. ECA Design

To design the agent, a meeting was set up between the ECA researchers and the clinical researchers–experts to collect inputs about the physical attributes expected to be acceptable to the end-users. The specification included:

• A female avatar/character, as females are associated with empathic caring dialogue [41];
• Approximately 40 years old to align with possible age of a facilitator;
• Dark hair, as that is the most common hair color globally;
• Casual clothing to facilitate bond and to avoid seeing the agent as a domain expert or substitute for a human clinician;
• The agent background scene should represent a home environment;
• The iTakeCharge ECA was given the name of TaCIA (Taking Charge Intelligent Agent), as using a name helps to build a social relationship with the user.

Taking into account the above specifications, twelve different options of agents were created with different hair styles, face shapes, and glasses (Figure 1). The options were discussed at a team meeting including the original Take Charge researchers. Further feedback was obtained from two stroke survivors, leading to a consensus to select avatar 1a.

3.1.2. Dialogues

To interact with users, ECAs’ dialogues can generally respond to two types of users’ inputs: constrained, where users have a pre-defined list of responses to interact with the agent, and unconstrained, where the user can freely interact with the agent in free text, speech, or any other natural language technique. Although unconstrained inputs could be considered ‘more natural’ by users, research has demonstrated that users prefer to select an option that helps them know what they can say and prefer to receive a response crafted by a human domain expert, which they believe is more accurate and empathic compared to the machine-generated response to their unconstrained natural language entry, as the system’s response has not been validated by a human and may not be specific to them [23]. Furthermore, unconstrained dialogues, where the speech from an agent is created in real time during the conversation, such as Alexa (Amazon assistant) or Siri (Apple), have presented safety issues to patients who seek lifestyle and healthcare recommendations [42]. To avoid errors, and also because stroke survivors may have cognitive and physical impairments that make coming up with and entering a response difficult, TaCIA’s dialogues use a constrained user’s multiple choice input model. We note some modifications to this approach following co-design feedback that will be presented in the results and discussion. This means that the dialogues were pre-defined and validated by the research team before being uploaded into the system. The dialogues are formed by two specific parts: a) menu of utterances to the end user in the form of multiple-choice responses and b) TaCIA’s responses and comments delivered using spoken audio that is synchronized with lip movements and presented as text in a speech bubble visible on the screen. A snippet of TaCIA’s state-based constrained dialogue input format is presented in Figure 2.

The iTakeCharge study, as a proof of concept, only sought to deliver the ‘Take Charge’ intervention’s ‘Looking at the big picture’ [43], with TaCIA’s dialogues designed to help the stroke survivor reflect on their self-identity and life. This stage of the ‘Take Charge’ intervention discussion has three steps:

• ‘MyStroke vs. MyIdentity’—The stroke survivor is assisted to think about how they feel their stroke has affected them and who they are as a person, regardless of their stroke;
• ‘Hopes&Fears’—The stroke survivor is asked to express and identify their hopes and fears for the future;
• ‘MyBestDay’—The stroke survivor is guided to visualize and describe their ‘Best Day’.

The ‘Looking at the big picture’ stage aims to assist the stroke survivor to reflect on what is most important to them. Each step poses questions that help the stroke survivor look beyond the stroke, allowing the person at the end to describe who they really are, giving context about where they want to be, and empowering them to make this happen: the self-rehabilitation. An example of a question with options is presented in Figure 3.

3.1.3. System Engine

We extended an in-house dialogue authoring tool that generates ECA dialogues using text-to-speech. To make dialogue entry and modification easier and logical, the tool reads a csv file with states and transitions, utterances, and a number of functions such as displaying web links. Generally, the user is presented a list of options (menu of utterances) that are displayed as individual text boxes, and they select at least one that best suits them to progress with the session. Some questions allow for multiple selections. A space for free text is also offered with most lists. To deliver that design, we combined the modules in

Table 2. System Engine main modules.

Module	Feature
ECA and environment design	Unity 3D game building platform
Graphic rendering	WebGL for native PC or Mac browsers.
Audio dialogues	Authoring tool for .avi audio files:
Dialogue tree module	Uses the components dialogue manager, agent’s action interpreter, and Salsa lipsync
Data storage and query	MySQL Database system

. The architecture for the Dialogue Tree Module is shown in Figure 4.

3.2. Recruitment

Participants in Phase I: The co-design of TACIA included the ‘Take Charge’ intervention researchers and former ‘Take Charge’ intervention facilitators. We first sought the feedback of the ‘Take Charge’ intervention researchers, as they had the vision and understanding of how and why the program was effective. If our design was not consistent with these, we would need to modify TACIA before evaluation with facilitators.

The next evaluation (Stage 1: Phase II) involved the ‘Take Charge’ intervention facilitators who were asked to play the role of a stroke survivor when interacting with TaCIA and while providing feedback during the testing session. Based on their experience in running the ‘Take Charge’ intervention multiple times with stroke survivors, the former facilitators could make a significant contribution by evaluating the system from the stroke survivor perspective. They were also able to provide feedback about any physical constraints that might be faced by stroke survivors that could impact the virtual session, dialogues, and session structure and if the options of menu utterances were realistic enough in covering most responses that stroke survivors might have. The iTakeCharge team had no direct contact with the ‘Take Charge’ intervention facilitators.

Stroke survivors were recruited for Stage 2: Phase III using several strategies: (i) Stroke survivors currently enrolled for research studies at Hunter Medical Research Institute (HMRI) were contacted by phone by a member of HMRI and were invited to join the project; (ii) Recruitment via social media advertisement as per the Human Research Ethics Committee (HREC) or through (iii) Word of mouth. Stroke survivors were eligible to participate in the iTakeCharge study if they: (i) had a stroke, (ii) were living in the community; (iii) could give informed consent (i.e., did not have language and/or cognitive impairments limiting their capacity to understand the participant information statement); and (iv) were discharged from the hospital to a non-institutional/community living situation.

No rewards or incentives besides contributing their valuable knowledge were offered to the participants in any of the phases.

3.3. Procedure

The procedure was as follows.

• Read Information sheet and give consent
• Pre-TaCIA Phase I Survey—answer:
  • a mix of closed and open-ended questions to capture the user’s profile (e.g., experience with the ‘Take Charge’ intervention) and demographic questions (e.g., gender, age)
  • AMP-C [44] (stroke survivors only): a self-reported scoring based on a model that focuses on enabling people to become intrinsically motivated—Autonomy, Mastery, Purpose, and Connectedness level.
• Interaction with TaCIA
• Second Interaction with TaCIA while responding to survey questions that reproduce the lists in TaCIA’s dialogue to allow for editing/additions
• Post-TaCIA Phase I Survey—answer:
  • AMP-C (stroke survivors only);
  • System Usability Scale (SUS) [45]: to measure system usability;
  • Working Alliance Inventory–revised–short form (WAI-SR) [46] (researchers and facilitators) OR Session Rating Scale (SUS) [47] (stroke survivors only) for measuring the therapeutic alliance with TaCIA;
  • a mix of closed and open-ended questions related to the TaCIA session.

Four weeks after their session with TaCIA, the stroke survivors (only) were followed up and again answered the AMP-C and questions relating to their post-stroke management.

The instruments mentioned in this procedure are described in Section 3.4.1. The whole study was designed to take 90 min. Surveys were accessed by the participant via the research survey system Qualtrics. To preserve the confidentiality of the participants, the data collected were de-identified for the analysis and publication of this study.

3.4. Data Collection and Analysis

3.4.1. Quantitative Data

The following data were collected in this study.

• Prior to their interaction with TaCIA, the participants answered demographic and profile questions.
• As the participants interacted with TaCIA, they were asked to identify how well TaCIA supported a conversation for each of the modules in the ‘Looking at the Big Picture’, including consideration of the response options provided.
• Following interaction, the participants were asked how well a stroke survivor could use TaCIA using a four-point Likert scale from one (Strongly Disagree) to four (Strongly Agree). The participants were also asked to answer five questions on TaCIA’s Rehabilitative Potential using a five-point scale: one (Strongly Disagree) to five (Strongly Agree).
• SUS: evaluates aspects of the user–system interaction with ten questions using a five-point Likert scale, from one (strongly disagree) to five (strongly agree), such as: likability, complexity, consistency, and learnability. To allow for scoring and interpretation of the SUS questionnaire, five questions were reversed and marked with [R] in the results presented. SUS scores can be interpreted as follows: >80.3—Excellent; 68–80.3—Good; 68—Okay; 51–68—Poor; <51—Awful.
• WAI-SR (researchers and facilitators): based on the original 36 questions from the Work Alliance Inventory (WAI) questionnaire (Horvath & Greenberg, 1989), the 12 statements/questions on a seven-point Likert scale, from one (never) to seven (always), are applied to evaluate patients’ feelings concerning a sense of working alliance across the categories Task, Goal, and Bond. The word Therapist was replaced with TaCIA. WAI-SR provides four scores: three for each category (ranging from 3 to 28 each) and an aggregated overall score ranging from 12 to 84. It is important to notice that there is no standardized measurement for the WAI, as the tool measures the opinion and self-perception of the relationship. However, higher scores reflect a stronger working alliance.
• SRS (stroke survivors only): To reduce the survey length in Phase III, instead of WAI, we chose to use SRS, as it is a brief measure of therapeutic alliance with four items: Relationship, Goals and Topics, Approach, and Overall, with a Cronbach’s alpha of 0.88, a test–retest reliability of 0.64 [47], and moderately strong correlation with the Working Alliance Inventory (r = 0.63) [48].
• AMP-C (stroke survivors only): includes four questions using a four-point Likert scale, from one (strongly disagree) to four (strongly agree). This was used to evaluate motivation across the four dimensions: Autonomy, Mastery, Purpose, and Connectedness. The overall score is the sum of the four components ranging from 4 to 16.
• System Logs: from the user’s actions during the session with TaCIA, we sought to measure statistics such as the frequency with which the options from the menu of utterances were chosen, the average time spent by the user on each question, and other statistics that will help in enhancing the system.

3.4.2. Qualitative Data

In Phases I and II, the survey presented open questions of general impressions about TaCIA’s performance in delivering a digital session inspired by the Take Charge intervention, recommendations for improvement, and any potential pitfalls to be avoided. The responses aimed to aid interpretation of the quantitative responses, identify any specific issues or recommendations, and identify common impressions across participants. Phase III surveys excluded the questions about comparing the iTakeCharge study with the Take Charge intervention. Given the small number of participants and the limited qualitative data, there was no need to conduct thematic analysis or another qualitative method to draw out themes.

4. Results

In Stage 1, feedback was collected in Phase I from the two researchers and developers (R1 and R2) who designed, implemented, and evaluated the ‘Take Charge’ intervention program and in Phase II from four ‘Take Charge’ intervention facilitators (F1–F4). The lengthy time commitment for participants contributes to the small number of facilitators recruited. We first present a summary of Stage 1 quantitative (

Table 3. Working Alliance Inventory *—Phase I and II.

Questions	Researchers		Facilitators		C
	M	s.d.	M	s.d.	M
Q1—As a result of this session, I am clearer as to how I might be able to change.	3.0	1.0	4.5	1.8	+1.5
Q2—What I am doing in therapy gives me new ways of looking at my problem.	1.5	1.5	3.8	1.5	+2.3
Q3—I feel that TaCIA likes me.	2.5	2.5	4.5	0.5	+2.0
Q4—TaCIA and I collaborate on planning my therapy.	3.0	1.0	4.3	1.5	+1.3
Q5—TaCIA and I respect each other.	3.5	0.5	4.3	1.1	+0.8
Q6—TaCIA and I are working towards the same things.	5.0	0.0	5.3	1.3	+0.3
Q7—I feel that TaCIA appreciates me.	4.0	1.0	4.8	1.5	+0.8
Q8—TaCIA and I agree on what is important for me to work on.	4.0	1.0	4.8	1.1	+0.8
Q9—I feel TaCIA cares about me even when I do things that she does not approve of.	2.5	2.5	3.0	2.2	+0.5
Q10—I feel that the things I do in therapy with T will help me accomplish the changes that I want.	4.5	0.5	5.5	1.1	+1.0
Q11—T and I have established a good understanding of the kind of changes that would be good for me.	4.0	0.0	4.5	1.1	+0.5
Q12—I believe the way we are working with my problem is correct.	4.5	0.5	5.0	0.7	+0.5
Average	3.5	1.6	4.5	1.5	+1.2

* 1 (Strongly Disagree) to 7 (Strongly Agree); C = Change, M = Mean.

,

Table 4. System Usability Scale * (SUS): Phase I and II.

Questions	Researchers		Facilitators		C
	M	s.d.	M	s.d.	M
Q1—I think that I would like to use TaCIA.	2.5	0.5	3.8	0.4	+1.3
Q2—I found TaCIA unnecessarily complex. [R]	3.0	1.0	4.0	0.0	+1.0
Q3—I thought TaCIA was easy to use.	3.5	0.5	4.0	0.0	+0.5
Q4—I think I would need the support of a technical person to be able to use TaCIA. [R]	3.5	0.5	4.0	0.0	+0.5
Q5—I found that, during my session with TaCIA, the tools and functionalities were well-integrated (e.g., buttons, links, speech bubbles, a list of options, transitions of sound and images).	4.0	0.0	4.3	0.4	+0.3
Q6—I thought there were too many errors (inconsistencies) in TaCIA. [R]	3.0	1.0	4.0	0.0	+1.0
Q7—I would imagine that most people would learn to use TaCIA very quickly.	4.0	0.0	4.0	0.0	-
Q8—I found TaCIA awkward to use. [R]	3.0	1.0	4.3	0.4	+1.3
Q9—I felt confident using TaCIA.	2.5	0.5	4.0	0.0	+0.5
Q10—I need to learn a lot of things before I could get going with TaCIA. [R]	4.5	0.5	4.3	0.4	−0.2
Average	3.4	0.9	4.1	0.3	+0.7
SUS Sum	58.8	-	76.3	-	+17.5

* 1 (Strongly Disagree) to 5 (Strongly Agree). R = reversed scoring

,

Table 5. Evaluating the iTakeCharge steps *—Phase I and II.

Questions	Researchers		Facilitators		C
	M	s.d.	M	s.d.	M
Overall, TaCIA did well in supporting a conversation to understand my stroke vs. my identity.	2.5	0.5	3.3	0.4	+0.8
Overall, TaCIA did well in supporting a conversation about the topic ‘knowing my hopes’.	2.0	-	3.3	0.4	+1.3
Overall, TaCIA did well in supporting a conversation about the topic ‘knowing my fears’.	2.0	-	3.0	0.0	+1.0
Overall, TaCIA did well in supporting a conversation about the topic ‘defining my best day’.	2.0	-	3.0	0.0	+1.0
Average	2.1	0.3	3.1	0.3	+1.0

* 1 (Strongly Disagree) to 4 (Strongly Agree).

,

Table 6. Perspectives about using TaCIA *: Phase I and II.

Questions	Researchers		Facilitators		C
	M	s.d.	M	s.d.	M
Q1—TaCIA would be able to be used by most stroke survivors who have no or very mild communication problems and/or thinking problems.	3.0	-	3.5	0.5	+0.5
Q2—I think that I would recommend using TaCIA to assist stroke survivors in managing their recovery.	2.5	-	3.5	0.5	+1.0
Q3—I believe that TaCIA succeeded in delivering the ‘Take Charge’ intervention’s ‘Looking at the big picture’ section.	2.0	-	3.5	0.5	+1.5
Average	2.0	0.5	3.5	0.5	+1.5

* 1 (Strongly Disagree) to 4 (Strongly Agree).

and

Table 7. Rehabilitative Potential * Phase I and II.

Questions	Researchers		Facilitators		C
	M	s.d.	M	s.d.	M
Q1—TaCIA fulfills the objective of helping stroke survivors.	3.5	0.5	4.5	0.5	+1.0
Q2—The interactions with TaCIA are relevant for the stroke survivor’s rehabilitation.	4.0	-	4.3	0.5	+0.3
Q3—The way the session was conducted by TaCIA involving questions and options was suitable for different kinds of stroke survivors.	3.5	0.5	4.0	0.5	+0.5
Q4—The dialogues with TaCIA could potentially support improvements to regain the sense of autonomy of stroke survivors.	3.5	0.5	4.0	0	+0.5
Q5—The dialogues with TaCIA could stimulate stroke survivors’ motivation in taking charge of their own recovery.	3.0	1.0	4.0	0.5	+1.0
Average	3.5	0.7	4.2	0.6	+0.7

* 1 (Strongly Disagree) to 5 (Strongly Agree).

) and qualitative results and changes made to TaCIA in Phases I and II. Then, we present the results from Stage 2 (Phase III), with eight (P1–P8) stroke survivors, and include a comparison with the Stage 1 results.

4.1. Phase I—The ‘Take Charge’ Intervention Researchers and Developers

Both R1 and R2 are PhD-qualified, have over 10 years of experience in practice, and deeply understand the goals of the ‘Take Charge’ intervention study. R2 declared having delivered the ‘Take Charge’ intervention to stroke survivors at the time of the survey.

4.1.1. Researcher Quantitative Analysis

As shown in Table 3, the average score for the combined participants of 3.5 points in the WAI questionnaire indicates that participants R1 and R2 have scored the collaborative relationship as slightly poor. In Table 4, the SUS combined score of 58.8 reveals a general perception that the tool is poorly designed for its intent in supporting stroke survivors. Participants R1 and R2 gave the iTakeCharge sessions 2.2 points for an average scoring on a 4-point Likert scale across 13 questions, which represents a general disagreement with the several lists of options suggested during the ‘MyStroke vs. MyIdentity’, ‘Hopes&Fears’, and ‘MyBestDay’ presented by TaCIA during these respective session with stroke survivors, as well as the overall feedback about TaCIA’s performance in delivering the three individual parts of the session at 2.1 points (Table 5).

The comparison of TaCIA with the ‘Take Charge’ intervention booklet scored two points (Table 6), moderately worse than the booklet. Researchers, on average, slightly agree that stroke survivors would be able to use TaCIA, that they would recommend it for stroke survivors management, and that TaCIA was able to deliver the ‘Take Charge’ intervention. TaCIA’s rehabilitative potential received an average score of 3.5 (Table 7), revealing that researchers slightly agree.

4.1.2. Researcher Qualitative Analysis

During a second interaction with TaCIA, the participants were asked to provide feedback on each section. The key feedback and recommendations can be found in the thesis [49]. In brief, we include a comment from R1:

‘Overall a very good start. Lots of potential. It will eventually deliver a fantastic product. The avatar works really well-I sometimes wanted her to hesitate longer before speaking, as if she was listening and considering my response. Currently it feels like I am being pushed along too fast.’

(R1)

4.1.3. Phase I Modifications Implemented in TaCIA Version 2.0

As captured in the above comment, the researchers could see potential value, but they expressed concern that TaCIA’s style was different from the delivery of Take Charge, where facilitators would not provide suggestions or comment on the patient’s responses. Due to TaCIA’s interaction method involving answer options with tailored responses, the researchers felt that TaCIA acted more as a coach than a facilitator. In response, the iTakeCharge research team decided that, in TaCIA V.2.0, we would have facilitators evaluate which is the best session format, free-text box exclusively or lists of options, to inform the best version to be tested by stroke survivors. The changes and evaluation process were as follows:

• For the session part ‘MyStroke vs. MyIdentity’, participants were presented with three options: ‘Enter my own response’; ‘See a list of options’; ‘I don’t want to answer this question’.
• For the session part ‘Hopes&Fears’, both a free text box AND a list of options appeared on the screen at the same time.

Two other key changes were made to TaCIA’s design. Additional text was included in the TaCIA dialogue when introducing each part of the sessions ‘MyStroke vs. MyIdentity’, ‘Hopes&Fears’, and ‘MyBestDay’. The objective was to facilitate the transition between these themes and improve the conversation flow. Further, to address researcher concerns around patient autonomy, we added the option ‘I don’t want to answer this question’ to all the questions asked by TaCIA to empower users to skip any questions they did not want to respond to. All modifications were implemented in TaCIA V.2.0 to be tested by the ‘Take Charge’ intervention Facilitators in Phase II.

4.2. Phase II—The ‘Take Charge’ Intervention Facilitators

Five ‘Take Charge’ intervention facilitators were invited to test TaCIA by considering the perspectives of stroke survivors, based on their experience in delivering a face-to-face ‘Take Charge’ intervention. Four agreed to participate and tested TaCIA V.2.0. All four participants were female, have a qualification in the healthcare field, and have delivered the ‘Take Charge’ intervention, on average, approximately 30 times. Three out of four participants declared having previously used the internet to obtain information about and/or to manage their own health condition.

4.2.1. Facilitator Quantitative Analysis

An average of 4.5 points across F1–F4 identified WAI as satisfactory, an improvement from the 3.5 points given by R1 and R2. SUS questions had an average result of 4.1 points and a combined score of 76.3 points, in contrast to the 58.8 points given in Phase I. The ‘Take Charge’ intervention facilitators agreed that stroke survivors would consider realistic the lists of options suggested by TaCIA for the sessions ‘MyStroke vs. MyIdentity’, ‘Hopes&Fears’ and ‘MyBestDay’, with an average scoring of 3.1 points across 14 questions, which is an improvement from the Phase I, which scored 2.2 points. The participants agreed that TaCIA did well in supporting a conversation about the topic of TaCIA’s performance in delivering the three individual parts of the session, scoring an average result of 3.1 points, an improvement from 2.1 in Phase I.

When asked if stroke survivors would be able to use TaCIA, if they would recommend TaCIA to stroke survivors for stroke management, and about TaCIA’s ability in delivering the ‘Take Charge’ intervention, the facilitators, overall, scored 3.5 points, on average, which contrasts with the Phase I score of 2.5 points. Concerning rehabilitative potential, the facilitators agree with an average of 4.2 points, an improvement from the previous 3.5 points in Phase I.

4.2.2. Facilitator Qualitative Analysis

The ‘Take Charge’ intervention facilitators evaluated whether the free text or list of choice format was preferable. The results are shown in

Table 8. Which method of conducting the session do you believe supports the patient most when TaCIA asks them a question? (P = participant, M ¹ = method).

P	M	Comment
F1	(1)	‘It helps the participant think more for themselves so the answer is a more personal one, coming from within.’
F2	(2)	‘It helps to have prompts to include something they may not have thought about. There is a natural anxiety present in people early in the session due to doing something completely new and feeling out of their comfort zone. A mix of free-text and suggested options is helpful when anxiety can impede clear thinking, and gives a sense of support and encouragement.’
F3	(2)	‘Good to have prompts/suggestions for guidance.’
F4	(2)	‘Looking at the list may help you start thinking about what the question means to you, you are then able to add more in the free text box.’

¹ Method Legend: (1) ‘I prefer just to give a free text box without showing any list of suggestions (like in the ‘MyStroke vs. MyIdentity’ session).’, (2) ‘I prefer to give them the free text box at the same time as showing them a list of suggestions (like in the ‘Hopes&Fears’ session)’; (3) ‘I do not have any preference between both ways’.

. Three out of four (75%) of the ‘Take Charge’ intervention facilitators have chosen to give stroke survivors a list of options combined with a free text box. These participants have argued that, based on their own experience in delivering the ‘Take Charge’ intervention to stroke survivors, the combination of lists of options with a free text box would be helpful in supporting stroke survivors to start to think about their own response before entering it in the free text box, as well as the benefit of reducing their natural anxiety present in this type of session. Other qualitative feedback that was provided can be found in the thesis [49].

Table 9. Overall impression from each facilitator.

Comment
F1—‘I think it is perhaps more helpful because she is not a real human. I did not feel that she could judge me for my true feelings and aims. I’m a fan!!’
F2—‘I enjoyed seeing this product and think it could be very helpful.’
F3—‘Overall reasonable alternative-would be better to have some element of human interaction (phone contact)’
F4—‘Easy to use, I like the content and list of options if stroke survivors are first unsure what to add in free text. (…) Good emphasis on how individuals are more than their stroke.’

provides a single comment from each facilitator which expresses their overall experience.

Table 10. Facilitators responses to the question ‘How does TaCIA compare with the ‘Take Charge’ intervention?’.

Facilitator Comment
F1—‘I prefer the TaCIA session, I think, because there was more ability to ponder the questions without feeling there was a real human listening who would know your true thoughts and feelings.’
F2—‘I think TaCIA does a good job of helping the person to understand that what they want/hope for is very important.’
F3—‘Different-less personal. Not able to go deeper i.e., be more empathetic or encouraging when needed.’
F4—‘Successful in getting stroke survivors to think about their stoke in the context of who they are and in bringing focus to what is most important for their future lives.’
F1—‘I thought her responses were a little trite and overly positive at times, e.g. when a fear was expressed, she initially gave an empathetic response but then immediately put a positive spin on how fears could be harnessed for good. I felt this undid the good work of helping the participant express their true fears.’
F2—‘TaCIA gives lots of positive feedback. Possibly a bit repetitive.’
F4—‘Less time spent on encouraging the setting of clear goals, meaning whilst stroke survivors may reflect on the content they may not feel as ‘in control‘’.

provides responses to the question comparing the ‘Take Charge’ intervention and TaCIA.

4.2.3. Phase II Modifications Implemented in TaCIA Version 3.0

Facilitator feedback confirmed the value of including answer options as well as an input box on the same screen but also identified the further need to prepare the patient, provide progress cues, and give stroke survivors more time and opportunity to revisit or skip suggestions. In response, the following list of changes were implemented in version TaCIA V.3.0 that was tested by stroke survivors:

• Implement the combination of lists and free text boxes for the three parts of the session (Figure 5).
• Add to TaCIA’s dialogue at the beginning that the session will be made of three different parts, which will help participants to align expectations of what is ahead of them.
• Ensure TaCIA reinforces her role as a computer agent who is only there to help.
• Include the session part number on the screen (see top-left side of Figure 5) to inform the user of their progress.
• In ‘Hopes&Fears’, ensure TaCIA only gives the option to suggest another way to look at the participant’s selected fear if requested by participants, by selecting the new option ‘Would you like to consider another way to think about that? Please take your time to think about it before we proceed’. If this option is taken, participants may provide feedback about TaCIA’s suggestion.
• Add the facilitators’ suggestions to the various lists of options.
• Change the speech pace for TaCIA to speak slowly.

4.3. PHASE III: ECA Co-Design with the System’s End-Users: Stroke Survivors

The eight participants are self-declared stroke survivors; two of them self-identified with Aboriginal or Torres Strait Islander backgrounds. All participants have English as their first language in their country of birth. Five participants are male. Their current age varies from 40 to 81 years old, and most participants have had their stroke for over two years, presenting with communication impairment and a need for a walking aid as a result of their disease. Despite the presence of stroke disabilities, all participants have declared themselves mobility-independent, even when two of them also present visual impairments. In terms of educational level, all participants have declared possessing a higher education degree. Appendix A 

Table A1. Summary of the eight participants’ characteristics and their stroke condition. Y = Yes, N = No.

	P1	P2	P3	P4	P5	P6	P7	P8
iTakeChargeSession format	FtF	FtF	FtF	FtF	Online	FtF	Online	Online
Assistance required for using TaCIA	N	N	N	Y	N	N	N	N
Age	81	55	44	51	51	64	40	58
Gender	M	M	F	F	M	M	F	M
Country of Birth	Australia	Canada	Australia	England	Australia	Australia	Australia	Australia
Aboriginal or Torres Strait Islander	Y	N	N	N	N	N	Y	N
Highest completed level of education	University degree	Tertiary Certificate Diploma	University degree	University degree	Tertiary Certificate Diploma	University degree	University degree	University degree
Rehabilitation Status	Outpatient	Outpatient	Neither	Outpatient	Outpatient	Neither	Outpatient	Neither
Years since Stroke	3.5	16	18.5	8.5	5.5	1.5	<1	1.5
Stroke Type	Ischaemic	Haemorr hagic	-	Haemorr hagic	Ischaemic	Ischaemic	Ischaemic	Ischaemic
Side of stroke	Left	Right	Left	Left	Right	Left	Left	Sub-cortical
Side affected	Left	Left	Right	Right	Left	Neither	Right	Left
Communication impairments	Y	Y	Y	Y	N	Y	Y	N
Visual impairments	Y	Y	N	N	N	N	N	N
Mobility status	Independ ent	Independ ent	Independ ent	Independ ent	Independ ent	Independ ent	Independ ent	Independ ent
Walking aid	Y	Y	N	Y	N	N	N	Y
Sit to stand	Independ ent	Independ ent	Independ ent	Independ ent	Independ ent	Independ ent	Independ ent	Independ ent
Rehab program	Y	Y	-	Y	Y	Y	Y	Y
Therapists program	Y	Y	-	Y	Y	-	Y	Y

provides a detailed summary of the participants’ characteristics and their stroke condition.

Five of them (P1–P4, P6) have taken the session in a face-to-face setting at HMRI, with support from the researcher, and completed the digital session with TaCIA and pre- and post-session questionnaires. For these participants, the researcher could confirm their condition as a stoke survivor and eligibility for the study. Three participants (P5, P7, P8) are self-declared stroke survivors and have taken the session with TaCIA online and without any support or assessment from the researcher.

4.3.1. Stroke Survivors Quantitative Analysis

Table 11. Pre-session questionnaire quantitative results for stroke survivors (Phase III).

Participants	Stroke Survivors
Computer and Technology Literacy	P1	P2	P3	P4	P5 *	P6	P7 *	P8 *	Mean	s.d.
Q1—I feel confident using a computer for basic activities (e.g., read emails, social networks, researching on Google).	4.0	4.0	4.0	2.0	4.0	4.0	4.0	4.0	3.8	0.7
Q2—I feel confident using a smartphone for basic activities (e.g., social networks, installing and using apps, taking photos).	4.0	4.0	4.0	3.0	4.0	4.0	4.0	4.0	3.9	0.3
Q3—I know how to use the internet.	4.0	4.0	4.0	1.0	4.0	3.0	4.0	4.0	3.5	1.0
Average	4.0	4.0	4.0	2.0	4.0	3.7	4.0	4.0	3.7	0.7
AMP-C (Pre-session)	P1	P2	P3	P4	P5 *	P6	P7 *	P8 *	Mean	s.d.
Q1—I feel in control of my life.	4.0	3.0	3.0	3.0	3.0	3.0	3.0	2.0	3.0	0.5
Q2—I have the skills to make the most of my life.	4.0	3.0	2.0	2.0	3.0	3.0	4.0	2.0	2.9	0.8
Q3—My life has a clear sense of purpose.	4.0	3.0	2.0	2.0	3.0	3.0	3.0	2.0	2.8	0.7
Q4—I feel connected with the important people in my life.	4.0	3.0	3.0	4.0	3.0	4.0	4.0	3.0	3.5	0.5
Average	4.0	3.0	2.5	2.8	3.0	3.3	3.5	2.3	3.0	0.7
Post-Stroke Self-Management	P1	P2	P3	P4	P5 *	P6	P7 *	P8 *	Mean	s.d.
Q1—I believe my stroke rehab plan will help me get to where I want to be.	4.0	3.0	2.0	3.0	3.0	2.0	3.0	2.0	2.8	0.7
Q2—I can see progress towards where I want to be.	4.0	3.0	2.0	3.0	3.0	3.0	4.0	2.0	3.0	0.7
Q3—I know what I must do to get where I want to be.	4.0	3.0	2.0	4.0	4.0	3.0	4.0	2.0	3.3	0.8
Q4—I aim to live my life how I want to despite having had a stroke.	4.0	4.0	2.0	4.0	4.0	2.0	3.0	1.0	3.0	1.1
Q5—I need help in setting goals to help me to get where I want to be. [R]	1.0	2.0	3.0	4.0	2.0	3.0	2.0	2.0	2.4	0.9
Q6—I feel confident in defining steps to get where I want to be.	4.0	3.0	2.0	2.0	4.0	2.0	3.0	2.0	2.8	0.8
Average	3.5	3.0	2.2	3.3	3.3	2.5	3.2	1.8	2.9	0.9

1 (Strongly Disagree) to 4 (Strongly Agree) * = Online Participant, [R] = reversed scoring.

presents the pre-session questionnaire results for the eight participants. The pre-session questionnaire shows that seven of the eight stroke survivors have strong confidence in using the basic functions inherent in computers, smartphones, and the internet, with P4 having the lowest average of two on a four-point Likert scale, indicating less IT experience. For the AMP-C questionnaire applied before the session with TaCIA, the participants have scored an average of three points on a four-point Likert scale across the four questions, indicating agreement that they have enough motivation in life to feel a sense of purpose, autonomy, mastery, and connectedness. Six questions evaluated the participants’ perception towards their post-stroke self-management, resulting in an average score of 2.9 points, which indicates they are generally positive about their rehab and future.

Two online participants (P7 and P8) did not complete the post-session questionnaires, so

Table 12. Post-session questionnaire quantitative results for stroke survivors (Phase III).

Participants	Stroke Survivors
Session Rating Scale (SRS)	P1	P2	P3	P4	P5	P6	Mean	s.d.
SRS TASK—‘TaCIA’s approach [0 = ‘is NOT’, 10 = ‘is’] a good fit for me’	10.0	7.0	9.0	7.0	10.0	10.0	8.8	1.3
SRS GOAL—‘We [0 = ‘did NOT’, 10 = ‘did’] work on or talk about what I wanted to work on and talk about’	10.0	8.0	2.0	8.0	10.0	10.0	8.0	2.8
SRS BOND—‘I [0 = ‘did NOT’, 10 = ‘did’] feel heard, understood and respected’	10.0	8.0	4.0	5.0	10.0	10.0	7.8	2.5.
SRS OVERALL 0 = ‘There was something missing in the session today’ up to 10 = ‘Overall, today’s session was right for me’	10.0	7.0	4.0	7.0	10.0	10.0	8.0	2.2
Average	10.0	8.0	5.0	7.0	10.0	10.0	8.0	2.3
AMP-C (Post-session)	P1	P2	P3	P4	P5	P6	Mean	s.d.
Q1—I feel in control of my life.	4.0	4.0 +1	3.0	3.0	3.0	2.0 −1	3.2 +0.2	0.7
Q2—I have the skills to make the most of my life.	4.0	3.0	2.0	3.0 +1	4.0 +1	2.0 −1	3.0 +0.1	0.8
Q3—My life has a clear sense of purpose.	4.0	3.0	3.0 +1	3.0 +1	3.0	3.0	3.2	0.4
Q4—I feel connected with the important people in my life.	4.0	3.0	3.0	4.0	3.0	4.0	3.5	0.5
Average (+ or – change from pre-test)	4.0	3.3 +0.3	2.8 +0.3	3.3 +0.5	3.3 +0.3	2.8 −0.5	3.2 −0.2	0.6
System Usability Scale (SUS)	P1	P2	P3	P4	P5	P6	Mean	s.d.
Q1—I think that I would like to use TaCIA.	5.0	4.0	3.0	4.0	3.0	4.0	3.8	0.7
Q2—I found TaCIA unnecessarily complex. [R]	5.0	3.0	2.0	3.0	5.0	4.0	3.7	1.1
Q3—I thought TaCIA was easy to use.	5.0	4.0	4.0	3.0	5.0	4.0	4.2	0.7
Q4—I think I would need the support of a technical person to be able to use TaCIA. [R]	5.0	4.0	2.0	2.0	5.0	4.0	3.7	1.4
Q5—I found that the tools and functionalities were well-integrated	2.0	2.0	4.0	3.0	4.0	4.0	3.2	1.2
Q6—I thought there were too many errors (inconsistencies) in TaCIA. [R]	4.0	4.0	4.0	3.0	5.0	4.0	4.0	0.9
Q7—I would imagine that most people would learn to use TaCIA very quickly.	5.0	4.0	4.0	3.0	4.0	4.0	4.0	0.6
Q8—I found TaCIA awkward to use. [R]	5.0	4.0	4.0	3.0	4.0	4.0	4.0	0.6
Q9—I felt confident using TaCIA.	5.0	4.0	4.0	3.0	4.0	2.0	3.7	0.9
Q10—I need to learn a lot of things before I could get going with TaCIA. [R]	5.0	4.0	4.0	3.0	5.0	4.0	4.2	0.7
Average	5.0	4.0	3.0	4.0	3.0	4.0	3.8	0.9
SUS Scoring	90	67.5	62.5	50	85	70	71	-
Evaluating the iTakeCharge session steps	P1	P2	P3	P4	P5	P6	Mean	s.d.
Q1—Overall results related to questions about how realistically each list of options presented by TaCIA on the screen could represent the stroke survivors’ real responses.	3.4	3.0	2.7	3.0	3.0	2.5	2.9	0.5
Q2—Overall, TaCIA did well in supporting a conversation to understand my stroke vs. my identity.	4.0	3.0	3.0	3.0	3.0	3.0	3.2	0.4
Q3—Overall, TaCIA did well in … about the topic knowing my hopes.	4.0	3.0	3.0	3.0	3.0	3.0	3.2	0.4
Q4—Overall, TaCIA did well in … about the topic knowing my fears.	4.0	2.0	3.0	3.0	3.0	3.0	3.0	0.6
Q5—Overall, TaCIA did well in the topic defining my best day.	4.0	3.0	3.0	3.0	3.0	3.0	3.2	0.4
Average	4.0	2.8	3.0	3.0	3.0	3.0	3.1	0.4
Perspectives about using TaCIA	P1	P2	P3	P4	P5	P6	Mean	s.d.
Q1—TaCIA would be able to be used by other stroke survivors like me.	4.0	3.0	2.0	4.0	4.0	4.0	3.5	0.8
Q2—I would like to keep using TaCIA to manage my stroke recovery.	4.0	3.0	2.0	2.0	2.0	4.0	2.8	0.9
Q3—I would recommend TaCIA to other people who are living with a stroke.	4.0	3.0	1.0	4.0	4.0	4.0	3.3	1.1
Average	4.0	3.0	1.7	3.3	3.3	4.0	3.2	1.0

[R] = reversed scoring; AMP-C + increase or − decrease compared to pre-session results.

only reports the results for participants P1–P6. Table 12 also shows the change (+, -, or not reported if no change) between the pre- and post-results for AMP-C. The average score in the combined participants for the SRS questionnaire of 8.0 points in a 10-point scale indicates that participants P1–P6 have scored the collaborative relationship as satisfactory. However, P3 has scored the lowest, with an average of five points and the question related to Task at nine points and the Goal at two points. The AMP-C post-session questionnaire score is 3.2 points on average, indicating no change in the motivation level of these participants after their session with TaCIA. Participants P2, P3, P4, and P5 increased their score, as opposed to P6, who has decreased their score. These changes were observed in the questions Q1–Q3 related to purpose, autonomy, and mastery, with no change to Q4 connectedness.

Among these six participants, the SUS combined score of 71 indicates the perception that the system’s usability is ‘Good’ but does suggest that improvement is possible. The participants also evaluated the lists of options provided on the screen by TaCIA during the iTakeCharge session as 2.9 points for an average scoring on a 4-point Likert scale, representing a general agreement that the lists of options were realistic options for stroke survivors to utilize as their response to TaCIA’s questions. Similarly, participants scored an average of 3.1 points for the questions that asked about TaCIA’s performance in supporting conversations over the themes ‘MyStroke vs. MyIdentity’, ‘Hopes&Fears’, and ‘MyBestDay’ with stroke survivors, which means an overall agreement that the ECA did well in delivering these three individual parts of the digital session. Finally, in the questions related to their perspectives about using TaCIA, the participants scored an average of 3.2 points on a 4-point Likert scale, indicating that they agree that TaCIA could be used by other stroke survivors, they would like to continue using TaCIA, and they would recommend it to other people living with stroke. Similar to the SRS questionnaire, participant P3, however, scored the lowest in this questionnaire (overall average of 1.7 points), disagreeing that they would recommend TaCIA to other stroke survivors (question Q3).

At the end of the face-to-face session at HMRI regarding the use of TaCIA, all participants declined taking home a hard copy of the iTakeCharge booklet filled with the responses they have provided to TaCIA in the sessions ‘MyStroke vs. MyIdentity’, ‘Hopes&Fears’, and ‘MyBestDay’. Online participants have not downloaded the booklet.

Four weeks after the session with TaCIA, stroke survivor participants who attended the face-to-face session or provided contact details in the online session received a follow-up phone call from the researcher.

Table 13. Follow-up phone call questionnaire results for stroke survivors (Phase III).

Binary Closed-Ended Questions: Y = Yes, N = No, NA = Not Applicable	P1	P2	P3	P4	P5 *	P6	Mean	s.d.
Q1—Has the session with TaCIA caused you any (emotional) distress?	N	N	N	N	N	N	NA	NA
Q2—Have you reviewed the material sent to you since your last session with TaCIA?	N	N	N	N	N	N	NA	NA
Post-Stroke Self-Management	P1	P2	P3	P4	P5 *	P6	Mean	s.d.
Q1—I believe my stroke rehab plan will help me get to where I want to be.	4.0	3.0	3.0 +1	4.0 +1	3.0	3.0 +1	3.3	0.5
Q2—I can see progress towards where I want to be.	4.0	3.0	2.0	4.0 +1	3.0	3.0	3.2	0.7
Q3—I know what I must do to get where I want to be.	4.0	3.0	3.0 +1	4.0	3.0 −1	3.0	3.3	0.5
Q4—I aim to live my life how I want to despite having had a stroke.	4.0	3.0 −1	3.0 +1	4.0	3.0 −1	2.0	3.2	0.7
Q5—I need help in setting goals to help me to get where I want to be. [R]	3.0 +2	3.0 +1	2.0 +1	4.0	3.0 +1	3.0	3.0	0.6
Q6—I feel confident in defining steps to get where I want to be.	4.0	3.0	4.0 +1	3.0 +1	3.0 −1	2.0	3.2	0.7
Average Change (+ or – from post-test)	3.8 +0.3	3.0	2.8 +0.6	3.8 +0.5	3.0 −0.3	2.7 +0.2	3.2	0.6
AMP-C (Follow-up session)	P1	P2	P3	P4	P5 *	P6	Mean	s.d.
Q1—I feel in control of my life.	4.0	3.0 −1	4.0 +1	3.0	3.0	2.0	3.2	0.7
Q2—I have the skills to make the most of my life.	4.0	3.0	3.0 +1	3.0	3.0 −1	2.0	3.0	0.6
Q3—My life has a clear sense of purpose.	4.0	3.0	2.0 −1	3.0	3.0	2.0 −1	2.8	0.7
Q4—I feel connected with the important people in my life.	4.0	3.0	2.0 −1	3.0 −1	3.0	3.0 −1	3.0	0.6
Average Change (+ or – from post-test)	4.0	3.0 −0.3	2.8	3.0 −0.3	3.0 −0.3	2.3 −0.5	3.0	0.6

* = Online Participant, [R] = reversed scoring; contain + increase or − decrease to both P-Stroke Self-Management vs. pre-session results and AMP-C vs. post-session results.

presents the results from the phone call, where we can observe that none of the participants had any distress caused by the interaction with TaCIA. We note that participants P1, P3, P4, and P6 have increased their original score in the Post-Stroke Self-Management Evaluation questionnaire, with P5 decreasing it, when compared against their scores taken in the same pre-session questionnaire. The overall score of 3.2 points for an average scoring on a 4-point Likert scale indicates that the participants are positive about their rehab and future. The AMP-C results in that follow-up phone call show that none of the participants have increased their scoring, when compared to the results in the post-session with TaCIA, with P2, P4, P5, and P6 having decreased their results. Overall, an average of 3.0 points on a 4-point Likert scale indicates agreement that stroke survivor participants have enough motivation in life to feel a sense of purpose, autonomy, mastery, and connectedness.

4.3.2. Stroke Survivors Qualitative Analysis

With the version TaCIA V.3.0, the stroke survivors have experienced a digital session where both lists of options and free text boxes were provided to capture their opinion when asked questions.

Table A2. Additional options that stroke survivors suggested to be included in the lists of responses.

Lists	Additional Options to Be Added to the List
‘My stroke effects’	P1: ‘drooling’ P2: ‘Change in my sensation | using my hands and thinking speed’ P3: ‘it suck’ P4: ‘talking, continence, fatigue, not driving’
‘Relationship Roles’	P3: ‘Stroke Sufferer’
‘Jobs’	P7: ‘Health care worker, single mum of two boys’
‘Personality Attributes’	P4: ‘Naughty, determined’
‘Hobbies and Interests’	P2: ‘woodworking, volunteering, movies, gardening, computer gaming, puzzles’ P6: ‘Environment improvement-landcare, community garden’
‘Hopes’	P4: ‘Holiday overseas since stroke’ P6: ‘Improve interactions in social situations’
‘Fears’	P1: ‘Risk of falling’ P3: [Not] ‘Be able to talk’ P5: ‘seizures, partner dying’
‘People you want to spend your best day with’	P1: ‘Wife’ P5: ‘alone, can be your best day. i enjoy alone time. less thinking required.’
‘Places you want to spend your best day at’	-

presents all the new options or changes the participants suggested for the lists. During the ‘Hopes&Fears’ part of the session, TaCIA V.3.0. gives participants the option of receiving a different way to look at their declared fear. Five participants opted for listening to TaCIA’s suggestion; P2 provided a comment on this approach (see Appendix A Table A2, ‘Fears’).

At the end of the questionnaire, the participants provided qualitative feedback through open-ended questions that explored the overall experience with TaCIA. Generally, the stroke survivors declared that the digital session had not caused them any emotional distress and that they have not experienced any limitations/constraints during the session with TaCIA. In terms of visible errors, two participants suggested to adjust the language utilized by TaCIA and review the dialogue texts’ accuracy. The improvements recommended regarded increasing the text font size, giving the user an option to turn on/off the sound, and improving TaCIA’s voice. Regarding the participants’ overall experience, four were positive about their experience with TaCIA, participant P3 evaluated the system as ‘very basic’, and P4 did not answer the question.

At the end of the digital session, TaCIA asked four closed questions about the conversation she and the participant just had. The results show that two participants had some challenges during the session, two have identified something that became obvious during the session, three have declared becoming aware of a particular hope for the future, and three have felt motivated to change or try something (

Table 14. Participants’ responses to the final closed questions asked by TaCIA at the end of the session.

Question	P1	P2	P3	P4	P5	P6	P7	P8	Y%
Q1—TaCIA: ‘Did talking with me raise any challenges for you?’	N	Y	N	Y	N	N	N	N	25%
Q2—TaCIA: ‘Did anything stand out for you?’	N	N	N	Y	N	Y	N	N	25%
Q3—TaCIA: ‘Did you become aware of a particular hope for the future?’	N	N	N	Y	N	Y	Y	N	37%
Q4—TaCIA: ‘Did it make you feel motivated to change or try something?’	N	Y	N	N	Y	Y	N	N	37%

N = No, Y = Yes.

).

Also, when asked about their feelings towards taking control of their stroke recovery, P4 chose ‘I feel I am in control’, P1 chose ‘I feel I do not have control of my stroke recovery’, and P3 chose ‘I feel my choices are limited’.

4.3.3. Session LOG Analysis

Eight participants had their full digital session with TaCIA logged for analysis. From the system logs, we can learn about participants’ engagement across the dialogue-based system and have highlighted the following observations:

• The average time spent by the participants during the complete digital session was 28 min.
• Thinking about fears was the only question asked by TaCIA where one of the participants (P5) decided not to answer, with that being the only time when a question was intentionally skipped during the session across all stroke survivor participants.
• For the five participants who opted to listen to TaCIA’s alternative thinking about their fear, P2 selected three distinct options of fear from the list provided and did not like the suggestions proposed by TaCIA to each of them, selecting the feedback options ‘I disagree’, ‘I don’t think you understand the problem’, and ‘I found that offensive’. Participant 4 (P4) had not provided any comments about it, while P6 and P7 were positive, as indicated by selecting the options ‘I like that suggestion’ and ‘I never thought about that in that way’, respectively. After selecting the option ‘Having another stroke’ in the list of fears and opting for listening to TaCIA’s response, Participant 8 has provided a justification for why TaCIAs’ comment would not be applicable to him, saying ‘I have a severe progressive CSVD. (…) I am quite likely to have more.’
• The log from the dialogue-based system shows how the eight participants have responded to TaCIA’s questions that were followed by a list of suggestions, detailed in Appendix A 

  Table A3. Total number of participants per response type to each question, from Q1 to Q8.

  Question	Lists Only	Free Text Only	Lists & Free Text
  Q1- TaCIA: ‘To help you think about the question: ‘How do I feel my stroke has affected me?’, you can enter your own opinion, tick any checkboxes that match how you feel or pick none of them and press the ‘Done’ button.’	3	-	5
  Q2- TaCIA: ‘In terms of relationship roles, what roles do you play within your family and community? Again, click any options that fit you, and you can also add your own option if you want or pick none of them and press the ‘Done’ button.’	6	-	-
  Q3- Tacia: ‘Thinking of your occupation or work as something important that helps to define you, select some options from the list below, add your own roles at the end or pick none of them and press the ‘Done’ button.’	1	1	1
  Q4- TaCIA: ‘From these personality attributes, how would you see yourself? Or add your own ideas at the end. You can also pick none of them and press the ‘Done’ button to skip the question.’	5	-	1
  Q5- TaCIA: ‘How about your hobbies and interests, or how do you like to enjoy your time?’	4	-	4
  Q6- TaCIA: ‘Ask yourself ‘what do I want to do in my life?’. You can enter your own option or pick anyone that you feel is relevant. Let‘s just focus on one at the time.’	6	-	2
  Q7- TaCIA: ‘Okay. Now, ask yourself ‘What am I afraid of?’ If you don‘t have any fears, just click ‘I don‘t want to answer this question’. If you do, you can write your own or pick one to focus on.’	7	-	1
  Q8- TaCIA: ‘Thinking about your best day, who would you be spending the day with?’	7	1	-
  Q9- TaCIA: ‘Here is a framed picture of the combination that you‘ve chosen. Did anything surprise you?’	7	1	-
  Total	46	3	14
  % of column over the Total	73%	4%	22%

  . Overall, 73% of the responses have utilized the list of options provided by TaCIA only, with participants usually providing their own thoughts in a combination of ‘lists of options + free text’ 22% of the time, and only 4% have used free text only without selecting anything from the lists that were presented to them.
• After including the button feedback in TaCIA V.3.0, allowing participants to give feedback at any time during the digital session (Figure 5, bottom right), the system log has captured the overall feedback from the online participants at the end of their session with TaCIA: P5: ‘I think this technology will be very helpful for some people’; P8: ‘(…) This program seems to be designed for someone who has had their first stroke. Doesn’t help someone like me who has a severe genetic cerebral small vessel disease (CADASIL) who has had multiple strokes and is likely to have more (…)’.

Table 15. Phase III participants’ overall impressions about the session and TaCIA.

Comment
P1: ‘Very neat and tidy’
P2: ‘looks like could be usefull’
P3: ‘Very basic’
P4: -
P5 (Online): ‘very good use of technology.’
P6: ‘It created a positive environment to explore some thoughts about what is being experienced in recovering from, and deal with outcomes from a stroke.’

shows the overall feedback provided as an open-ended question ‘Please provide your overall feedback about the session and TACIA’ to participants P1–P6 in the post-session questionnaire.

4.3.4. Phase III Modifications Recommended for Future Implementation

The co-design sprint with stroke survivors in Phase III resulted in identifying changes that aim to improve the user’s experience with the ECA–Patient dialogue system and should be implemented in the next version of TaCIA. The changes concern the content and usability.

From a content perspective, it was recommended to complement the introduction, with TaCIA clarifying that stroke survivors are the primary target of the system who will engage in a talk session about their stroke journey and motivation, without defining rehab goals. It was also recommended to include an extra dialogue where TaCIA would recommend the participant to talk with their carer, GP, family, or support person in case the participant declares being in distress, especially in the session Hopes and Fears. Finally, participants suggested changes to the ‘MyBestDay’ part of the session by allowing participants to change or have more than one best day picture and selections of people and places that would make their best day.

From a usability perspective, participants suggested providing opportunity for users to provide feedback at the end or any other point to give feedback to the research team about the experience. From an accessibility perspective, it was suggested to allow changing the font size, potentially addressing a variety of visual impairments that stroke survivors could have, and to add a switch off/on facility for TaCIA’s speaking (mouth and voice) at any time during the session, as some stroke survivors with aphasia found the text, sound, and lip movement to be overwhelming.

5. Discussion

This project explored the high-level question ‘Is it feasible to deliver the evidence-based ‘Take Charge’ intervention via an ECA–Patient dialogue?’ An ECA–patient dialogue was created based on the ‘Take Charge’ intervention materials and the patient booklet and informed by instructions included within the instructor manual. The next subsection discusses the co-design process to develop and evaluate TaCIA. Then, the research sub-questions are answered in Section 5.2, Section 5.3, Section 5.4 and Section 5.5.

5.1. Co-Designing the Dialogue-Based System: Discussing the Feedback Provided in Each Phase

Consideration of key stakeholders’ perspectives and motivations is essential when developing ECAs, supporting the best uptake and utilization by the system end-users [50]. Once the initial prototype of TaCIA was designed by the iTakeCharge researchers based on the ‘Take Charge’ intervention materials, the co-design approach adopted in the study had started, with each one of the phases collecting feedback from key stakeholders and informing the relevant system changes and adaptations to the following stage.

In Phase I, we contacted the ‘Take Charge’ intervention researchers and invited them to evaluate TaCIA V.1.0. Their feedback was used to develop TaCIA V.2.0, which was further evaluated with four experienced Take Charge session facilitators in Phase II to produce TaCIA V.3.0, evaluated by eight stroke survivor participants in Phase III. The findings from each phase are discussed next.

5.1.1. Findings from Phase I: The ‘Take Charge’ Intervention Researchers

Quantitative and qualitative data gathered through testing the first version of TaCIA with the ‘Take Charge’ intervention researchers revealed two key points. The first is that, to align with the face-to-face delivery of the ‘Take Charge’ intervention, TaCIA should perform active listening, with minimum intervention and suggestions (or opinions) expressed by the ECA to the participants. The iTakeCharge study did not aim to convert the Take Charge intervention to be completely ECA-delivered with stroke survivors. It was inspired by the Take Charge session and should not be considered a digital version of it. It had its own objectives including that the ECA, TaCIA, acted as a coach using a positive, encouraging, and engaging personality. These objectives are based on previous studies that have demonstrated that ECAs delivering encouragement while coaching for behavioral change has a positive impact on the attitudes of patients across many areas of healthcare [35], including chronic conditions [51].

Second, in contrast to the Take Charge session delivered by a human facilitator, TaCIA suggests a list of options for users to choose as their own response to TaCIA’s questions during the iTakeCharge session. The ‘Take Charge’ intervention researchers strongly disagreed that lists would be beneficial to stroke survivors, highlighting the contrast with the original Take Charge session and the potential risk that the lists could influence participants in thinking about their own answers. Instead, the ‘Take Charge’ intervention researchers recommended that lists should be replaced by a free-text box for the participants to enter their own thoughts without any suggestions, even though simultaneously with the display of each list of options, users could enter their own responses by selecting the option ‘Add my own option to the list’, which prompted a free text box on the screen for them to enter their own words. This design was based on the hypothesis, explicitly tested in the next phase, that stroke survivors would find it helpful to see some possible responses for the questions they have been asked, facilitating their reasoning, which is also the novelty adopted uniquely in the iTakeCharge session.

The general perception from the ‘Take Charge’ intervention researchers was that TaCIA V.1.0 did not possess the elements necessary to conduct a similar Take Charge session between a human facilitator and stroke survivors. These participants also believed that TaCIA did not deliver a session format that could collaborate with stroke survivors and help them to improve their understanding about their health condition and empower them to self-rehabilitate.

5.1.2. Findings from Phase II: Take Charge Session Facilitators

Testing TaCIA V.2.0 with the ‘Take Charge’ intervention facilitators (i.e., people who had delivered the ‘Take Charge’ intervention in person with at least 30 stroke survivors) revealed a general perception that the utilization of an ECA may be a feasible tool by which to deliver aspects of the ‘Take Charge’ intervention to support stroke survivors in regaining control of their life after having a stroke (Table 6). Additionally, TaCIA’s designed dialogue and behavior were successful in demonstrating multiple human-like attributes that were recognized by the users during their interaction. Characteristics such as positive, encouraging, helpful, and non-judgemental, listed by participants in their feedback, have reinforced that TaCIA had demonstrated human-like attributions that have good alignment with the ‘Take Charge’ intervention format, supporting the feasibility of the ‘Take Charge’ intervention-inspired session to be delivered in a human–computer interaction format. Table 3, Table 4, Table 5, Table 6 and Table 7 revealed substantial improvements in scores following our improvements to TaCIA from Phase I to Phase II.

The major finding of Phase II was that most participants considered having TaCIA provide a combination of lists of options and a free text box for stroke survivors to be appropriate (Table 8), as they perceived that offering a list of options may prompt stroke survivors to think about their own responses as opposed to a free text box only. Additionally, the options presented in the lists were generally accepted by the ‘Take Charge’ intervention facilitators to be sufficient to predict most responses that stroke survivors might have to each question. Suggestions of new options were also made. Participants in Phase II were positive about generating a pdf booklet with all the responses collected during the session and being able to print it for continued use after the session with TaCIA.

Reinforcing findings from previous research [52], the facilitators also advised us about possible physical and cognitive constraints that stroke survivors would have in using TaCIA. Their feedback informed the inclusion criteria for Phase III and the final iteration of TaCIA, TaCIA V.3.0.

5.1.3. Findings from Phase III: Stroke Survivors

The major finding in Phase III is that, after testing TaCIA V.3.0, most stroke survivor participants were satisfied with the overall experience in the digital session (Table 12). From the six participants who interacted with TaCIA alongside a researcher (HJ), five of them believed that the usability of the ECA–Patient dialogue system was good, that it could be used by other stroke survivors like them, and that they would recommend it to other people living with stroke (Table 12—‘Perspectives about using TaCIA’ questions). These are promising results, as they support the feasibility of a human–computer interaction session format delivering the ‘Take Charge’ intervention-inspired digital session to stroke survivors.

The session log from Phase III, where participants had the choice of using either lists of options or free text entry, reveals that participants selected multiple-choice options from provided lists about 73% of the time (and free text with lists 23% of the time) to interact with TaCIA (Table A3). This indicates the acceptance and relevance of the provided lists as suitable and representative of options in real life.

The feedback provided in the quantitative and qualitative analysis of the pre- and post-session questionnaires indicates that the TaCIA–participant collaboration during the digital session was respectful, appropriate, and generally a good fit for them. Similarly, other studies have found that the relational and empathic characteristics of ECAs contribute to building rapport and developing a bond with patients [11,15,53]. Participants also positively evaluated TaCIA’s performance in delivering the ‘Take Charge’ intervention-inspired digital session over the themes ‘MyStroke vs. MyIdentity’, ‘Hopes&Fears’ and ‘MyBestDay’.

5.2. Stroke Survivor Engagement

Stroke survivors in Phase III have also demonstrated a satisfactory engagement with TaCIA during their session, favorably answering the research sub-question RQ1.2 ‘Has the stroke survivor engaged positively with the materials and the ECA, in terms of the therapeutic alliance?’ Participants 1, 5, and 6 scored the highest score (10 on a 10-point scale) for all of the four related questions regarding the perceived relationship with the ECA, session goals/topics, and approach/method by which the digital session was delivered. This is surprising given that the SRS has been designed for delivery by the therapist at the conclusion of a face-to-face client–patient session [47]. Participants 2 and 4 were also positive, but moderately, with both scoring seven points in the overall session. Participant 3 was the lowest scorer, with goals/topics and approach/method questions being the lowest. Despite that, P3 still declared feeling respected by TaCIA (Q1—relationship with the ECA). We treat these findings cautiously because TaCIA does not work on goals or tasks, and some researchers have voiced concerns over the use of current measures of therapeutic alliance for digital contexts [11,54,55].

Looking more broadly at engagement with TaCIA, in Phase III, all face-to-face participants completed the full session, testing TaCIA and responding to the questionnaires. Two of the three online participants did not respond to the post-TaCIA session questionnaire, which could indicate either that this step in the procedure was not completely clear to online participants (i.e., it being unclear that they should go back to continue responding to the questionnaire after running the session with TaCIA) or that participants did not wish to continue the study after having their session with TaCIA. Regardless, the overall engagement between online participants and the system was satisfactory, with all participants deciding not to skip any questions and going through all three sessions—‘MyStroke vs. MyIdentity’, ‘Hopes&Fears’, and ‘MyBestDay’—of the iTakeCharge, while actively interacting with the lists of options and the free text box, comparable to the face-to-face participants. While we were unable to validate that the online self-declared stroke survivors had in fact had a stroke (i.e., confirmation by clinical assessment, imaging, or medical doctor assessment), online usage without the support of a facilitator is the end goal of iTakeCharge, and thus, there is no validation under those circumstances. From the perspective of whether we can rely on the data of these three participants, the results reported clearly identify these users, and the results are consistent with the results from participants using TaCIA with a facilitator present.

5.3. The Feasibility of Delivering the ‘Take Charge’ Intervention-Inspired Session to Support Stroke Recovery Self-Management

Although the iTakeCharge study has data from only eight participants in Phase III, the general results indicate that the ECA was successful in delivering elements of the ‘Take Charge’ intervention to stroke survivors, specifically answering RQ1.3: ‘Have the dialogues and multiple-choice menu utterances available for each patient’s interaction been suitable and representative of options in real life?’ Concluding from the discussion provided in Section 5.1.2 and Section 5.1.3, there was general agreement that the lists of options provided in these sessions were realistic in representing their own opinions, confirming the opinion of the facilitator group from Phase II.

Next, we consider RQ1.4 ‘Do the data collected via the questionnaires indicate self-awareness and motivation to change and take charge among stroke survivors that have taken the virtual ECA session?’ An important element of the ‘Take Charge’ intervention, ‘Looking at the Big Picture’, is to support stroke survivors’ self-reflection to shift thinking about themselves as a stroke person to a person who happens to have had a stroke. Questions at the end of the session sought to capture any shift in thinking. Table 14 reported that five participants, more than half, became aware of a particular hope for the future and/or felt motivated to change or try something different after having the session guided by TaCIA. Given that the participants only had one session with TaCIA, changing motivations, awareness, or intentions in any participants is a promising outcome. While we anticipated people would feel the system had helped them in managing their stroke recovery, we note that these questions had the response options ‘yes’ and ‘not really’. The response options of ‘yes’ and ‘no’ could have elicited different responses, as ‘not really’ could have been interpreted to mean ‘nothing substantial’ and ‘yes’ could have been interpreted to mean ‘something substantial’.

In the questions related to ‘Perspectives about using TaCIA’ (Table 12), the participants evaluating TaCIA positively demonstrate a good level of acceptance of TaCIA conducting these specific discussions. Five out of six participants have agreed that other stroke survivors would be able to use TaCIA (Q1), with the same number also agreeing that they would recommend TaCIA to other people living with stroke (Q3). When asked if they would like to continue using TaCIA to manage their own stroke recovery, only 50% of them have agreed with that, which could indicate that the current format of the TaCIA Session alone, inspired by the ‘Looking at the big picture’ component of the Take Charge intervention, is not perceived as useful for repeated use by the same stroke survivor. This is also in line with our expectation and intended use of TaCIA, where component two would be used once, but component three involving setting goals and tasks would be accessed repeatedly.

The results from the SUS applied to the three phases indicate that the system usability is considered good. Phase III with face-to-face participants shows that five of the six stroke survivors who tested TaCIA have provided positive quantitative feedback in the questions related to SUS, ‘Evaluating the iTakeCharge session steps’, and ‘Perspectives about using TaCIA’ (Table 12). Positive feedback has also appeared in qualitative data with participants’ overall impression with TaCIA (Table 15). Participants also suggested that the types of stroke survivors considered to be most suited to use TaCIA were those who are computer-literate or who have a support person to assist them with TaCIA and those with mild communication or cognitive problems.

It is also evident that not everyone will benefit from or want to use TaCIA. Participant 3 gave low scores to TaCIA concerning the system (Table 8), with the same reflected in the qualitative data (Table 15), describing the system as ‘Very basic’. In their response to SRS (Table 12 SRS questions), Q4—Please rate today’s session from ‘0—There was something missing in the session today’ to ‘10—Overall, today’s session was right for me’, they chose a score of 4, indicating the session was not appropriate for them.

When considering participants from the Phase III study specifically, we observed a possible association between the participants’ computer literacy and scores in the SUS questionnaire, where participant P4 had self-assessed as having poor computer literacy and later provided the lowest SUS score. That result could indicate that participants with low computer literacy could experience some challenges in using the dialogue-based system. However, P4 did not give low scores concerning their experience with TaCIA and was positive about recommending TaCIA to other stroke survivors. This indicates TaCIA was able to overcome low computer literacy, similar to Bickmore’s [12] finding that ECA can overcome barriers due to low health literacy.

5.4. Comparison of the ‘Take Charge’ Intervention and TaCIA

To further answer our feasibility research question, Phase II included asking the Facilitators to answer the question RQ1.5 ‘How does TaCIA compare with the ‘Take Charge’ intervention?’, specifically seeking to determine whether the TaCIA session had succeeded in supporting stroke survivors to think about their condition. From our Phase II study with the ‘Take Charge’ intervention facilitators, we conclude that TaCIA is a feasible approach to delivering the ‘Take Charge’ intervention program. Furthermore, some comments (Table 10) demonstrate that TaCIA can fill some gaps besides the resource and availability issues, such as being non-judgemental and consistent, in line with benefits found by other researchers (e.g., [9,54]). However, strong attention is needed to ensure TaCIA is not overly positive or discounting the hardship being faced by the stroke survivor, which would potentially negatively impact stroke survivors. This element was previously pointed out by participants in the Phase I study and was considered in the changes taken in Phase III.

All stroke survivors have, overall, evaluated that TaCIA did well in delivering the three elements of the iTakeCharge session that were inspired in the TCS (Table 12—‘Evaluating the iTakeCharge session steps’ questionnaire) regarding ‘MyStroke vs. MyIdentity’ at 3.2 points, ‘My Hopes’ at 3.2 points, ’My Fears’ at 3.0 points, and ‘MyBestDay’ at 3.2 points, all combined at an average of 3.1 points in the 4-point scale, meaning it was a positive experience and similar to the previous evaluation from the ‘Take Charge’ intervention facilitators at, overall, 3.1 points in Table 5. Participant 2 was the only one scoring one of the questions below three points, disagreeing that TaCIA did well with the ‘knowing my fears’ session. P2 suggested improving the experience of this particular session by making TaCIA offer ‘…a fallback just incase the user doesnt agree with the suggestion, as in ‘Try discussing this with your GP, family member, or friends’ or other advice’ (Table A3, ‘Fears’).

We had anticipated that participants would express concerns that TaCIA would not and/or should not fully replace a human therapist, since neither the ECA’s dialogues or the provided user options could adapt and respond in the same manner as a human–human interaction. We did not receive such comments, but in any case, it was never our intention to replace facilitators and human delivery or support of the program, and our expectation was never that her options would be exhaustive.

5.5. Final Components of TaCIA to Be Used in Future Trials

Finally, we can answer the important question: RQ1.1: ‘What are the essential and desirable elements as perceived by stakeholders of an ECA that aims to deliver the ‘Looking at the big picture’ session of the ‘Take Charge’ intervention?’ First, we take a moment to discuss the three principles that underpin the design of the ECAs developed in our lab, Empathic, Empowering, and Ethical, to understand the design of TaCIA and her essential elements. In the case of TaCIA, the unique goal was to leverage the evidence supporting the Take Charge intervention to see if the concept can be delivered by an ECA to allow more stroke survivors to access these benefits. The consumer self-management philosophy behind the Take Charge intervention aligned with our philosophy of patient empowerment embedded in the design of our ECAs. Recognizing that domain expertise and evidence-based content are essential in any health application, the co-design and co-production process, as reported in this article concerning TaCIA, is one of the key ways we ensure the ethical development, evaluation, and usage of our ECAs. As part of the initial phases of TaCIA’s co-design, we shared our principles with the Take Charge researchers and developers to ensure our philosophies were compatible and that we had good alignment in our goals.

An example of how our philosophy distinguishes the design of our ECAs from many other ECAs is our inclusion of dialogue statements to clarify that our ECA is not a human and is not a replacement for human care and contact. In fact, our ECAs clarify to users the importance of health professionals in their care and include triggers when seeking human support is advisable. From an ethical perspective, we ensure our ECAs do not ‘pretend’ to be human, while also using empathic language that draws on multiple relational cues such as mutual knowledge, empathetic understanding, and affirmation and shared goals to build a working alliance to encourage adherence to the advice and psychoeducation provided. We have conducted many studies exploring empathic versus neutral dialogue [56] to understand which relational cues are preferred [57,58] and explanations based on the user’s beliefs and goals for improving behavior change intention [59]. While the goal of most of our agents is related to behavior change, as a persuasive technology, we focus on the empowerment of the individual user rather than the manipulation or use of persuasion techniques.

From a principal-based ethical perspective [60], empowerment focuses on the ethical principle of autonomy by providing choices. Related to the ethical principles of explicability and non-maleficence, in health applications with vulnerable populations, such as children and patients, we do not use the relational cue of self-disclosure by the ECA or design the agent with biographic memory or ‘false memories’, where the ECA may claim to have experienced something or to have preferences, e.g., ‘I love chocolate too’. The use of an ECA which is not constrained by physical embodiment also seeks to address the ethical principles of beneficence and justice through increased accessibility.

While we could not find any ECAs aimed at stroke self-management, we compared our design with two applications designed to assist stroke survivors. We did not find any discussion of design principles in the paper describing the mobile conversational agent by Griol and Callejas [20]. After a review of their presented design, we found that the ‘conversation’ consists of guiding the user through tasks such as testing their memory and a recognition of items. There are no reflective activities. The design principles of the empathic avatar created by Aljaroodi, Adam, Chiong, Cornforth and Minichiello [30] includes the use of: customizable self-avatars, animations with the empathic self-avatar in a familiar environment, gestalt (round face-like shapes), and high-hue colors. These design principles do not concern conversation design and thus are not relevant to the implementation of the Take Charge intervention.

As a result of the three co-design phases in the iTakeCharge study, we were progressively able to improve our design to better align with our three design principles. It is worth noting that co-designers in all phases provided feedback that aimed to increase user empowerment through more options, more explanations and introductions, opportunities to change choices and provide feedback, and increased control over the conversation content, how the content should be delivered, which included elements such as the speed of speech, the font size, and the use of multiple modalities such as text, lip synching, and audio, and what information was provided, as presented in the Phase III modifications in Section 4.3.4. Specific recommended design elements that an ECA system should have in order to deliver self-managed stroke recovery support to stroke survivors include:

• Maintain the approach of a free text box combined with lists of responses during the digital session.
• Using the declared gender of participants for more tailoring of the dialogues and lists of options was perceived as an element to make the ECA–Patient conversation feel less ‘generic’.
• Make sure the ECA does not behave overly positive in such a way that downplays the stroke survivors’ challenges and fears: participants in Phases I and II expressed that such behavior could impact stroke survivors’ perceptions and outcomes of the session.
• Improve TaCIA’s human-like behavior, such as including pauses in between the talks and repeating what was said when requested.
• Further review of the session content flow, vocabulary used, and amount of talking from TaCIA, ensuring the session is kept within 30 min and the discussion is appropriate.

6. Study Limitations and Future Directions

The iTakeCharge study is not without some limitations, which we discuss here. The current system is a constrained format dialogue with a decision tree system of pre-set texts, including lists of options and ECA responses to the user’s selection. While all the free text entered by the users is saved, it was not processed by TaCIA to provide tailored responses during the interaction. This design aimed to ensure patient safety and the mitigation of harmful, incorrect, or invalid ECA responses due to natural language processing errors [61]. Not all the changes suggested were possible to be incorporated during the reported iTakeCharge study. These future changes are provided as a list of recommended changes from Phase III, which we are currently in the process of implementing.

We had a very limited number of participants with demographics not representing the stroke survivor general population, with some limitations observed: all participants had English as their first language and had a declared tertiary educational level, which does not include the representation of the lower-literacy population. Further, online recruitment had not allowed the researcher to execute the participant screening, resulting in a potential recruitment of stroke survivors with a different health condition testing TaCIA and evaluating the iTakeCharge session, such as Participant 8, who declared having a genetic disorder affecting the small blood vessels in the brain.

The iTakeCharge study does not focus on evaluating participants’ motivation to take charge, despite some elements being captured during the feedback sessions. The original ‘Take Charge’ intervention study had focused on delivering a Take Charge Session to stroke survivors in their subacute phase (no more than 16 weeks following their stroke) [4]. In the iTakeCharge study, we did not exclude participants outside this phase, with all eight stroke survivors reporting being in their post-stroke chronic phase (had their stroke for over 6 months) before joining the study. It was recognized that the system, as currently presented, would not be feasible for stroke survivor users with cognitive impairments.

Due to the current limitations and the initial focus on the system feasibility, further research for understanding the digital session effectiveness in supporting self-determined stroke recovery is needed. As a first step, we are in the process of making TaCIA 4.0 using the recommendations from Phase III in Section 4.3.4. Next, we will explore the need for and feasibility of following up the current digital session inspired by the ‘Take Charge’ intervention, ‘Looking at the Big Picture’, with the ‘Breaking it down into do-able pieces’ of the ‘Take Charge’ intervention session, which aims to support stroke survivors in taking the next step in their stroke recovery journey, allowing them to set goals based on their own understanding of the big picture of their lives. Following the investigation, researchers should decide if that next phase should be delivered by TaCIA herself or a human facilitator, or a combination.

Further research needs to be conducted to evaluate if the iTakeCharge session is appropriate for stroke survivors at any stage of their post-stroke condition or if it should focus only on stroke survivors at the acute and/or subacute post-stroke stages. We also potentially need to adapt the system to be mobile-friendly, as it is not guaranteed which medium will be used by participants to access TaCIA online. Importantly, future designs and features must consider ethical issues [60] and elements of patient safety in the online environment [62]. Other modalities such as speech input and integration with various sensors and the use Artificial Intelligence-Guided Communication (AIGC) or virtual reality may also have a place in future versions, but issues around use of Head-Mounted Displays and cognitive overload would be a concern. Due to these concerns, particularly for some users such as stroke survivors with aphasia, currently, we are exploring whether some current modalities might be removed, such as audio and/or lip syncing or the provision of key words or summarized text in TaCIA’s speech/text boxes.

7. Conclusions

The iTakeCharge study represents the first attempt to use an ECA to support stroke survivors in the self-management of their stroke recovery. The study addresses the gap found in the literature concerning the possible use of ECAs to deliver self-determined stroke recovery (Section 2.1) and utilizes the evidence-based ‘Take Charge’ intervention tool (Section 2.2) as inspiration for the content to be delivered in a digital session with stroke survivors.

The primary objective of the study was to collaborate with key stakeholders (former deliverers and end-users (stroke survivors)) and adapt the ‘Looking at the big picture’ portion of the ‘Take Charge’ intervention from a collocated human-to-human therapy into a virtual session involving human–ECA interaction. Stroke survivor participants who received the iTakeCharge session have demonstrated acceptance in interacting with TaCIA, as well as some indication of developing self-awareness and motivation to take charge of their self-recovery and life, therefore contributing to reducing the global burden caused by the stroke [1].

Stroke survivors face many challenges in their recovery. The ‘Take Charge’ intervention study demonstrates the importance of applying the principles of self-management in stroke recovery and its impact on stroke survivor independence and health-related quality of life. The ‘Take Charge’ intervention-inspired digital session with TaCIA aims to increase motivation, self-awareness, and behavior change among stroke survivors for the purposes of improving their quality of life after stroke. The fact that three participants were recruited via iTakeCharge promotional material on social media and have completed a full online session without the support from a researcher suggests it is feasible to deliver such an ECA–Patient dialogue system online. This initial iTakeCharge study is a first step in realizing the possibility of delivering the ‘Take Charge’ intervention-based session by an ECA. The results from this study suggest such a medium has the potential to reduce current barriers to the delivery of self-management interventions including time and the associated costs and burden of a face-to-face session. These results suggest great potential in scaling up a completed ECA version of the ‘Take Charge’ intervention to reach many more stroke survivors nationally and internationally.