This section presents findings from three data collection methods: researchers’ observations, participants’ evaluation questionnaire, and ideas developed and detailed during the workshop.
4.1.1. Observations of the Workshop Participants
Participant observations were carried out by the facilitators throughout the problem exploration, idea generation and idea detailing stages of the codesign process. The aim was to collect qualitative insights on participants’ behaviour during the use of the toolkit to assess the design of the codesign workshop. The facilitators moved between the tables listening to group conversations and noting down all noticeable insights. Insights from the facilitators’ personal observations were linked to the workshop design and facilitation itself, group work dynamics and participant engagement, and the process of using the GENS codesign toolkit.
Workshop design and facilitation. First, it was observed that during the workshop, the participants kept on referring to the contents of the presentations given by the facilitators. Second, even though the facilitators guided the participants through the different elements of the toolkit throughout the workshop, the participants required extra clarification on how to select household energy issues and adapt the idea generation questions listed on the cards to a specific design brief they chose to address. Third, it was observed that two participating groups used some elements of the toolkit slightly differently from how it was explained by the facilitators: one group started by reading all energy issue cards instead of first selecting the issues from the problem exploration diagram to focus on, while the other group generated ideas for all three selected issues simultaneously rather than one at a time. However, both groups achieved the required results by the end of the workshop.
Groupwork and participant engagement. Energetic discussions between the group members were easy to notice, as the whole venue became noisy once the first group task—the selection of a design brief—was introduced. It was observed that the toolkit helped participants from different backgrounds to communicate: the contents of both types of cards worked as a conversation starter and helped participants to open up and share ideas. Even though the contents of the toolkit were written in English, the participants discussed them in both English and Swahili. It was observed that the involvement of Mathare community members made the group work particularly lively because other stakeholders were interested in better understanding the context by questioning the community members. The community representatives also informed the choice of household energy issues. However, not all the participants were fully engaged in the group work. RS1 was more interested in reading through the toolkit’s materials on his own rather than engaging in the group discussions. Later, he shared that the toolkit was very relevant to his personal work, and thus, he was fascinated by the amount of information it provided. In another group, the participation of the energy provider was vague, which was most likely caused by the very talkative researcher who took the lead. The lack of participant engagement could be addressed by supportive and sensible facilitation.
The use of the GENS codesign toolkit. The researcher who praised the contents of the toolkit was not the only participant who found the data of the toolkit interesting. It was observed that the case studies included in the cards helped the participants to understand the rest of the contents of the cards without having sufficient English language skills. PC9 had previous experience using various design tools and complimented the case studies as a particularly informative element of the GENS toolkit. In addition, group conversations and presentations considered gendered aspects, showing that the toolkit encouraged gender-focused thinking. During informal conversations after the workshop, energy providers from two companies expressed their interest in applying the toolkit in their companies’ work.
Despite the positive comments and enthusiasm from the participants, observations also revealed a few limitations of the toolkit. Some participants struggled to understand the data collection and idea generation questions written on the cards; they had to read the same question more than once. Some participants had questions on how to complete the energy innovation canvas and suggested that the canvas could include possible options to choose from: e.g., PC10 was not sure what to include in the “Service” section. Finally, it was observed that the problem exploration diagram and the idea generation template printed on the A3-sized paper were too small, as participants found them difficult to share and read.
4.1.2. Toolkit’s Evaluation Questionnaire
The evaluation questionnaire was designed to assess the completeness, usability and effectiveness of the GENS codesign toolkit and its individual elements. The questionnaire consisted of quantitative closed-ended scale questions and qualitative open-ended questions. Each participant was handed a printed copy of the questionnaire to be completed at the end of the workshop. Qualitative feedback was divided into positive comments praising the toolkit and its individual elements, suggestions for improvements and negative comments criticising the toolkit without suggesting improvements.
- (1)
All Elements of the GENS Codesign Toolkit
Gendered perspective included in the toolkit. All aspects of the whole toolkit received very high evaluation scores (
Table 3). The toolkit’s ability to provide a gendered perspective was praised by the participants: “
So far this is the most integrated gender toolkit for product design I’m so excited to keep using it” (PC9). The feature of how the toolkit incorporated gendered perspectives was evaluated 4.56 out of 5, meaning there was still room for improvement. The participants noticed the absence of considerations regarding how energy affects different age categories of both genders. In terms of the contents, the participants found the case study descriptions too generic: “
Some innovations, concepts are a little generalised. I think we can only be intentional by intentionally involving both genders” (PC2). The community members suggested focusing more on gender involvement in the energy value chain, particularly in the installation and maintenance stages. In terms of the workshop design, the participants suggested that both genders should have equal rights to speak about their issues and express their views on design.
Energycontext of informal urban settlements presented in the toolkit. The toolkit was found to be a useful source for helping understand the energy context of the informal settlement: “I think I got to understand the energy situation in poor urban areas in a way that has not been explored before. Having actors from those regions was a huge bonus” (PC7). The participants requested more context-specific information to be included in the toolkit, such as images from the settlement and descriptions of current “initiatives in place to address energy issues or any upcoming initiatives” (PC8). The participants were also interested in seeing the context-specific statistics. In terms of the workshop design, the participants would like to see more actors from each role in the energy sector and a stronger representation of the community members.
Facilitation of group discussion. The toolkit’s ability to facilitate group discussion received the highest evaluation score (4.75/5): “Very useful for getting diverse perspectives to collaborate” (RS3). Discussions were so engaging that the participants found the workshop too short for the number of activities and discussion points they had to complete: “Having enough time for discussion in order to allow everybody to give his/her opinions” (MC5). In addition, the participants suggested having a skilled facilitator in every group to guide the discussion and ensure the inclusivity of ideas.
- (2)
Problem Exploration Diagram
Completeness. Some participants found the problem exploration diagram complete: “The diagram was very comprehensive in content” (MC9), while others suggested adding more details, such as real-world examples and images: “it could be more specific and detailed” (MC8). However, the participants agreed that “with proper guidance received from the facilitator it was easier to comprehend and use [it]” (PC11). Finally, the issue of sustainability, which was perceived as critical by the participants, was not directly considered in the diagram.
Usability. The colours and overall layout of the diagram were complimented by the participants: “
The diagram was very easy to use by an ordinary person in the urban poor areas” (MC1). The usability feature of the problem exploration diagram received the lowest evaluation point of all elements of the toolkit, yet still high (4.24/5) (
Table 4). The participants expressed their dissatisfaction regarding the small text size that made the legibility of the diagram difficult and requested more than one copy per group. The participants also suggested testing the diagram with colour blind and short-sighted people.
Effectiveness. Despite the contents and usability limitations, the diagram was effective in helping to explore and select energy issues (4.6/5): “I think the diagram covers a large scope and it’s been compressed to cover everything nicely and still be understood” (PC1). The participants emphasised the importance of having sufficient use guidelines and facilitation since the diagram “might seem overwhelming at first” (PC3).
- (3)
Energy Issue Cards and Additional Data Collection Template
The energy issue cards were the most successful part of the toolkit according to the quantitative evaluation score given by the workshop participants (
Table 5).
Completeness. The participants’ opinions differed on the contents of the energy issue cards. Some of them found that the cards contained too much information and too many questions: “Look at reducing the areas, especially questions to answer given the limited time” (PC10). The other participants requested additional information, such as more roles of each gender, the aspects of government involvement in the energy sector and more context-specific case studies.
Usability. The community members found the cards easy to understand because of the simple English language used to explain their contents. However, similarly to the problem exploration diagram, the participants requested larger text sizes and more copies of the cards to be provided to each group to facilitate legibility and sharing. Additionally, the usability could be improved by including use guidelines: “My eyes weren’t quite sure where to start so perhaps there could be an arrow just for easier/quicker comprehension” (PC9). One community member suggested combining the cards with the additional data collection template by adding a context-specific comment section to the cards.
Effectiveness. The cards’ ability to help generate ideas received the highest evaluation score (4.84/5) out of all the elements in the toolkit. The participants found the case studies particularly insightful and helped to trigger ideas: “The global examples on the cards were excellent in helping teams think out of the box” (PC7); in terms of discussing the idea generation questions: “The specific questions really focused the conversation” (PC4).
- (4)
Energy End-Use Cards and the Additional Data Collection Template
It was observed during the workshop that the persona stories made the energy end-use cards educational and engaging: “Best part of the toolkit!” (RS1). “I think I had a better experience with these cards than the previous” (MC5).
Completeness. Several participants agreed that the energy end-use cards were complete: “
They should remain just like as they are” (MC7), “
No additions are needed” (PC4). The contents of the cards received an evaluation score of 4.6 out of 5 (
Table 6), and thus, could be improved. The participants highlighted the lack of gender considerations in the cards: “
Let the questions have a gender component. Ask the question for a woman and the same question as it applies to men” (PC2); they also demanded more context-specific case studies.
Usability. The overall layout and the colours used in the cards were complimented by the workshop participants: “The colours are bright enough for one to read” (MC2). However, the small text size and the lack of copies provided to each participating group were criticized by the participants. The additional data collection template could include more instructions on how to use it.
Effectiveness. The educational aspect of the persona stories included in the cards received the highest quantitative evaluation (4.8/5). The participants requested more stories from other African countries: “The stories help us understand the energy situation in different countries and so we are able to develop innovations that cut across” (PC8). However, the participants pointed out the lack of linkage between the specific challenge they chose to address and the questions listed on the cards: “It was a little confusing when we had to address the challenge and didn’t really quite figure out the questions in relation to persona and the challenge being addressed” (RS4).
- (5)
Idea Generation Template
All features of the idea generation template received relatively high evaluation scores (
Table 7), making it the third most successful tool across both types of cards.
Completeness. The participant feedback regarding the contents of the template was very positive, with most of the comments expressing satisfaction about its completeness: “Nothing can be improved” (MC4), “It is good and organised no need to change anything” (MC8). One researcher suggested including the identification of the actor network as the next step after idea generation.
Usability. In terms of usability, the workshop participants requested that the template be larger, followed by increased text size, so that more ideas can be recorded. The participants, used to colourful tools they used previously, suggested including colours in the template: “Stop making it look like an exam paper” (MC2).
Effectiveness. In addition to idea generation, the template was also made to describe a challenge and help to select a promising idea(s) for further development. RS2 highlighted that it is a “good way to group ideas by solution. Sticky notes really helped”. However, the template’s ability to support idea selection could be improved: “Use stage wise selection of ideas-start with a bigger number and applying a set criterion reduce them until you have the most critical ones.” (GV1). The participants suggested that the workshop design could ensure that everybody explains their opinions alone, while the presentations are recorded for future use.
- (6)
Energy Innovation Canvas
Completeness. The workshop participants complimented the comprehensiveness of the canvas and compared it to the Business Model Canvas, with which most of them were familiar: “The content is elaborated, nothing to be improved” (MC2), “I like this better than the business model canvas” (RS3). However, several participants found the contents of the canvas repetitive. The customers’ section was named as ambiguous for some ideas, and the participants suggested that the canvas could be customised depending on the ideas. Furthermore, the participants highlighted the absence of the aspect of sustainability and social issues.
Usability. The participants found the canvas “great and easy to use. It has a natural flow” (PC10). The comments regarding the usability of the canvas were similar to the ones directed to the idea generation template: the participants requested more colours “to make it exciting” (PC3) and more space in each section. PC10 suggested that the usability could be facilitated by numbering a suggested route for completing the diagram.
Effectiveness. The canvas feature to help detail the selected idea received the highest evaluation score (4.68/5) (
Table 8) compared to other features: “
The canvas adequately captured value proposition for the innovation and a benefit matrix” (PC4). However, some sections included in the canvas could not be completed during the allocated time: “
We were unable to exhaust our discussions for each section” (PC5). Furthermore, an additional template for the problem statement could be introduced to support concept descriptions.
4.1.3. Ideas Developed and Detailed during the Workshop
To triangulate with the findings from the observations of participants and the evaluation questionnaire, we also analysed the ideas recorded by each group on the idea generation diagrams. This enabled us to evaluate the outcomes that the toolkit helped to produce. We aimed at understanding which household energy issues were selected by the participants and the amount, relevance and originality of the developed and detailed ideas.
Each of the participating groups recorded between 18 and 45 ideas in one hour. Each group consisted of between four and six participants, meaning that each participant developed between 3 and 11 ideas. Prior to the idea generation session, the facilitators introduced the most common brainstorming principles and reminded the participants to record all ideas on Post-It notes. However, the participants often forgot to write down the ideas they shared verbally within the group, especially if they engaged in energetic discussion.
The participants developed ideas addressing three household energy issues they selected at the beginning of the workshop. Eleven different issues were selected by the participants out of the seventeen included in the toolkit, while four of them were selected twice. The participants selected issues from all six categories: accessibility (one issue); affordability (two issues); reliability (two issues); health and safety (three issues); income generation (two issues); social culture (one issue). Therefore, the ideas developed by the participants suggested energy solutions addressing a great range of energy-related issues in informal urban settlements.
The GENS codesign toolkit was developed to combine gender, energy and informal urban settlements. Therefore, we looked at whether and how the recorded ideas included all three elements. Most of the recorded ideas included the energy aspect and focused on low-income urban areas, e.g., “Community kitty to maintain and service the street lighting, facilitated by community leader living in the area” (group 3) and “Smart systems to track illegal connections” (group 5). In terms of gender inclusion, there were some ideas recorded that included the terms “women”, “men”, and “gender”, e.g., “Role reversals. Fun ways to feel what it is like to be the opposite gender” (group 4) and “Awareness on how to create innovations using waste collected from the community. Involve youth and women” (group 3). Several ideas included youth and communities, which referred to both genders: “Training/sensitisation/awareness on how to create innovations using waste collected from the community. Involve youth and women” (group 3). Even though gender was not specified in most of the ideas recorded on Post-It notes, group discussions and presentations included gender considerations. During the idea detailing stage, the energy innovation canvas encouraged participants to consider both genders, energy and informal urban settlements, and thus, the final detailed ideas sufficiently combined all three elements.
In terms of creativity and originality, the ideas ranged from generic: “Energy efficient appliances” (group 2), to elaborated and context-specific: “Feasibility study for wind power to generate electricity to power outdoor lighting (Kosovo is located on a hill with potential for wind power generation)” (group 3). Group 4, which addressed a communal entertainment facility challenge and included a design expert, developed particularly original ideas that did not repeat any information included in the toolkit: “Self-defence classes for women-Kenya Karate Grannies”, “Mural/artwork for education on the electrical safety” and “Recruitment events/contests for sharing skills” to name a few. In fact, the expertise of group members was reflected in the recorded ideas. Group 5, which also focused on a communal entertainment facility and included a policymaker, developed technical, cost and regulations-oriented ideas: “Different billing schemes based on income level”, “Set standards for imports manufacturing of energy devices” and “Government to: formulate policy to provide incentives to enterprises to lower cost of energy devices”.