SNUXH: A Set of Social Network User Experience Heuristics

Abstract

With the growth and overcrowding of the internet, the use of online social networks has been increasing. Currently, social networks are used by a wide variety of users–with different objectives and in different contexts of use–, so it is essential to design intuitive and easy to use social network applications that generate a positive user experience (UX). The heuristic evaluation is a well-known evaluation method that allows detecting usability problems; a group of experts evaluates a product and/or system using a set of heuristics as a guide. Although the heuristic evaluation is oriented to evaluate the usability, it can be useful to evaluate other aspects related to the UX. Due to the specific features of social networks, it is necessary to have a specific set of heuristics to evaluate them. Sets of specific heuristics for social networks have been proposed, but they focus on evaluating the only usability. This article presents a set of heuristics that attend not only usability issues, but other UX factors as well, social network user experience heuristics (SNUXH). The new set of heuristics was developed, validated, and refined in four iterations. The results obtained in the experimental validation indicate that the SNUXH set is useful and more effective than generic heuristics (Nielsen’s heuristics) when evaluating social networks.

1. Introduction

Nowadays, social networks are part of people’s daily lives. In a study conducted by GlobalWebIndex in 2018 to 113,932 users from several countries [1], it was concluded that 98% of digital consumers are social media users, and adoption is high even among 55–64 years (94%). Social networks have evolved into multimedia platforms, and for this reason, users are now as likely to use them for keeping up with the news (40% of users) to stay in touch with friends (40% of users). On the other hand, 36% of users use social networks for entertainment. In a lower percentage, users also use social networks to share photos or videos with others (33%), to research/find products to buy (29%), to share their opinions (29%), and to create networks for work (23%). In the same report, it is showed that the most popular social platforms are Facebook (with an 85% of users with an account created on Facebook); followed by YouTube (79% of users), Facebook messenger (72% of users), and WhatsApp (66% of users). According to the study, there are 20 most used social networks.

It is a fact that social networks are widely used. A social network can be defined as a social structure formed by people or entities connected and linked together by some type of relationship or common interest [2]. We can maintain this social network using devices, such as smartphones, laptops, tablets, etc., and applications. There is a large variety of social networks, and users use those that meet their expectations; social networks that allow them to perform the tasks they want to do (sell products, share with users, read the news, etc.), and social networks where the interaction to be intuitive and pleasant, generating a good experience. If a social network is difficult to use or the users do not understand how to use it, they can get frustrated and look for another one. If the use of a social network evokes negative experiences among users, it will hinder the product’s success on the market [3]. It is important to detect problems that prevent users from performing their tasks in a simple and intuitive way, and to understand what emotions are important for them in the interaction.

In this sense, it is crucial to evaluate the usability and the experience that users have when using social networks, detect problems that may generate negative experiences, and correct them to generate greater user satisfaction. One of the most used methods to evaluate usability is the heuristic evaluation [4], which detects usability problems using usability heuristics [5]. Although the heuristic evaluation is oriented to evaluate the usability, it can be useful to evaluate other aspects related to the user experience (UX) [6,7,8,9]. UX is not the same as usability. The UX is composed of several dimensions, such as accessibility, communicability, usability, among others. Thus, usability is part of UX, and depending on the set of heuristics used, it is possible to evaluate both usability and UX. In this sense, several authors have developed heuristics to evaluate UX aspects [9]. Well-known Nielsen’s heuristics [10] are widely used to evaluate applications in a general way. However, social networks have specific features that differentiate them from other applications, so a generic set of heuristics may not be effective in detecting specific usability/UX problems [6,11,12].

Sets of heuristics have been proposed to evaluate social networks [13,14,15]. However, these sets focus on evaluating usability [13,15] or specific features for a particular discipline (security and health-oriented social networks [14]) so other relevant aspects related to the UX are not evaluated with the existing heuristics. In this article, we present SNUXH—a set of user experience heuristics for social networks. The set was proposed to evaluate several aspects of the UX, including usability. The heuristics were developed in four iterations using the methodology proposed by Quiñones et al. [6,7] and were specified, including 13 elements: id, priority, name, definition, explanation, social network feature, examples, benefits, problems, checklist, usability attribute related, UX attribute related, and set(s) of existing heuristics related. SNUXH was validated and refined through several experiments with experts (expert judgment and heuristic evaluation) and users (co-discovery test). The version of SNUXH developed in the third iteration can be reviewed in Reference [16] with preliminary validation results.

This article is organized as follows: Section 2 explores the theoretical background; Section 3 briefly explains the methodology applied to develop SNUXH; Section 4 shows the process followed to develop SNUXH; Section 5 explains the SNUXH validation and results; Section 6 presents the final set of SNUXH; and Section 7 presents the conclusions and future works.

2. Theoretical Background

The concepts of social network, UX, usability, and heuristic evaluation are briefly presented below. In addition, related work is described.

2.1. Social Networks

A social network is a social structure formed by people or entities connected and linked together by some type of relationship or common interest [2]. Currently, a social network is related to websites or mobile applications used by users to connect with people with common interests. This “online” social network provides services through the Internet and allows users to generate a public profile; include data and personal information; and interact with other users using different tools depending on the type of network [17]. Users can build a public (or semipublic) profile within a delimited social network [18], manage a list of other users with whom they have a connection, and thus, share experiences, content, images, videos, among others. The nature of interactions between users differs between different types of social networks (Facebook, Instagram, Twitter, LinkedIn, YouTube, etc.). According to the ONTSI [19], social networks are classified into two large groups: direct and indirect.

On the one hand, in direct social networks, there is a collaboration between people who share common interests. Users interact with each other on equal terms and can control the information they share. Users create profiles through which they manage their personal information, their degree of privacy, and the relationship with other users (e.g., Facebook, YouTube, Twitter). Direct social networks are classified according to four approaches: (1) Purpose (leisure or professional use); (2) Mode of operation (of content, based on personal or professional profiles, or microblogging); (3) Degree of openness (public or private); and (4) Level of integration (vertical or horizontal integration). On the other hand, in indirect social networks, users do not usually have a visible profile for everyone. There is an individual or group that controls and directs information or discussions around a specific topic. Indirect social networks are classified into forums and blogs (e.g., Reddit, 9gag).

A social network can be classified into more than one type. For example, Twitter is a type of direct social network that can be of leisure or professional use (according to its purpose) and also public (according to its degree of openness). Depending on the type of social network, these may have different functionalities and features. However, all online social networks share the following six features [2,17,19,20]:

• Security: Social network must maintain the confidentiality, integrity, and availability of data (personal information, images, videos, etc.), avoiding unauthorized access.
• Connectivity: Social network allows users to establish links with other people, generating different types of connections. The social network provides tools for users to manage their contacts, their relationships, and maintain contact with other users.
• Interaction: Through the social network, the user can interact and relate to each other through different mechanisms (audio, video, text). Social network allows users to share content and experiences.
• Customization: Users can adjust different configurations of the social network based on their interests and needs. For example: Profile personalization, notification, permissions, language, interface settings, etc.
• Content management: Social network provides tools to users to manage the content and information they wish to display and visualize within their network space, independently if other users will interact with it or not.
• Help Center: Social network offers users a help service to solve problems or doubts about its use and functionalities.

2.2. User Experience and Usability

UX refers to “person’s perceptions and responses resulting from the use and/or anticipated use of a product, system or service” [21]. UX includes all the emotions, beliefs, preferences, perceptions, physical and psychological responses, behaviors, and achievements of users that occur before, during, and after use [21]. UX involves how the user feels when using a product or how they will feel when using a product in the future. The UX is different for each user, so it is crucial to know who the users that will use the product are, their features, needs, and what motivates them to interact with a specific product. It is very important to try to meet the majority of potential users.

Several factors influence the UX, and different authors have proposed various models to describe the UX. Peter Morville [22] proposes seven factors that influence the UX: useful, usable, desirable, findable, accessible, credible, and valuable. Morville states that the core of the UX is that users find a value to what is being provided to them. On the other hand, Arhippainen and Tähti [23] decompose the UX into several factors classified into five groups: specific user factors, social factors, cultural factors, the context of use factors, and product factors. For Kankainen [24], the UX is the result of a motivated action in a specific context, which is influenced by the user’s expectations and previous experiences. The above influences the user’s current experience, and with it, their expectations and future experiences. Each proposal is different, and which one to use will depend on the type of product (system or service), its purpose, the context of use, and the type of user.

As described by Morville proposal [22], usability is part of the UX, and they are not the same. Usability is a quality attribute and is defined as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use” [21]. The ISO 9241 standard [21] states three attributes for usability: effectiveness, efficiency, and satisfaction. Since usability is part of the UX, then UX extends these three usability attributes [25]. On the other hand, Nielsen [26] proposes five attributes: learnability, efficiency, memorability, errors, and satisfaction. Although the UX is a broader concept than usability, evaluating usability also evaluates the UX partially. For the development of SNUXH, we used the UX factors proposed by Morville [22], and the usability attributes proposed by the ISO 9241 standard [21]. The reason for this selection is explained in detail in Section 3.

2.3. Heuristic Evaluation

There are many methods for evaluating usability and UX. The AllaboutUX website gathers more than 80 evaluation methods [27]. In general, the evaluation methods are classified as follows:

• Inspection methods: Methods in which expert evaluators (no users) inspect a product, system, or service to detect usability/UX problems. These inspections can be carried out before, during, and/or after the product development. These methods include: Heuristic evaluation, cognitive walkthrough, formal inspections [28], among others.
• Test methods: Methods in which end users participate. Users interact with a product, system, or service to gather relevant information about usability/UX problems. These methods include: thinking aloud [29], co-discovery [30], emocards [31], among others.

Heuristic evaluation is an inspection method proposed by Nielsen and Molich [5,29]. This method involves having between 3 and 5 expert evaluators who examine a user interface to find (potential) usability/UX problems and judge its compliance with recognized usability principles (“heuristics”). Heuristics are “general rules that seem to describe common properties of usable interfaces” [10,32]. Depending on the set of heuristics used, it is possible to evaluate both usability and other attributes related to the UX [6,7,8,9].

Nielsen proposed 10 general usability heuristics (NH) for evaluating the interaction design [10]: NH1: Visibility of system status; NH2: Match between system and the real world; NH3: User control and freedom; NH4: Consistency and standards; NH5: Error prevention; NH6: Recognition rather than recall; NH7: Flexibility and efficiency of use; NH8: Aesthetic and minimalist design; NH9: Help users recognize, diagnose, and recover from errors; and NH10: Help and documentation.

Nevertheless, Nielsen’s heuristics [10] may be too generic to evaluate specific application domains [11]. For this reason, several authors have proposed new sets of usability/UX heuristics to evaluate different domains considering those specific features that differentiate them from other domains [11,12]. Two systematic reviews [11,12] analyze in detail the process of development and validation of new sets of heuristics. Most authors establish a new set of usability/UX heuristics adapting existing heuristics and creating new heuristics to evaluate specific features of the domain, in addition to usability/UX attributes. Other authors apply a methodology to establish and validate a new set of heuristics [11].

2.4. Related Work

Esteves et al. [13] propose eight usability heuristics oriented to social networks (EH). Each heuristic has a name and a brief explanation. The heuristics were developed based on usability problems and heuristics for virtual worlds. The set evaluates relevant features of social networks, such as: The disclosure of personal data, visualization of user information, and interaction. However, it does not consider general aspects (or elements) of any application, such as “aesthetic and minimalist design” (Nielsen’s heuristics [10]), does not have heuristics to evaluate the creation or publication of content, and does not present experimental validation.

Yeratziotis et al. [14] present 11 heuristics to evaluate the security of social networks (Online Health Social Networking, YH). Although they present an interesting set of heuristics, the proposal focuses on a specific type of social networks for health, which makes it difficult to evaluate online social networks more generally.

Dubois [15] presents an interesting and complete analysis of social networks and their relationship with psychology (psychosocial components). It proposes a method of evaluating usability in social networks through heuristics with specific metrics, grouped in categories. However, this study does not present a set of heuristics with a name and definition, but proposes aspects and categories to be evaluated using checklists, which makes it difficult to use in a heuristic evaluation.

3. Methodology Applied to Establish SNUXH

Even though there are several sets of heuristics to evaluate specific application domains [11,12], there are few formal methodologies to develop and validate new heuristics [6,33,34,35,36,37,38]. The heuristics’ validation process is a critical task to determine if the new set allows detecting both general and specific usability/UX problems related to the domain [11,12].

We developed SNUXH using the methodology proposed by Quiñones et al. [6,7]. We decided to apply this methodology since we consider that its stages are well explained and that it proposes concrete methods of heuristics’ validation (both quantitative and qualitative methods). Although other methodologies are interesting, some of them do not explain in detail how to apply the proposed stages [33,34,35,36], or do not propose quantitative and qualitative validation methods that allow evaluating the effectiveness of the heuristics [33,36,37,38].

The methodology used has eight stages that can be applied iteratively to refine and improve the new set of heuristics proposed. The methodology allows developing heuristics to evaluate the usability and other attributes related to the UX. Each stage of the methodology presents: (1) A definition; (2) the inputs that are needed to start the stage; (3) the activities that are performed in the stage; (4) the outputs that are obtained at the end of the stage; and (5) a BPMN diagram that shows the activities of the graphically stage. The methodology stages are shown in

Table 1. Stages of the methodology proposed by Quiñones et al. (adapted from References [6,7]).

Step	Name	Definition
1	Exploratory stage	Perform a literature review to collect information about the specific domain, their features, the usability/UX (user experience) attributes, and existing set of heuristics (and/or other relevant elements).
2	Experimental stage	Analyze data that are obtained in different experiments to collect additional information that has not been identified in the previous stage.
3	Descriptive stage	Select and prioritize the most important topics of all information that was collected in the previous stages.
4	Correlational stage	Match the features of the specific application domain with the usability/UX attributes and existing heuristics (and/or other relevant elements).
5	Selection stage	Keep, adapt, and/or discard the existing sets of usability/UX heuristics that were selected in Step 3 (and/or other relevant elements).
6	Specification stage	Formally specify the new set of usability/UX heuristics using a standard template.
7	Validation stage	Validate the set of heuristics through several experiments (heuristic evaluations, expert judgments, user tests) in terms of their effectiveness and efficiency in evaluating the specific application.
8	Refinement stage	Refine and improve the new set of heuristics based on the feedback that was obtained in Step 7.

.

4. Applying the Methodology to Establish SNUXH

We established SNUXH through four iterations. Figure 1 shows the stages applied in each iteration. Iterations are marked as “It. N”.

In iteration 1, all stages were performed, and the heuristics were developed and validated. Iterations 2, 3, and 4 mainly focused on validating and improving the set of heuristics. The detail of inputs, outputs, and activities performed in each step can be reviewed in Appendix A (first iteration), Appendix B (second iteration), Appendix C (third iteration), and Appendix D (fourth iteration). Appendix E shows the set of heuristics for social networks developed at each iteration. Each set has been abbreviated differently for each iteration: SNH (first version, iteration 1), SNWH (second version, iteration 2), SNXH (third version, iteration 3), and SNUXH (final version, iteration 4). Iterations performed are explained below.

4.1. First Iteration

In the first iteration, all stages were performed. In “Step 1: Exploratory stage”, we conducted a literature review to collect information about: Social networks and features (see Section 2 and Section 2.1); usability/UX attributes (see Section 2 and Section 2.2); and existing set of heuristics (see Section 2 and Section 2.4). We performed the activities proposed in the methodology [6,7] for conducting the literature review. In “Step 2: Experimental stage”, we performed a heuristic evaluation of Facebook using three sets of heuristics (Nielsen [10], Esteves et al. [13], and Yeratziotis et al. [14]). Based on the results, we: (1) Analyzed how each set cover (or not cover) the usability/UX problems detected by evaluators; (2) analyzed correct/incorrect associations among problems and heuristics; (3) detected heuristics with unclear definitions; and (4) identified what heuristics are necessary to create to evaluate specific features of social networks not evaluated by the existing sets. For more detail about the results obtained in the experiment performed, see Appendix F.

In “Step 3: Descriptive stage”, we grouped the information collected in steps 1 and 2 (features of social networks, usability/UX attributes, existing sets of heuristics, usability/UX problems), and we sorted and prioritized the information of each topic using a three-level scale. Finally, based on the value assigned, we selected the next information to develop the heuristics: Six social network features (security, connectivity, interaction, customization, content management, help center); three usability attributes (effectiveness, efficiency, satisfaction); five UX attributes (useful, usable, desirable, findable, credible); and three existing sets of heuristics (Nielsen [10], Esteves et al. [13], and Yeratziotis et al. [14]). The detail of selected information can be reviewed in Appendix G. In “Step 4: Correlational stage”, we matched the social network features, usability/UX attributes, and existing sets of heuristics. Only the “Help center” feature is completely covered by the existing sets of heuristics; while the “Security” feature is partially covered. The features “Connectivity”, “Interaction”, “Customization”, and “Content management” are slightly covered, so it is necessary to create new heuristics to evaluate these features. We decided not to group the heuristics into categories. For more detail about the matching process, see Appendix H.

In “Step 5: Selection stage”, using the three existing sets of heuristics selected in step 3 [10,13,14], we kept zero heuristics without changes; adapted 25 heuristics to evaluate both general and specific features; and discard nine heuristics (since they did not fit to evaluate social networks). We created one new heuristic for evaluating the specific social network feature “Content management”. For more details about the selection process, see Appendix I. In “Step 6: Specification stage”, we proposed the first version of social network heuristics (SNH, see Appendix E). This version included 16 heuristics, and the template proposed in the methodology was used for its specification, including all elements except the “checklist” item (“checklist” was added in the second iteration).

In “Step 7: Validation stage”, we validated the first version of heuristics through heuristic evaluations, analyzing Facebook. Evaluations were carried out by two separate groups of evaluators, in equal conditions. Each group was composed of three evaluators, of a similar level of expertise. One group used only the SNH set (experimental group), while the other group used only Nielsen’s heuristics [10] (control group). All evaluators have moderate to high expertise in HCI and have experience performing heuristic evaluations (they performed a heuristic evaluation on a regular basis). The results obtained by the two groups were then compared. The details of heuristic evaluations performed, and results obtained are presented in Section 5 and Section 2.1. Finally, in “Step 8: Refinement stage”, we refined the heuristics based on the results and feedback obtained in step 7. We refined and improved all heuristics and eliminated two heuristics. We decided to perform a second iteration, repeating steps 6, 7, and 8. For more detail about the refinements performed, see Appendix J.

4.2. Second Iteration

In the second iteration, we performed steps 6, 7, and 8. In “Step 6: Specification stage”, we proposed the second version of social network heuristics (SNWH, see Appendix E). This second version included 14 heuristics, and we completed the entire template, adding checklists. In addition, we improved the specification of each heuristic and eliminated the heuristics “SNH2: Visibility of system elements and important information” and “SNH6: Consistency in design and web symbology”, since these heuristics were similar to SNH1 and SNH5.

In “Step 7: Validation stage”, we validated the second version of heuristics (SNWH) through expert judgment. We surveyed the three evaluators who participated in the heuristic evaluation performed in the previous iteration and who used the SNH set. We asked them about the utility, clarity, ease of use, and need for additional elements of SNWH to evaluate social networks. The results obtained are presented in Section 5 and Section 2.2. Finally, in “Step 8: Refinement stage”, we refined the heuristics based on the results and feedback obtained in step 7. We refined eight heuristics, eliminated two heuristics, joined two heuristics into one, and create two heuristics. We decided to perform the third iteration, repeating steps 6, 7, and 8 to validate if the changes performed improve the effectiveness of SNWH. For more detail about the refinements performed, see Appendix K.

4.3. Third Iteration

In the third iteration, we performed once again steps 6, 7, and 8. In “Step 6: Specification stage”, we proposed the third version of social network heuristics (SNXH, see Appendix E). This third version included 12 heuristics, improving the heuristics specification. We presented the SNXH version of the third iteration in Reference [35], including the ID, name, and description for the 12 heuristics. In Reference [16], we present the heuristics proposed in the third iteration, with preliminary validation results.

In “Step 7: Validation stage”, we validated the third version of heuristics (SNXH) through user tests and expert opinion. We performed a Co-discovery test to: (1) Validate that the problems identified in the heuristic evaluation (first iteration) are perceived as real problems for the users, and (2) identify new usability/UX problems that arise in the test (not detected in the heuristic evaluation), and then review if these problems are evaluated by the SNXH set. The test was designed to evaluate Facebook in its desktop application (website version). A total of eight users participated in the test forming four groups (two users per group). The participants were undergraduate students of the School of Informatics Engineering of the Pontificia Universidad Católica de Valparaíso (PUCV), Chile. All the participants had knowledge of the use of websites and the social network Facebook. The experiments were performed in the usability laboratory of the School of Informatics Engineering.

In addition, we conducted an expert opinion to validate if the SNXH set is correctly specified. We asked two experts in HCI what they think about the SNXH set. Based on the feedback obtained, the SNXH set was refined. All the comments and recommendations were taken into account to improve the set. The detail of the results obtained in both the Co-discovery test and the expert opinion is presented in Section 5 and Section 2.3.

Finally, in “Step 8: Refinement stage”, we refined the heuristics based on the results and feedback obtained in step 7. We refined four heuristics, eliminated one heuristic, separate one heuristic into two, and create four heuristics. We decided to perform a final iteration, repeating step 6 to propose the final version of SNXH. For more detail about the refinements performed, see Appendix L.

4.4. Fourth Iteration

In the fourth and final iteration, we performed step 6 to prepare the final version of SNUXH. In “Step 6: Specification stage”, we proposed the fourth version of social network heuristics (SNUXH, see Appendix E). This version included 14 heuristics, improving the heuristics specification. We refined the heuristics based on the results obtained in validations performed in the third iteration. The final version of SNUXH is presented in Section 6.

5. Validation of SNUXH

The validation of a new set of usability/UX heuristics is one of the most complex tasks identified by several authors [4,11,12]. The methodology applied to develop SNUXH proposes three validation methods: heuristic evaluation, expert judgment, and user tests [6,7]. We experimentally validated SNUXH in three iterations using the three methods proposed. The results of each iteration are described below.

5.1. First Iteration: Validation through Heuristic Evaluations

We selected six evaluators to perform a heuristic evaluation to evaluate SNH against a set of control heuristics (Nielsen’s heuristics, NH [10]). We decided to use Nielsen’s heuristics as control heuristics, given the results obtained in the “Step 2: Experimental stage”. In step 2, a heuristic evaluation was performed using three sets of heuristics [10,13,14]. Of the three sets, NH obtained better results in terms of correct associations and a number of covered problems related to social networks. An association is correct when the unfulfilled heuristic is properly linked to the identified usability/UX problem. The researcher must perform a critical and constructive analysis of the results and determine if the association of problems with heuristics was correct or incorrect. For more detail, see Appendix F.

Facebook was evaluated (in its 210–240 version). We decided to use Facebook as a case study since it is one of the most used social networks [1]. The protocol of the heuristic evaluation proposed by Nielsen was followed [5,32]. Facebook was evaluated by two separate groups of evaluators of similar experience under the same conditions. One group uses only the SNH set (three evaluators, experimental group), while the second group uses only NH set [10] (three evaluators, control group). The groups worked separately. Then, the usability/UX problems identified by the two groups were compared to evaluate the effectiveness of SNH in terms of five criteria [6,7]. The description and formula of the criteria are presented in

Table 2. Five criteria for evaluating the effectiveness of a new set of usability/UX heuristics (adapted from References [6,7]).

Criterion Description	Formula
(1) Numbers of correct and incorrect associations of problems to heuristics	$CA=\frac{{\sum }_{n=1}^{T}CAHn}{TP}\times 100$ $IA=\frac{{\sum }_{n=1}^{T}IAHn}{TP}\times 100$ where: - CA: correct associations - IA: incorrect associations - T: total number of heuristics of the set - CAHn: number of correct associations of the problems to the heuristic “n” - IAHn: number of incorrect associations of the problems to the heuristic “n” - TP: total usability/UX problems identified
(2) Number of usability/UX problems identified	P1 = Problems that are identified by both groups of evaluators (common problems identified by both groups) P2 = Problems that are identified only by the group that used the new set of heuristics (without considering the common problems) P3 = Problems that are identified only by the group that used control heuristics (without considering the common problems)
(3) Number of specific usability/UX problems identified	$ESS=\frac{NSP}{TP}\times 100$ where: - ESS: effectiveness - NSP: number of specific usability/UX problems identified - TP: total usability/UX problems identified
(4) Number of identified usability/UX problems that qualify as more severe (this is, how catastrophic the usability/UX problem detected is)	$ESV=\frac{NPV}{TP}\times 100$ where: - ESV: effectiveness - NPV: number of usability/UX problems identified qualified with a severity greater than 2 - TP: total usability/UX problems identified
(5) Number of identified usability/UX problems that qualify as more critical (this is, how severe and frequent the problem detected is; understanding frequency as the number of times a problem occurs. Criticality is the sum of frequency and severity)	$ESC=\frac{NPC}{TP}\times 100$ where: - ESC: effectiveness - NPC: number of usability/UX problems identified qualified with a criticality greater than 4 - TP: total usability/UX problems identified

. As stated in the methodology [6,7], SNH performs well, and it is an effective instrument when better results than the control heuristics are obtained in terms of (1), (2), (3), (4), and (5) criteria.

Table 3. Effectiveness of social network user experience heuristics, SNH (first iteration).

	Experimental Group	Control Group	Observations
Amount of evaluators	3	3	-
Set of heuristics used	Heuristics for social networks (SNH)	Nielsen’s heuristics (NH)	-
Amount of heuristics (T)	16	10	-
Total of problems identified (TP)	42	34	-
Number of specific problems identified (NSP)	17	9	-
Number of problems identified and qualified with a severity greater than 2 (NPV)	18	12	-
Number of problems identified and qualified with a criticality greater than 4 (NPC)	14	8	-
Problems identified by both groups (P1)	12		Given that (P1) and/or (P2) include the highest amount of problems, it is concluded that SNH works better than NH
Problems identified by the experimental group (P2)	30	-
Problems identified by the control group (P3)	-	22
Total of the correct associations ($\sum \mathrm{CAHn}$)	32	18	-
Total of the incorrect associations ($\sum \mathrm{IAHn}$)	10	16	-
Percentage of the correct associations (CA)	CA1 = 76.19%	CA2 = 52.9%	Given that CA1 > CA2 it is concluded that SNH works better than NH (SNH has a higher percentage of correct associations)
Percentage of the incorrect associations (IA)	IA1 = 23.8%	IA2 = 47.05%	Given that IA1 < IA2 it is concluded that SNH works better than NH (SNH has a lower percentage of correct associations)
Effectiveness in terms of number of specific problems identified (ESS)	ESS1 = 40.47%	ESS2 = 26.47%	Given that ESS1 > ESS2 it is concluded that SNH works better than NH (SNH finds more specific usability/UX problems than NH)
Effectiveness in terms of number of problems identified and qualified with a severity greater than 2 (ESV)	ESV1 = 42.85%	ESV2 = 35.29%	Given that ESV1 > ESV2 it is concluded that SNH works better than NH (SNH finds that more usability/UX problems qualify as more severe than NH)
Effectiveness in terms of number of problems identified and qualified with a criticality greater than 4 (ESC)	ESC1 = 33.33%	ESC2 = 23.52%	Given that ESC1 > ESC2 it is concluded that SNH works better than NH (SNH finds that more usability/UX problems qualify as more critical than NH)

shows the results obtained in the heuristic evaluations performed by both experimental and control groups. In addition, the effectiveness of SNH in terms of the five criteria is shown. As shown in Table 3, SNH obtained better results than NH in the five criteria, indicating that they work better and are more effective. SNH detected more usability/UX problems than NH, and moreover, detected more specific, severe, and critical problems.

In both evaluations, SNH and NH had associated at least one usability/UX problem, except the heuristic “SNH9: Minimize the user’s memory load” of SNH set, which had zero associated problems. The specification of SNH9 should be reviewed and determine why it has no associated problems. The heuristics with more incorrect associations for SNH set were: “SNH4: Match between system and the real world”, “SNH5: Consistency and standards between system elements”, and “SNH6: Consistency in design and web symbology”. It is necessary to review and improve the specification of these heuristics. However, it is important to notice that the percentage of incorrect associations obtained for SNH is lower than for NH. After analyzing the results, each heuristic for SNH set was reviewed to improve its specification.

5.2. Second Iteration: Validation through Expert Judgment

In the second iteration, we applied a survey to the three evaluators who used SNH set in the heuristic evaluation performed in the previous iteration. Specific scales have been proposed to validate the quality of heuristics [4,39]. We developed (and refined) our own scale to validate the quality of the heuristics through a survey [6,39]. The survey was designed to capture the evaluators’ perception about SNWH set in four dimensions: D1—“Utility”, D2—“Clarity”, D3—“Ease of use”, and D4—“Need of additional elements (checklists)”. A five-point Likert scale was used (where 1 means “worst” and 5 means “best”). D4 has somehow an inverse scale. One would expect that if heuristic is perceived as useful, clear, and easy to use, then it does not need of additional elements. In addition, evaluators added comments for each heuristic.

Table 4. Average perception scores for SNWH are set in the four evaluated dimensions (second iteration).

Heuristic	D1—Utility	D2—Clarity	D3—Ease of Use	D4—N. of Add. Elem.
SNWH1: Visibility of elements and system status	4.25	4.5	4.5	4
SNWH2: Perception and user status	4.5	4.25	4.5	3.5
SNWH3: Match between system and the real world	4	3.25	3.25	3.75
SNWH4: Consistency and standards between system elements	4.5	4	4.25	3.5
SNWH5: User control and freedom	4.5	4.5	4.25	3.5
SNWH6: Error prevention	4.25	4	4	3.75
SNWH7: Minimize the user’s memory load	3.5	3	4	4.5
SNWH8: Flexibility and efficiency of use	4	3.25	3.5	2.75
SNWH9: Aesthetic and minimalist design	4	3	3.75	4
SNWH10: Help user to recognize, diagnose, and recover from errors	4.5	4.25	4	3.25
SNWH11: Help and documentation	4.5	4.5	4	3.5
SNWH12: Privacy and exposure control	4.25	4.25	4	4.75
SNWH13: Content control published	4.25	3.25	3.25	5
SNWH14: Security and recovery of user account	4.5	4.25	3.75	3.75
Average	4.25	3.88	3.93	3.82

shows the average obtained for each dimension per heuristic. In general, heuristics obtained high grades (higher than 3.00). The dimension D1—“Utility” obtained the highest average (4.25), which means that heuristics are perceived as very useful for evaluating social networks. The dimension D4—“Need of additional elements (checklists)” obtained the lowest average (3.82); however, it is higher than the middle point (3.00). The above means the evaluators consider that it is necessary to add additional elements in the heuristics’ specification to help in the detection of usability/UX problems; especially for SNWH7: “Minimize the user’s memory load”, SNWH12: “Privacy and exposure control”, and SNWH13: “Content control published”. The heuristics SNWH12 and SNWH13 are new and specific for evaluating social networks, so it is reasonable that evaluators consider it important to have additional elements to better understand the heuristics. On the other hand, the dimension D2—“Clarity” obtained an average of 3.88, which means that the heuristics’ specification is not so clear; hence, the wording and explanations should be improved; especially for SNWH7: “Minimize the user’s memory load” and SNWH9: “Aesthetic and minimalist design”.

5.3. Third Iteration: Validation through User Tests and Expert Opinion

In the third iteration, we performed a Co-discovery test to evaluate Facebook (in its 210–240 version). Co-discovery is a user testing method that offers valuable user thinking/thoughts insides [30]. Users explored Facebook in pairs, freely discussing it, while performing together predefined tasks. A total of eight users participated in the test forming four groups (two users per group). Participants were undergraduate students of 21–26 years old, from the Computer Science program at Pontificia Universidad Católica de Valparaíso (PUCV), Chile. All participants have experience using social networks and Facebook.

The experiment was designed to validate that the usability/UX problems identified in the heuristic evaluation (first iteration) are perceived as real problems for the users, and to identify new problems and review if these problems are evaluated by the SNXH set. The test was divided into three parts: (1) A pre-experiment questionnaire, aimed to broadly identify user’s profile and his/her previous experience using Facebook; (2) a list of tasks to complete working in pairs (the experiment); and (3) a post-experiment satisfaction questionnaire, aimed to know the user’s perception using Facebook.

We included questions concerning users’ perceptions about task completion, orientation through the website, overall satisfaction regarding Facebook, and intention of future use. We also included open questions that allowed users to comment on Facebook’s (perceived) positive and negative aspects.

Table 5. Co-discovery test quantitative results (third iteration).

Task (T)	Usability/UX Problems Related (P)	Percentage of Task Fulfillment	Description	Heuristic Related (Third Iteration Version, SNXH)
T1: Search for the “event” word in the quick help section	- P1: The quick help section is hard to find. - P2: It is not possible to search for help through the main search engine.	75% (3 of 4 groups)	- Although for three groups, it was difficult to find the quick help section, after several attempts, they were able to locate the section and complete the task. - 1 group could not complete the task since they used the main website search engine, which did not show the information they were looking for.	- P1 is covered by SNXH1 - P2 is covered by SNXH11
T2: Create an event by adding name, location, date, time, and description	- P3: The creation of events is confusing. - P4: The website does not indicate what information is required.	100% (4 of 4 groups)	- All groups were able to correctly perform the task. - For a group, the creation of the event was complicated, but then they understood how to use the functionality.	- P3 is not covered by SNXH - P4 is covered by SNXH5
T3: Invite three friends to the event	- P5: The option to add more guests to an event (apart from friends) is hard to find and use. - P6: Additional options for events are hard to find.	50% (2 of 4 groups)	- 2 groups could not complete the task, since they did not find the option to invite additional friends. - It was difficult for one group to understand how to invite additional friends, but after a couple of tries, they were able to complete the task.	- P5 is covered by SNXH5 - P6 is covered by SNXH1
T4: Create a publication in the event and associate a sentiment with the publication	- P7: Overload of elements and options when publishing content in an event. - P8: The use of icons to associate feelings is unintuitive.	100% (4 of 4 groups)	- All groups were able to correctly perform the task.	- P7 is covered by SNXH4 - P8 is not covered by SNXH
T5: Delete the created event	- P9: Limited options to remove information. - P10: The option to eliminate an event is difficult to find (option not visible).	100% (4 of 4 groups)	- All groups were able to correctly perform the task. - Although all groups to completed the task, it was difficult to find the option to delete the event, since this functionality is not clearly visible in the interface.	- P9 is covered by SNXH6 - P10 is covered by SNXH1

shows the tasks performed by the users in the Co-discovery test and its results. The results of the questions asked were not included because the responses received were not significant to generate additional changes.

Based on the results, shown in Table 5, we concluded that the problems P1, P2, P3, P5, P6, and P10 are perceived as real problems for users since it was difficult for the groups complete the related tasks. We reviewed if SNXH allows evaluating the usability/UX problems detected in the test. As shown in Table 5, all problems detected in the Co-discovery test are covered by SNXH, except the problems P3 and P8, which means that it is necessary to create a new heuristic that evaluates the intuitive use of elements that do not confuse the user.

We also conducted an expert opinion for validating SNXH. We asked two experts in HCI what they think about the SNXH set. The experts had experience performing heuristic evaluations and using sets of usability/UX heuristics. They gave their points of view regarding whether the SNXH set is correctly specified, whether it is useful, what elements they would add or remove. Based on the feedback obtained, the SNXH set was refined. All the comments and recommendations were taken into account to improve the set. Experts said:

• Some heuristics refer to systems or websites, while others refer to applications. For consistency, it is recommended to use only one term (for example, “applications”).
• SNXH5 (Prevention and recovery from errors) may cover many usability/UX problems. Prevent errors is not the same that recover from them. SNXH5 should be separated into two heuristics.
• The name of some heuristics may confuse the evaluator, since it does not make clear what aspects it evaluates. For example, the name “Design and interface” (SNXH2) does not make clear if what is evaluated with the heuristic is the interface aesthetic part (“design”), or the interface type and its elements (“interface”). On the other hand, for the SNXH9 (Privacy and exposure control), it is not clear what is “exposure control”. The heuristic description is related to privacy. It is recommended to simplify the heuristic name so that it is clearer what aspect it evaluates.
• SNXH12 (Consistency between platforms) specification implies that the heuristic evaluates aspects related to standards in different platforms, but its name does not indicate it.
• SNXH8 (Alert and notification control) is too specific. The heuristic is oriented to evaluate only notifications and alerts, so it is very likely that it has few (or no) associated usability/UX problems. It is recommended to modify the heuristic to evaluate more aspects that can be customizable (such as messages, sounds, etc.).
• The explanation of SNXH10 (security and recovery of user account) does not cover the “recovery of user account” aspect. The explanation should be improved to be consistent with the heuristic name.
• There is no heuristic that evaluates the flexibility of use. This is a critical element in social networks, since users use them in different contexts to meet different objectives. In addition, users may (or may not) have experience using similar social networks, so they should be tolerable at different levels of user experience.

6. SNUXH: A Set of Social Network User Experience Heuristics

Based on the iterations and validations described in the previous sections, the SNUXH set was refined and improved.

We develop 14 heuristics. Of these, four are considered critical (with priority 3: SNUXH3, SNUXH12, SNUXH13, and SNUXH14), because they evaluate specific and crucial aspects of social networks; seven heuristics are considered important (with priority 2: SNUXH2, SNUXH4, SNUXH5, SNUXH6, SNUXH7, SNUXH9, and SNUXH11), since they evaluate relevant aspects of social networks; and three heuristics are considered useful (with priority 1: SNUXH1, SNUXH8, and SNUXH10), since they evaluate elements that improve the UX. Regarding usability attributes: Six heuristics evaluate effectiveness, six heuristics evaluate efficiency, and nine heuristics evaluate the satisfaction. Regarding the UX attributes: Two heuristics evaluate the useful factor, nine the usable factor, six the desirable factor, five the findable factor, and seven the credible factor.

Of all the proposed heuristics, SNUXH12 is a brand-new heuristic created to evaluate specific features of social networks. It focuses on evaluating customization, content management, and interaction with a social network regarding the published content that users want (or do not want) to see. Users should feel comfortable and confident when browsing a social network, so they should have options to configure what they want to see.

The 14 heuristics are presented below. Each heuristic is presented as a table that includes 13 elements (

Table 6. SNUXH1: Visual feedback and social network status.

Id	SNUXH1
Name	Visual Feedback and Social Network Status
Priority	(1) Useful
Definition	The social network must inform the user application’s status in response to the actions that he/she performs.
Explanation	The social network must inform the user through feedback (such as visual or auditory feedback), the state of the application, and the events that occur while the user interacts with it. The social network must inform the user when the application waits for an action from him/her.
Social network feature	(1) Connectivity, (2) Content management, and (3) Interaction
Examples	Figure 2 shows an example of heuristic compliance in Twitter, the number of new publications that have emerged while the user interacts with other elements of the application.
Benefits	The user will notice that the application informs about the operations performed during the interaction, which generates a higher degree of satisfaction, and confidence in the social network.
Problems	This heuristic is related to the feedback, i.e., that the social network informs its state to the user. The evaluator may confuse this heuristic with SNUXH6 (“Minimize the user’s memory load”), which is related to not forcing the user to remember information. Remembering information is not the same as giving feedback.
Checklist	The social network offers loading elements that inform users of the state of processes that he/she performs (e.g., loading bars). The user must be able to visualize the content loaded to the social network (videos, images, texts) through a representative visual element, especially when the files to upload will take longer than expected. The social network indicates if a process is taking longer than expected. The social network executes the functionality expected by the user when performing an action. The social network communicates to the user the number of new publications. The social network is updated in real-time (when it applies).
Usability attribute related	Satisfaction, Efficiency
UX attribute related	Usable, Findable
Sets of heuristics related	Nielsen, “Visibility of system status” [10] Esteves et al., “Privacy and exhibition control” [13] Yeratziotis et al., “Visibility”, “Availability” [14]

,

Table 7. SNUXH2: Match between the social network and real world.

Id	SNUXH2
Name	Match between the Social Network and Real World
Priority	(2) Important
Definition	The social network should use a language familiar and understandable to the user and use icons that clearly represent their meaning.
Explanation	The social network should show the information, features, and messages in the user’s language, avoiding technical terms. The icons used must clearly represent what they mean, without generating confusion.
Social network feature	(1) Interaction
Examples	Figure 3 shows an example of heuristic compliance in Facebook. The social network icons represent “real life” emotions (like, love, happiness, sadness, and anger).
Benefits	The user will understand the features, icons, and messages presented by the social network, making it easier to interact with and use.
Problems	The evaluator could confuse this heuristic with “SNUXH4: Consistency and standards in multiplatform”. Problems related to the use of icons or symbols familiar to the user and that make sense of what they represent are related to SNUXH3. SNUXH4 refers to consistency and standards between platforms and sections, not (only) standard iconography.
Checklist	The icons shown by the social network clearly represent the functionality associated with them. The social network groups functionalities and/or icons in related categories. The social network highlights elements or functionalities through representative colors for the user. The social network allows the user to choose between more than one language. The social network shows the information and functionalities in the language selected by the user.
Usability attribute related	Satisfaction
UX attribute related	Usable, Desirable
Sets of heuristics related	Nielsen, “Match between system and the real world” [10]

,

Table 8. SNUXH3: User control and freedom.

Id	SNUXH3
Name	User Control and Freedom
Priority	(3) Critical
Definition	The user must feel that he/she manages the social network, being able to undo or redo his/her actions and use the social network freely.
Explanation	The social network must allow the user to control the actions performed, providing options, such as go back, go to home, cancel, confirm (among others). The user must be able to freely use the functionalities of the social network without fear of permanent errors.
Social network feature	(1) Content management, (2) Interaction
Examples	Figure 4 shows an example of heuristic compliance in Facebook. The social network allows actions to edit and/or delete the comment.
Benefits	Social networks that allow controlling actions make users feel comfortable and confident.
Problems	This heuristic should not be confused with the concept of flexibility and customization (SNUXH8). SNUXH3 refers to the control that the user has on the actions performed and the freedom to manage them (such as “undo” actions), while SNUXH8 refers to the social network allows the user to adapt their elements easily according to his/her needs.
Checklist	The social network allows us to always return to the homepage (through its main logo). The social network does not publish content that the user has not previously authorized. The social network must provide options to undo and redo actions. The social network must allow the user to cancel options selected by mistake. The social network must include options to go back, edit or delete publications or comments made by the user.
Usability attribute related	Satisfaction, Effectiveness
UX attribute related	Usable, Credible
Set of heuristics related	Nielsen, “User control and freedom” [10] Yeratziotis et al., “Revocability” [14]

,

Table 9. SNUXH4: Consistency and standards in multiplatform.

Id	SNUXH4
Name	Consistency and Standards in Multiplatform
Priority	(2) Important
Definition	The social network must be consistent in the several platforms that support it. The same functionalities must be present, there should not be differences (e.g., visual differences, behavior differences, etc.), and the standards must be followed for each platform.
Explanation	Several social network platforms (for example, desktop and mobile version) must present elements and actions that have the same meaning. The design, standards, and functionalities must be maintained along with the platforms to not alter the user interaction.
Social network feature	(1) Customization, (2) Connectivity, (3) Content management, and (4) Interaction
Examples	Figure 5 shows the consistency between the desktop and mobile versions of Facebook. The colors, icons (some), and type of letter are consistent on both platforms. However, while some features are maintained on both platforms (such as “Search” and “What’s on your mind?”), others change and are only present in the mobile version (the icon to take photos, located in the upper left corner of the screen, where usually the logo of the social network is located).
Benefits	The user will easily interpret the representative elements (icons) used in the social network, facilitating the use of the available functionalities, and helping their learning process.
Problems	The evaluator may confuse the concept of consistency (SNUXH4) with aesthetics (SNUXH7 “Aesthetic and minimalist design”). Consistency refers to the fact that there must be coherence between the elements of the social network, while aesthetics refers to combining (interface) elements that together generate a visually pleasing appearance.
Checklist	The social network uses the same kind of symbols to show information and actions in the different sections and/or platforms. The social network has the same content distribution in different sections and/or platforms. The social network presents the same functionalities in the different sections and/or platforms. The social network presents the same colors in the different sections and/or platforms. The social network presents the same type of letter in the different sections and/or platforms.
Usability attribute related	Effectiveness, Efficiency
UX attribute related	Usable, Findable
Set of heuristics related	Nielsen, “Consistency and standards” [10] Esteves et al., “Uniform interface” [13]

,

Table 10. SNUXH5: Error prevention.

Id	SNUXH5
Name	Error Prevention
Priority	(2) Important
Definition	The social network must prevent errors from occurring, providing warning messages to the user with useful information and without technical terms.
Explanation	The social network must prevent errors and warn the user before performing an action that leads to unwanted situations or status. The social network should not make available to the user functionalities that are not yet operational.
Social network feature	(1) Security, (2) Connectivity, (3) Interaction, and (4) Help center
Examples	Figure 6 shows an example of heuristic compliance in Twitter. The social network displays a warning message to the user, asking if he/she is sure to discard the tweet to publish.
Benefits	Social networks that prevent errors and that do not publish unwanted content make the user interaction more effective, avoiding user frustrations.
Problems	This heuristic refers to detecting an error in advance and preventing problems from occurring. The evaluator may confuse SNUXH5 with SNUXH9 (“Help users recognize, diagnose, and recover from errors”); however, the difference should be clear: SNUXH5 helps to prevent errors, while SNUXH9 helps to recover from them, that is, returns to a normal state after an error.
Checklist	The social network prevents errors from occurring by displaying warning messages to the user. The social network informs the user that the action he/she is doing will be canceled before changing the section. The social network clearly indicates which fields are mandatory for the continuation of any process. The social network presents its functionalities and icons with adequate separation between them, to prevent the user from pressing the wrong element.
Usability attribute related	Effectiveness, Efficiency
UX attribute related	Usable, Credible
Set of heuristics related	Nielsen, “Error prevention” [10] Yeratziotis et al., “Errors” [14]

,

Table 11. SNUXH6: Minimize the user’s memory load.

Id	SNUXH6
Name	Minimize the User’s Memory Load
Priority	(2) Important
Definition	The social network must minimize the user’s memory load, without forcing him/her to unnecessarily remember information.
Explanation	The social network should not force the user to remember previous information or from one section to another; so, the common actions or settings should be saved and automatically loaded.
Social network feature	(1) Customization
Examples	Figure 7 shows an example of heuristic compliance in Twitter. The concept searched by the user (“world wide web consortium”) and the most relevant functions (“top”, “latest”, “people”, etc.) are always visible, while the user scrolls through the social network to review the results.
Benefits	The user will have the necessary information for the use of the social network at all times, without the need to return to previous states or sections for review. The above should generate better user satisfaction while interacting with the network.
Problems	The evaluator may detect a problem related to SNUXH5 (“Error prevention”) and think that it is related to SNUXH6. It is important to remember that SNUXH6 refers to not forcing the user to remember information. If “showing little information to the user” generates errors, the problem detected is related to SNUXH5, not to SNUXH5.
Checklist	The social network maintains a history of conversation among users (for social networks of instant messaging). The social network shows the section name in which the user is located. The social network saves the user’s previous settings. The login of the social network offers the option “remember login data”. The social network indicates to the user its current location within a navigation hierarchy (breadcrumbs).
Usability attribute related	Effectiveness, Efficiency
UX attribute related	Usable, Desirable
Set of heuristics related	Nielsen, “Recognition rather than recall” [10]

,

Table 12. SNUXH7: Aesthetic and minimalist design.

Id	SNUXH7
Name	Aesthetic and Minimalist Design
Priority	(2) Important
Definition	The social network should show only the relevant elements for the user, without overloading the interface with less usual functionalities.
Explanation	The main functionalities of the social network (such as publishing content and comment) must be differentiated from each other, hiding less usual options that may distract the user. The social network should show information according to the user preferences, hiding information that he/she has not decided to visualize.
Social network feature	(1) Interaction
Examples	Figure 8 shows an example of heuristic compliance in Facebook. The most relevant options and information are displayed, providing options to the user to see more details of the publications or comments (e.g., “view all comments”, “view details”) and to access to more functions or sections (e.g., “see more”, additional option menu “3 dots”).
Benefits	The user will be able to find important information more easily, so he/she will find out more content in less time and without difficulty, providing a better experience.
Problems	The evaluator may confuse SNUXH7 with SNUXH6 (“Minimize the user’s memory load”). SNUXH7 refers to maintain a “clean” interface, showing only the relevant elements of the social network to the user. If the problem of showing few information forces the user to have to remember elements, the problem is related to SNUXH6, not to SNUXH7.
Checklist	The social network shows important information for the user, highlighting it properly without disturbing visually. The social network highlights the publications most commented by users. The social network appropriately uses the different sizes and types of letters depending on the importance of the information. The social network separates the information making good use of the space. The social network avoids information not relevant to the user (e.g., in “Home” or “Timeline”). The social network shows only the most important functionalities in the main interface, separating them from the restless relevant.
Usability attribute related	Satisfaction
UX attribute related	Findable, Usable, Desirable
Set of heuristics related	Nielsen, “Aesthetic and minimalist design” [10] Yeratziotis et al., “Aesthetic and minimalist design” [14]

,

Table 13. SNUXH8: Flexibility and customization.

Id	SNUXH8
Name	Flexibility and Customization
Priority	(1) Useful
Definition	The social network should allow configuring frequent actions and be flexible to adapt its interface based on the users’ preferences and their interests.
Explanation	The social network should be flexible and allow the user to adapt the content to visualize and the functionalities, according to their preferences. The social network should allow the user to modify the order of the elements that are displayed in the interface, the colors he/she wants to use, and the type of information he/she wants to view.
Social network feature	(1) Customization and (2) Interaction
Examples	Figure 9 shows an example of heuristic compliance in Twitter. The social network allows the user to change and personalize the trends to be visualized.
Benefits	Users will feel that they have enough freedom to personalize the social network based on their interests, so they will be able to visualize content based on personal needs, improving their experience in the network.
Problems	The evaluator may confuse this heuristic with SNUXH11 (“Perception and user status”) and/or SNUXH12 (“Content control published”). It is important that the evaluator is clear about the purpose of each heuristic: - SNUXH11 refers to how the user wants other users to see him/her status. SNUXH8 refers to configuring elements of the social network (its interface and content). - SNUXH12 refers to controlling the published content that the user wants to see and report the inappropriate. SNUXH8 refers to configuring elements of the interface (frequent functionalities, colors, how to visualize the content).
Checklist	The social network allows customizing elements, interface colors, profile image, and most frequent functionalities, according to the user preferences. The social network allows the user to create direct access to functionalities or specific content. The social network allows the user to configure which notifications to receive and which not. The social network allows the user to select what type of content to visualize.
Usability attribute related	Effectiveness, Satisfaction
UX attribute related	Usable
Set of heuristics related	Nielsen, “Flexibility and efficiency of use” [10] Esteves et al., “Flexibility” [13] Yeratziotis et al., “User suitability” [14]

,

Table 14. SNUXH9: Help users recognize, diagnose, and recover from errors.

Id	SNUXH9
Name	Help Users Recognize, Diagnose, and Recover from Errors
Priority	(2) Important
Definition	The social network should help the user to recover from errors by indicating the problem and suggesting a solution.
Explanation	When an error occurs, the social network must indicate the problem occurred through error messages expressed in an understandable language. The messages should accurately indicate the problem and suggest a constructive solution.
Social network feature	(1) Security, and (2) Help center
Examples	Figure 10 shows an example of heuristic compliance in Twitter. The social network displays error messages indicating that a problem has occurred (“something went wrong”, “cannot retrieve messages”), its possible reason (“connection problems”), and solution (“please try again”).
Benefits	Social networks that guide the user to solve problems easily and quickly generate better experience.
Problems	This heuristic should not be confused with SNUXH5 (“Error prevention”). If an error occurred (it was not prevented), it is necessary to recover from it, and it is related to SNUXH9 (return to a normal state after an error).
Checklist	The social network immediately informs the user if he/she made a mistake when completing a field (e.g., “username” or “password” field). Error messages displayed by the social network are written in a simple and easy to understand language for users. Error messages include the reason why a problem occurred and its possible solution. The social network informs the user on how to leave the unwanted situation in which he/she is.
Usability attribute related	Efficiency
UX attribute related	Useful, Credible
Set of heuristics related	Nielsen, “Help users recognize, diagnose, and recover from errors” [10] Yeratziotis et al., “Errors” [14]

,

Table 15. SNUXH10: Help center.

Id	SNUXH10
Name	Help Center
Priority	(1) Useful
Definition	The social network should provide help and documentation on how it works, providing accurate information, and oriented on the tasks performed by the user.
Explanation	The social network should provide access to detailed information on how to use the functionalities in a clear and simple way through a specific section, online help, tutorials, or tooltips.
Social network feature	(1) Help Center
Examples	Figure 11 shows an example of heuristic compliance in Facebook, the frequently asked questions about how to use the social network, and help with popular topics. In addition, it allows the user to search for help for specific topics through a search bar.
Benefits	The user will solve easily and quickly the problems that occur during his/her interaction with the social network. The user will feel that the company (or brand) cares about the experience of its customers by incorporating different help mechanisms.
Problems	The evaluator may have difficulty determining if the detected problem is related to this heuristic (SNUXH10) or with SNUXH5 (“Error prevention”). It is important to remember that SNUXH10 is related to providing help on how to use the social network and its functionalities, and not to show messages to prevent errors (SNUXH5).
Checklist	The social network explains how to use its functionalities, either through a help menu, tutorials, or tooltips. The social network informs the user of how to use new features added after an update. Each functionality of the social network is explained, indicating what the steps to follow are, in order to use it correctly. The information presented is understandable for all types of users and useful for solving problems or clarifying doubts. The social network provides (at least) a way to communicate with network administrators in order to resolve problems.
Usability attribute related	Satisfaction
UX attribute related	Findable, Useful, Credible
Set of heuristics related	Nielsen, “Help and documentation” [10] Esteves et al., “Documentation” [13] Yeratziotis et al., “User assistance” [14]

,

Table 16. SNUXH11: Perception and user status.

Id	SNUXH11
Name	Perception and User Status
Priority	(2) Important
Definition	The social network should allow the user to perceive if other users are available to interact and/or communicate. In addition, the social network must allow the user to define how and when his/her status is perceived by other users in the network.
Explanation	The social network should allow the user to change at any time how he/she wants to be perceived by other users within the social network (“Available”, “Busy”, “Disconnected”, etc.), and should be able to easily visualize those contacts that are available to interact with (this applies to social networks that include a communication system between users). In addition, the social network should allow the user to choose who can interact with him/her (and who does not).
Social network feature	(1) Customization, (2) Connectivity, and (3) Interaction
Examples	Figure 12 shows an example of heuristic compliance in Facebook, the users who are online and available to chat (green circular icon), and disconnected users (minutes taken offline).
Benefits	The user will easily and quickly distinguish those users with whom he/she can communicate and interact, as well as decide how he/she wants to be perceived by his/her contacts. This will allow the user to feel free to use the social network, based on their wishes and needs.
Problems	This heuristic should not be confused with SNUXH8 (“Flexibility and customization”). SNUXH11 refers to how other users of the social network perceive the user (their status). SNUXH8 refers to flexibility regarding the interface and to customize the visualization of functionalities and content (not customization of user status).
Checklist	The social network allows the user to decide how he/she want to be perceived by all or some of their contacts. The social network allows changing user availability (status) at any time. The social network allows visualizing the status of the other contacts of the user for their interaction (through representative icons, colors, or text). The social network allows blocking contacts with which the user does not want to interact. The social network allows the user to silence contacts who he/she does not want to read his/her comments or publications.
Usability attribute related	Satisfaction
UX attribute related	Usable, Desirable
Set of heuristics related	Esteves et al., “Perception of Users” and “User’s status” [13]

,

Table 17. SNUXH12: Control the published content.

Id	SNUXH12
Name	Control the Published Content
Priority	(3) Critical
Definition	The social network should control the published content to not affect the sensitivity of users, through filters and regulations. The user should be able to report content published by other users in the network, indicating the reason.
Explanation	The social network should allow the user to enjoy the content without inconvenience or worries. For this reason, the user should be able to mark out content that is not suitable for all public or block the content to not visualize it. The social network should allow reporting content, indicating the reasons why the user believes that the publication should not be shown.
Social network feature	(1) Customization, (2) Content management, and (3) Interaction
Examples	Figure 13 shows an example of heuristic compliance in Twitter. The social network allows reporting content, asking the user the reason why he/she wants to report.
Benefits	Users will avoid seeing content that they do not like or affect their sensitivity. Users will feel comfortable and confident, navigating the social network.
Problems	This heuristic should not be confused with SNUXH8 (“Flexibility and customization”). SNUXH12 refers to controlling the content that the user views (what content to see) and reporting inappropriate or uninteresting content to the user. SNUXH8 refers to flexibility regarding the interface and to customize how the content will be displayed (not what content to see).
Checklist	The social network allows reporting publications indicating the reason. The social network responds in a reasonable time to the reported content or publications. The social network keeps in constant monitoring the content that is published within it. The social network warns the user about content considered offensive or violent before showing it, informing what it means to enter it. The social network allows to censorship, stop following or block the content that the user does not want to see.
Usability attribute related	Satisfaction
UX attribute related	Credible, Findable, Desirable
Set of heuristics related	No sets of existing heuristics related.

,

Table 18. SNUXH13: Privacy control.

Id	SNUXH13
Name	Privacy Control
Priority	(3) Critical
Definition	The social network should allow the user to have control over the information that he/she wants to share and who can access it.
Explanation	The social network should allow the user to restrict access to their information (personal data, publications, photos, comments, etc.), giving them the possibility to decide who can view their information. The social network should predefine minimum privacy for new users and allow changing the privacy settings at any time.
Social network feature	(1) Customization, (2) Security, and (3) Interaction
Examples	Figure 14 shows an example of heuristic compliance in Facebook, how to control who can see a publication shared by the user (“public”, “friends”, “friends except…”, “specific friends”, and “only me”).
Benefits	The user will feel safe using the social network, knowing that he/she can control who visualizes his/her information and who can access their personal content.
Problems	The evaluator may confuse this heuristic with SNUXH14 (“Security and recovery of user account”); however, privacy is not the same as security. SNUXH13 refers to the authorized use and access to the user’s personal information; whereas, SNUXH14 refers to protecting the user’s personal information, maintaining the confidentiality, integrity, and availability of their data.
Checklist	The social network allows selecting the audience that can view the user profile and its content. The social network has a reasonable privacy configuration of information for new users. The social network maintains minimum privacy between users who interact with each other through instant messaging, ensuring that their information will not be disclosed on the network. The social network allows users to choose who can (or cannot) access to their profile and its content.
Usability attribute related	Satisfaction
UX attribute related	Credible, Desirable
Set of heuristics related	Esteves et al., “Privacy and exhibition Control” [13] Yeratziotis et al., “Privacy” and “Confidentiality” [14]

and

Table 19. SNUXH14: Security and recovery of user account.

Id	SNUXH14
Name	Security and Recovery of User Account
Priority	(3) Critical
Definition	The social network must include security measures, account recovery account protection, and personal data of the user.
Explanation	The social network must include security options that assure the user a total protection of their personal information. In addition, the social network must notify the user about situations that could affect the security of its content and personal information and provide mechanisms to recover the account.
Social network feature	(1) Security, and (2) Customization
Examples	Figure 15 shows an example of heuristic compliance in Facebook. The social network displays where the user is logged in, different methods to login, and options such as: “get alerts about unrecognized logins”, “use two-factor authentication”, and “choose friends to contact if you get locked out”.
Benefits	The user will feel safe to include their personal information in the social network and to share contents. The above generates loyal customers with the brand or company.
Problems	The evaluator may confuse this heuristic with SNUXH13 (“Privacy control”). It is important that the evaluator is clear about what each heuristic evaluates: SNUXH13 refers to controlling who can see the user’s personal information (who is authorized to see it); whereas, SNUXH14 refers to protecting information (storing the user’s data securely, that the information shown by the social network is reliable and that it is available when the user requires it).
Checklist	The social network has at least one mechanism for account recovery (e.g., by email or text message to smartphone). The social network includes a section to review the user logon history on different devices or platforms. The social network allows modifying the user’s password. The social network suggests security measures for the password of the account associated with the user, as a minimum number of characters. The social network includes an option that allows the user to receive a notification when a new login is made from his account on other platforms.
Usability attribute related	Effectiveness, Efficiency
UX attribute related	Credible
Set of heuristics related	Yeratziotis et al., “Visibility” and “Revocability” [14]

): id; priority (how important the heuristic is: useful, important or critical); name; definition; explanation; social network feature (feature(s) evaluated with the heuristic); examples (examples with images of compliance with the heuristic); benefits (expected usability/UX benefits when the heuristic is satisfied); problems (anticipated problems of confusing heuristics and misunderstanding); checklist; usability attribute related (attribute evaluated with the heuristic); UX attribute related (attribute evaluated with the heuristic); and set(s) of existing heuristics related (set on which the heuristic is based, if exist, and the heuristic that evaluates to some degree a certain general feature or a specific feature of social networks).

7. Conclusions

Social networks are used for different purposes, but users always expect the interaction to be natural, pleasant, and intuitive, without the need for instructions or explanations, and above all, without problems. Evaluating the UX is essential to detect which elements hinder the interaction or bother the user, to then correct those problems and improve the user experience. Since social networks are applications with specific features, it is necessary to use a specific instrument. Even though there are sets of heuristics to evaluate usability in social networks [13,14,15], those sets evaluate only some attributes of the UX (such as usability).

We proposed a set of 14 heuristics to evaluate the UX for social networks: SNUXH. The heuristics were developed using a standard template and are oriented to evaluate both UX aspects (useful, usable, desirable, findable, and credible) and usability attributes (effectiveness, efficiency, and satisfaction). Evaluators decide what elements of the heuristic’s specification use in a heuristic evaluation, depending on their knowledge, experience, time, and needs. We recommend using at least the id, name, description, and examples in the evaluation process, for obtaining better results.

SNUXH were developed following a formal process using the methodology proposed by Quiñones et al. [6,7], and with a greater level of detail compared to other existing sets [13,14,15]. Applying the methodology allowed us to obtain a set of well-defined heuristics. We experimentally validated SNUXH in four iterations, through expert judgment, heuristic evaluation, and co-discovery tests.

Based on the results obtained in the validations performed, we conclude that SNUXH are effective, since they obtained better results than the control heuristics (Nielsen’s heuristics [10]) in the heuristic evaluation carried out in the first iteration, for all criteria evaluated [6,7]. In addition, in the second iteration, the experts perceived SNUXH as useful and easy to use. The feedback received by the experts in the third iteration allowed us to refine and improve SNUXH.

Since usability heuristics are principles for interaction design [10,32], we believe that heuristics can be used as a guide for social network design. To create social networks, SNUXH can be used as a support to avoid designing interfaces that generate usability and/or user experience problems. On the other hand, to improve the design of existing social networks, SNUXH can be used to detect usability/UX potential problems and then correct these problems, and thus, improve the user experience.

The current work has some limitations. In the first iteration, we validated SNUXH through heuristic evaluations using a case study the social network Facebook. Even though we obtained useful information to refine and improve the heuristics; it is necessary to evaluate the effectiveness of heuristics evaluating other social networks, such as Twitter, Instagram, among others. In the third iteration, we validated SNUXH through a user test (a Co-discovery test). In this test, participated users of 21–26 years old. This age range represents a type of user profile (young people), so it is necessary to work with users of other ages to determine what other problems they have used social networks.

In future work, we plan to validate and refine SNUXH by evaluating other social networks (in addition to Facebook), and selecting case studies for both desktop and mobile.