Next Article in Journal
Artificial Intelligence on Food Vulnerability: Future Implications within a Framework of Opportunities and Challenges
Previous Article in Journal
Comparative Analysis of Stakeholder Integration in Education Policy Making: Case Studies of Singapore and Finland
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

‘I Just Don’t Trust Them’: Reasons for Distrust and Non-Disclosure in Demographic Questionnaires for Individuals in STEM

1
Learning Engineering Institute, Arizona State University, Tempe, AZ 85287, USA
2
Human Systems Engineering, Arizona State University, Mesa, AZ 85212, USA
*
Authors to whom correspondence should be addressed.
Societies 2024, 14(7), 105; https://doi.org/10.3390/soc14070105
Submission received: 31 May 2024 / Revised: 27 June 2024 / Accepted: 27 June 2024 / Published: 29 June 2024

Abstract

:
Demographic data pertain to people’s identities and behaviors. Analyses of demographic data are used to describe patterns and predict behaviors, to inform interface design, and even institutional decision-making processes. Demographic data thus need to be complete and correct to ensure they can be analyzed in ways that reflect reality. This study consists of interviews with 40 people in STEM and addresses how causes of relational (dis)trust in demographic data collection contribute to pervasive problems of missing and incorrect responses and disobliging responses (e.g., non-disclosure, false responses, attrition, and hesitancy to use services). The findings then guide a preliminary set of recommendations for cultivating trustworthiness based on recent developments in trust theory and designing for responsive and trustworthy systems. Specifically, we explore how demographic questionnaire design (e.g., item construction and instructions) can communicate necessary reassurances and transparency for users. The ongoing research provides interview-based recommendations for improving the quality and completeness of demographic data collection. This research adds to other recommendations on improving demographic questionnaires.

1. Introduction

The collection of demographic data (i.e., self-reported identity categories such as age, gender, race, veteran status, disability status, and more) is standard practice for numerous services, institutions, and companies [1,2,3,4]. Daily examples of demographic identity data collection can include registering for online social media accounts, accessing product reviews or orders, completing intake forms at medical offices, or responding to market research surveys. Respondents to demographic questionnaires sometimes omit responses or do not respond truthfully, leading to missing or incorrect values [5]. Trust, or lack thereof can be a cause for non-disclosure [6]. Datasets with missing or incorrect demographic information can be problematic as such data are frequently used to generate statistics about people (e.g., [7]); characterize populations and research samples; develop algorithms, automation and machine learning functions (e.g., [8]); and guide legal and policy decisions [3,4,9]. Demographic data are also critical for social decision-making processes in education [10], government, and law enforcement [11].
Importantly, many institutions that gather demographic data (e.g., universities, governments, and employers) must contend with at least two demands that may occasionally conflict. First, these services often need demographic data to understand clients and customers. For example, such data are often essential for businesses to profile consumers, demarcate different markets, and then provide tailored or personalized advertising. Second, these services must also convey credibility and legitimacy that encourage people to engage and respond. The extent to which people perceive institutions to be trustworthy influences whether they engage with those systems, and whether engagements are positive or negative [12]. Within this context, the ways in which organizations solicit and manage demographic data can have substantial consequences on trust.
One example occurs when people develop suspicion or distrust towards institutions that request demographic data and/or how their data will be used [13,14]. Such attitudes might stem from people’s prior beliefs about the institutions but may also emerge from problems within the questionnaires. Trust in the data request may be undermined by missing identity descriptors, incorrect labels, personal questions that seem irrelevant, and other factors that hinder people’s ability to accurately express their identity. Moreover, disclosure of demographic variables in certain organizations (e.g., sexual orientation, disability, and race) could lead to negative experiences of discrimination or bias [15,16,17]. Alternatively, members of hegemonic groups may suspect that minoritized people will receive priority due to institutional hiring or reputational goals.
Despite their utility or necessity, problematic demographic data collection methods can also exacerbate and mask social inequities leading to distrust. For example, large data sets may exclude one or more populations (e.g., excluding non-binary people from gender categories and excluding North African or Middle Eastern identities from race categories). Missing populations cannot then be accurately or validly analyzed (e.g., their behaviors, beliefs, preferences, or other trends) to draw trustworthy conclusions. Another challenge is that many identity category labels within questionnaires are coarse and do not encompass appropriate variance [18,19]. For instance, people who select “White” as their race category may hail from distinct backgrounds (e.g., a 10th-generation American of Irish heritage and a recent Moroccan immigrant). Unfortunately, institutions also commonly generalize based on flawed group aggregation. As a result, descriptions of group behavior may not address intracategorical variance [20] or apply to all persons who share a label. Respondents cannot trust that they are fairly represented.
Researchers have proposed ways to improve granularity and ability to express one’s identity in demographic questionnaires, focusing on variables like gender [21], sexual orientation [22], and disability [23]. Some studies focused on a more holistic approach to improving demographic questionnaire as a whole [19,24]. This work had outlined specific ways in which labels can be added to demographic questions, and sensitivity can be employed in asking about variables that may be disclosed to a lesser extent (e.g., sexual orientation, disability). However, the proposed solutions are still unable to capture intersectionality and have not achieved a consensus status among researchers or demographic questionnaire designers and data collectors.
Crucially, negative experiences, attitudes, and distrust with a demographic questionnaire may lead people to misrepresent or avoid disclosing their personal information [25]. Individuals’ misunderstanding or mistrust of demographic identity collection and usage undermines perceived credibility and legitimacy, and these attitudes lead people to quit, opt out, or intentionally provide inaccurate data (e.g., backgrounds and preferences). For instance, a person who identifies as “non-binary” might reasonably distrust institutions whose demographic questionnaires offer only “male” and “female” as response options—these respondents may then skip questions or provide inaccurate information. Such actions contribute to incorrect datasets and missing values, which drive faulty user profiles that are non-representative of authentic populations.
Notably, despite valid reasons for distrust and non-disclosure, many people do often share demographic information about themselves (e.g., in online systems and surveys) despite knowing that their information might be compromised. This contradictory behavior is referred to as the privacy paradox [26,27,28,29,30]. Specifically, people may choose to share personal information with an organization because they believe that the benefits of using a service or product outweigh the risks of disclosure. Similarly, they may believe that because their information is already compromised and sharing it again will not make a big difference. In other cases, people may be unaware of how their data will be used or may feel like they do not have a choice (e.g., when applying for a job or a degree). Finally, some individuals may respond partially—they may only respond to a limited number of items or skip questions perceived as optional or threatening.

1.1. Demographic Identity Data Collection in STEM

Within the broader discourse on demographic identity data, the collection of such data in science, technology, engineering, and mathematics (STEM) organizations is a meaningful area of focus. Ample studies and reports have documented deep disparities in STEM participation—there have been well-documented “gaps” in college enrollment, degree attainment, job hiring, and career promotion in STEM organizations related to race, ethnicity, gender, sexual orientation, and historically underserved populations [30,31,32,33]. Likewise, researchers have carefully investigated how lived experiences of STEM students and professionals are impacted (e.g., bias and discrimination) by demographic factors [34,35]. For example, research reports a gender gap still existing in math-intensive STEM fields [31]. Another example [34] reveals that engineering students and professionals with disabilities report being excluded and less respected than their colleagues. Accurate demographic identity data collection has been an important component of such scholarship—disparities and experiences cannot be revealed or reported if the data are not even collected.
Relatedly, another thread of work has sought to investigate and support the formation of STEM identities. In brief, STEM identity refers to one’s sense of self as a participant in STEM institutions and activities (e.g., scientist, engineer, or mathematician) along with the ways that identity intersects with other aspects of one’s life (e.g., hobbies, family, and goals) [36,37,38,39]. Research has shown that STEM identities are constructed through interactive and relational ways [36,37] and include one’s perception regarding not only their STEM knowledge and skills, but also one’s work ethic, relationships within one’s field, and the potential effects of one’s work on the world. The construction of STEM identities is thus influenced by disciplinary and societal hierarchies and perceptions related with demographic variables like race [38] and gender [40].
Finally, efforts to broaden participation in STEM—to increase fair representation in STEM institutions that mirrors broader populations [41,42,43,44,45]—have also relied upon meaningful demographic identity data collection. Efforts to broaden participation often focus on the inclusion of populations who have been historically excluded, based on gender (e.g., [32]), race [46,47], and ability (e.g., [35]). Interventions include introducing STEM faculty to culturally responsive teaching, which had resulted in faculty motivation to foster a more equitable learning environment [45]. Other efforts are generated by STEM faculty who make conscious efforts to increase participation through pedagogy [44], and through larger-scale frameworks like programs and institutions that have the goal of including members of marginalized group and incorporating them in both research and non-academic jobs in STEM fields and have shown promising results for the students and faculty involved [42,43].
Despite many enlightening findings and subsequent successful interventions, the collection and use of demographic data in STEM organizations must nonetheless contend with the same trust and response issues noted previously. Trust in organizations (and demographic measures) is undermined when respondents lack the ability to identify themselves accurately, appropriately, or fairly, or when they possess meaningful reasons to question the motives of the organization(s) requesting the data (e.g., [15,17]). Such threats might emerge from missing identity or incorrect labels, misleading aggregate analyses and conclusions, or perceived dangers of disclosure (e.g., being fired).

1.2. Relational Trust: Situation, Semiotics, Interaction Sequence, and Strategy

Demographic identity data are necessary and useful, yet the collection of such data is hindered by distrust stemming from fears and perceived threats about institutional data collection and (mis)use. Distrust contributes to incomplete or inaccurate information, which in turn directly undermines data quality. Improving the trustworthiness of demographic identity data collection is thus a worthwhile endeavor for businesses, developers, researchers, educators, and any organization that relies on such data. As a specific case of interest, improved data collection should facilitate the study and remediation of inequities in STEM organizations—revealing demographic disparities and opportunities, understanding participation in STEM, and broadening participation in those institutions.
To explore issues of trust toward the collection of demographic identity data and the requesting institutions, we consider a framework of relational trust [6,48]. In brief, building and sustaining any trusting relationship (e.g., with an organization) is a process that requires responsivity while considering different stakeholders’ goals. Such responsivity is established and maintained through attention to characteristics of the decision situation, semiotics, interaction sequences, and strategies (i.e., 4Ss) [6]. This framework provides a lens for analyzing perceptions of trust and how components of demographic questionnaire design may influence such perceptions.
Responsivity refers to the ability of any system (e.g., a healthcare institution collecting patient information) to adjust and adapt to altered conditions while continuing to function correctly and smoothly. Importantly, responsivity is not a purely objective feature of a system or relationship. Instead, participants’ goals or perceptions determine what counts as “adaptive” or “correct”, and thus what is considered “trustworthy”. Different people may possess overlapping or distinct parameters for determining trustworthiness that depend on their previous experiences, self-placement within a system (e.g., sense of belonging [49] or sense of self-efficacy [50,51]), and expectations. Nonetheless, we can examine responsivity in several ways.
One relevant consideration is the context in which trust manifests—the situations in which people are making decisions in or about the system. Understanding of the decision situation can help to determine whether and to what extent trust is an important variable, and any perceived tradeoffs among stakeholders. In the case of demographic data, stakeholders include both the respondents and the organizations collecting the data. The situation encompasses both the immediate activity of completing the questionnaire as well as the environment or goals for responding (e.g., job interview, account creation, research study, or government census).
Another consideration is the semiotics (i.e., signals, signs, and symbols) embedded in the system [52,53], and how they are interpreted in ways that impact trust. For example, in some cases, certain symbols (e.g., official logos, certifications, and other credentials) might be perceived as indicators of trustworthiness [54]. If an institution has built a good community relationship and reputation (i.e., a “trustworthy brand”), then their visible logo might imply that their data requests are similarly trustworthy. In other cases, website errors (e.g., slow loading, missing pages, or typos) might be interpreted as signs of negligence that undermine trust. Thus, semiotics can be both intentional and unintentional ways that trustworthiness is signaled. In collecting demographic data, semiotics might entail not only questionnaire design (e.g., logos and layout) but also content—the types of questions asked, their wording, and allowable responses.
Interaction sequence refers to how trusting relationships evolve across multiple interactions and situations. Attitudes are rarely formed in a single instance; instead, trust is built (or lost) through a series of interactions wherein people interact with systems and experience responsive or nonresponsive outcomes. In demographic data collection, a typical interaction sequence might encompass (1) the inciting purpose (e.g., job interview), (2) the request to complete a questionnaire (e.g., online resume), (3) completing the questionnaire itself, and then (4) any follow-up based on those data (e.g., job offers). Each step in this sequence contributes to the decision situation and comprises numerous semiotics that may impact trust (e.g., reputation of the employer or interviewers’ body language).
Finally, strategy is operationalized as how institutions and people intentionally navigate decision situations. Regarding demographic questionnaires, organizations might intentionally author questionnaire items according to inclusive policies (e.g., person-first language) or feedback from users (e.g., certain questions reported as “confusing”). Such changes might be intended to “cultivate trust” by signaling that the organization is compassionate and wants people to have a pleasant experience. Another design strategy may be to create dynamic questionnaires that provide respondents with nested options for nuanced reporting. For instance, if individuals choose “Black” as their race, additional options may then appear to further specify national origin or other heritage (e.g., African American or Afro-Cubano). Likewise, many questionnaires include “Prefer not to Say” options that allow for responding with nondisclosure, and “Prefer to Describe Myself” options that enable free response text boxes. Such designs influence interactions sequences by asking respondents to respond to only relevant questions and giving them more control over how and when they respond.
In sum, relational trust offers a comprehensive lens for understanding numerous aspects of trust toward demographic data collection and requesting institutions. Understanding respondents’ beliefs or fears about the purposes of data collection (i.e., the decision situation) and their responses to questionnaire items (i.e., interaction sequences) may enable designers to craft more trustworthy measures that will, in turn, elicit more complete and valid data.

1.3. The Current Study and Research Questions

The purpose of the current study is to further understand reasons for trust, distrust, disclosure, and non-disclosure in response to demographic data collection. To do so, we analyze qualitative interview data on diverse individuals’ perceptions of data collection and usage, which are analyzed via a relational trust theoretical lens. This work occurs within a STEM context that reflects the authors’ interests in STEM identities, disparities, and participation. However, we anticipate that findings might offer insight beyond the boundaries of STEM. This work is guided by two overarching questions (RQs) elaborated below.
We first (RQ1) explore: what considerations contribute to trust and distrust toward organizations (e.g., university researchers or employers) who are collecting the demographic data? Do people trust the institutions collecting the data and/or how the data will be used? In this context, what factors might contribute to trust and distrust toward the demographic identity data questionnaires?
Our second question (RQ2) considers responding behaviors. We ask: what trust-related factors influence STEM respondents’ response strategies, such as choosing “Other,” choosing “Prefer Not to Say,” or intentionally responding incorrectly to demographic questionnaire items? Given such behaviors and concerns, we also explore demographic questionnaire design changes that may increase trustworthiness and thus valid responding.

2. Materials and Methods

The current study employed a qualitative, semi-structured interview methodology to elicit diverse STEM individuals trust and attitudes toward demographic data collection, potential data use, and the institutions that solicit such data. The focus on STEM participants and disciplines reflected the authors’ interests in STEM identities, disparities, and participation. The benefits and limitations of this design are discussed in Section 4.5. Semi-structured interviews were conducted virtually (via Zoom) as part of a larger project on improving demographic data collection practices to better represent identity. The interviews ranged from 30 to 75 min in duration.

2.1. Participants

Adult (age 18+) participants with STEM backgrounds (e.g., via education or work history) were recruited via word of mouth and emails to listservs of specialized STEM departments and interest groups. Recruitment messaging broadly explained that the purpose of the study was to explore identity and demographic questionnaire responses. Study participants were compensated via a $10 USD Amazon gift card. A total of 39 participants were recruited and interviewed.
Participants were invited to describe their age, gender identity, race, and ethnicity (see Table 1) in their own words. If participants were unsure of the meaning of a demographic term or wanted clarification, working definitions were provided. These definitions were very intentionally worded to avoid academic or technical jargon. For instance, a clarification of gender might state that “gender has been defined as how you experience and define your gender, based on how much you align (or don’t align) with what you understand the options for gender to be”. Similarly, a definition of ethnicity stated “ethnicity has been defined as a grouping of people who identify with each other on the basis of shared attributes distinguishing them from other groups. Those attributes can include common sets of traditions, ancestry, language, history, culture, nation, religion, or social treatment within their residing area”. Disability was described as “having physical or mental characteristics that substantially limit one or more major life activities”.
As summarized in Table 1, it is worth noting that open-ended definitions of ethnicity allowed participants describe themselves in fairly nuanced ways such as “Japanese”, “Chinese”, “South Indian”, or “2nd gen. immigrant from Hong-Kong”, instead of simply “Asian”. Other participants provided descriptors that do not align with typical questionnaire ethnicity categories (e.g., “A poor person”, “Utah Mormon”, and “Army Brat”).

2.2. Probing Trust and Responding Patterns

The complete interview comprised a number of questions about perceived identity and salience of numerous demographic variables. For the current research questions, we selected and analyzed a subset of questions that were specifically related to trust in demographic data collection, institutions that collect such data, and how data might be used. The first selected question asked: Do you trust those who ask the questions (universities, employers) to use that information in a way that benefits you or at least doesn’t discriminate against you? Responses to this question revealed participants’ feelings of (mis)trust related to organizations that collect demographic data, which could stem from decision situations, semiotics, interaction sequences, and strategies (e.g., the nature of the questions and subsequent uses of the data). The second targeted question asked: Are there questions you choose not to answer if you can; why? Answers to this question revealed participants’ responding behaviors, such as their decisions to ignore or skip questions.
Procedurally, brief or vague responses were further probed with follow up questions to clarify statements and ensure that participants’ intentions were fully communicated. This process also helped to mitigate bias and over-interpretation by the researchers. These clarifying questions first revoiced participants’ responses and then invited correction and elaboration. For brief responses, prompts such as “could you say more about that?” were also often effective for eliciting more detail.

2.3. Analysis

Participants’ responses related to trust and disclosure behaviors were transcribed and thematically analyzed [55,56,57]. Transcriptions were automatically generated by the virtual meeting software (Zoom, v. 5.13.4), and transcripts were reviewed and corrected by both the interviewer and a graduate research assistant. After this editing process, the transcripts were shared with the interviewees. The interviewees could then implement further edits, including redactions of responses they no longer wished to share. Thus, the final, analyzed transcripts confidently reflected the participants’ own words and intentions. In addition to the transcripts, interviewers recorded detailed notes immediately following each interview. These notes facilitated interpretation of responses.
Inductive thematic analyses were guided by the 4S relational trust framework [6], including considerations of decision situations, semiotics, interaction sequences, and strategies related to the collection of demographic data and the requesting institutions. The relational trust framework informed interpretations of participants’ trust-related attitudes, while allowing for the emergence of insights about improving responsivity and supporting trust. Inspired by thematic analysis methodologies [55,56,57] individual or small sets of interviews were analyzed in a continuous fashion as interviews were completed. This process informed the identification of emerging patterns, which then enabled focused probing of these trends in subsequent interviews (and follow-up communications with participants) with saturation. Upon completion of data collection, the complete set of interviews were reviewed again to refine and triangulate observed patterns.

3. Results

Inductive qualitative coding, informed by the 4S relational trust framework, revealed 12 themes associated with trust and distrust in organizations related to demographic data collection. Interviewees broadly discussed topics spanning (a) how demographic data should be used, (b) trust and distrust in institutions’ integrity and ability to safeguard personal data, (c) fear of identity-based marginalization or identity misrepresentation, and (d) the well-intentioned and appropriate use of demographic data to work towards inclusion and diversity. Specific themes are summarized in Table 2 and aligned to the 4S concepts. Importantly, the 4Ss are interconnected and overlapping. Trust is formed over a series of interactions within situations that are influenced by semiotics and strategies. Thus, while attitudes expressed by participants related to more than one component, they are reported under a single component for convenience and clarity.
The 12 reported themes pertain to seven over-arching topics: (1) experiences of bias, which includes themes of bias from other institutions (T1), and bias from the questionnaire (T2); (2) positive trust stances (T3); (3) not understanding how the queried information is relevant (T4); (4) identity misrepresentation in questionnaire, which includes themes of missing relevant labels (T5) and labels too coarse (T6); (5) not trusting institutional integrity and ability to overcome hegemony, which includes the themes: believing data collection is meant for virtue signaling (T7), experiencing bias related to demographic information (T8), and concerns of data security (T9), and lastly; (6) decision not to disclose, which includes the themes response option (T10), having ‘prefer not to say’ or ‘other’ options (T11), and fear of disclosure leading to bias (T12). The following section reviews relevant excerpts to illustrate these themes. The analysis will consider the themes through Chiou and Lee’s [6] relational trust framework as a way to consider demographic questionnaire design and response experience. These perspectives provide an applied basis to consider implementations to questionnaire design that address respondent concerns and data collector goals.
Importantly, although each theme is presented within the scope of one of the 4Ss, the themes are related with different elements of the demographic questionnaire system. For example, T7 is related with the belief that inclusive data collection is motivated by virtue signaling, rather than affecting positive change and is discussed relative to ‘sequence’. The theme of perceived virtue signaling is also directly related to ‘semiotics’, the signs and signals provided by the questionnaire.

3.1. Three Trust Themes Related to Situation

Responding to demographic questionnaires is embedded within multiple situations, including the purpose of the data collection (e.g., for hiring or research) and even the modality (e.g., web form or paper). However, the most salient situation was responding to the questionnaire itself. Respondents noted that when they are asked to disclose their personal information, it creates a state of vulnerability regardless of their levels of trust or distrust. Respondents raised questions about how institutions might use the data inappropriately in hiring, practices, and culture, along with the impact on the trajectories of workers, users, or students within the institution.
In this context, two themes emerged for reasons to distrust organizations and demographic data collection (Theme 1: Unskilled Implementation and Theme 2: Lack of Follow-Through), and one theme emerged for positive trust (Theme 3: Good Faith). First, one reason for distrust captured beliefs that that institutions have good intentions but are unskilled at achieving those goals (Theme 1). Although organizations may seek to collect meaningful demographic data, their methods or measures are flawed. Flaws such as reductionist categories undermine trust in the organization and data collection. For example, Participant 13 (P13, italics added for emphasis) stated:
P13:
I don’t think that, generally speaking, that these bodies actually include people of these, you know, quote unquote “categories’ […] in the process of just deciding or like of defining what […] fair treatment even means for them, and defining what it means to be a part of that category. I think oftentimes it feels like you know, you get reduced to a label, especially in any point kind of quantitative analysis of this kind of data. And I think it just it completely misses the actual experience. It’s like these categories […] gesture at like broader trends and experience, but they themselves like do not contain, you know, like you can’t say that these things happen because of you know, like my queerness, or something, right? It’s like these things happen because of some aspect of it, and like we need to be like better about delving into what these things are. Otherwise, you just have a story that is very, very manipulable.
A second theme captured beliefs that institutions only request demographic identity data to appear as if they care about or intend to improve “diversity, equity, and inclusion” (DEI) metrics. However, the organization never does anything beyond asking; the request is distrusted because it is performative with no follow-through (Theme 2). In the following excerpt, P17 expressed this sentiment:
I:
so why do you think those questionnaires are used at all? If you think the information is not used, where do you think people have the -
P17:
like I said. I think it’s I think it’s just to make people feel better about themselves. I feel like the most that I’ve ever like really seen out of those types of questionnaires is maybe an increased diversity in advertising for positions or jobs or colleges and then like historically lacking demographic area regions so like if they’re lacking on minorities, they’ll try and increase advertising to minority stronghold centers across the US. But that […] that’s largely it. I feel like they don’t necessarily use it to increase admissions or decrease admissions or increase access into programs. They just do it to feel like they’re doing good, and that’s it.
Themes 1 and 2 were not mutually exclusive. Participants noted that organizations could be well-intentioned yet still require significant pressure to actually use the collected data in positive ways to achieve change. P27 expressed:
P27:
they could be well intentioned, but it may just end up on a desk and go nowhere, and I’ve heard this from individuals directly I’ve seen it myself. Where the system is, it protects […] itself. And when you start talking about things that are a little too far beyond the norm, you will get pushed back because I think at the root of it is that a lot of these maybe well, meaning initiatives have to deal with funding sources and being able to continue to survive off of their constituents, and […] if you say stuff that aggravates their base you’re not going to be well received. So that means that I don’t trust these questionnaires, because there’s limits to what they can do, even if they wanted to.
Importantly, distinctions were sometimes made based on the type of organization or individuals conducting the data collection, such as university researchers, companies, employers, or other institutions. For example, P8 expressed trusting the interviewer in this study (i.e., our intentions and data usage) but not others:
P8:
Definitely depends on the place. if we’re talking universities or employers specifically, absolutely not. Like I’ve been around the university enough to know that they care because they are required to care. That it’s about the culture of like audit compliance more than it is any actual interest […] I’m in a department where, like the old chair of the department, like bragged […] to new students about how like I’d won this amazing fellowship right. And like that I was trans, and I was doing all this cool work, and like was using it to advertise like how cool and […] welcoming the department is. Meanwhile, like he gave me zero fucking help with that application. When I sent it to him, he actually said, ‘Okay, when It’s rejected, you should submit it to the NSF’. […] it’s like they’re willing to claim credit for successes. But I don’t think they give a shit about like the work.
In contrast to concerns about institutional capacity and motives, it is worth noting that a small number of interviewees expressed positive trust toward demographic questionnaires and those who collect the data (Theme 3: positive trust stance). For example, P16 expressed trust that their data will not be used in biased ways against them, despite knowing that other people may experience bias:
P16:
So, I trust them not to use it to discriminate against me.
I:
Okay, how about against others?
P16:
Mostly trust.
I:
Okay, can you say more about that cause that seems like a slightly different answer.
P16:
I just, I think most of the people that are collecting that kind of information have good intentions and are trying to use it to see where they’re at, and if there are ways they can try to get a better mix of people together
P26 expressed a similar attitude that institutions have positive intentions and obligations that they act upon:
P26:
Yeah, in general, I don’t worry that I’m being discriminated against for my characteristics. And I think, in general, that those questionnaires exist because they’re trying to push back on, or the government at least somewhat, is trying to push back on 400 or 2000. Whatever years of culture, of people liking people who look like them. So, yeah, I’ve never felt negative from that
Several participants described having distinct trust stances towards for-profit companies and industry employers compared to universities and researchers. For example, P38 described trusting most organizations who collect demographic information, but specifically not trusting for-profit companies:
P38:
Most of the time, and I say that because my agency [their workplace] is one that uses that data. So I know in general how it’s supposed to be used. Yes.
I:
So you said mostly so alright. What other instances when you don’t? And what’s causes that distrust? Is it the agency, like [the entity] who’s asking, or something about the questionnaire?
P38:
Yes, yeah. It depends on who’s asking?
I:
Okay. So who do you trust less?
P38:
Generally it will be companies that are trying to get me to buy something
A similar preference of trusting and sharing information with researchers was expressed by P39, who particularly valued clear regulations and accountability:
P39:
Depends on the context, but in general no[…] I’m […] in the midst of applying for jobs at other universities and that particular part of interacting with a big bureaucracy has a lot more controls around it, because there’s HR laws […] and they get a lot of trouble for discriminating in that setting. But in many less structured settings, I think, for sure I don’t trust the organizations that I give my demographic details to maybe I’m just a cynical person, […] I think in many less structured settings that is less sort of regulatory oversight then [I’m] less inclined to divulge information about myself.
In sum, within the context of demographic data collection situations (e.g., institutional purposes), trusting attitudes emerged from skepticism about the ability and/or intent to follow through on the stated goals of demographic data collection. These attitudes imply a decision structure in which disclosure of information is perceived as an action that may not lead to any direct or indirect benefits for the discloser. However, skepticism about organizations was not universal; participants noted the good intentions of institutions regarding demographic data and its collection.

3.2. Three Trust Themes Related to Semiotics

Signs and signals related to trust can emerge from a variety of sources, such as the nature and reputation of the requesting institution, and interpersonal interactions with organization representatives. In our interviews, participants focused on several signs and signals instantiated through questionnaire design, contents, and phrasing (Theme 4: Perceived Irrelevance, Theme 5: Missing Labels, and Theme 6: Coarse Labels).
Several participants expressed a strong preference for questionnaires to address only “relevant” variables (Theme 5: Missing Labels). For example, in hiring situations, participants stated that they wanted themselves and others to be judged only on variables that directly relate to their education, skills, and ability to perform at a certain position. P18 (italics added for emphasis) expressed this preference while questioning the motives for hiring:
P18:
the goal of a company should not be what do our employees look like. you know, the goal of a university or a business should be to succeed. And that should be based on the skill of your employees. The effectiveness of your employees, and I also think that choosing certain people based on that ethnicity really takes out your own. It takes out human free will out of the decision making process altogether, because your pool is only available only open to like who’s naturally making the choices to apply to your organization in the first place […]. If you say we must have this number of this of you know black women or Asian homosexual men, or you know we can only have 10 Caucasians in our entire system. You know, but that limits like you’re only dependent, you know. You’re dependent on who’s applying to your job or who’s applying to that university, and if the kind of people you think should be on your staff are not naturally applying to be on your staff you can’t fill that position and no fault […] my kids’ elementary school has 2 male teachers out of like 60 teachers. Now is that because there bad at hiring men? Or is that because predominantly, women apply to be elementary school teachers, you know it doesn’t mean that they’re sexist.
Other participants discussed the importance of “relevant” questionnaire items based on the belief that collecting demographic data is not the right way to achieve diversity:
P5:
[…] i’ve been applying for jobs. I get really annoyed when I get to that section. The whole ‘do you have a disability?’. ‘have you served in that in the army?’ There’s 3 questions that they ask at the end of something, and I always wonder like if we’re creating equitable hiring processes. Are you hiring like for an engineering job are you hiring me based on the interviews and my technical skills which you’ll assess, or what I really a lot of times I don’t understand what role those like last 3 questions play. Am I at a disadvantage for not, you know, answering quote unquote “correctly” to […] those questions?
Several participants expressed desire for more detailed signals about how data would be used. In the excerpt below, P12 questioned why a stranger would need to know that someone is Jewish, resulting in decisions to skip such questions:
P12:
like it’s none of their business cause It’s not something I open […] with […]. Like Jewish is kind of like a personal thing to me, […] I’m not gonna walk up to a stranger, and be like ‘hey, I’m a Jew’, so like when it’s on a questionnaire, and I’m like, ‘why are you asking?’ then I, if I can say nothing, I say nothing
P30 similarly wanted to avoid sharing information that seemed irrelevant to the purpose of the data collection, or which seemed irrelevant to participants’ goals in completing the questionnaire:
P30:
Anything where it’s not relevant for them to need to know that information. So if I’m signing up for like a new random account on a website, or maybe a social media, or you know, or just any, I think I’ve seen it even one time when I was signing up to receive like a gift card on a website, and I was like, why, you know what I mean, like this is completely like I mean, I know they want the info, you know, they want the data, but it is just didn’t seem relevant at all and automatically if I don’t feel like it’s relevant. then I don’t trust it.
Two additional themes addressed the availability of appropriate descriptors for self-identification, including missing labels (Theme 5: Missing Labels) or labels that lacked nuance (Theme 6). Several interviewees raised concerns about their identities being misrepresented in standard demographic questionnaires, such as coarse demographic categories that lack intersectionality [58,59,60,61], individuals reduced to labels in quantitative analyses, and ignoring personal experiences. For example, P11 noted the lack of attention to intersectionality and of intracategorical variance:
P11:
because, like it’s just like every yeah, everybody is just so many combinations of things that if you pick one thing out it doesn’t necessarily mean that like there can be patterns between people with like one specific category in common but like picking that one thing out doesn’t necessarily mean that any experience is just because of that one thing. And you also might be missing a lot of yeah.
P27 described how coarse labels prevent them from expressing their identity. When organizations use or adopt coarse labels at face value, without additional explanation, it may signal that the institution cannot be trusted to appreciate individuals or diversity:
P27:
Understanding that I’m categorized in a larger group of you know, if I check off this box of ‘black’, it’s a larger group or African American which can include Africans who come here… So. Hmm. I am, I would say, I guess, protesting the box in in some sense, because it needs to change,
In sum, attitudes related to semiotics highlighted how trust might be undermined by questions that seem irrelevant or seem to neglect appropriate identity labels. These concerns and oversights signal that institutional motives may be insincere or misaligned to diversity goals. Thus, as noted previously, components of the 4S framework are intertwined. Many of the responses quoted in this section corroborate trust issues related to decision situations (e.g., institutional motives or capacity to use data wisely).

3.3. Three Trust Themes Related to Sequence

Diverse sequences of interactions can inform trust-related attitudes, such as the history of interactions preceding the questionnaire, interactions with other similar measures, and histories of personally experiencing biases or threat (e.g., a data breach). Similarly, institutional actions and policies may be informed by prior interactions with data collection and respondents (e.g., prior surveys with poor response rate or response quality). In the current study, participants articulated a variety of themes emerging from their actual and imagined interactions with demographic data measures.
One theme addressed perceptions that activities of collecting or reporting demographic data served only to perform “DEI” without intent to change (Theme 7: Virtue Signaling, see also Theme 2: Bias from questionnaire, and see P17 and P27 for relevant excerpts). More unique to interaction sequences were themes emerging from a history of discrimination (Theme 8: Experienced Bias) and worries about data privacy or security (Theme 9: Data Security).
For example, P2 expressed a fear of discrimination informed by their awareness of both past and current systemic and individual biases (e.g., [3,4,62]), along with their own personal experiences of stigma and bias (Theme 8). Such anxieties could manifest as individuals perceiving themselves as having demographic traits that might be targeted:
P2:
So like, for example, specifically with the disability thing that I said, It seems like actually they might work to your advantage, because you would be considered kind of a minority, like if you said that you have a condition. But I always think that if I say something they might use that against me, or yeah, discriminate against me, or be like this person is not fit for the job, and I think they wouldn’t even have to say why they can like find out. Another reason. So yeah, I think I can say that. … I don’t trust them.
Several people with identities that were perceived as hegemonic or privileged sometimes feared that “DEI efforts” might negatively affect them. That is, such efforts might prioritize members of minoritized groups. P19 felt that they were not selected for a position because members of minoritized race groups were preferred in the job search. The sequence of advertising the job, soliciting applications and demographic data, and then making hiring decisions did not contribute to their trust in the organization:
P19:
I’m, not 100% sure about that. […] I’ve had instances for I do fill out the document. And it’s like I don’t check a box where I am not identified with a specific minority, that they’re trying to hire. Which you know they don’t really say it should count on the application, but in reality, it does sometimes cause when the, you know, for these examples about applying those jobs. it said nowhere on there that we’re looking for somebody from this conference it didn’t say we’re looking to hire somebody from this particular organization. It was simply ‘fill this out, send it in’. So you know, that was a little frustrating it’d be nice if they’re a little more transparent exactly what they’re hiring with I have no problem with them hiring from a Specific Group that’s just you know what they what they do but you know it’s for the most part I don’t think it affects me largely, but I can see how it affects a lot of other people. […] I don’t have any particular issue with trusting them to use it correctly, because they have their own motives for who they’re trying to hire what they’re trying to do.
Additional sequence-related concerns pertained to institutional integrity and data security (Theme 9: Data Security). For example, P23 expressed the fear that personal information may be compromised, which was based on their prior interactions with security breaches:
P23:
[…] I feel fine talking to you right now about this stuff, and I believe you that you’re gonna protect my information and not, you know I’ll be de-identified in things. But […] when they ask you on your like HR portals for your jobs in the hospitals and in the, you know, packets and all these different things, I don’t know truly how they’re using that information. I don’t think they’re publishing it necessarily, you know, but I also worked for a company that got […] like all of our social security numbers and things, and it was a major health system, and it was like 11,000 people, so I guess I also just have mistrust in general, especially because of stuff like that.
In sum, participants’ prior and imagined sequences of interactions with demographic data collection (i.e., before, during, and after) informed their trust attitudes. Such interactions included direct experiences with demographic data along with relevant lived experiences.

3.4. Three Trust Themes Related to Strategy

Strategies refer to deliberate actions to gauge, communicate, establish, or maintain trust. For example, organizations may use press releases to establish their reputation as a trustworthy brand or their commitment to societal causes. Similarly, organizations may design data collection efforts in ways respect diverse identities (e.g., implementing bias-free and person-first language in all documents) and protect demographic data (e.g., implementing encryption to prevent data loss). Interviewees focused on their strategies when responding, such as choosing specific types or answers (e.g., “Prefer Not to Say”) or not responding at all. Participants appreciated “optional” questions that gave them control over how and whether to respond (Theme 10: Optional Responses). More specifically, several participants noted the need for “Prefer Not to Say” or “Other” choices (Theme 11: Response Options). Participants also described how avoiding self-disclosure could be a strategy for self-protection (Theme 12: Fear of Bias).
Participant 7 (P7) described how certain options were preferable to selecting an incorrect descriptor (Theme 11). This except also exemplifies Theme 5 (missing labels):
P7:
Yeah. …i’ll put ‘prefer not to say’ if that’s the option, because those are the only 3 options this man woman prefer not to say and so, like So it’s because the actual answer is just missing from It yeah
Several interviewees (e.g., P2) mentioned not wanting to disclose available labels due to fear of bias (Theme 12: Fear of disclosure leading to bias):
P24:
yeah, when it comes to the disability question, I definitely put ‘prefer not to say’.
Similarly, P36 expressed conflicts towards appreciating having more identity options available (e.g., gender) versus skepticism about date use and sharing. Ultimately, they chose to not disclose, when possible:
P36:
Truly depends on the organization. If it’s a survey to understand, believe it’s experience of people. I often I trust them, if it’s to, deal with my data and my name. I have my doubts. And sometimes I’ve been happy to see non-binary listed, and then said that I did like I wasn’t comfortable, feeling that because it was for work related stuff so. Sometimes. Yeah, I’m not. I feel like people and organizations don’t explain enough what we’ll do with the gender and who is gonna see that information. Yeah. And I say gender because it’s the most often asked and most sensitive for me, and they’re like, Oh, give me your information. It’s confidential on someone. We’re not gonna sell the information. Okay, I’m gonna send the information.

4. Discussion

Demographic identity data are ubiquitously collected for a wide range of purposes. Companies may gather such information to understand consumers or profile markets for advertising, and employers may use these data to characterize their work force or detect gaps to be addressed via hiring. Demographic data are also broadly employed as descriptive variables in research analyses, such as studies to reveal or mitigate discrepancies in STEM participation [31,32,33,34,35]. Importantly, because such data may be deeply personal—with ramifications for experiences of bias or personal safety—there are substantial concerns about trust in demographic identify data collection and data use.
The current study sought to better understand how persons conceptualize reasons to trust or distrust demographic data collection. We specifically considered two overarching questions that asked: what considerations contribute to trust and distrust toward organizations (e.g., university researchers or employers) who are collecting the demographic data? (RQ1) and what trust-related factors influence STEM respondents’ response strategies, such as choosing “Other”, choosing “Prefer Not to Say”, or intentionally responding incorrectly to demographic questionnaire items? (RQ2). To this end, qualitative interviews were conducted and analyzed via the 4S relational trust framework [6]. That framework considers how trust is formed and maintained through responsivity in decision situations, semiotics, interaction sequences, and strategies.
Inductive, thematic qualitative analyses surfaced 12 themes associated with components of relational trust. Many of these themes affirm or elaborate trends identified in the literature. In this discussion, we summarize major findings and contributions. We also begin to consider how these findings might tentatively inform recommendations for better practices.

4.1. Situation

Themes related to situation (Themes 1–3) captured concerns about how and whether institutions could appropriately collect demographic data and their motives for doing so. Participants questioned whether data collection might lead to discrimination toward individuals based on their identities, such as further marginalizing underserved populations via biased hiring or firing practices (e.g., firing a disabled person).
Demographic data collection occurs within institutional contexts that can either build trust via clear and sincere communication of their goals and actions, or undermine trust by creating environments (i.e., situations) of ambiguity use of the data collection, and clear commitments to use the data to benefit respondents and relevant communities. The former recommendation may build trust by allowing respondents to make informed decisions about participation while having faith that the organization will use the data fairly (i.e., Theme 3). The latter recommendation may support trust formation by authentically following through on stated “good intentions”. If the data are supposed inform new policies or contribute to more inclusive cultures, then clear actions should be taken to achieve those goals using the data.
Exemplar design questions regarding decision situation for demographic data might thus include: Who will receive or use the data? What is the purpose of the survey? What are the possible consequences of sharing this information? Who is completing the survey? Can respondents describe themselves the way they want to? Answers to these questions shape trust. For example, organizations that deploy questionnaires to collect and sell personal information might be deemed untrustworthy.

4.2. Semiotics

Themes related to semiotics (Themes 4–6) revealed how questionnaire design and related communications serve as signals that institutions can (or cannot) be trusted with demographic data. One prominent theme emphasized the importance of perceived relevance; participants wondered “why” certain data were requested or how they would be used. This finding corroborated themes related to situations (i.e., institutional motives). Two other themes highlighted the presence or lack of meaningful identity labels. In sum, asking seemingly irrelevant questions or providing poor response options were signs that institutions lacked awareness or expertise in soliciting such data. These actions could also signal that institutions did not care about identity.
Similar to clarifying institutional motives and goals (i.e., situations), one might recommend developing trust semiotically via transparent information and explanations [63] along several dimensions. Such transparent signaling might clarify (a) the goals of demographic data collection, (b) data use (past, present, and future) policies and practices, and (c) the underlying rationale for questions asked and available response options. For example, questionnaire designers might have thoughtful reasons for offering only “Hispanic/Non-Hispanic” options for reporting ethnicity, even though participants may feel such options are too coarse. Communicating such constraints may alleviate distrust; the communication itself is also a signal of trustworthiness.
Additional semiotic recommendations to build trust within questionnaires could be (d) allowing respondents to express answers in their own words (e.g., free response field) or (e) to leave feedback for improving the questionnaire. Both features directly increase questionnaire quality, but also signal to respondents that their authentic ideas and reactions matter (if the feedback is heeded, of course). Similarly, in digital environments, (f) questionnaire structure may dynamically transform based on prior responses. For instance, if respondents choose “Asian” as their race, additional descriptions might appear to elicit nuance tailored to that broader category (e.g., further details regarding nationality, ethnicity, or language). Respondents might also opt out of certain classes of questions (e.g., all questions about disability); these questions might thus never appear instead of forcing respondents to repeatedly select “Prefer Not to Answer” or skip questions.
Example design questions that could arise regarding semiotics might thus include: What are the credentials of the organization requesting these data? Do the layout and wording demonstrate attention to detail? Do the questions use respectful and bias-free language? Do the questions include appropriate and complete ways for people to respond? Answers to these questions may influence trust to the extent that they reveal—explicitly or implicitly—perceived motives or threats from an organization.

4.3. Sequence

Three findings (Themes 7–9) addressed how sequences of interactions before, during, or after completing a demographic questionnaire could contribute to feelings of trust or distrust. For example, prior experiences with discrimination, biased hiring practices, or compromised data directly contributed to reticence towards disclosing demographic data. Participants could readily imagine future cases where such events could occur again, perhaps with even worse outcomes. Likewise, participants could envision future events where the demographic data are ignored or misused. Finally, interactions with the questionnaires themselves also mattered; the experience of responding to irrelevant questions or incorrectly worded items (see prior themes) could undermine trust.
Importantly, the interaction sequences component of relational trust denotes that individuals experience a series of decision situations and semiotics. Thus, recommendations based on sequences pertain to cumulative experiences that lead to trust or distrust. A single factor (e.g., improved question wording or transparency about data use) may be insufficient to cultivate trust if other interactions undermine it (e.g., inability to save data or skip questions). In other words, recommendations for improved semiotics leverage more trustworthy situations, which in turn create more signs of trustworthiness. Demographic data collectors must be mindful of the flow of experiences over time. Likewise, organizations must be cognizant of prior attitudes or experiences that might affect trust. Respondents from vulnerable, historically marginalized, or underserved communities may possess highly valid reasons to be skeptical or hesitant from the outset. Such concerns must be respected and assuaged over successive trust-building interactions.
Design questions about situations and semiotics are asked and answered continuously over time (i.e., over multiple sequences of interactions), which dynamically update beliefs or judgments about trust in an organization.

4.4. Strategy

A final set of themes (Themes 10–12) considered the strategies that respondents used to disclose information or protect themselves within the context of trust or distrust. Interviewees expressed strategic behaviors including non-disclosure in response to institutional distrust, lack of knowledge about how demographic data are used, and public perceptions and experiences related to security. Respondents also expressed the strategy of relying on broader categories or open text fields (when available) to describe themselves more accurately. Such findings thus reinforce existing recommendations to give respondents control over how and whether to respond (e.g., optional responses) and a broad range of potential descriptions (see also Themes 5 and 6).
Strategies are also institutional; organizations can enact policies and practices (i.e., strategies) for communicating goals and data plans, creating opportunities for dialog and feedback, and so forth. Organizations can strategically consider questionnaire design, steps to alleviate existing concerns, and methods of communication that establish trust and allow respondents to be informed and agentic. Thus, the recommendations outlined in prior sections also serve as organizational strategies related to trust.
Example design questions that might be asked regarding strategies might include: How can the questionnaire enable accurate self-descriptions? How will data security and privacy be maintained? How can users control their level of disclosure? How can users complete the questionnaire in a timely manner, including saving their progress to return later? Answering these questions helps designers articulate concrete actions that positively elicit trust.

4.5. Limitations and Future Research

Although the current study collected rich qualitative data from nearly 40 participants, the sample size was nonetheless small for deriving highly generalizable conclusions. Similarly, although we attempted to recruit participants that varied according to numerous demographic identity variables, it was not possible to fully represent (i.e., larger subsamples) all populations within society. The findings summarized in this paper thus do not capture all possible concerns about demographic identity data collection and individuals’ experiences. Nonetheless, this work was able to surface several robust themes that seem unlikely to be unique or idiosyncratic to this dataset; many individuals possess fears about bias and discrimination, along with skepticism about how companies, employers, university researchers, or commercial service providers use personal data. The findings from the current study contribute to understanding experiences and attitudes relevant to trust in demographic identity data collection contexts. Future work can expand data collection to additional populations and contexts.
Another limitation is that our tentative recommendations for cultivating trust require testing and validation. Prior research has shown that cultivating trust is an important condition for response to demographic questionnaires. Non-disclosure can be caused by inappropriate response options (e.g., too coarse, missing labels), unclear explanation regarding why questions are asked and how data will be used [17,18,19,64]. However, a contribution of the current paper is that a relation trust theoretical framework—including attention to decision situations, semiotics, interaction sequences, and strategies—can be a provide lens for (re)interpreting research on trust and data collection practices.
Finally, the current study focused on participants in STEM. Although the perspectives of STEM participants contribute to our understanding of the experiences of people from different backgrounds (i.e., they are unlikely to be entirely unique). This study’s findings can aid in better design for demographic questionnaires that can aid efforts to broaden participation in STEM. However, it is likely that people in different contexts (e.g., business or art) have particular experiences related to demographic identity data that were not covered in this work. Future research should consider populations from different educational and work spheres to further map out the variance people display in their attitudes toward demographic data collection.

5. Conclusions

The interview data analyzed in this study provided insights into the motivations for dis/trust in demographic questionnaires, the institutions that collect that data, and the societal structures and biases within which these situations occur. Examining the emerging themes through the 4s framework allowed us to situate trust concerns and attitudes within the demographic questionnaire situation, the semiotics involved in shaping understanding, the sequence of related interactions and experiences that inform decisions to disclose, and strategies for disclosure that stem from the above, as well as strategies for cultivating trustworthiness. This research adds to other recommendations on improving demographic questionnaires (e.g., [19,20,21,22,23,24,65]) by typologizing concerns raised by a heterogenous (though not fully representative) sample of the population. Designers of demographic questionnaires must be responsive to concerns relevant to all populations they wish to serve, along with historical and current systemic and anecdotal causes for distrust and non-disclosure. Future research should outline (e.g., via situation matrices [66]) and empirically test how trust and non-disclosure practices are affected by implementing suggestions like transparency with respect to goals, previous, current, and future uses of demographic data, dynamic questionnaire structures.

Author Contributions

Conceptualization, M.G., R.D.R. and E.K.C.; methodology, M.G., and R.D.R.; formal analysis, M.G.; resources, M.G. and R.D.R.; data curation, M.G.; writing—original draft preparation, M.G.; writing—review and editing, M.G., R.D.R. and E.K.C.; visualization, M.G.; supervision, R.D.R.; project administration, M.G.; funding acquisition, M.G. and R.D.R. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by ACM: Special Interest Group for Design of Communication (ACM: SIGDOC), given in 2022 and by The Gates Foundation (INV-006213) Given in 2020. This material is based upon work supported by the National Science Foundation under grant number 2231874.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board (or Ethics Committee) of Arizona State University(protocol code STUDY00016331, date of approval: 1 August 2022).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study. Verbal and written informed consent has been obtained from participants to publish this paper.

Data Availability Statement

The datasets presented in this article are not readily available because participation in the study was contingent upon protecting interviewee identity and only publishing relevant excerpts from interviews. Requests to access additional parts of the datasets should be directed to [email protected]

Acknowledgments

Special thanks given to Nicole Darmawaskita and Jessica Sturgis for help with transcribing interview recordings.

Conflicts of Interest

The funders had no role in the design of the study; in the collection, analyses, or interpretation of the data; in the writing of the manuscript; or in the decision to publish the results.

References

  1. Ajakwe, S.O.; Nwakanma, C.I.; Lee, J.M.; Kim, D.S. Machine Learning Algorithm for Intelligent Prediction for Military Logistics and Planning. In Proceedings of the 2020 International Conference on Information and Communication Technology Convergence (ICTC), Jeju, Republic of Korea, 21–23 October 2020; pp. 417–419. [Google Scholar] [CrossRef]
  2. Yang, Q. The role of design in creating machine-learning-enhanced user experience. In Proceedings of the AAAI Spring Symposium Series, Stanford, CA, USA, 27–29 March 2017; pp. 406–411. [Google Scholar]
  3. Eubanks, V. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor; St. Martin’s Press: New York, NY, USA, 2018. [Google Scholar]
  4. O’Neil, C. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy; Crown: New York, NY, USA, 2017. [Google Scholar]
  5. Emmanuel, T.; Maupong, T.; Mpoeleng, D.; Semong, T.; Mphago, B.; Tabona, O. A survey on missing data in machine learning. J. Big Data 2021, 8, 140. [Google Scholar] [CrossRef] [PubMed]
  6. Chiou, E.K.; Lee, J.D. Trusting automation: Designing for responsivity and resilience. Hum. Factors J. Hum. Factors Ergon. Soc. 2023, 65, 137–165. [Google Scholar] [CrossRef]
  7. Kietzmann, J.; Paschen, J.; Treen, E. Artificial Intelligence in Advertising: How marketers can leverage artificial intel-ligence along the consumer journey. J. Advert. Res. 2018, 58, 263–267. [Google Scholar] [CrossRef]
  8. Knighton, J.; Buchanan, B.; Guzman, C.; Elliott, R.; White, E.; Rahm, B. Predicting flood insurance claims with hydrologic and socioeconomic demographics via machine learning: Exploring the roles of topography, minority populations, and political dissimilarity. J. Environ. Manag. 2020, 272, 111051. [Google Scholar] [CrossRef] [PubMed]
  9. Aitamurto, T.; Landemore, H. Five Design Principles for Crowdsourced Policymaking: Assessing the Case of Crowdsourced Off-Road Traffic Law in Finland. J. Soc. Media Organ. 2015, 2, 2. [Google Scholar]
  10. Hasan, R.; Fritz, M. Understanding utility and privacy of demographic data in education technology by causal analysis and adversarial-censoring. Proc. Priv. Enhanc. Technol. 2022, 245–267. [Google Scholar] [CrossRef]
  11. Kaye, K. Police can Use Facial Recognition again after Ban in New Orleans, Home to Sprawling Surveillance [In-ternet]. Available online: https://www.protocol.com/enterprise/new-orleans-surveillance-facial-recognition (accessed on 26 July 2022).
  12. Kruse, D.; Schur, L. Employment of people with disabilities following the ADA. Ind. Relat. A J. Econ. Soc. 2003, 42, 31–66. [Google Scholar] [CrossRef]
  13. Van De Walle, S.; Six, F. Trust and distrust as distinct concepts: Why studying distrust in institutions is important. J. Comp. Policy Anal. Res. Pract. 2014, 16, 158–174. [Google Scholar] [CrossRef]
  14. van Prooijen, J.-W.; Spadaro, G.; Wang, H. Suspicion of institutions: How distrust and conspiracy theories deteriorate social relationships. Curr. Opin. Psychol. 2022, 43, 65–69. [Google Scholar] [CrossRef]
  15. Durso, L.E.; Meyer, I.H. Patterns and predictors of disclosure of sexual orientation to healthcare providers among lesbians, gay men, and bisexuals. Sex. Res. Soc. Policy 2013, 10, 35–42. [Google Scholar] [CrossRef]
  16. Tran, A.G.T.T.; Miyake, E.R.; Martinez-Morales, V.; Csizmadia, A. “What are you?” Multiracial individuals’ responses to racial identification inquiries. Cult. Divers. Ethn. Minor. Psychol. 2016, 22, 26–37. [Google Scholar] [CrossRef]
  17. Pearson, H.; Boskovich, L. Problematizing disability disclosure in higher education: Shifting towards a liberating hu-manizing intersectional framework. Disabil. Stud. Q. 2019, 39, 1. [Google Scholar] [CrossRef]
  18. Andrus, M.; Spitzer, E.; Brown, J.; Xiang, A. What we can’t measure, we can’t understand: Challenges to demographic data procurement in the pursuit of fairness. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, Virtual, 3–10 March 2021; pp. 249–260. [Google Scholar] [CrossRef]
  19. Fernandez, T.; Godwin, A.; Doyle, J.; Verdín, D.; Boone, H.; Kirn, A.; Benson, L.; Potvin, G. More comprehensive and inclusive approaches to demographic data collection. In Proceedings of the ASEE Annual Conference & Exposition, New Orleans, LA, USA, 26–29 June 2016. [Google Scholar] [CrossRef]
  20. Bauer, G.R.; Scheim, A.I. Advancing quantitative intersectionality research methods: Intracategorical and intercategorical approaches to shared and differential constructs. Soc. Sci. Med. 2019, 226, 260–262. [Google Scholar] [CrossRef] [PubMed]
  21. Spiel, K.; Haimson, O.L.; Lottridge, D. How to do better with gender on surveys: A guide for HCI researchers. Interactions 2019, 26, 62–65. [Google Scholar] [CrossRef]
  22. National Academies of Sciences, Engineering, and Medicine. Measuring Sex, Gender Identity, and Sexual Orientation; The National Academies Press: Washington, DC, USA, 2022. [Google Scholar] [CrossRef]
  23. Blaser, B.; Ladner, R.E. Why is data on disability so hard to collect and understand? In Proceedings of the 5th International Conference on Research in Equity and Sustained Participation in Engineering, Computing, and Technology (RESPECT), Portland, OR, USA, 10–11 March 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 1–8. [Google Scholar] [CrossRef]
  24. Hughes, J.L.; Camden, A.A.; Yangchen, T. Rethinking and updating demographic questions: Guidance to improve descriptions of research samples. Psi Chi J. Psychol. Res. 2016, 21, 138–151. [Google Scholar] [CrossRef]
  25. Furr, J.B.; Carreiro, A.; McArthur, J.A. Strategic approaches to disability disclosure on social media. Disabil. Soc. 2016, 31, 1353–1368. [Google Scholar] [CrossRef]
  26. Brown, B. Studying the Internet Experience (HPL-2001-49). 2001. Available online: https://www.hpl.hp.com/techreports/2001/HPL-2001-49.pdf (accessed on 15 May 2024).
  27. Hoffmann, C.P.; Lutz, C.; Ranzini, G. Privacy cynicism: A new approach to the privacy paradox. Cyberpsychology J. Psychosoc. Res. Cyberspace 2016, 10, 7. [Google Scholar] [CrossRef]
  28. Kokolakis, S. Privacy attitudes and privacy behaviour: A review of current research on the privacy paradox phenomenon. Comput. Secur. 2017, 64, 122–134. [Google Scholar] [CrossRef]
  29. Young, A.L.; Quan-Haase, A. Privacy Protection Strategies on Facebook: The Internet privacy paradox revisited. Inf. Commun. Soc. 2013, 16, 479–500. [Google Scholar] [CrossRef]
  30. Colnago, J.; Cranor, L.F.; Acquisti, A. Is there a reverse privacy paradox? an exploratory analysis of gaps between privacy perspectives and privacy-seeking behaviors. Proc. Priv. Enhancing Technol. Symposium. 2023, 455–476. Available online: https://ssrn.com/abstract=4607259 (accessed on 28 June 2024).
  31. Wang, M.-T.; Degol, J.L. Gender gap in science, technology, engineering, and mathematics (stem): Current knowledge, implications for practice, policy, and future directions. Educ. Psychol. Rev. 2017, 29, 119–140. [Google Scholar] [CrossRef]
  32. Cheryan, S.; Ziegler, S.A.; Montoya, A.K.; Jiang, L. Why are some STEM fields more gender balanced than others? Psychol. Bull. 2017, 143, 1–35. [Google Scholar] [CrossRef]
  33. Riegle-Crumb, C.; King, B.; Irizarry, Y. Does stem stand out? examining racial/ethnic gaps in persistence across postsecondary fields. Educ. Res. 2019, 48, 133–144. [Google Scholar] [CrossRef]
  34. Cech, E.A. Engineering ableism: The exclusion and devaluation of engineering students and professionals with physical disabilities and chronic and mental illness. J. Eng. Educ. 2023, 112, 462–487. [Google Scholar] [CrossRef]
  35. McCall, C.; Shew, A.; Simmons, D.R.; Paretti, M.C.; McNair, L.D. Exploring student disability and professional identity: Navigating sociocultural expectations in U.S. undergraduate civil engineering programs. Australas. J. Eng. Educ. 2020, 25, 79–89. [Google Scholar] [CrossRef]
  36. Stewart, C.O. STEM identities: A communication theory of identity approach. J. Lang. Soc. Psychol. 2022, 41, 148–170. [Google Scholar] [CrossRef]
  37. Agne, R.R.; Muller, H.L. Discourse strategies that co-construct relational identities in STEM peer tutoring. Commun. Educ. 2019, 68, 265–286. [Google Scholar] [CrossRef]
  38. Chang, M.J.; Eagan, M.K.; Lin, M.H.; Hurtado, S. Considering the impact of racial stigmas and science identity: Persistence among biomedical and behavioral science aspirants. J. High. Educ. 2011, 82, 564–596. [Google Scholar] [CrossRef]
  39. Simpson, A.; Bouhafa, Y. Youths’ and adults’ identity in STEM: A systematic literature review. J. STEM Educ. Res. 2020, 3, 167–194. [Google Scholar] [CrossRef]
  40. Ahlqvist, S.; London, B.; Rosenthal, L. Unstable identity compatibility: How gender rejection sensitivity undermines the success of women in science, technology, engineering, and mathematics fields. Psychol. Sci. 2013, 24, 1644–1652. [Google Scholar] [CrossRef]
  41. London, J.S.; Lee, W.C.; Phillips, C.; Van Epps, A.S.; Watford, B.A. A systematic mapping of scholarship on broadening participation of African Americans in engineering and computer science. J. Women Minor. Sci. Eng. 2020, 26, 199–243. [Google Scholar] [CrossRef]
  42. Saw, G.K. Leveraging social capital to broaden participation in STEM. Policy Insights Behav. Brain Sci. 2020, 7, 35–43. [Google Scholar] [CrossRef]
  43. Burt, B.A.; Stone, B.D.; Motshubi, R.; Baber, L.D. STEM validation among underrepresented students: Leveraging insights from a STEM diversity program to broaden participation. J. Divers. High. Educ. 2023, 16, 53–65. [Google Scholar] [CrossRef]
  44. Goering, A.E.; Resnick, C.E.; Bradford, K.D.; Othus-Gault, S.M. Diversity by design: Broadening participation through inclusive teaching. New Dir. Community Coll. 2022, 2022, 77–91. [Google Scholar] [CrossRef]
  45. O’leary, E.S.; Shapiro, C.; Toma, S.; Sayson, H.W.; Levis-Fitzgerald, M.; Johnson, T.; Sork, V.L. Creating inclusive classrooms by engaging STEM faculty in culturally responsive teaching workshops. Int. J. STEM Educ. 2020, 7, 32. [Google Scholar] [CrossRef]
  46. Canning, E.A.; Muenks, K.; Green, D.J.; Murphy, M.C. STEM faculty who believe ability is fixed have larger racial achievement gaps and inspire less student motivation in their classes. Sci. Adv. 2019, 5, eaau4734. [Google Scholar] [CrossRef]
  47. Sovero, V.; Buchinsky, M.; Baird, M.D. Playing catch up: A term-level investigation of the racial gap in STEM retention. Econ. Educ. Rev. 2021, 83, 102146. [Google Scholar] [CrossRef]
  48. Bryk, A.; Schneider, B. Trust in Schools: A Core Resource for Improvement; Russell Sage Foundation: New York, NY, USA, 2002. [Google Scholar]
  49. Hagerty, B.M.; Lynch-Sauer, J.; Patusky, K.L.; Bouwsema, M.; Collier, P. Sense of belonging: A vital mental health concept. Arch. Psychiatr. Nurs. 1992, 6, 172–177. [Google Scholar] [CrossRef]
  50. Bandura, A. Self-Efficacy: The Exercise of Control; Macmillan: New York, NY, USA, 1997. [Google Scholar]
  51. Maddux, J.E.; Gosselin, J.T. Self-Efficacy; The Guilford Press: New York, NY, USA, 2012. [Google Scholar]
  52. De Saussure, F. Course in General Linguistics; Columbia University Press: New York, NY, USA, 2011. [Google Scholar]
  53. Peirce, C.S. Collected Papers of Charles Sanders Peirce; Harvard University Press: Cambridge, MA, USA, 1974; Volume 5. [Google Scholar]
  54. Riegelsberger, J.; Sasse, M.A.; McCarthy, J.D. The mechanics of trust: A framework for research and design. Int. J. Hum. Compu. Stud. 2005, 62, 381–422. [Google Scholar] [CrossRef]
  55. Braun, V.; Clarke, V. Toward good practice in thematic analysis: Avoiding common problems and be(com)ing a knowing researcher. Int. J. Transgender Health 2023, 24, 1–6. [Google Scholar] [CrossRef]
  56. Byrne, D. A worked example of Braun and Clarke’s approach to reflexive thematic analysis. Qual. Quant. 2022, 56, 1391–1412. [Google Scholar] [CrossRef]
  57. Braun, V.; Clarke, V. Using thematic analysis in psychology. Qual. Res. Psychol. 2006, 3, 77–101. [Google Scholar] [CrossRef]
  58. Crenshaw, K. Demarginalizing the intersection of race and sex: A black feminist critique of antidiscrimination doctrine, feminist theory, and antiracist politics. In Feminist Legal Theory; Bartlett, K.T., Kennedy, R., Eds.; Routledge: London, UK, 1989; pp. 57–80. [Google Scholar]
  59. Collins, P.H. Intersectionality as Critical Social Theory; Duke University Press: Durham, NC, USA, 2019. [Google Scholar]
  60. Else-Quest, N.M.; Hyde, J.S. Intersectionality in quantitative psychological research: I. Theoretical and epistemological issues. Psychol. Women Q. 2016, 40, 155–170. [Google Scholar] [CrossRef]
  61. Bauer, G.R.; Churchill, S.M.; Mahendran, M.; Walwyn, C.; Lizotte, D.; Villa-Rueda, A.A. Intersectionality in quantitative research: A systematic review of its emergence and applications of theory and methods. SSM Popul. Health 2021, 14, 100798. [Google Scholar] [CrossRef]
  62. Buolamwini, J.; Gebru, T. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on Fairness, Accountability and Transparency; Friedler, S.A., Wilson, C., Eds.; PMLR: New York, NY, USA, 2018; pp. 77–91. [Google Scholar]
  63. Miller, T. Explanation in artificial intelligence: Insights from the social sciences. Artif. Intell. 2019, 267, 1–38. [Google Scholar] [CrossRef]
  64. Slade, T.; Gross, D.P.; Niwa, L.; McKillop, A.B.; Guptill, C. Sex and gender demographic questions: Improving methodological quality, inclusivity, and ethical administration. Int. J. Soc. Res. Methodol. 2021, 24, 727–738. [Google Scholar] [CrossRef]
  65. Chen, J.H.; Gardner, A.K. Promoting inclusive environments through best practices in demographic survey design. Glob. Surg. Educ. J. Assoc. Surg. Educ. 2022, 1, 47. [Google Scholar] [CrossRef]
  66. Kelley, H.H.; Holmes, J.G.; Kerr, N.L.; Reis, H.T.; Rusbult, C.E.; Van Lange, P.A.M. An Atlas of Interpersonal Situations; Cambridge University Press: Cambridge, UK, 2003. [Google Scholar]
Table 1. Participants’ self-reported demographic characteristics.
Table 1. Participants’ self-reported demographic characteristics.
ParticipantAgeGender IdentityRaceEthnicity
P128MalePart White, part Native-American, Half-KoreanA poor person
P232womanMiddle Eastern (white in questionnaires)Iranian
P323womanAsianChinese Indonesian
P431maleWhiteGreek
P523femaleWhiteHispanic
P627femaleWhite (cause that’s the only option)Arab, Muslim
P727genderfluidWhiteAmerican Southern
P832non-binaryWhiteWhite, Ashkenazi Jewish
P926maleSouth Asian/IndianSouth Indian
P1025maleAsianIndian
P1123femaleEast-Asian/Chinese2nd gen. immigrant from Hong-Kong
P1230womanWhiteJewish
P1327non-binaryWhiteUtah Mormon
P1448femaleWhiteArmy brat
P1547femaleCaucasianProtestant
P1647femaleWhiteWhite, American, Texan
P1727maleMexicanMexican
P1844womanWhiteCatholic Texan
P1926maleCaucasian WhiteMidwestern, German ancestry
P2039femaleWhite, Caucasiannot Hispanic; background Italian, and Irish, and Scottish, and Czechoslovakian
P2137womanBlackAfro-Caribbean
P2245femaleBiracial- Asian and CaucasianKorean-American
P2336womanWhiteIrish American
P2552MaleJewishJewish
P2637ManWhiteWhite
P2735ManAfrican American descendent of chattel slavesHebrew Israelite
P2822womanWhiteItalian American
P2937FemaleAsianChinese
P3032womanWhiteAmerican
P3125FemaleWhiteWhite
P3228MaleSouth AsianHindu
P3323Non-binaryFilipinoFilipino, American
P3425FemaleAsianJapanese
P3531MaleWhiteChristian
P3626Non-binaryWhiteWhite and European
P3736womanWhiteEuropean
P3846FemaleWhite and Native Americannone
P3932MaleWhiteJudeo-Christian, Anglo-Saxon
P4023I prefer to think of it as femaleWhiteProbably like 50% Swedish
Table 2. Emerging themes on dis/trust through the 4S lens. Clarifications in brackets added by the authors.
Table 2. Emerging themes on dis/trust through the 4S lens. Clarifications in brackets added by the authors.
4S ItemDescriptionTrust ThemesExample (with Participant #)
SituationThe interdependent decision structure and perceived tradeoffs of that interaction for all parties involvedT1: Bias from other institutions
T2: Bias from the questionnaire
T3: Positive trust stance
“I always think that if I say something [on disability] they might use that against me, or yeah, discriminate against me, or be like ‘this person is not fit for the job’” (P2)
SemioticsSignals, signs, and symbols that affect initial perceptions, trusting decisions, and interaction outcomesT4: Not understanding how queried information is relevant
T5: Missing relevant labels
T6: Labels too coarse
“The goal of a company should not be what do our employers look like. […] the goal of a university or a business should be to succeed […] should be based on the skill of your employees.” (P18, example of T4)
SequenceTrust evolution across repeating interactions, multiple situations, and how interaction patterns shape trustingT7: Believing data collection meant for virtue signaling
T8: Experiencing bias related to demographic information (minoritized/hegemonic)
T9: Concerns of data security
“I feel like they don’t necessarily use it to increase admissions or decrease admissions or increase access into programs. They just do it to feel like they’re doing good, and that’s it.” (P17, example of T7)
StrategyHow people, institutions, and other agents navigate decision situationsT10: Response optional
T11: Having ‘prefer not to say’ or ‘other’ options
T12: Fear of disclosure leading to bias
“i’ll put ‘prefer not to say’ if that’s the option, because those are the only 3 options this man woman prefer not to say and so, like So it’s because the actual answer is just missing” (P7, example of T11)
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Goldshtein, M.; Chiou, E.K.; Roscoe, R.D. ‘I Just Don’t Trust Them’: Reasons for Distrust and Non-Disclosure in Demographic Questionnaires for Individuals in STEM. Societies 2024, 14, 105. https://doi.org/10.3390/soc14070105

AMA Style

Goldshtein M, Chiou EK, Roscoe RD. ‘I Just Don’t Trust Them’: Reasons for Distrust and Non-Disclosure in Demographic Questionnaires for Individuals in STEM. Societies. 2024; 14(7):105. https://doi.org/10.3390/soc14070105

Chicago/Turabian Style

Goldshtein, Maria, Erin K. Chiou, and Rod D. Roscoe. 2024. "‘I Just Don’t Trust Them’: Reasons for Distrust and Non-Disclosure in Demographic Questionnaires for Individuals in STEM" Societies 14, no. 7: 105. https://doi.org/10.3390/soc14070105

APA Style

Goldshtein, M., Chiou, E. K., & Roscoe, R. D. (2024). ‘I Just Don’t Trust Them’: Reasons for Distrust and Non-Disclosure in Demographic Questionnaires for Individuals in STEM. Societies, 14(7), 105. https://doi.org/10.3390/soc14070105

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop