A Construct Validity Study for the Union Intolerance Scale: Convergent-Discriminant Validity and Concurrent Criterion-Related Validity

Abstract

Stemming from union intolerance as a psychological construct, defined as sustained hatred by nonunion employees when unions are perceived to encourage intolerable vices in employees and, notably, in union employees, we sought to broaden the empirical basis to claim scale validity for the union intolerance scale. In doing so, we targeted two forms of construct validity: convergent-discriminant validity and concurrent criterion-related validity. Based on survey data drawn from a sample of American nonunion employees (N = 162), we found supportive correlational evidence for both forms. In reference to convergent validity, the scores yielded by the intolerance scale showed expected convergence with the scores yielded by measures of general intolerance of stigmatized groups and union reliance. In reference to discriminant validity, the scores yielded by the intolerance scale showed expected divergence with the scores yielded by a measure of self-reported negative social emotions linked to vices. As for concurrent validity, the scores yielded by the intolerance scale were associated with the scores yielded by a measure of support for the Protecting the Right to Organize Act (the PRO Act), a proposed labor law passed by the U.S. House of Representatives, in which employees indicated whether they will or will not urge their two State Senators to cast a vote in favor of the Act when it is brought to the floor of the U.S. Senate. Included are discussion points about why the scale can be suggested as valid and how scale validity can be extended.

1. Introduction

In the interdisciplinary field of union–management relations, the union intolerance scale was introduced in 2021 as a measure of the union intolerance construct [1] (see also [2,3,4] for overviews of the field and example studies). Shown to have content and construct scale validity, further demonstrations of validity are needed to firmly establish the scale as a sound empirical tool to forecast outcomes that are germane to the success of labor unions in their endeavor to enhance the working lives of union and nonunion employees. As such, the present study was constructed to broaden the empirical basis to claim scale validity. As empiricists, with a decidedly focused eye on measurement, we take the position that evidence-based predictions must be well-grounded in measures with evidence-based validity (for similar views, see [5,6,7,8]).

As a guide to our study, we first review the union intolerance construct and the extant validity data for the union intolerance scale. We next introduce convergent-discriminant validity and concurrent criterion-related validity as two forms of construct validity yet to be added to the empirical basis to claim scale validity. Next, we formulate research questions specific to the intolerance scale, asking whether the scores yielded by the scale provide supportive data to claim convergent-discriminant validity and concurrent validity. Our sample drawn from nonunion employees is fully documented, as are the scales used in our validity tests. In reference to concurrent validity, we selected as a criterion support for a contemporary outcome critical for the success of American unions which has yet to be passed (or not passed) into U.S. law: the Protecting the Right to Organize Act of 2021 (the PRO Act), a Bill passed by the U.S. House of Representatives that has yet to be brought to the floor of the U.S. Senate.

1.1. Union Intolerance Construct

Against the background of a documented rise in intolerance directed at American labor unions in corporate circles and in the public at large (see [9,10] for thorough discussions; see also [11,12] for early warnings), union intolerance was developed as a psychological construct, defined as sustained hatred by nonunion employees when unions are perceived to encourage intolerable vices in employees [1]. Importantly, it is suggested that union employees are positioned as first-line recipients of such encouragement, in which vices—alternatively referred to as unacceptable emotions—are keyed to numbered lists recorded by religious institutions (e.g., the nine deadly thoughts, the six enemies, the five thieves, the five poisons, the seven egos). Translated and summarized for English speakers, the numbered vices are represented as anger, apathy, envy, gluttony, greed, lust, and pride [1] (see also [13,14] for encyclopedic discussions).

With culture used as a deductive framework, it is suggested that unacceptable emotions are associated with socialized inhibitions. Driven by inhibitions, the enactment of unacceptable emotions can induce hatred turned inward (e.g., self-loathing; [15]) or outward (e.g., stereotyping; [16]). Once induced, defense mechanisms emerge which are intended to mask intrapsychic conflict stemming from conscious impulses to reenact the emotions (see [17] for a detailed discussion). Although some mechanisms are deemed mature in that they resolve the conflict (e.g., sublimation; [18]), other mechanisms are deemed less mature in that they require a continuous investment of psychic energy to maintain.

A less mature mechanism often associated with intolerance of others is projection (also known as displacement of attribution or scapegoating; [19,20]), in which conflict is masked by projecting impulses to reenact unacceptable emotions onto others. Adapted to the experience of nonunion employees, union intolerance is conceived as a subjective state of readiness to project unacceptable emotions onto unions when unions are perceived to encourage such emotions in employees and, notably, in union employees [1].

1.2. Union Intolerance Scale

To demonstrate the effect of projected vices onto unions, union intolerance as a construct was hypothesized as a predictor variable, in which modeling predictor-outcome relationships required the construction of a union intolerance scale (see [1] for a full account). With a focused eye on content validity, the construction of the scale began with search engines (e.g., Google) keyed to such phrases as “why I hate unions.” First-person narratives were gathered from nonunion employees as author/bloggers/commentators reporting personal experiences with unions and like experiences by friends and family. Extracted from the narratives were verbatim statements that indicated union intolerance. Edited only for clarity (e.g., grammar) and repetition, the retained statements were compiled and matched to the represented vices (i.e., anger, apathy, etc.) by a sample of naïve participants performing a sorting task in which vices (i.e., elements of the construct) were presented as feelings. Subsequent samples of nonunion employees performing checklist screening tasks were used to: (a) discriminate elements by identifying which statements suggested feelings most encouraged by unions, (b) confirm elements by identifying which statements factored into a willingness to tolerate or not tolerate a union, and (c) reduce the number of statements by eliminating which statements indicated a union that no nonunion employee would be willing to tolerate or that any nonunion employee would be willing to tolerate. Using agreement criteria for the performed tasks, 12 of the original-worded statements were retained as scale items.

Following this inductive method of item construction, two subsequent samples (an initial sample and a replication sample) of nonunion employees were asked to respond to the items, with response options of willing to tolerate or not willing to tolerate a union that, for example, “urges members of the union to pride themselves as the best trained, the most responsible, the most reliable, and the most dedicated employees that an employer could ever hope for.” To evaluate the construct validity of a unidimensional scale, principal components and confirmatory factor analyses were performed on the item responses gathered in the samples. In both the initial and replication samples, nearly identical component matrices and fit indices were found for a single factor.

Using the initial sample responses, construct validity for the union intolerance scale was claimed in that the scores yielded by the scale provided support for a hypothesized predictor-outcome relationship. As expected, the intolerance scores were inversely associated with the scores yielded by a measure of willingness to join a union (as were the item scores and the willingness to join scores), suggesting that more intolerance was associated with less willingness to join. Further, construct validity was illustrated by a hypothesized moderated relationship between union intolerance and willingness to join, in which the inverse relationship was stronger for employees who claimed to have experienced union discrimination (e.g., felt “passed over” for a promotion because they were not a union member).

1.3. Two Forms of Construct Validity

1.3.1. Convergent-Discriminant Validity

Evidence of convergent-discriminant validity is a demonstration of construct validity, with the caveat that evidence of convergent validity without discriminant validity (or vice versa) represents a deficient validity claim [21] (see also [22] for example studies). Convergent validity is shown if the scores yielded by a focal measure—in this case, the union intolerance scale—show covariation or convergence with the scores yielded by a measure of a variable that is thought to be related to the variable of the focal measure. That is, given two variables that are conceptualized as related, the scores yielded by the measures of the variables should show strong covariation. Discriminant validity is shown if the scores yielded by a focal measure do not show covariation or discrimination with the scores yielded by a measure of a variable that is thought to be unrelated to the variable of the focal measure. That is, given two variables that are conceptualized to be unrelated, the scores yielded by the measures of the variables should show weak covariation. In this context, it is also possible with respect to convergent validity to suggest that two variables are conceptualized as positively or inversely related, in which the scores yielded by measures of the variables should show expected positive or negative covariation.

To demonstrate all aspects of convergent-discriminant validity, a minimum of three additional variables with three measures is required [21]. To the extent that strong and weak covariation may induce disagreement about what is strong and what is weak, we evoked statistical probability to separate random from nonrandom covariation. That is, assuming linear covariation, we relied on zero-order correlations (r coefficients or variants of r) to suggest, in statistical terms, the presence or absence of covariation. Specifically, we used a probabilistic basis to suggest that nonzero coefficients may suggest the presence of nonrandom positive or negative covariation (see [23] for a relevant discussion).

In reference to convergent validity, we selected two scalable variables to demonstrate positive and negative covariation with the scores yielded by the union intolerance scale. Tapping into the nomological network set by Halman and Vloet [24] for general intolerance of stigmatized groups (hereafter referred to as general intolerance), we conceptualize that union intolerance is a special case of general intolerance. As such, the scores yielded by the union intolerance scale should relate to the scores yielded by a measure of general intolerance. Specifically, the scores should show convergence; the coefficient for union intolerance and general intolerance should be nonzero and positive (i.e., more union intolerance is associated with more general intolerance).

Within the same nomological network extended by Asbrock et al. [25], reliance on a collective of others—such as a labor union—for pay and benefits versus non-reliance (e.g., striving for benefits on one’s own) suggests a tolerance versus an intolerance of others. As such, the scores yielded by the union intolerance scale should relate to the scores yielded by a measure of union reliance. Specifically, the scores should show convergence; the coefficient for union intolerance and uni on reliance should be nonzero and negative (i.e., more union intolerance is associated with less union reliance).

In reference to discriminant validity, we selected a scalable variable to demonstrate absent covariation with the scores yielded by the union intolerance scale. Reviewing studies keyed to anger, apathy, and other vices, we noted that vices were alternatively referred to as negative social emotions and were indexed with self-report measures (e.g., [26,27]). As such, we asked ourselves whether there might be a difference between the scores yielded by a self-report measure, in which respondents are asked about the extent to which they experience feelings associated with vices, and the scores yielded by a tolerance scale, in which respondents are asked about the extent to which they are tolerant of others who encourage feelings associated with vices? That is, we asked ourselves whether the scores should show covariation? Among many possible answers, we took to heart one suggested by [1] (p.14):

“… suggesting that the scores yielded by a self-report attitude scale should correlate with the scores yielded by a tolerance scale may be untenable. Consider tolerance of others who encourage greed. Hypothesizing that individuals who report a positive attitude toward greed should also be more tolerant of others who encourage greed may simply reflect the assumed ubiquity of greed in a culture (Erlich, 2017 makes the same point about envy). That is, without additional inductive work to identify the normative strength of greed in a culture, it would seem that the relationship between a positive (or negative) attitude toward greed and tolerance (or intolerance) of others who encourage greed is indeterminable.”

As such, we conceptualize that union intolerance based on a tolerance scale is unrelated to negative social emotions based on a self-report measure, in which the scores yielded by the union intolerance scale and the scores yielded by a self-report measure of negative social emotions keyed to anger, apathy, and other vices should show absent covariation. Specifically, the scores should show divergence; the coefficient for union intolerance and self-reported negative social emotions should be zero or, less stringently, near zero (i.e., more union intolerance is not associated with more or less self-reported negative social emotions).

1.3.2. Concurrent Criterion-Related Validity

Additionally, as a demonstration of construct validity, evidence of concurrent criterion-related validity involves the selection of a criterion to show an intended or subsequent outcome, a variable in which the scores yielded by a measure of the variable are thought to show covariation with the scores yielded by a measure of a focal variable [28] (see also [6] for example studies). With union intolerance positioned as a predictor, we selected a contemporary variable of importance to the success of American unions as an outcome (i.e., as a criterion): support for the passage of the PRO Act, a Bill passed in the U.S. House of Representatives that has yet to be brought to the floor of the U.S. Senate.

The Bill is a proposed U.S. law that amends previous labor laws, such as the 1935 National Labor Relations Act. The intent of the Bill is to “expand various labor protections related to employees’ rights to organize and collectively bargain in the workplace” (see [29]). The items of the Bill include: (a) it prevents employees who seek to organize or to join a labor union from being fired or suspended, (b) it makes mandatory employee meetings used by employers to counteract or to discourage attempts at organizing a union illegal, (c) it allows the National Labor Relations Board (a federal government review board) to fine employers up to USD 50,000 for every violation of labor law and up to USD 100,000 in cases of repeat violations, (d) it reclassifies some “independent contractors” as “employees,” giving them the right to organize a union, and (e) it permits labor unions to encourage their members to participate in strikes that are initiated by members of other labor unions (so-called “secondary strikes”).

Given the intent of the Bill, we conceptualize that union intolerance is related to support for the PRO act. As such, the scores yielded by the union intolerance scale should relate to the scores yielded by a measure of whether nonunion employees will or will not urge their two State Senators to cast a vote in favor of the Pro Act when it is brought to the Senate floor. Specifically, independent of the scores yielded by measures of other scaled variables in the study (e.g., union reliance), the scores yielded by the union intolerance scale and the scores yielded by the support measure (scored toward “will urge”) should show negative covariation; the coefficient for union intolerance and support for the Act should be nonzero and negative (i.e., more union intolerance is associated with less support for the Act).

2. Research Questions

Whether data from a new sample of nonunion employees will be sufficient to broaden the empirical basis to claim convergent-discriminant validity and concurrent criterion-related validity for the union intolerance scale is a question represented by four research questions.

Research Question 1: Do measures of union intolerance and general intolerance yield scores that show positive covariation?

If so, a claim of convergent validity for the union intolerance scale is supported.

Research Question 2: Do measures of union intolerance and union reliance yield scores that show negative covariation?

If so, a claim of convergent validity for the union intolerance scale is supported.

Research Question 3: Do measures of union intolerance and self-reported negative social emotions yield scores that show absent covariation?

If so, a claim of discriminant validity for the union intolerance scale is supported.

Research Question 4: Do measures of union intolerance and support for the PRO Act show negative covariation independent of the measures of other scaled variables?

If so, a claim of concurrent validity for the union intolerance scale is supported.

3. Method

3.1. Procedure

Anonymous survey data were collected from American employees beginning in June 2021 and ending in December 2021. The survey sites were community gatherings and public transportation areas (e.g., farmers’ markets, licensed bingo halls, tourist information centers, commuter train stations).

With permission obtained at each site, the researchers circulated flyers with the following information:

“Can you volunteer to take this survey? You can if you are employed in the United States and are not a full-time student. The survey is anonymous—no names. The survey takes less than 10 min to complete. The survey cannot be mailed. USD 5 is given for taking the survey. Please ask the researcher for a survey.”

Employees who responded to the flyer were given a no-name informed consent form, a survey, a pencil, and an unmarked envelope. The researchers collected sealed envelopes, paid participants, and conducted onsite debriefing.

The intent of the USD 5 payment was provided to offset asking participants for their time in exchange for nothing. Against the prospect that the payment may have violated their voluntary status (i.e., may have been viewed as coercive), true to the wording of the flyer, and underlined by informed consent statements, the payment was given for “taking” the survey rather than for “completing” the survey. Of note, many participants chose to forgo the payment.

3.2. Sampling

To ensure that the sampling resulted in data appropriate to address the research questions, the survey was embedded with eligibility items (e.g., “Are you currently employed in the U.S.?”; “Are you currently a union member?”). An additional check for careless responses resulted in excluded surveys if the responses indicated the same scale anchor for long strings of consecutive items.

Based on 302 collected surveys with no missing data, 162 surveys were counted as eligible. The eligible surveys included employees from three U.S. States: Connecticut, Massachusetts, and Rhode Island.

The assessed demographics included age (indicated in years), gender (coded as either man (0) or woman (1)); ethnic group (coded as either nonethnic (0, White, European American) or ethnic (1, African American, Asian, Pacific Islander American, Latinx American, Middle Eastern, Arabian American)); employment status (coded as either part-time (0, less than 35 h a week) or full-time (1, 35 h or more a week)), and past union membership (coded as either no (0) or yes (1)).

Ninety percent of the employees were aged 25 years or older (the median age was 47; the range in years was 18 to 75). Sixty-one percent were women employees. Fourteen percent identified themselves as ethnic. Seventy percent were full-time employees. Twenty-seven percent indicated past union membership.

To estimate the representativeness of the sample with respect to the 2021 population of American nonunion employees, the 2022 January issue of the U.S. Bureau of Labor Statistics was consulted [30]. In doing so, we compared the percentages in the sample for age group, gender, ethnic group, and employment status with the reported national percentages. The results indicated that employees 25 years or older and women employees were oversampled by 13% or less: +0.0275 and +0.1298, respectively. The results also indicated that ethnic employees and full-time employees were undersampled by 19% or less: −0.1914 and −0.1354, respectively.

3.3. Measures

Union intolerance. To assess union intolerance, we asked the employees to respond to 12 items from the union intolerance scale developed by Mellor [1] (see [31] for a list of items).

The items were prefaced with the following statements:

“We are interested in what you think about labor unions.

Below are statements taken from various Internet blogs, in which bloggers expressed their willingness or unwillingness to tolerate a union should they be required to join a union or should they be given a choice to join a union.

Please read each statement carefully and decide for yourself.”

The statements were followed with a response instruction (“Check (√) one blank”) and an item stem:

“I am willing to tolerate a union, or I am not willing to tolerate a union…”

An example item keyed to anger is:

“…that tells members of the union that when an employer issues a statement like “here at this company, we are one team, with one goal to make our company the best it can be,” that members should ask about how many seats will be reserved on the board of directors for union members.”

The response options were “willing to tolerate” or “not willing to tolerate”.

A principal components analysis was performed on the union intolerance items. The analysis produced one eigenvalue greater than 1.00 (λ = 4.228, % of variance explained = 44.495, item loadings ≥0.447). The Cronbach’s α for the items was 0.83.

As a result of these analyses, the responses were averaged, yielding continuous union intolerance scale scores from 0 (less intolerance) to 1.00 (more intolerance).

General intolerance. To assess general intolerance, we asked the employees to respond to 14 items drawn from intolerance scales of stigmatized groups (e.g., [24,25]). Excluded were items that implied or indicated ethnicity or specific religions (see [31] for a list of items).

The items were prefaced with the following statement:

“We are interested in what you consider to be an ideal community for you to live in by considering what groups of people you would rather not have as neighbors.”

The statement was followed by a response instruction (“Check (√) = yes”) and the item stem:

“I would not like as neighbors…”

An example item is:

“…immigrants”

The response options were coded as 0 for blank and as 1 for yes.

A principal components analysis was performed on the general intolerance items. The analysis produced one eigenvalue greater than 1.00 (λ = 1.918), but four items (anti-vaccinationists, convicted felons, psychiatric out-patients, religious fundamentalists) did not load the factor due to low extraction values. An analysis performed on the remaining 10 items produced one eigenvalue greater than 1.00 (λ = 4.317, % of variance explained = 43.166, item loadings ≥0.466). The α for the items was 0.84.

As a result of these analyses, the responses were averaged, yielding continuous general intolerance scale scores from 0 (less intolerance) to 7 (more intolerance).

Union reliance. To assess union reliance, we asked the employees to respond to an item about reliance “on a collective of others—like a labor union” (see [32,33] for similar one-item measures).

The item was prefaced with the following statement:

“We are interested in how often you have experienced the feeling listed below during the past several months.”

The statement was followed by a response instruction (“For the item on this page, write in one number in the blank using the following scale:”) and an item stem:

“When I think about how I felt during the past several months…”

Followed by the item:

“I have felt that relying on a collective of others—like a labor union—for pay and benefits is unwise—that I’d be better off pursuing what I need on my own.”

The response options were based on a Likert scale, using 1 to 7 anchors (none of the time to all of the time).

The responses were “reverse coded” and averaged, yielding continuous union reliance scale scores from 1 (less reliance) to 7 (more reliance).

Self-reported negative social emotions. To assess self-reported negative social emotions linked to vices, we asked the employees to respond to seven items drawn from self-reported measures of anger, apathy, envy, gluttony, greed, lust, and pride (e.g., [26,27]; see [31] for a list of items).

The items followed the same preface statement, response instruction, item stem, and response options used for the union reliance item.

An example item keyed to anger is:

“I have felt resentful of others, wishing to enact vengeance or revenge on others.”

A principal components analysis was performed on the negative social emotion items. The analysis produced one eigenvalue greater than 1.00 (λ = 4.017, % of variance explained = 57.387, item loadings ≥0.436). The α for the items was 0.85.

As a result of these analyses, the responses were averaged, yielding continuous negative social emotions scale scores from 1 (less negative) to 7 (more negative).

Support for the PRO Act. To assess support for the PRO Act, we asked the employees to respond to an item about whether they will or will not urge their two State Senators to cast a vote in favor of the Act when it is brought to the Senate floor.

The item was prefaced with the following statements:

“We are interested in what you think about the Protecting the Right to Organize Act or PRO Act.

The Bill passed in the U.S. House of Representatives by a vote of 225 to 206 on 9 March 2021.

The Bill now advances to the U.S. Senate.

Main items of the Bill include:

It prevents employees from being fired or suspended who seek to organize or to join a labor union.

It allows labor unions to override “right-to-work laws,” allowing unions to collect “fair share” fees from non-union employees who are covered by a union contract.

It makes illegal mandatory employee meetings used by employers to counteract or to

discourage attempts at organizing a labor union.

It prevents employers from holding immigration status against employees in hiring or

retention decisions.

It allows the National Labor Relations Board (the NLRB, a federal government review board) to fine employers up to $50,000 for every violation of labor law.

It allows the NLRB to fine employers up to $100,000 in cases of repeat violations of labor law.

It brings monetary compensation to employees suspended or fired in violation of labor law.

It reclassifies some “independent contractors” as “employees,” giving them the right to

organize a labor union.

It permits labor unions to encourage their members to participate in strikes that are initiated by members of other labor unions (so-called “secondary strikes”).”

The statements were followed by a response instruction (“Check (√) one blank”) and an item stem:

“When the Bill is brought to the floor of the Senate for a vote, I will urge my two State Senators to vote…”

The response options were “yes, for the Bill” or “no, against the Bill” and were coded as 0 for no and as 1 for yes.

3.4. Controls

To enhance the generalizability of the statistical results, we used the assessed demographics as covariates in all of the multivariate tests.

4. Results

The raw score zero-order correlations, means (Ms), and standard deviations (SDs) for all study variables are presented in

Table 1. Zero-Order Correlations (rs), Means (Ms), and Standard Deviations (SDs).

Variable	1	2	3	4	5	6	7	8	9	10
1. Age	-
2. Gender	0.12	-
3. Ethnic group	−0.22 **	−0.02	-
4. Employment status	−0.27 **	−0.07	0.14	-
5. Past union membership	0.22 **	−0.05	0.04	0.03	-
6. General intolerance	0.12	0.07	0.10	0.07	0.06	0.84
7. Union reliance	−0.18 *	0.03	0.16 *	0.02	0.07	−0.19 *
8. Negative social emotions	0.07	−0.19 *	0.00	0.13	0.04	0.32 **	−0.33 **	0.85
9. Union intolerance	0.18 *	0.03	−0.08	−0.02	0.07	0.18 *	−0.26 **	0.09	0.83
10. Support for the PRO Act	−0.12	0.11	−0.07	−0.13	−0.06	−0.27 **	−0.24 **	−0.14	−0.45 **	-
M	46.14	0.61	0.14	0.70	0.27	0.20	4.59	2.05	0.55	0.64
SD	15.66	0.49	0.34	0.46	0.45	0.25	2.16	1.14	0.27	0.48

Note: N = 162. Diagonal entry is a scale reliability (α). Age: in years; Gender: man = 0, woman = 1; Ethnic group: nonethnic = 0, ethnic = 1; Employment status: part-time = 0, full-time = 1; Past union membership, Support for the Pro Act: no = 0, yes = 1; General intolerance [of stigmatized groups], Union intolerance: less = 0, more = 1; Union reliance, [self-reported] Negative social emotions: less = 1, more = 7. * p < 0.05. ** p < 0.01.

.

4.1. Descriptive Statistics

The descriptive statistics in reference to the scaled variables indicated that the average for union intolerance was 0.55 on a metric of 0.00 to 1.00. On the same metric, the average for general intolerance was 0.20. The average for union reliance was 4.59, an average higher than the midpoint of the 7-point Likert scale (4). On the same metric, the average for negative social emotions was 2.05, an average lower than the midpoint of the scale.

In reference to whether employees will or will not urge their two State Senators to cast a vote in favor of the PRO Act, 64% responded “yes” to indicate that they will.

4.2. Demographic Tests

To discern mean differences in demographics in relation to the scaled variables, we performed t-tests, using median splits for non-dichotomous variables to construct demographic subgroups. On average, women employees (vs. men employees) were less likely to indicate negative social emotions (t(160) = −2.413, p < 0.05). Additionally, on average, ethnic employees (vs. nonethnic employees) were more likely to indicate union reliance (t(160) = 2.033, p < 0.05).

4.3. Convergent-Discriminant Validity Tests

In reference to the convergent-discriminant validity of the union intolerance scale, the relevant zero-order correlations are union intolerance in relation to general intolerance, union reliance, and negative social emotions. As seen in Table 1, union intolerance and general intolerance were positively correlated (r = 0.18, p < 0.05; convergent coefficient), suggesting that more union intolerance is associated with more general intolerance. Union intolerance and union reliance were inversely correlated (r = −0.26, p < 0.01; convergent coefficient), suggesting that more union intolerance is associated with less union reliance.

Union intolerance and negative social emotions were not correlated (i.e., the coefficient was nonsignificant, r = 0.09, p > 0.05; discriminant coefficient).

These results provide support for the convergent-discriminant validity of the union intolerance scale. In reference to convergent validity, the scores yielded by the scale and the scores yielded by the measures of general intolerance and union reliance show expected positive and negative covariation. In reference to discriminant validity, the scores yielded by the scale and the scores yielded by the self-report measure of negative social emotions show expected absent covariation.

4.4. Concurrent Criterion-Related Validity Tests

To test the concurrent validity of the union intolerance scale, we performed a stepwise hierarchical regression, in which union intolerance was positioned as a predictor of support for the PRO Act independent of the other scaled variables.

The regression analysis required three steps. At step 1, support for the PRO Act was regressed onto the demographics (as covariate effects). Next, at step 2, support for the PRO Act was regressed onto general intolerance, union reliance, and negative social emotions (as main effects). At step 3, the final step, support for the PRO Act was regressed onto union intolerance (a main effect).

The results of the analysis are presented in

Table 2. Regression results for support for the Pro Act.

	At Step		Final
Predictor	B a	Β b	B a	Β b	R2/∆R2	F/∆F	df
Step 1					0.064	2.133	5, 156
Age	−0.006	−0.201	−0.003	−0.085
Gender	0.116	0.117	0.130	0.132
Ethnic group	−0.132	−0.094	−0.151	−0.108
Employment status	−0.168	−0.159	−0.134	−0.127
Past union membership	0.002	0.002	0.006	0.006
Step 2					0.091	5.489 **	3, 153
General intolerance	−0.416 **	−0.220	−0.316 *	−0.167
Union reliance	0.045 *	0.202	0.025	0.113
Negative social emotions	0.018	0.043	0.012	0.029
Step 3					0.141	30.314 **	1, 152
Union intolerance			−0.711 **	−0.396
Overall					0.295	7.083 **	9, 152

Note: N = 162. ^a Unstandardized coefficient. ^b Standardized coefficient. * p < 0.05. ** p < 0.01.

. As shown in the final step, independent of general intolerance as a main effect (unstandardized B = −0.316, standardized β = −0.167, p < 0.01), union intolerance as a main effect was significant (B = −0.711, β = −0.396, p < 0.01).

The effect size associated with union intolerance as a main effect is indicated by a change in the squared multiple correlation (∆R²) at the final step. At this step, union intolerance explained 14% of the variance in support for the PRO Act independent of the covariate effects and the main effects (∆R² = 0.141; ∆F(1, 152) = 30.314, p < 0.01).

These results provide support for the concurrent criterion-related validity of the union intolerance scale. With support for the PRO Act as an outcome, the scores yielded by the intolerance scale and the scores yielded by the measure of support for the Act show expected negative covariation independent of other scaled variables, suggesting that more union intolerance is associated with less support for the Act.

4.5. Supplemental and Exploratory Analyses

In reference to the concurrent validity of the union intolerance scale, we examined the concurrent validity of other scaled variables with support for the PRO Act as a criterion. We did this by running the three-step regression for each scaled variable entered at the final step. Only the regression with general intolerance at that step produced a significant change in the squared multiple correlation (∆R² = 0.023; ∆F(1, 152) = 4.960, p < 0.05; B = −0.316, β = −0.167, p < 0.05).

Additionally, to explore the nonsignificant zero-order correlation for union intolerance and self-reported negative social emotions, we ran moderated regressions, examining interactions between union intolerance and demographics in relation to negative social emotions as an outcome variable. The two-step hierarchical regressions (one regression for each two-way interaction term, with nonfocal demographics as covariates and union intolerance as a main effect) indicated a significant interactive effect for union intolerance and past union membership (∆R² = 0.045; ∆F(1, 154) = 7.663, p < 0.01; B = 1.918, β = 0.215, p < 0.01).

The subgroup regressions for past union membership indicated that when employees had past membership, the relationship between union intolerance and negative social emotions was significant and positive (n = 44), in which the explained variance in negative social emotions was 10% above and beyond the demographic covariates (∆R² = 0.100; ∆F(1, 38) = 5.085, p < 0.05; B = 1.610, β = 0.325 p < 0.05). In contrast, when employees had no past membership, the relationship between intolerance and negative social emotions was nonsignificant (n = 118), in which the explained variance in negative social emotions was 1% above and beyond the demographic covariates (∆R² = 0.005; ∆F(1, 112) = 0.636, p > 0.05; B = −0.287, β = −0.075, p > 0.05).

5. Discussion

5.1. Overall Summary

In reference to our research questions about the construct validity of the union intolerance scale, our data are sufficient to answer each question in the affirmative. Using a sample of American nonunion employees and a probabilistic basis to discern random from nonrandom covariation, the data are supportive of adding convergent-discriminant validity and concurrent criterion-related validity to the empirical basis to claim scale validity. Specifically, in reference to concurrent validity, empirical support for a predictor–outcome relationship of importance to the success of American labor unions is shown in the data. To wit, with union intolerance positioned as a predictor, the scores yielded by the intolerance scale indicate whether nonunion employees will or will not urge their two State Senators to cast a vote in favor of the Pro Act when it is brought to the floor of the U.S. Senate. As seen in the data, independent of the other scaled variables in the study, employees who indicated more union intolerance are predictively less likely to indicate that they will urge their Senators to cast a favorable vote for the Act.

5.2. Why Valid?

Explaining the construct validity of the union intolerance scale begins with the inductive methods used in item construction. To ensure item quality—that items are relevant to and representative of the construct—the expression of item content did not rely on expert judgment. Rather, the expression of item content was sought from nonunion employees who were envisioned as representative employees. We think this switch to actual words and phrases used by representative employees to construct items opens a less disrupted channel through which elements of the construct can be represented and recorded numerically, as, for example, in our effort to represent projected impulses to reenact unacceptable emotions.

By less disrupted, we mean less interference due to item construction favored by experts, in which items are worded to suggest abstract concepts thought to have high generalizability (e.g., “Unions keep workers safe.”). However, in striving for such generalizability, at risk is item irrelevance, the downside of which is revealed by absent responses (i.e., missing data), and/or indifferent responses (i.e., midpoint responses), and/or “who knows, who cares responses” (i.e., random responses). In this vein, we cannot imagine a more worst-case scenario in collecting survey data than asking employees to respond to items that they discount as “the imagined reality of outsiders” who appear to be clueless about their situation or their viewpoints (see [34,35,36,37] for relevant discussions).

Additionally, to be explained is the demonstrated prediction of support for the PRO Act, as seen in our data in reference to the union intolerance scale but not seen in reference to a self-report measure of negative social emotions. Our view is that the scaling formats for self-report and tolerance tap decidedly different representations of emotions, a difference that we think makes a difference in matters of prediction when negative emotions are assessed (see [38,39] for comprehensive discussions). In the case of a self-report, we expect that channel distortion is associated with defense mechanisms that turn intrapsychic conflict inward (e.g., denial). This may explain why, when negative emotions are assessed by self-report measures, it is recommended that yielded scores be adjusted or corrected by scores yielded by broad measures of distortion (e.g., self-deception; see [40,41] for discussions). Although often labeled as “lie scales,” we do not think that respondents are actually motivated to falsify their feelings. Rather, we think they are motivated to mask the intrapsychic conflict they feel when asked to indicate the extent to which they experience impulses to reenact unacceptable emotions. What comes through the channel are adjusted representations of emotions in which the direction of masking may be suggested (as inward), but the precise masking mechanism that may be used to interpret the represented emotions is unidentified.

In contrast, in the case of a tolerance scale, we expect that channel distortion is associated with defense mechanisms that turn intrapsychic conflict outward. Asking respondents to indicate the extent to which they are intolerant of others who encourage negative emotions provides an interpretative basis to identify what is “coming through the channel.” Seen through the lens of projection as an interpretable masking mechanism, it is possible to suggest that intolerance scores represent what respondents are asked to represent: their intolerance of emotions they feel should not be enacted and, therefore, should not be encouraged by others. As such, when negative emotions are assessed by a tolerance scale, the yielded scores need not be adjusted nor corrected by the scores yielded by broad (shot-in-the-dark) measures of masking mechanisms. Rather, the scores can be interpreted as indicated by the respondents; they are correct representations of projected impulses to reenact unacceptable emotions.

5.3. What’s Next?

The external validity of the union intolerance scale is yet to be added to the empirical basis to claim scale validity in terms of generalizability beyond extant samples of American nonunion employees. In this respect, we hope our study will inspire and encourage other researchers to collect samples of nonunion employees to model predictor–outcome relationships relevant to the success of unions, positioning union intolerance as either a predictor or outcome and using the intolerance scale as an indicator. What would be most desirable to demonstrate external validity is a representative sample of nonunion employees targeted for union membership by a single union (e.g., all nonunion employees in a shared work environment with union employees). Such a study could demonstrate not only which vices are projected onto the union but also which are the most projected, leading to an evidence-based intervention, the object of which is to clarify the union’s agenda in plain emotional terms. Intriguing for additional sampling and scale use is the role of past union membership as a moderator of suggested relationships involving union intolerance. Questions about why past membership (vs. no past membership) brings into focus a positive relationship between union intolerance and self-reported negative social emotions, and why claims of past union discrimination intensify the inverse relationship between union intolerance and willingness to join a union (as seen in [1]), call for expanded empirical answers.

As we see it, the matter of the external validity of the union intolerance scale as an indicator of union intolerance beyond American nonunion employees is first and foremost a matter of content validity. As indicated, inductive methods were used to construct item content for the union intolerance scale. That is, effort was expended to capture elements of the construct as expressed by American nonunion employees, with such employees envisioned as representative employees. The question of whether the experience of American nonunion employees with unions is sufficiently similar to the experience of non-American nonunion employees with unions is an open question (see [42] for an example discussion on the question). At minimum, we think that a demonstration of external validity that includes samples of non-American nonunion employees should begin with an inductive analysis similar to the one conducted for the union intolerance items developed for American nonunion employees, in which extracted narratives are gathered to ensure that the items are both relevant to and representative of construct elements from the viewpoint of envisioned employees.

6. Conclusions

Our study results offer an expanded basis to claim construct validity for the union intolerance scale, adding evidence of convergent-discriminant validity and concurrent criterion-related validity. Additionally, in reference to concurrent validity, the scores yielded by the intolerance scale linking union intolerance to support for the PRO Act offer a demonstration of scale utility in modeling predictor–outcome relationships of importance to the success of American unions. Contingent on representative sampling of American nonunion employees and inductive item construction involving non-American nonunion employees, next to be added to the empirical basis to claim validity for the intolerance scale are demonstrations of external validity.