Social Desirability Bias and the Prevalence of Self-Reported Conservation Behaviour Among Farmers

Abstract

Agriculture is recognised globally as a major contributor to environmental degradation, habitat loss, and climate change. Having reliable data on the conservation behaviour of farmers is crucial to the evaluation of policy measures intended to reduce the harmful environmental effects of agriculture and promote sustainability. In responses to direct questions about conservation behaviours, the biasing of responses to appear more socially responsible has been found to be commonplace. From a policy perspective, the degree to which farmers might overstate the frequency with which they engage in behaviours that are desirable from a conservation perspective and understate the frequency with which they engage in behaviours that are undesirable because of social desirability bias is a matter of practical importance. In this paper we use, for the first time, crosswise questioning to investigate the influence of social desirability bias on self-reporting of conservation behaviour by farmers in New Zealand. We found that the effect of social desirability bias on self-reports of conservation behaviour by farmers in New Zealand is small. Consequently, self-reporting of conservation behaviour by farmers may be relied on when evaluating policies intended to promote sustainable development.

1. Introduction

Agriculture is recognised globally as a major contributor to environmental degradation, habitat loss, and climate change. Having reliable data on the conservation behaviour of farmers is crucial to the decades-long effort to understand the reasons why farmers do, or do not, adopt conservation behaviours and, consequently, the decades-long effort to develop policy measures intended to improve the sustainability of agriculture. However, the primary source of data on the conservation behaviour of farmers is their self-reporting of their behaviour in surveys [1,2,3,4,5]. This is because direct, continuous observation of farmers’ actual behaviour on any scale is, to put it mildly, impractical. In this context, the extent to which farmer reports of their behaviour are biased because they overstate the frequency with which they engage in behaviours that are desirable from a conservation or sustainability perspective (and understate the frequency with which they engage in behaviours that are undesirable) is a matter of importance.

The potential for bias in survey responses has long been recognised and, over the past two decades, a variety of questioning techniques have been developed to ameliorate a particular kind of bias: social desirability bias [6]. This bias arises when respondents are motivated to answer direct questions about their opinions or behaviour falsely to avoid revealing opinions, or behaviours, that they believe others (whose opinions they value) will find objectionable. If respondents feel that certain behaviours are strongly socially desirable (or undesirable), then securing reliable self-reported data on those behaviours using direct questioning becomes problematic. Respondents may overstate the frequency with which they engage in behaviours they perceive that others regard as desirable while understating the frequency with which they engage in behaviours that they perceive others regard as undesirable.

While the potential for social desirability bias to influence farmers’ reporting in surveys of their conservation behaviour is real, the extent of this bias has rarely been investigated [7,8]. In this paper, we quantify the influence of social desirability bias on farmers’ reporting of their conservation behaviour using a questioning technique designed to overcome social desirability bias. To the best of our knowledge, ours is the first ever application of the technique, crosswise questioning (CWT), to sustainable farming and the conservation behaviour of farmers. CWT has been applied in a range of fields, including racism [9,10,11], charitable behaviours [12], health behaviours [13], tax evasion [14], and behaviours relevant to conservation and sustainability such as illegal wildlife hunting [15].

We reasoned that, while farmers may be aware of the social desirability (or otherwise) of conservation practices such as fencing or clearing native vegetation, and fencing or draining wet areas, their behaviour is driven by the practical and commercial exigencies of farming. For farmers, these exigencies may justify engaging in practices that, from a sustainability perspective, appear socially undesirable. For example, while draining wet areas may be undesirable from an environmental perspective, it may be desirable from a production and stock management perspective. Our reasoning was that this implies that farmers will have a strong engagement with the adoption of environmental conservation practices that may impinge on farm production and costs [16,17,18,19] and that this will trigger extensive decision-making processes that lead to a carefully evaluated decision which, being so, will leave little scope for perceived social norms about sustainability, and therefore social desirability bias, to dominate reports of actual behaviour [20,21,22,23].

To summarise, the purpose of this study was to investigate the influence of social bias on farmers’ reports of their behaviour. Our hypothesis was that this influence would be limited, an implication being that farmer responses to direct questions (of the kind normally asked in surveys) about their conservation behaviour are substantially likely to be valid. Our intention was not to identify the causes of their actual conservation behaviour.

In the next section, we describe crosswise questioning and how it is used to estimate the prevalence of behaviours. We then describe our application of the technique to farming-related conservation behaviours and the report on the results. We then go on to discuss the implications of our findings.

2. Methods

Two questioning techniques have proved to be relatively popular in seeking to mitigate social desirability bias: the randomised response technique, of which there are several variants, and the crosswise questioning technique [6]. Recent research suggests that the results generated using the randomised response technique are unreliable because respondents fail to correctly observe instructions when questioned using this technique [6,9,24]. This has prompted, to a degree, interest in alternatives such as CWT which, in principle, is simpler to implement as it does not require a randomisation device [9]. Furthermore, unlike the randomised response technique, it does not offer a self-protective response strategy [10] with which respondents can mask the truth.

While CWT has received a lot of attention, most studies have analysed behaviour that was negatively connoted (that is, undesirable) and, using the principle of ‘more is better’ [12], have concluded that CWT is successful in reducing social desirability bias because it yields higher ‘prevalence estimates’ than direct questioning (DQ); that is, higher counts of reports of undesirable responses (prevalence estimates) are presumed to imply less desirability bias in, and thus greater validity of, the data. While there is the possibility that higher prevalence rates could simply be the result of systematic bias due to the framing of questions [10] or randomness in respondents’ answers [25], there are tests for detecting the influence of systematic bias due to framing, and randomness, in respondents’ answers on estimates of prevalence rates obtained using CWT [10,25]. The results of several comparative validation studies suggest that CWT outperforms competing approaches with respect to limiting the influence of social desirability bias in surveys [26].

We designed a questionnaire seeking information from farmers on a range of behaviours relating to sustainability and conservation, four of which directly related to farming. (see

Table 1. Conservation related behaviours.

Behaviour
Cleaned up litter in public space, park, or forest 1
Signed an online petition in support of protecting the environment 1
Taken part in hearings or consent processes about the environment 1
Fenced off wet areas on property 1
Fenced off land for native bush on property 1
Cleared any native trees or bush on property 2
Drained any wet areas on property 2

Notes: ¹ indicates socially desirable behaviour; ² indicates socially undesirable behaviour.

). We also included three conservation behaviours available to all members of the public in New Zealand to investigate whether the influence of social desirability bias varied between farm and non-farm conservation behaviours.

The farming behaviours we expected would be thought by farmers to be socially desirable from an environmental perspective were fencing native bush and wet areas to exclude livestock. These behaviours are recognised as critical to improving water quality and promoting biodiversity on agricultural land in New Zealand [27,28,29]. The behaviours that we thought farmers would view as socially undesirable from a conservation perspective were clearing native vegetation and draining wet areas.

The non-farm conservation behaviours were signing an online petition in support of protecting the environment; cleaning up litter in a public space, park, or forest; and participating in hearings or consent processes about the environment. (A consent process is a regulatory process that is used by central, regional, and local government to control, among other things, the use of natural resources in New Zealand [30]).

To begin with, we questioned farmers about their interest in, attitudes towards, and intentions and subjective norms regarding the act of conserving native vegetation (the relevant survey questions are reproduced in Supplement A). Farmers were asked directly (direct questioning—DQ) about the conservation-related behaviours. Finally, CWT was used to question farmers in relation to the conservation behaviours (see

Table 2. Examples of direct and crosswise questions.

Direct Question
Have you fenced off land for native bush on your property? ☐ Yes, all of it ☐ Yes, some of it ☐ No, but I plan too ☐ No and I don’t plan too ☐ I don’t have any native bush on my property
Crosswise question
Here are two questions…
1     Is your birthday in January (October)? a,*
2     Have you ever fenced off land for native bush on your property?
Are your answers to each question the same or different? If your answers are the SAME for both questions (both are YES or both are NO) then choose “A” If your answers are DIFFERENT (one is YES and the other is NO) then choose “B” ☐ A—my answers are the SAME for both questions ☐ B—my answers are DIFFERENT
Extended Crosswise question
Here are two questions…
1     Is your birthday sometime during the year from March to December)? a,*
2     Have you ever fenced off land for native bush on your property?
Are your answers to each question the same or different? If your answers are the SAME for both questions (both are YES or both are NO) then choose “A” If your answers are DIFFERENT (one is YES and the other is NO) then choose “B” ☐ A—my answers are the SAME for both questions ☐ B—my answers are DIFFERENT

Notes: ^a indicates non-sensitive attribute with known probability of occurrence. * Respondents were randomly allocated between January and October variants. Answers to the direct question were coded as 1 for ‘Yes, all of it’ or ‘Yes, some of it’, 0 otherwise. Answers to the crosswise questions were coded as 1 for A, 0 for B.

for example questions).

We also questioned farmers about their interest in, attitudes towards, and intentions and subjective norms regarding the act of conserving native vegetation (the relevant survey questions are reproduced in Supplement A).

An indicator of respondents’ interest in conserving native vegetation was formulated based on a scale developed by Laurent and Kapferer [31] to measure involvement with a subject. Respondents’ attitudes and intentions towards biodiversity conservation were based on the extent of their agreement with the following statements:

• They think that protecting native plants and wildlife is the right thing to do;
• They feel some responsibility for protecting native plants and wildlife;
• They would change their normal behaviour to protect native plants and wildlife;
• They were prepared to make sacrifices to protect the native plants and wildlife;
• They were willing to work with others to protect the native plants and wildlife.

We measured subjective norms regarding biodiversity conservation based on their agreement with the same statements but with respect to ‘nearly everyone they know’ and ‘nearly every farmer they know’. Respondents scored their agreement with involvement, attitude, intention, and subjective norm statements using a five-point rating ranging from strongly disagree (1) to strongly agree (5). Statements were randomised to avoid systematic bias in responses that may arise from the presentation order of statements. Significant differences in respondents’ assessments of their attitudes and intentions with respect to conserving biodiversity and the attitudes and intentions of other farmers and other people were identified using paired sample t-tests [32].

The survey was approved for distribution by Manaaki Whenua—Landcare Research’s social ethics process (application 2223/30) and the questionnaire was administered online to members of a commercial internet survey panel who were farmers. For each survey a member completed, a 10 NZD donation was made to a charity of their choosing. Surveying commenced on 11 March 2024, following piloting with a small sample of farmers (n = 30) and closed on 5 April 2024, when the required number of responses (n = 350) had been obtained. Note that respondents self-selected based on their interest in completing the survey. The size of the sample was based on the definition by Ibbett et al. [7] of a medium sample size for behaviours that are thought to be relatively common (that is, not rare).

To test our hypothesis that social desirability bias would have a limited influence on farmers’ answers to questions about farm-related environmental behaviours, we estimated prevalence rates for each of the farm-related behaviours using direct questioning (DQ) and the crosswise technique (CWT). Prevalence rates using DQ were estimated simply as the proportion of respondents indicating that they had engaged fully or partly in the behaviour. Prevalence rates for CWT were estimated as follows [33]:

(1)

Pr(A) = λ = (1 − π) · (1 − p) + π · p

(2)

π = (λ + p − 1)/(2 · p − 1)

where Pr(A) is the probability that the same answer apples to both statements, p is the known probability that the answer to the non-sensitive statement (i.e., not subject to normative judgement) is affirmative, and π is the estimated probability that the answer to the sensitive statement is affirmative. We computed 95% confidence intervals for both the DQ and CWT prevalence estimates to identify any statistically significant differences between them using the formulae provided by Yu et al. [33]. To test for systematic preference effects or non-adherence to the instructions regarding CWT questions, we implemented a modified CWT (the ‘extended CWT’ model) proposed by Meisters et al. [10]. This was carried out in two ways: first, by partitioning the sample so that respondents were offered two different versions of the CWT question regarding, for example, fencing native vegetation, and second, by randomly offering two different versions of the non-sensitive attribute (see Table 2). We then tested for statistically significant differences in estimated prevalence rates across the different versions of the CWT questions using the formula provided by Yu et al. [33].

Using the formulae provided by Walzenbach and Hinz [25] to estimate the proportion of respondents that would need to answer randomly to generate the observed CWT prevalence rate, we also tested for the influence of random responses to the CWT questions. We hypothesised that respondents who answered the CWT questions randomly would do so for all the behaviours, which implies that the proportion of respondents who answered randomly would be similar across the behaviours (given that the ordering of CWT questions was randomised). This implies that the proportion of random responses required to generate the observed difference between the DQ and CWT prevalence rates should be similar across the behaviours. Consequently, if the proportion of random responses required to generate the observed difference between the DQ and CWT prevalence rates differs across behaviours, the difference in these rates is unlikely to be attributable to randomness in respondents’ answers to CWT questions.

Statistical analyses were conducted using IBM SPSS Statistics v28, Windows [34].

3. Results

We found that respondents had moderate interest in, strongly favourable attitudes towards, and moderately strong intentions with respect to protecting native plants and wildlife. Respondents believed that their attitudes, and the attitudes of other farmers, towards conserving biodiversity were significantly more favourable than the attitudes of most people they knew (see

Table 3. Attitudes, intentions, and subjective norms.

Item (1)	Item (2)	Mean Difference (1)–(2)	Paired t-Test
Own attitude towards protecting native plants and wildlife	Other farmer’s attitude towards protecting native plants and wildlife	0.30	7.18, p < 0.001
Own attitude towards protecting native plants and wildlife	Other people’s attitude towards protecting native plants and wildlife	0.52	15.66, p < 0.001
Other farmers’ attitudes towards protecting native plants and wildlife	Other people’s attitude towards protecting native plants and wildlife	0.40	9.14, p < 0.001
Own intention to protect native plants and wildlife	Other farmers’ intentions to protect native plants and wildlife	0.07	1.93, p = 0.05
Own intention to protect native plants and wildlife	Other people’s intentions to protect native plants and wildlife	0.52	14.53, p < 0.001
Other farmers’ intentions to protect native plants and wildlife	Other people’s intentions to protect native plants and wildlife	0.45	13.52, p < 0.001

Notes: Statistically significant differences in involvement were identified using paired samples t-test [32].

). Respondents also believed that their intentions, and the intentions of other farmers they knew, towards conservating biodiversity were significantly stronger than those of most people they knew (see Table 3).

The estimated prevalence of the behaviours using DQ and CWT is reported in

Table 4. Estimated prevalence of conservation behaviours (DQ and CWT).

	Questioning Technique
Behaviour	Direct	Crosswise
Cleaned up litter in public space, park, or forest a	53.6	53.8
Signed an online petition in support of protecting the environment a	5.2	16.9 *
Taken part in hearings or consent processes about the environment a	23.2	29.9 *
Fenced off wet areas on property a	77.1	67.1 *
Fenced off land for native bush on property a	58.3	65.5 *
Cleared any native trees or bush on property b	11.9	27.6 *
Drained any wet areas on property b	28.2	38.8 *

Notes: Values are percentage of the sample. ^a indicates anticipated to be a desirable behaviour; ^b indicates anticipated to be an undesirable behaviour; * indicates DQ and CWT proportions statistically significantly different (p < 0.05).

. The results are somewhat mixed. The estimated prevalence of cleaning up litter in a public space, park, or forest was similar for the DQ and CWT. However, the estimated prevalence using DQ for signing an online petition in support of protecting the environment, and for taking part in hearings or consent processes about the environment, were lower than the estimated prevalence using CWT. We expected the opposite result on the basis that these behaviours would be presumed by farmers to be socially desirable by the wider community.

The results for the primarily farm-related behaviours were as expected, with the estimated prevalence of (desirable) fencing off wet areas being higher using DQ compared to CWT, and the estimated prevalence of (undesirable) draining wet areas and clearing native vegetation being lower using DQ compared to CWT. However, the estimated prevalence using DQ for fencing off land for native bush was lower than the estimated prevalence using CWT. We expected the opposite result on the basis that this behaviour would be expected to be regarded as socially desirable by the wider community.

The opportunity to engage in, and therefore the prevalence of, farm-related behaviours will depend on possibly unique mixes of contextual factors such as the presence of native bush, waterways, or wet areas on properties. Proceeding on the assumption that respondents truthfully replied to questioning about the presence of native bush, waterways, or wet areas on their properties, we adjusted the prevalence estimates for the farm-related behaviours accordingly. The resulting estimates (see

Table 5. Adjusted estimated prevalence of conservation behaviours (DQ and CWT).

	Questioning Technique
Behaviour	Direct	Crosswise
Fenced off wet areas on property a	93.3	77.8 *
Fenced off land for native bush on property a	86.5	85.2
Cleared any native trees or bush on property b	16.0	32.5 *
Drained any wet areas on property b	33.8	46.4 *

Notes: Values are percentage of the sample. ^a indicates anticipated to be a desirable behaviour; ^b indicates anticipated to be an undesirable behaviour; * indicates DQ and CWT proportions statistically significantly different (p < 0.05).

) are consistent with our expectations, with the estimated prevalence of (desirable) fencing-off of wet areas and native vegetation being higher using DQ compared to CWT, and the estimated prevalence of (undesirable) draining wet areas and (undesirable) clearing native vegetation being lower using DQ compared to CWT.

In Figure 1, we present differences in the estimated prevalence rates using DQ and CWT for fencing bush and wet areas, for clearing bush, and for draining wet areas, partitioned according to respondents’ interest in protecting native plants and wildlife (and adjusted for farm context). Given that most respondents had a neutral-to-favourable attitude towards protecting native plants and wildlife, we expected the difference between the DQ and CWT estimates would be least among respondents with mild interest in protecting native plants and wildlife and greatest among those with high interest in protecting native plants and wildlife. Our assumption here was that those with least interest in conservation issues would be the least influenced by social desirability bias.

Inspection of Figure 1 reveals that the difference between DQ and CWT estimates of prevalence regarding draining wet areas, fencing native vegetation, and clearing native vegetation is less for respondents with mild interest than for respondents with greater interest in protecting native plants and wildlife. The results are mixed regarding fencing wet areas, with the difference in prevalence estimates being similar for respondents with mild and moderate interest and smaller for respondents with high interest. The difference between DQ and the extended CWT estimates of prevalence for fencing native vegetation was greatest among respondents with mild interest in protecting native plants and wildlife.

In

Table 6. Estimated prevalence of conservation behaviours for different framings of non-sensitive attribute.

	Birth Month January			Birth Month October
Behaviour	DQ	CWT	Difference	DQ	CWT	Difference
Cleaned up litter in public space, park, or forest a	56.5	55.4	1.1	50.3	52.1	1.8
Signed an online petition in support of protecting the environment a	7.3	17.7	10.4	2.9 d	16.2	13.3
Taken part in hearings or consent processes about the environment a	24.1	34.6	10.5	22.2	24.6 e	2.4
Fenced off wet areas on property a	96.2	82.0	14.2	90.6 d	72.9 e	17.7
Fenced off land for native bush on property a	88.0	86.6 f	1.4	85.5	83.8	1.7
Fenced off land for native bush on property a,c	88.0	76.5 f	11.5	85.5	78.6	6.9
Cleared any native trees or bush on property b	17.5	37.1	19.6	14.4	27.5	13.1
Drained any wet areas on property b	40.3	52.8	12.5	31.8	41.8 e	10.0

Notes: Values are percentage of the sample. ^a indicates anticipated to be a desirable behaviour; ^b indicates anticipated to be an undesirable behaviour; ^c indicates extended CWT derived from Meisters et al. [4]; ^d indicates statistically significant difference in direct question estimates of prevalence for January and October framings (p < 0.05); ^e indicates statistically significant difference in crosswise estimates of prevalence for January and October framing (p < 0.05); ^f indicates statistically significant difference between CWT and extended CWT estimates of prevalence for January framing (p < 0.05).

, we present estimated prevalence rates using two approaches to the extended CWT model. In the first approach, we compare prevalence estimates for respondents with different birth months. In the second approach we compare prevalence estimates using a different framing of the non-sensitive attribute. Again, the estimates for the farm-related behaviours have been adjusted for farm context. The difference between the birth months in the CWT estimates of prevalence were only statistically significant for participating in consents, fencing wet areas, and draining wet areas. The differences between the birth months in the estimated prevalence rates for fencing and draining wet areas appears parallel with the differences in the estimated prevalence rates using DQ (see the final column in Table 6), suggesting that the differences in prevalence rates are not attributable to the questioning technique. Overall, the results indicate the CWT procedure generated reasonably consistent estimates of the prevalence of both desirable and undesirable behaviours irrespective of birth month.

The prevalence estimates for fencing native vegetation using the two different framings of the non-sensitive attribute (fifth and sixth rows of Table 6) were not statistically significantly different, indicating consistent estimates of the prevalence of both desirable and undesirable behaviours irrespective of the framing of the non-sensitive attribute. Note that the difference in prevalence estimates for fencing native vegetation comparing the two different framings for the non-sensitive attribute was not statistically significantly different for respondents who answered the CWT questions using October birth month but was (though barely) statistically significantly different for respondents who answered the CWT questions for the January birth month.

Finally, we estimated the proportion of respondents that would be needed to answer the CWT questions randomly to generate the difference in the estimated prevalence rates using DQ and CWT for each behaviour. We had hypothesised that respondents who answered the CWT questions randomly would do so for all the behaviours, which implies that the proportion of respondents who answered randomly would be similar across the behaviours. We found that the percentage of random responses needed varied from −6.6 for picking up litter through to 77.8 for draining wet areas (see

Table 7. Estimated prevalence of potential random responses to CWT.

Behaviour	DQ	CWT	Proportion Random Responses Required d
Cleaned up litter in public space, park, or forest a	53.6	53.8	−6.6
Signed an online petition in support of protecting the environment a	5.2	16.9 *	26.1
Taken part in hearings or consent processes about the environment a	23.2	29.9 *	24.8
Fenced off wet areas on property a	93.3	77.8 *	35.8
Fenced off land for native bush on property a	86.5	85.3	3.4
Fenced off land for native bush on property a,c	86.5	77.4 *	25.9
Cleared any native trees or bush on property b	16.0	32.5 *	48.5
Drained any wet areas on property b	33.8	46.4 *	77.8

Notes: Values are percentage of the sample. ^a indicates anticipated to be a desirable behaviour; ^b indicates anticipated to be an undesirable behaviour; ^c indicates extended CWT derived from Meisters et al. [4]; ^d proportion of respondents that would have had to answer randomly to obtain estimated CWT prevalence rate; * indicates statistically significant difference between DQ and CWT estimates of prevalence (p < 0.05).

). This result suggests that random responses did not consistently affect the difference between the DQ and CWT estimates of the prevalence of behaviours.

4. Discussion

Several implications follow from our results: First, our results indicate that the influence of social desirability bias on farmers’ answers to questions about farm-related environmental behaviours (at least those that we considered) is limited. This indicates that the DQ method provides reasonably reliable estimates of the prevalence of behaviours. This is consistent with our supposition that while farmers may be aware of the social desirability (or otherwise) of farm practices such as fencing or clearing native vegetation and fencing or draining wet areas, their behaviour is deliberate and purposive, driven by the practical and commercial exigencies of farming (for example, see [16,17,18,19]). The latter may justify engaging in farm practices that, from a sustainability perspective, may appear socially undesirable. For example, while draining wet areas may be believed by the farmer to be, and to be thought generally to be, personally undesirable from an environmental perspective, it may be judged as more desirable from a production and stock management perspective [35].

Having reliable data on the conservation behaviour of farmers is crucial to understanding the reasons why farmers do, or do not, adopt conservation behaviours, and to the development and evaluation of policy measures intended to improve the sustainability of agriculture. Since the primary source of data on the conservation behaviour of farmers is through surveys, the fact that we found that social desirability bias had a limited influence on farmers’ reporting of their conservation behaviour is reassuring. However, our results differ markedly from those of Moore and Rutherfurd [8], who found that 60% of farmers in northern Victoria, Australia inaccurately reported that they prevented cattle from grazing riverbanks. These estimates were based on comparisons of farmer reports with visual evidence of cattle access such as hoof marks, eaten vegetation, and the presence of cows. The reasons for the differences in the findings of the two studies are unclear. On the one hand, the inaccuracies found by Moore and Rutherfurd [8] included over-reporting as well as under-reporting; on the other, it is not clear that their survey question referred to permanently preventing stock from grazing riverbanks, while our questions did, referring specifically to capital work such as fencing and clearing rather than a management practice (i.e., grazing).

The second implication is that we did find, contrary to our expectations, that the estimates of prevalence for signing online petitions and for participating in hearings and consenting processes was lower for DQ than for CWT. These results raise the possibility that some farmers in the sample might privately regard these behaviours as desirable but believe that their peers regard these behaviours as socially undesirable, and therefore are less likely to admit to the behaviour when questioned directly. Alternatively, some farmers in the sample may regard these behaviours as undesirable but have engaged in them because of social pressures. Respondents may be unwilling to admit to acquiescing to social pressures and so may under-report in response to direct questioning. Relatedly, these results suggest that CWT studies could be improved by including questions regarding participants’ attitudes [11] and subjective norms regarding the specific behaviours of interest where subjective norms (and therefore social desirability) may vary among groups within the population.

Third, the results for the extended CWT indicated that the framing of CWT questions did not appear to influence farmers’ answers to those questions, suggesting that question framing did not create systematic bias in responses [10]. This is consistent with our hypothesis that while farmers may be aware of the social desirability of farm practices, they also know that their behaviour is driven by the practical and commercial exigencies of farming and thus will be less susceptible to that bias. This is an area for further research to check the reliability and generalisability of our results.

Fourth, unlike Walzenbach and Hinz [25] we found evidence indicating that prevalence estimates using the CWT were not the product of random responses to CWT questions. This suggests that the farmers in our sample were sufficiently engaged with reporting on their conservation behaviour to devote the cognitive effort required to comply with the instructions regarding CWT questions. This seems consistent with farmers having strong interest (involvement) in the adoption of practices that may impinge on farm production and costs [16,17]. This implies that random responses to CWT questions may reflect disinterest in the question topic rather than the apparent cognitive capacity of respondents as indicated by their educational background [10]. It also implies that disinterest—that is, low interest—in the topic may lead to false positive responses [13,36]. If this is the case then providing more detailed instructions to respondents may increase, rather than decrease, the rate of false positive responses. It is relevant to observe here that there is no reason to expect that highly sensitive topics are also necessarily highly involving.

The potential for interest in a topic to influence respondents’ propensity to fail to comply with questionnaire instructions and answer cognitively demanding questions randomly, and for this effect to be exacerbated by providing more detailed instructions, raises concerns about the reliability of randomised response and crosswise techniques and other complex questioning techniques such as those employed in contingent valuation studies. We obtained mixed results regarding our expectation that farmers with a low level of interest, as measured by involvement, in protecting native plants and wildlife would be influenced less by social desirability bias than farmers with a greater interest. The difficulty here is that, in the absence of information on relevant characteristics of the agricultural production context, the extent to which interest in conserving biodiversity is relevant is difficult to ascertain. However, the results do point to the possibility that a respondent’s interest with a topic may moderate the influence of attitudes and subjective norms on their behaviour and so their susceptibility to social desirability bias. This is an important area for further research.

Finally, we note some limitations of our research. First, the present study is based on a relatively small sample of farmers, so the results may not generalise to the broader farming population. Second, we were unable to identify a reliable source of external validation data to evaluate the accuracy of the respondents’ answers. Third, we did not gather data on respondents’ attitudes and subjective norms with respect to the specific behaviours covered in the crosswise questions. Consequently, we were unable to use estimation techniques such as those described by Jann et al. [37] to indirectly assess the validity of our results. This is also an important area for further research.

5. Conclusions

We investigated the effect of social desirability bias on farmers’ self-reported behaviour related to environmental conservation practices. Based on a comparison of DQ and CWT prevalence estimates, the effect of social desirability bias on self-reports of conservation behaviour among farmers in New Zealand appears to be relatively small. This is consistent with our hypothesis that, while farmers may be aware of the social desirability (or otherwise) of farm practices, their behaviour is driven by the practical and commercial exigencies of farming. For farmers, these commercial and practical exigencies justify engaging in practices that, from a sustainability perspective, may appear socially undesirable. The negative consequences, for output quantity or quality and costs of production, are effectively perceived by farmers to comprise too great an opportunity cost in business activity that is central to their sense of identity. Consequently, they are not overly susceptible to social desirability bias when reporting on their conservation behaviour (or the lack of it), which suggests that self-reporting of conservation behaviour by farmers in surveys is unlikely to be wildly inaccurate and so may be relied on when formulating, and evaluating, policies intended to promote sustainability.