An Ex-Post Evaluation of Agricultural Extension Programs for Reducing Fertilizer Input in Shaanxi, China

Abstract

It is difficult to control agricultural Non-point Source (NPS) pollution, caused by overusing of chemical fertilizer, through laws alone. An agricultural extension program is an alternative way to reduce fertilizer input. This paper estimates the average change in fertilizer use of farmers, which is attributable to their participation in the agricultural extension program. The Inverse-Probability Weighted matching method is used to evaluate the average treatment effect for the farmers who engaged in the program and, to a limited extent, evaluate the impact of individual elements of the program. Surprisingly, we find that farmer-to-farmer training may have had a counterproductive effect while the posters may be the most effective program element in reducing fertilizer use in the context.

1. Introduction

Overuse of fertilizers leads to agricultural Non-point Source (NPS) pollution. Nitrogen (N) and phosphorus that cannot be absorbed by crops leave the farm into underground water, lakes and rivers. This can result in significant environmental and human health impacts [1,2,3]. Agriculture contributes 0.76 million tons of NH3-N, accounting for 31.7% of the annual total emission to China’s waters [4]. Since 1999, various national laws and rules have been passed with the goal of reducing agricultural NPS pollution. However, because of challenges associated with regulation and enforcement, it is difficult to control agricultural NPS pollution through laws alone. Researchers have pointed out that one important reason for widespread NPS pollution in China is that farmers have limited knowledge of “rational fertilization”, which refers to the proper balance between yield gain and fertilizer use [5].

A change in farmers’ fertilizer decisions is essential to the reduction of agricultural NPS pollution in China. Researchers have found that Chinese farmers are more likely to use environmentally friendly methods if they have received training or guidance of agricultural technologies and practices [6,7]. Effective agricultural extension program is an attractive and alternative way to reduce fertilizer input. The effectiveness of an education and extension program depends on whether the information from the program induces a positive change in farmers’ behavior. Measuring the benefit or effectiveness of such education or extension programs is a challenge. There are many compounding factors affecting farmers’ behavior; separating these factors and attributing the specific benefit to a particular extension program is difficult. Yet, proper evaluation of these programs is important in guiding the design of these programs and in the allocation of scarce resources. Here we propose to use matching methods to estimate the effects of one such program.

In the seminal paper on propensity score analysis, Rosenbaum and Rubin presented both the theory and application principles for a variety of propensity score models and proposed propensity score matching as a way to reduce the bias in estimates of treatment effect when using observational data [8]. Matching methods have increasingly been used to estimate treatment effect in a wide range of programs targeting farmers. Godtland et al. used Propensity Score Matching (PSM) methods to examine the impact of a pilot program on farmers’ knowledge from a survey of potato farmers in Cajamarca, Peru, finding that the program had the potential to raise productivity by about 32% of the average value in a normal year [9]. Mezzatesta et al. used PSM to estimate how much additional conservation is achieved by federal cost-share programs in the U.S. by looking at six types of conservation practices. They found that cost-share programs achieve statistically significant levels of additionality for each practice, but that there was significant variation in the levels of additionality across the different practices [10]. Woodward et al. used PSM to estimate the extent to which a water quality trading program affected farmers’ adoption of conservation practices [11].

To the best of our knowledge, matching methods have not been widely used to evaluate the effects of agricultural extension programs for controlling agricultural NPS pollution. This is especially true in China. However, matching on covariates is well suited for the analysis on the effects of agricultural extension program if the number of characteristics over which participant and non-participant differ is limited and the differences can be represented by categorical variables [12]. Several studies have used matching methods to measure treatment effect in other Chinese contexts. Wu et al. evaluated the effects of improved upland rice technology on rice farmers’ income in southwestern Yunnan province by using the non-parametric PSM method [13]. Wang et al. estimated the treatment effect of vocational training in rural Guangdong province [14]. Zhang and Wang estimated the effects of job training on rural laborers’ earnings by using PSM method [15].

In this paper, we use the Inverse-Probability Weighted (IPW) matching method to evaluate effects of an agricultural extension program focused on the reduction of farmers’ fertilizer use. Our paper uses a 2011 survey of farmers in Shaanxi province in China to measure their change in fertilizer usage and participation in the China-UK program. By using the matching method, we are able to control for farmers’ participation decisions to obtain an unbiased comparison between the fertilizer use of the participating farmers and those who did not participate. We are unable to find statically significant evidence that the China-UK program was reduced farmers’ fertilizer use. We do find some evidence that one aspect of the program, farmer to farmer training, may have actually been counterproductive, while the posters may be the most effective program element.

2. Materials and Methods

2.1. Introduction of Agricultural Extension Program

In this paper, we focus on a program that was implemented in Shaanxi Province called “Improving Livelihoods on Farms by Reducing Non-point N Pollution through Improved Nutrient Management”. It was funded by both the Foreign and Commonwealth Office of the United Kingdom and Ministry of Agriculture of the People’s Republic of China from January 2007 to December 2009 (This program was led by Dr. Tong Yanan from Northwest A&F University in Shaanxi Province of China and Dr. David Powlson from Rothamsted Research in UK.). Hereafter we will refer this as the China-UK program. Program participants included agriculture scientists, local extension workers, farmers associations and individual farmers. The objective of the China-UK program was to provide farmers information that will enable them to avoid overuse of N fertilizer without compromising their crop yields and incomes. The program sought to reduce environmental pollution while at the same time improving income of the farmers.

The China-UK program consisted of the following parts: (1) An assessment of farmer and community perceptions to understand reasons for current N overuse; (2) Collection and analysis of relevant technical data on rates and timing of N fertilizer application relevant to the local environment (climate, soils, and cropping systems) of Shaanxi Province; (3) Farm-based experiments using the results from step 2 to measure how fertilizer changes can affect farm productivity and profitability; (4) Using results from points 2 and 3 above, to develop information delivery programs that inform farmers of the benefits of rationalized N use; and (5) Analysis of information delivery programs will be tested in collaboration with farmers. This paper is not intended to evaluate the whole program. Instead we focus on assessing the success or failure of the information delivery system (Step 4) of the program, and further evaluate which component of information dissemination is more effective.

Table 1. Changes of the farm based experiment for winter wheat and summer maize in 2010.

	Village	Fertilizer Changes	Yield Changes	Input Cost Changes *	Revenue Change *	Total Profit Changes *
Unit		Kg/ha	Kg/ha	Dollars/ha	Dollars/ha	Dollars/ha
wheat	Village 1	−70	+185	−43.4	+47.1	+90.6
	Village 2	−18	+117	−11.2	+31.3	+42.5
	Village 3	−22	−39	−13.6	−9.9	+3.7
	Average	−37	+88	−22.7	+22.8	+45.6
maize	Village 1	−167	+208	−103.3	+44.3	+147.6
	Village 2	−155	+298	−95.6	+63.4	+159.0
	Village 3	−160	+261	−98.4	+55.6	+154.0
	Average	−161	+256	−99.1	+54.4	+153.5

Data Source: Zhang Shulan, Report in 2010 conference of China-UK program [16]. * In the experiment results, the input cost, revenue and profit changes were calculated with unit Yuan. Here, we changed the unit to Dollars with average 2010 exchange rates for Yuan (7.05¥/$).

summarizes the evidence gathered by the China-UK program in Step 3, the experimental analysis of the effect on N application and crop yields. The China-UK program conducted farm-based experiments for winter wheat and summer maize in three villages. The content of the experiments was as follows. Selected farmers changed their fertilizer input rates according to the program recommendation, while holding other farming practices unchanged. It should be emphasized, that unlike later voluntary participants discussed below, farmers in this experimental phase received subsidies for participation and were compensated if they suffered a yield loss due to their participation in this program.

As seen in Table 1, the average fertilizer input for both wheat and maize decreased in all three villages. Despite the N reduction, except for wheat in Village 3, yields increased for both crops. Notably, the reduction of fertilizer application for maize is about 72% of past application rates. Despite decreased fertilizer use, profits increased in all three villages for both wheat and maize. Even in Village 3, famers’ savings from reduced fertilizer input for wheat were more than made up for the loss in yield. The average increase of profits in three villages was 45.6 Dollars/ha for wheat and 153.5 Dollars/ha for maize.

The experimental results from these three villages suggest that there was a great potential to reduce farmers’ fertilizer use and agricultural NPS pollution without economic losses. Hence, a naïve prediction would be that the program’s goal of changing farmer behavior could be achieved through an agricultural extension program in which farmers were informed of the experimental results. The remainder of the paper attempts to estimate whether the China-UK’s extension activities led to the desired behavioral changes.

2.2. Data Description

In this paper, we use data from 450 face-to-face surveys carried out from May to October 2011, in nine villages located in Yangling District, Wugong County and Jinyang County of Shaanxi Province. Farmers in each village were randomly selected for interviews. Only two of the villages were sites of the China-UK program, but information about the program was spread to other villages by radio or word of mouth among villagers. The surveys were implemented by 20 graduate students majoring in Agricultural Economics and Management at Northwest A&F University.

In the survey area, winter wheat and summer maize were the main crops, while some farmers also planted kiwis, tomatoes and other vegetables. The average family had 5.3 members with 2.5 participating in farming. Farmers were mostly middle age (average age of 52) and only 18% of farmers’ education level was above high school. The average field was small, only 0.3 ha. There was substantial variation of farmers’ reported annual net profit, ranging from 141.8 Dollars and over 7092.2 Dollars. The average N fertilizer input for the whole year 2010 from our survey data was 151.7 kg/ha.

2.3. Methodology

In this paper, we evaluate the effects of the agricultural extension program on farmers’ fertilizer using. We evaluate the treatment effect using the potential-outcomes framework, also called the counterfactual framework.

Our outcome variable of interest is fertilizer use per hectare. Farmers in our survey are divided into two groups, group I and group J. Farmers who received a treatment are in group I, and those who did not are in group J. Define an indicator variable, D, which equals one if a farmer received a treatment (i.e., participated in one or more components of the China-UK programs) and zero if not. Further, define the outcome variables Y for each farmer, fertilizer usage in our case. We will sometimes write $Y^i\left(D=1\right)$ to emphasize that the ith farmer is defined as a farmer who had received a treatment.

There are two possible outcomes of fertilizer use for each farmer: the fertilizer input for the farmer if he or she received the treatment, Y₁, and Y₀ for the case in which the farmer did not receive the treatment. The program’s average treatment effect on the treated (ATET) is the difference between Y₁ and Y₀ for the set of treated farmers:

$$ATET=E[Y_1^i-Y_0^i]=E[Y_1^i-Y_0^i|D=1]=E[Y_1^i|D=1]-E[Y_0^i|D=1].$$

(1)

That is, the treatment effect of the agricultural extension program is the difference between the fertilizer use by treated farmers, $E[Y_1^i|D=1]$, and what they would have used without participating, $E[Y_0^i|D=1]$. Note that the condition D = 1 appears throughout Equation (1) to emphasize that the ATET is sought for the subset of participating farmers.

Unfortunately, because our survey data were obtained at only one time, after the China-UK program had already been implemented, we only observe Y₁ for the treated farmers and Y₀ for the control group. To estimate the ATET, therefore, we must estimate $E[Y_0^i|D=1]$, what the treated farmers would have done if the China-UK program was not available to them. It is not acceptable, however, to simply use the difference between the sample means for the treated and untreated subjects to estimate ATET. Because participation is voluntary, there are typically systematic differences in these two groups. Hence, the average choice of the non-treated group is a biased indicator of the choices that the treated group would make when they are unable to participate in the China-UK program. Treatment-effect estimators seek to obtain unbiased estimates of ATET.

Two conditions must be satisfied for matching estimators to yield unbiased estimates of the ATET. The first condition, also called the unconfoundedness assumption, was proposed by Heckman et al. [17]. If unconfoundedness is satisfied, then conditional on the set of observable covariates, the outcome without treatment, Y₀ is independent of treatment, i.e., $Y_0\perp D|X$. In other words, this condition is satisfied if, once we have taken into account the covariates X, what farmers would have done in the absence of the China-UK program is independent of whether the individual is in the treated or untreated group. Rosenbaum and Rubin showed that one can use the individuals’ propensity scores, i.e., $P(D=1|X)$, as the conditioning statistic in order to calculate the ATET [6]. In this paper, the propensity score is our estimation of the probability that a farmer participates in the China-UK program as a function of a vector of covariates, X.

The second required condition is that the data set has sufficient overlap. This means that for both treated and control farmers the propensity score is neither zero nor one, $0<P(D=1|X)<1$. If satisfied, then for each treated farmer there is a positive probability of finding a non-treated farmer with essentially the same propensity to be treated. In practice, the overlap condition is imposed by dropping those observations for which it is not satisfied. That is, if for some farmers there are no untreated farmers with a similar set of covariates, then those farmers would be dropped from the data set and we would simply have to accept that we can not estimate the ATET for that group.

The matching estimators for $E[Y_1^i|D=1]$ and $E[Y_0^i|D=1]$ in Equation (1) are:

$$E[Y_1^i|D=1]=\frac{1}{I}{\displaystyle \sum _i(Y_1^i)},$$

(2)

$$E[Y_0^i|D=1]=\frac{1}{I}{\displaystyle \sum _i(Y_0^i)=\frac{1}{I}{\displaystyle \sum _i}}{\displaystyle \sum _{j}W(i,j)(Y_0^j)},$$

(3)

where $Y_0^j$ is the observed outcome for the farmers who did not receive treatment, W(i, j) is the weights given to the jth non-treated farmer used for estimating the ith farmer’s fertilizer use if he or she had not treated. In the Inverse Probability Weighting approach as implemented in Stata Statistical Software 14.0 [18], the weights W(i, j) vary inversely with the difference between the propensity score for the ith treated observation and the jth control observation. In this way, $Y_0^i$ is estimated placing more weights control observations that are most similar to the ith observation. Using (2) and (3), the matching estimator for the ATET in Equation (1) is

$$\stackrel{\wedge }{{\displaystyle ATET}}=E[Y_1^i|D=1]-E[Y_0^i|D=1]=\frac{1}{I}{\displaystyle \sum _i\left\{(Y_1^i-{\displaystyle \sum _{j}W(i,j)(Y_0^i)}\right\}.}$$

(4)

There are a variety of matching algorithms that can be used to calculate W(i, j) including Inverse-probability weighted (IPW) and Nearest-Neighbor matching (NNM) (We use the NNM method to check the robustness of our estimation results in the Appendix A.) using either the propensity scores or a nonparametric Mahanbolis. In this paper, we use the IPW algorithm to evaluate the effects of the agricultural extension program on farmers’ fertilizer usage. The IPW method involves three-steps. First, a Logit or Probit model is used to estimate the probability, conditional on a set of covariates, that each observation receives the treatment; Second, using the estimated probabilities, the propensity scores, weights are calculated for each observation. The weights W (i, j) are based on the inverse of the difference between the propensity score of the ith treated and the jth control. Using these weights, a pseudo-control data set is created in which each non-treated observation is multiplied by its weight. Balancing is tested by confirming that the weighted distributions of the covariates of the treated and pseudo-control sets are nearly the same. Third, once balancing has been established, the average outcome for the pseudo-control group is used to estimate Y₀, which can then be subtracted from Y₁ to calculate the estimate of ATET (While both Heckman and PSM might control for selection bias, the PSM method mitigates selection bias due to observables and the 2SLS Heckman method addresses selection bias due to unobservables [12]. Our balance tests in the later section found a good match of the treated and control farmers, supporting the use of PSM and the inferences we obtain. A key advantage for us in the PSM approach for this application is that we are interested in estimating the average treatment effect on the treated (ATET), while the Heckman method estimates average treatment effect for all farmers (ATE).).

3. Results

3.1. Outcome and Independent Variables

The outcome variable of interest is the farmers’ fertilizer input (kg/ha), which was obtained from our survey. The treatment variable is farmers’ participation in the China-UK program. There were five different ways for farmers to participate in China-UK program. The farmer field schools delivered information on proper fertilization and taught planting technology to farmers in the field. Demonstration zones showed the experimental results and details of fertilizer input in the demonstration area located in the village to farmers. Farmer meetings gathered farmers together to deliver information about fertilizer applications by scientists of the program. Farmer to farmer training consisted of smaller meetings at which the speaker is a farmer who has experience with reduced fertilizer use. Finally, posters contained information of appropriate amount of fertilizer input and were pasted on walls beside the street in the villages where they were quite visible to farmers.

The survey asked farmers if they had received each of the treatments.

Table 2. Participation in different components of China-UK program.

Program	Participation Number % of Sample
Components of China-UK program
Farmer Field School	16 3.5%
Demonstration zone and Farmers Viewing	12 2.6%
Farmer Meeting	11 2.4%
Farmer to Farmer training	17 3.7%
Poster, leaflet for reducing fertilizer use	12 2.6%
Participation rates
Participation in one or more components	45 10.0%
Participation in two or more components	17 3.7%
Participation in three or more components	4 0.8%

reports farmers’ participation in different components of the China-UK program. The participation rate for each treatment is relatively low; only 10% of the survey respondents reported receiving one or more of the program’s components. Farmer to farmer training, Demonstration zones and poster were the most popular ways for farmers to participate in the China-UK program.

While our primary focus is on the effects of the whole program on farmers’ fertilizer input, it is also interesting to evaluate the effects of each element of the program. Unfortunately, we do not have enough observations for each element to carry out such analysis. Instead, as discussed below, we drop one component of the China-UK program at a time to obtain an indication of which elements of the program had the greatest impact on fertilizer use.

In

Table 3. Average fertilizer input of treated and untreated farmers participating in different components of China-UK program.

Participation Component		Treated Group (kg/ha)		Untreated Group (kg/ha)
		Fertilizer Input	Std. Dev.	Fertilizer Input	Std. Dev.
Any one or more components of China-UK program		318.82	196.80	293.93	215.08
Any component	except farm field school	336.14	197.82	292.70	214.61
	except demonstration zone	322.67	204.76	293.85	214.26
	except farmer meeting	332.89	203.37	293.08	214.22
	except farmer to farmer training	303.95	198.14	295.42	214.96
	except poster	321.75	198.21	294.06	214.71

, we report a naïve measure of the treatment effect, the simple difference of average fertilizer input per hectare between the treated and untreated farmers. As we can see, the simple differences suggest that the China-UK program may have actually increased the fertilizer input. However, this maybe a result of endogenous selection—perhaps farmers who participated were those already tending to use a lot of fertilizer—so the simple difference in means would result in biased estimates. Matching methods offer a way to estimate the treatment effect that controls for the selection bias created by the fact of voluntary participation in the China-UK program.

3.2 Covariates

The remaining variables used in our analysis are listed in

Table 4. Balance statistics for IPW matching model for the treatment of participating in at least one Program component.

Covariates	Standardized Difference *		Variance Ratio *
	Raw	Weighted	Raw	Weighted
Personal Characteristics
Gender	−0.0182	−0.0374	1.0247	1.0016
Age	0.0023	0.0023	0.5952	0.5084
Education	0.0634	0.0076	1.1447	1.0127
Years for farming	0.0130	−0.0019	0.6780	0.7660
Farm Characteristics
Using machine	0.6577	0.0409	0.5512	0.9290
Land area	0.1140	−0.0342	1.0654	0.8346
Labor ratio	0.0481	0.0331	0.7823	0.7619
Farming income ratio	0.6342	−0.0605	0.9332	0.9155
Planting others crops	0.1619	−0.0340	1.5659	0.9308
Planting kiwi	−0.3267	0.0587	0.5520	1.1751
Planting corn	0.1546	−0.0485	0.6764	1.1743
Planting wheat	0.1468	−0.0355	0.6888	1.1228
Policies
Village has program	0.4027	−0.0176	1.2644	0.9986
Communication with friends	−0.2734	0.0450	0.8341	1.0527
Accessible training class	0.3951	0.0036	2.2193	1.0046

* The standardized difference and variance ratio for a variable x are calculated $d(x)=\left[\overline{x_1}-\overline{x_0}\right]/\sqrt{({\sigma }_1+{\sigma }_0)/2}$ and ${{\sigma }_1}^2/{{\sigma }_0}^2$, where σ² is the variance and the subscripts 1 and 0 refer to treated and control respectively.

. These variables were used in the propensity score equation (Probit model) in which treatment is the dependent variable. In the “raw” columns, we compared the variables for the two groups, those who participated in one or more components of the China-UK program and those who did not.

The demographic covariates are gender, age, education, and years of farming experience. There are intuitively plausible reasons to expect these demographic variables affect the propensity to participate in the program. Older farmers tend to refuse receiving new farming methods because they are risk averse and do not want to change their planting habits dramatically [19,20,21]. Thus, we expect that age will be negative to farmers’ participation. We also expect that well educated farmers will be better in dealing with the information from the program, making them more likely to participate in the program.

Eight variables capture characteristics of the farmers’ operations of agricultural production. An important distinguishing feature for farmers is whether they use of any machinery for farming, typically small machines; this is captured using a dummy variable that equals one if farmer used any machinery. Land area represents the farmer’s total arable land in hectares. The labor ratio is the numbers of family members involved in farming divided by the total number of family members. The farming income ratio is calculated by dividing the family’s income from farming by their total income. Finally, crop types affect both the farmers’ participation in the program and the effects of the extension program. A higher ratio of farming income or bigger land area means that a farmer’s life depends more heavily on agricultural production. Since the China-UK program seeks to control agricultural NPS pollution by reducing farmers’ fertilizer input, we expect that farmers who depend more heavily on agricultural production are more likely to participate in the program to minimize the cost of production. Similarly, we expect that the labor ratio and machine variables will be positively correlated with participation in the program.

The final set of variables presented in Table 4, capture attitudes toward the various policies. Data are collected on whether the respondents lived in a village where the program was implemented, their access to training classes (other than the China-UK program) and whether they get fertilizer information from friends or relatives. These variables are captured using dummy variables based on yes-no responses to questions.

A critical step in the matching procedure is to ensure that the matching algorithm creates a balanced sample, which means that the covariates for treated and control group have the same distribution. In Table 4 we present the balancing test for the treatment of participating in at least one of the China-UK Program’s component. Balancing test statistics for the other treatments are presented in the Appendix A. Two statistics are presented, the standardized difference in the means and the ratio of the variances for before and after the creation of a matched sample. As is seen, for all of the covariates the weighted average after matching is closer to that of the treated group and, with one exception, the variance ratio is closer to 1.

We also use a test derived by Imai and Ratkovic to check whether the above treatment model balanced the covariates [22]. The null hypothesis for the balance test is that covariates are balanced. The chi-squared statistic for our model is 12.51 with 23 degrees of freedom, and the probability is 0.9800. Therefore, we cannot reject the null hypothesis that the treatment model balanced the covariates. Combined with the output in Table 4, we conclude that the matching algorithm was quite successful in creating a balanced pseudo-control data set. Similar balancing tests were carried out for every model and the test results are presented in the Appendix A.

3.3. Estimation Results of Propensity Score Model

Before we obtain our matching estimator for the ATET, we must first estimate the propensity score using a Probit model, where the dependent variable is the treatment decision. As in Table 3, we consider six different treatment specifications and a separate propensity score model is estimate for each. Model 1 estimates the probability that farmer participated in one or more component of the China-UK program. The next five Models (2–6) are the same as Model 1 except we drop one element of the program. For example, a farmer is considered to be treated in Model 2 if he or she participated in any element of the China-UK program except farm field school. In this paper, our principal interest in the Probit results is to calculate the propensity score to carry out the matching estimation of the ATET. Therefore, the estimated parameters of the Probit model are presented in the Appendix A.

Some results from our Probit models are worth highlighting. Four variables are significant in more than five models: Using machine, Farming income ratio, Village has program, Accessible training class. Consistent with our expectations, these variables are positively correlated with participation in the China-UK program.

3.4. Estimation Results of Average Treatment Effect on the Treated (ATET)

In this section, we provide the estimation results on ATETs for different treatments. The ATET is estimated based on the Equation (1) by using IPW method. We are interested in looking at the effect of treatment on the fertilizer input per hectare per year, so the outcomes variables are the fertilizer input from our survey. This effect that we estimate is a net effect and does not capture how this change occurs. For example, a farmer exposed to the China-UK program could reduce fertilizer use by changing practices, changing crops or substituting off-farm income for agricultural output; we are not able to identify how the fertilizer changes are achieved.

As indicated above, we estimate the ATET for six treatments and

Table 5. ATETs for different treatments.

Treatment Model	Outcome
	Total Fertilizer Use (kg/ha)
	ATET	Std. Err	t-Stat
1. Full China-UK program	19.54	34.12	0.57
2. all components except farm field school	36.43	36.64	0.99
3. all components except demonstration zone	24.01	36.75	0.65
4. all components except farmer meeting	29.63	37.87	0.78
5. all components except Farmer to farmer training	−2.85	34.93	-0.08
6. all components except poster	36.58	35.94	1.02

below shows all the ATETs estimates. A positive ATET value estimated for the outcome indicates that the farmer who received that treatment applied more fertilizer (in kilograms per hectare) than they would have without treatment. The hope of the extension programs is that the ATET will be negative, i.e., that participation in the program led to a reduction in fertilizer use.

The results in Table 5 are not favorable with regard to the China-UK program’s success in reducing fertilizer use. None of the estimates are statistically different from zero at the 10% level and most models in Table 5 estimate a positive value for ATET, i.e., an increase in fertilizer use. When full China-UK program (any components participation of China-UK program were included) is evaluated (treatment 1), we estimate that participating farmers applied 19.54 kg/ha more fertilizer than they would have applied had they not participated, though we cannot statistically distinguish this from no effect at all. The only treatment that yielded a negative estimate of ATET was treatment 5, in which farmer to farmer training is dropped. That is, excluding those who only participated in the farmer to farmer training, the treatment effect is a reduction in fertilizer usage by 2.85 kg/ha, though again this is not significantly different from zero. This suggests that the farmer to farmer training component is the least effective of the components in terms of its effect on farmers’ fertilizer inputs. There is a plausible explanation as to why farmer to farmer training might be counterproductive: if the farmer teacher did not totally understand the technology from the training class given by scientists, the result could be a second phase of instruction that creates misunderstanding or delivers incomplete information to other farmers.

At the other extreme, if the poster component is excluded (treatment 6), the effect is largest, suggesting that the posters may have been the most effective component of the China-UK program in reducing fertilizer use. Nonetheless, the most important and only statistically valid conclusion that we are able to draw from the ATET estimates in Table 5 is that we find no statistically significant effect of the China-UK program on fertilizer use. As we present in the Appendix A, we also estimated the treatment effect using several alternative covariate sets and an alternative matching algorithm and the results were qualitatively unchanged.

4. Discussion

Our results may have important implications for countries that are trying to control agricultural NPS pollution by using education and training mechanism. Despite being widely promoted throughout the region, we find relatively limited uptake of the program and no statistically significant evidence that the program changed nutrient applications rates.

It is important to note that the effectiveness of education and training programs may diminish quickly, and this may provide an explanation for our findings. Our results did not show significant effects on farmers’ behavior of reducing fertilizer input. This is strikingly different from the program’s own assessment during the implementation period (from 2007 to 2009) when they did find significant effects on fertilizer use. However, when we surveyed the village in 2011, most farmers could not clearly remember the details of the program, even when the posters of the program were still displayed on the street. Hence, one possible explanation for our results is that farmers have poor recall of such programs and, change their farming practices only reluctantly and slowly. Furthermore, there is a huge difference between the program’s own data and ours. All the farmers in their sample were treated and received a subsidy to participate the program. In our sample, most farmers are in the control group and treated farmers did not receive the subsidy.

It appears that education and training alone are not sufficient to achieve the types of outcomes achieved 2007–2009 phase of the program. We suggest that the policy makers should pay attention to increasing farmers’ participation and, in addition to education and training, incentive tools be applied. Of course, that too requires careful ex-post evaluation in real-world settings.

5. Conclusions

In order to reduce agricultural NPS pollution, Chinese agricultural extension programs are increasingly trying to educate farmers to reduce fertilizer use. The evaluation of the effects of such programs is crucial because it will determine whether the programs contribute to the reduction of agricultural NPS pollutions and are worth continuing. This paper uses matching methods to estimate the impacts of the China-UK program on farmers’ fertilizer use.

From a random sample of farmers in nine villages, we find that only 10% of the farmers participated in one or more of the elements of the China-UK program. Despite the fact that the program experimentally found that farmers could reduce fertilizer and improve yields and/or farm income, we do not find any strong evidence of a favorable effect of the extension components of the programs. In part this may be due to the small number of farmers who actually participated in the programs; because of the small number of farmers receiving treatment our statistical confidence is substantially diminished. We also are able to evaluate to a limited extent the impact of individual elements of the program. We find some evidence that farmer-to-farmer training may have had a counter-productive effect while the posters may have been the most effective program element. These estimates are not very precise however, and not significantly different from zero at the 10% level.

A significant limitation of our paper is the small size of sample overall and the limited number of participating farmers. This probably contributed to the problem of statistically insignificant ATET estimates for most of our treatments. With this caveat, however, we can conclude that the effects of the China-UK program are quite limited. Hence, if resources continue to be dedicated to extension programs like this, there should be a simultaneous research effort studying such programs’ effectiveness and such research should strive to have much larger sample of farmers. Further, our results give rise to additional questions about why farmers do not voluntarily reduce their fertilizer use even when experimental results indicate the value of doing so.

An additional contribution of this paper is to demonstrate the potential to use matching methods to evaluate agricultural extension programs that seek to achieve environmentally beneficial outcomes. This paper can serve as a model for future such evaluations that would yield valuable information about the effects of agricultural extension programs in China and help improve the design of such programs in the future. Further research, in line with recent scientific literature [23], should be addressed to investigate the factors which increase farmers’ sensitiveness to fertilizers saving and encourage the adoption of environmental friendly measures.