Intensification and Upgrading Dynamics in Emerging Dairy Clusters in the East African Highlands

Abstract

Based on farmer and value chain actor interviews, this comparative study of five emerging dairy clusters elaborates on the upgrading of farming systems, value chains, and context shapes transformations from semi-subsistent to market-oriented dairy farming. The main results show unequal cluster upgrading along two intensification dimensions: dairy feeding system and cash cropping. Intensive dairy is competing with other high-value cash crop options that resource-endowed farmers specialize in, given conducive support service arrangements and context conditions. A large number of drivers and co-dependencies between technical, value chain, and institutional upgrading build up to system jumps. Transformation may take decades when market and context conditions remain sub-optimal. Clusters can be expected to move further along initial intensification pathways, unless actors consciously redirect course. The main theoretical implications for debate about cluster upgrading are that co-dependencies between farming system, market, and context factors determine upgrading outcomes; the implications for the debate about intensification pathways are that they need to consider differences in farmer resource endowments, path dependency, concurrency, and upgrading investments. Sustainability issues for consideration include enabling a larger proportion of resource-poor farmers to participate in markets; enabling private input and service provision models; attention for food safety; and climate smartness.

1. Introduction

Upgrading of dairy farming and value chains has been promoted by policy makers and development practitioners as a promising pathway to deal with the sustainability challenges of mixed crop–livestock systems [1,2]. These challenges include alleviation of rural poverty, supply of sufficient and safe food to growing urban populations, alleviation of rural poverty, and making farming climate-smart [3]. Of all livestock farming systems in the world, mixed crop–livestock systems produce the majority of livestock output and constitute the majority of livestock-keeping households, often smallholders [3,4]. Therefore, prospects for these systems to become more market-oriented and sustainably intensify are matters of academic, political, and societal interest [3,5].

Studies on the commercialization of milk production repeatedly show the complexity of the transition from semi-subsistence to market-oriented dairy farming, which is often associated with intensification and specialization [6,7,8,9,10]. For this transition, many farm practices may need innovation in areas such as feeding, housing, and output marketing. These innovations contribute to upgrading, defined as changes in the production process to increase productivity and added value and to improve product quality [1,2]. They require higher input levels, for which farmers need sufficient access to external resources, inputs, and services, both pre- and post-production [3,6]. In practice, upgrading occurs in so-called agribusiness clusters, i.e., geographic concentrations of producers and other actors engaged in the same subsector that facilitate the required linkages to input and output markets [11]. In clusters, the range and types of input–output connections for dairy farms and small and medium enterprises are increased, positively influencing knowledge creation and transfer between actors, enabling them to benefit from economies of scale (e.g., volumes of inputs and outputs) and scope (e.g., use of imported semen and sale of milk to new markets) [12,13,14,15].

Many studies have focused on understanding the drivers and bottlenecks affecting upgrading of dairy farming systems and value chains. These drivers include breeds; farm size; access to capital, inputs, and services; demand for dairy products; collective action; infrastructure and policies [7,8,9,10,16,17,18]. Literature yields limited analysis, however, of how these upgrading processes facilitate dairy cluster emergence and transformation to more market-oriented dairy farming, as most studies focus on a particular type of upgrading, on partial processes, or on single cases. Moreover, various authors have indicated that looking at the socioeconomic context aids comprehension of changes in agricultural practices and upgrading of farming systems [19,20,21]. It is apparent that understanding the complex dynamics of dairy farming systems requires assessment of upgrading in three domains: farming system, market, and context (including biophysical, institutional, and social conditions) [22,23]. However, empirical analysis of these dynamics remains limited. A comprehensive analysis of multiple clusters in comparable transition trajectories is expected to offer insights into the upgrading dynamics, causes of variation, and interactions between the three domains.

The present study, therefore, explores how interactions of the farming system with market and context determine upgrading pathways and outcomes. In particular, it (1) describes the present status of regional clusters; (2) assesses upgrading pathways; and (3) analyzes how interactions affect pathways and outcomes of upgrading. It compares five emerging clusters in the Kenyan and Ethiopian tropical highlands that vary in upgrading status. In all these clusters, dairy farmers face the question of whether or not to transition from ‘marketing of small surplus to local markets’ to ‘commercial supply to wholesale chains’ [3].

By looking systemically at these interactions, this paper contributes to the debates about upgrading in clusters, value chains, and farming systems; inclusion of smallholders in markets; system jumps; and pathways to sustainable intensification. The results can be used in devising future scenarios for system development and in co-design of interventions, as outlined by Martin et al. [24]. They inform strategic upgrading options for farmers and other value chain actors by pointing at the future shape of farm operations and the markets to supply to.

2. Methodology

2.1. Analytical Framework—Two Subsystems in Context

The analytical framework for this study considers that farming systems evolve because of the interaction with the market and context within a cluster (Figure 1). We take the dairy farming system within an emerging cluster as the main unit of analysis (A), from which we analyze linkages with and influences from the other two domains—market system [25] (B) and context (C)—taking into account inter-farm variation within clusters. Upgrading, defined above, can occur in all three domains and in this study is respectively distinguished as technical, value chain, and institutional upgrading [1,2]. Upgrading leads to system change (transition) and ultimately to alternative system state (transformation). The three forms of upgrading collectively can lead to commercial dairy farming and to the emergence of dairy clusters [26]. Transformation to a next development stage requires significant upfront investments in new practices, technologies, innovation system, etc. [3,27].

We view the farming system and market system as two interacting, co-evolving systems within dairy clusters, each of which may experience ‘system jumps’ between development stages [3]. Various system behaviors can be expected, as described by Schiere et al. [27], depending on the specific farm, market, and context factors that influence farmers’ livelihood strategy choices. These may include ‘adaptive cycles’, where change is episodic and periods of slow accumulation of capital (e.g., nutrients) are punctuated by release of capital and reorganization, for example by a forest fire or an epidemic; and ‘lock-in’, where systems get used to particular routines [27].

We build on two approaches for farming system analysis: (1) The farmers’ perspective of Oosting et al. [3], who in their LIVCAF model describe the transition from ‘rural farmers supplying to rural consumers’ to ‘rural farmers supplying to urban consumers’; and (2) The market quality perspective of Duncan et al. [28], who found that well-developed markets with good procurement and support service arrangements are key to sustainable dairy intensification, and that better market quality is associated with a higher proportion of improved cows that are better fed (sustainability here is used in the blended approach advocated by Mockshell and Kamanda [5]).

In all clusters, the primary driver for upgrading is the decline in livelihood due to diminishing farm size, mainly as a result of population growth [29]. This requires intensification, i.e., the increased use of external inputs and services to increase outputs per unit of input [6], in this case land use. We analyze upgrading dynamics by identifying and exploring changes in farming and marketing practices, as well as the secondary drivers that influence these; these act as accelerators of upgrading if present and as inhibitors if absent.

Analysis of upgrading dynamics thus includes three components:

• Farming system factors—Technical upgrading of the farming system is explored based on the sustainable livelihoods framework [30]. This considers how farmers combine the different types of livelihood resources they own or can get access to into livelihood activities, such as food and cash cropping, livestock-keeping, and off-farm activities, using a variety of practices, which often reinforce each other [31,32]. Farmers optimize several objectives into a livelihood strategy [33]. We thus assessed dynamics in the current mixed crop–dairy farming systems by looking at changes in the livestock and crops grown and at their functions in the farm, e.g., livestock for meat, milk, manure, draft power, social functions, household food, or sale; crops for food or sale [6].
• Farm–market interaction—Value chain upgrading changes the way a farm interacts with the market. Following the Windmill approach of Leonardo et al. [34], we explored the influence of the various service arrangements that determine farmers’ options for marketing their produce. We looked at farmers’ access to markets, associated transaction costs, and fit of service arrangements with particular degrees of market integration [13,17]. The service arrangements offer varying degrees and combinations of the horizontal (between farmers) and vertical (with input and output side chain actors) coordination that are necessary to effectively integrate smallholders into markets [11,35]. Market-integrated dairy requires a large variety of pre-production inputs and professional services, so this typology needs to cover service arrangements on both the input and output side.
• Context influence on farm–market interaction—Lastly, several context factors significantly influence farm–market interaction and determine the need for institutional upgrading, i.e., the improvement of institutional voids that constrain value chain operations [1,23]. We considered three types of factors: (1) factors in the biophysical environment, which include land-use patterns, infrastructure (roads and utilities), climate and weather, animal and crop pests and diseases, risks of natural and human-induced disasters (such as droughts and wars), seasonality of production, and environmental impact of farming, including effects of agro-chemical use [18,36]; (2) factors in the enabling context, i.e., the regulatory framework elements and their enforcement (such as agricultural policies, subsidies, access to finance, property rights, and quality standards) that determine whether the institutional context enables upgrading [16,23,29,37,38]; and (3) factors in the social environment, i.e., social identity and (dairy) farming history [39].

2.2. Data Collection and Analysis

Case study sites were selected from the highlands of Ethiopia and Kenya, home to significant dairy production on a large number of smallholder mixed farms and a smaller number of medium- and large-scale dairy farms. The two countries differ in terms of sociopolitical context. The presence, reliability, and attractiveness of market service arrangements for pre- and post-production inputs and services vary between and within countries, leading to differences in market quality [28].

Sub-regional administrative units of roughly similar size were chosen as starting points for cluster selection: Ethiopian Zones and Kenyan Counties. Based on a scoping exercise and team knowledge, in each country two emerging clusters were selected that have good and comparable agro-ecological potential for dairying (located between 1750 and 3000 m above sea level) but differ in market quality (see Figure 2). Milk production differs widely between clusters. For example, while Nyandarua and Nandi counties are roughly equivalent in terms of arable land, human population, and cattle herd size, the annual milk production in Nyandarua is nearly three times that of Nandi [40,41] (see Supplementary Material S1 for more detail). Due to two distinctly different milk-marketing situations within Nandi County, Nandi was divided into two clusters. To capture the within-cluster variety in market quality, six villages were selected per cluster, with the exception of East Shoa and Nyandarua clusters, where three and nine villages were selected respectively (see Figure 2 for location of study sites). Villages vary in access to rural service centers and end markets, with one-third each having good, medium, and poor access to a service center, located at zero, one, and two hours’ walk from a service center respectively.

Interviews with farmer groups and with other value chain actors occurred between September 2016 and May 2017. Dairy farmer group interviews (FGIs) were held in all thirty villages, with group numbers ranging from five to eleven participants, averaging eight. In Arsi, East Shoa and Nyandarua clusters, all farmers who had been interviewed as part of a previous study [42] were invited; in Nandi North and Nandi South, a new sample was invited to participate in FGIs. Farmers were purposively sampled to represent the range of dairy farm sizes in the village. The FGIs used a questionnaire with open questions for discussion and a number of participatory ranking exercises, focusing on both current situation and historic developments. The latter used either importance ranking or the ten seed technique [43], which was modified to use twenty seeds in case answers exceeded five items. Farm classification categories offered by FGIs were harmonized, as categories such as ‘small scale’ and ‘medium scale’ are context-specific; some categories were combined. Questions about dairy experience, farm acreage, number of dairy cows and main crops grown were included in the FGIs in Nandi; for other clusters, these data were derived from previous dairy farmer interviews [42].

For value chain actor interviews (VCAIs), dairy actors were selected by using information from earlier farmer interviews [42] and by snowballing. A broad range of value chain actors was covered: private and public suppliers of pharmaceutics, agro-chemicals, semen, feeds, forage, and equipment; private and public providers of artificial insemination (AI), veterinary, extension, and financial services; milk and butter traders, transporters, and dairy processors; cooperative societies and farmer groups; and development agencies and knowledge institutes (see Supplementary Material S1 for portrayal of dairy value chains in Ethiopia and Kenya). VCAIs numbered 118 in total (18 in East Shoa, 20 in Arsi, 43 in Nyandarua, 18 in Nandi North, 10 in Nandi South and 9 with multi-county actors in Kenya).

Secondary factors assessed in the FGIs and VCAIs—which act as drivers of upgrading and transition if present and as bottlenecks if absent—were derived from literature [11,16,18,28,30,44,45,46]:

• Farming system internal factors: Changes in farmer livelihood strategies, practices, outcomes, and resources (also called capitals or assets) including natural (land acreage and soils, water, climate and weather, herd size and genetics, functions of and interaction between livestock and crops used); economic (capital); physical (farm structures, equipment); human (labor, knowledge and skills); and social resources (networks, groups)
• Market factors: Dairy pre- and post-production service arrangements and service offer; farmer utilization and satisfaction; demand for dairy products (product, price, place); scarcity of inputs, services, and production factors; key marketing institutions, such as competition, role division in service supply, availability of market information, actor relationships, and milk quality assurance
• Context factors: Collective action; dairy history and identity; consumer preferences; conducive infrastructure; access to production factors; regulatory space for private services; policy priority/instruments, public services, and subsidies; social inclusion and environmental impact.

Analysis—FGI and VCAI recordings were transcribed. Along with notes made during FGIs, they were analyzed in Atlas.ti using secondary factors as codes. Differences between clusters were rated by the first author based on data analysis. Results from FGI ranking exercises were translated into percentages and tabulated along with quantitative data; simple statistics were calculated.

3. Results

3.1. Cluster Description

The five clusters selected are briefly described using the schematic positioning of their specialization and upgrading dynamics along two axes (Figure 3): feeding system and cash crop types. These axes denote the variation and recent upgrading in farming systems that, under pressure of land shortage, intensify in different ways along two directions (as observed in clusters studied): a feeding system transition from ‘grazing with crop residue use’ (low dairy intensity—L_d) to ‘zero-grazing with planted forage’ (high dairy intensity—H_d) and a cash crop transition from ‘grains’ (low cropping intensity—L_c) to ‘horticulture and/or perennials’ (high cropping intensity—H_c).

The clusters are thus characterized as (

Table 1. Key characteristics of dairy farming and marketing in five Ethiopian and Kenyan clusters.

Country:		Ethiopia		Kenya
Cluster Type:		LdLc	LdHc	LdHc	HdHc	HdHc
Characteristics	Cluster Name:	Arsi	East Shoa	Nandi S	Nandi N	Nyandarua
Average farm size (ha)		3.2	4.0	0.8	1.6	2.9
Proportion improved cattle		55%	34%	n.a.	95%	98%
Feeding system (1)		grazing and residues	grazingand residues	grazing and residues	residues + planted fodder	residues + planted fodder
Main cash crop(s) (2)		grains	various	tea	various	Potatoes
Main marketing channel		traders	processors and coops	traders	coops	coops
Milk demand		low	medium	low	medium	high
Average est. milk sales (US$/yr)		859	2,384			1,621
Input service offer		low	low–med.	low	med–high	high
Main service providers		public	public	private	private	private

(1) In all clusters, urban farms mostly practice zero-grazing. (2) ‘Various’ indicates that no crop is dominant.

):

• Dairy clusters—H_dH_c Nandi North and Nyandarua gradually specialize to dairy and become increasingly market-oriented; there is significant milk collection by cooperatives and processors; increasingly sophisticated types of service arrangements exist; other cash crops or livestock products are produced as a second activity; Nyandarua enjoys high demand for milk from processors and traders; 98% of the dairy farm herd is either crossbred or purebred exotic; potatoes come second after dairy; Nandi North has more non-dairy farmers and more medium- and large-scale farms; the choice of dairy over horticulture or perennials is still tentative.
• Grain and fattening cluster—L_dL_c Arsi specializes in barley and wheat as cash crops, enabled by farm sizes that still allow such relatively extensive crops; for a long time, poor roads limited market access for dairy; just before roads improved around 2012, farmers adopted improved grain crop packages promoted by government and agribusiness; as a result, farmers focus on livestock activities, other than dairy, that utilize cash crop residues, but do not require daily marketing, i.e., beef, mutton, and heifer production; dairy development interventions have been occurring since the 1950s.
• Perennial and horticultural crop cluster—L_dH_c Nandi South saw a diminishing role for dairy, as a move to high-value/ha activities occurred; farmers specialize in tea due to better support services; milk collection is almost only informal; cattle are being replaced by small livestock; semi-subsistence farming with extensive livestock and off-farm labor continues in areas unsuitable for tea and vegetable marketing.
• Mixed cluster—L_dH_c East Shoa, some farmers specialize in dairy (Type I), others in horticulture (Type III), while in more remote areas grains prevail (Type II). In the dairy herds of interviewed farmers, only 34% of animals are crossbred or purebred exotic; both subsectors benefit from fresh food demand in the nearby metropolis; competition for land occurs between the two and with export-oriented flower farming and urban development.

In all of the five clusters, intensification pressure is high. Over the past decades, farm sizes have shrunk due to customary intergenerational subdivision of land. In addition, the Ethiopian clusters reported land scarcity due to significant withdrawal of farm land for town and infrastructure development (past two decades) and due to allocation of land to state farms (L_dL_c Arsi cluster, 1980s) and flower farms (L_dH_c East Shoa cluster, 1990s–2000s).

3.2. Analysis of Upgrading in Three Domains

Figure 4 lists the main secondary factors that were identified in this study as influencing upgrading dynamics in the clusters. Upgrading in all three domains is most advanced in H_dH_c clusters, especially in Nyandarua, as

Table 2. Technical, value chain and institutional upgrading in five clusters.

Country:	Ethiopia		Kenya
Cluster type:	LdLc	LdHc	LdHc	HdHc	HdHc
Cluster Name:	Arsi	East Shoa	Nandi South	Nandi North	Nyan darua
Upgrading Type
Technical upgrading
- specialization in ‘dairy as business’	+	++	+	++	+++
- investments in dairy genotypes	++	++	+	++	+++
- investments in feeding	+	+	+	++	++
- investments in housing	+	+	+	++	+++
- investments in animal health care	+	++	+	++	+++
- specialization in high-value crops/livestock i.o. dairy	++	++	+++	++	+
Value chain upgrading
- more sophisticated input and output service arrangements		+	+	++	+++
- contracting and quality assurance		++	+	+++	+++
- competition in service provision		++	+	++	+++
- transformation farmer organizations		+	+	++	+++
Institutional upgrading
- role redefinition private–public			+	++	+++
- enabling private sector services	+	+	+	++	+++
- infrastructure development	+	++	++	++	+++
- upgraded financial services	+	+	++	+++	+++
- quality standards for products		+		++	++

N.B. Number of + denotes degree of upgrading: one + means some upgrading, additional + means more upgrading than in other clusters; no + means no upgrading identified.

shows. While a number of context conditions in L_dH_c Nandi South are good, specialization toward high-value cash crops is at the cost of upgrading in dairy. In L_dH_c East Shoa, competition with cash crops explains upgrading limitations for dairy. In L_dL_c Arsi, market constraints clearly affect dairy prospects. In the latter two clusters, less favorable context factors also dampen upgrading. Observed dynamics related to these factors are described in the next sections, following steps A–C from Figure 1. Factors with less apparent effect on upgrading dynamics were considered, but generally not described. A more detailed description of upgrading dynamics in each cluster is included as Supplementary Material S2.

The examined clusters are under land-scarce conditions, which means that farm acreage and stocking rate (livestock units per hectare) are key indicators to observe when assessing intensification and upgrading status. A number of additional parameters—suggested by this study as potential indicators for upgrading in the three domains that score resource base, intensity of production, and market—are shown in Figure 4.

3.2.1. Farming System Factors (A)

This section describes technical upgrading dynamics identified in the farming systems domain. The data in

Table 3. Farm characteristics for the five clusters.

Clusters (with sub-Counties/Districts Where Study Sites Are Located)			No. of Villages (FGI Part.)	Management Level (% of Dairy Farms) (1)				Farm Size (ha) (2)	Herd Size	Milking Cows as % of herd	% Improved Cattle in Herd (2)	Main crops (2),(3) (Dominant Crops in Bold)	% hh w/Off-farm Activ. (2)	Non-Dairy Farms (% of hh.)
				Low	Medium	High
		Breed type		Local	Local and improved	Improved
		Market focus		Within village	Mixed	Outside village
LdLc	Arsi (Limu-Bilbilo and Digalu-Tiyo)		6 (42)	61		39		3.2	6.5	31	55	Cereals, pulses, vegetables	42	13
LdHc	East Shoa (Ada’a)		3 (21)	67		33		4.0	7.6	31	34	Cereals, pulses, oil seed, veg.	35	40
Average for clusters Ethiopia				64		36		3.4	6.9	31	45		40	27
				Farm Size (% of Dairy Farms)
				Very sm.	Small	Medium	Large
		Farm size (ha)		<2.5	<2.5	2.5–8	>8
		No.of dairy cows		1–2	3–5	6–20	>20
LdHc	Nandi South (Aldai, Nandi South)		6 (63)	78	16	6	0	0.8	2.2	66	n.a.	Tea, maize, RTB, veg., fruits	n.a.	19
HdHc	Nandi North (Chesumei, Nandi North)		6 (47)	76		20	4	1.6	3.3	64	n.a.	Maize, sugarc., tea, veg., fruits	n.a.	25
HdHc	Nyandarua (Ol Kalou, Kipipiri)		9 (63)	92		8	0	2.9	4.3	65	98	Potatoes, cereals, pulses, veg.	40	13
Average for clusters Kenya				87		11	1	1.8	3.3	65				19

(1) In the Ethiopian clusters, FGIs distinguished between ‘low’,‘medium’ and ‘high’ management levels; numbers for the first two categories have been combined, as not all villages identified an intermediate management level.

(2) From dairy farmer interviews for East Shoa (n = 37), Arsi (n = 85), Nyandarua (n = 91) [42]; from focus group discussions for Nandi North and South; different methodologies may affect results; farm size in Nyandarua was 2.88 ha according to FGI, 3.59 ha according to dairy farmer interviews [42].

(3) RTB = roots/tubers/bananas; here it includes Irish potatoes, sweet potatoes, arrowroot, cassava and bananas.

offers insight into the ongoing changes in farming and the similarities and differences between clusters.

Specialization in dairy: smaller herds and less cattle functions—With farm size dropping to an average of three to four hectares, farmers in the Ethiopian FGIs reported that they specialize and reduce herd sizes, focusing on productivity rather than number of animals by crossbreeding with exotic dairy types: ‘Two improved cows compare to ten local cows, but they need intensive care.’ Farmers did not consider classification based on cattle number or land acreage to be meaningful; rather, they classified dairy farms based on market orientation and management level (see Table 3). This points to the ongoing transition in cattle functions in the farming system, from multipurpose (with local cattle for draft power, beef, manure, savings, social functions such as dowry, household consumption, and a small surplus for market) to more dairy-oriented, with fewer but specialized dairy cows. In Kenya, where average farm size is already well below three hectares and nearly all dairy cows have exotic blood, farmers specialize further to increase income per hectare. Breed choice is mainly between Friesian (higher producer) and Ayrshire (more disease-resistant and less heavy feeder). Entrepreneurial entrants, who have accumulated resources through employment or business, are investing in medium- to large-scale commercial farms and in advanced technology for feeding, housing, reproduction, etc., but often without commensurate investment in high quality farm labor.

Specialization in high-value crops/livestock/off-farm activities—Due to ongoing pressure on land, farmers reported that they choose livestock types and cash crops with shorter maturation time and higher margin per hectare, to offset rising land costs. Choice of crops/livestock types depends on how available options ‘fit’ within the farm, market, and context, including personal preferences and identity: especially in the Nandi clusters, farmers consider cattle-keeping an inalienable part of their identity. This brings important experience and skills, but also explains why farmers continue with dairy cattle even where the farm size barely allows for it (see Table 3) and when competitive advantages of other livestock and crops as livelihood options outweigh those of dairy. Until some decades ago, sale of fresh milk and dairy products was subject to taboos (e.g., in L_dL_c Arsi cluster) that are only gradually losing their impact as milk undergoes commodity individuation [47].

While dairy is being upgraded in H_dH_c Nyandarua, H_dH_c Nandi North and L_dH_c East Shoa clusters, it is being replaced by smaller species (such as goats, sheep, chickens, or rabbits) in L_dH_c Nandi South and by heifer production and/or fattening in L_dL_c Arsi and remote parts of Nandi and Nyandarua. Farmers increase roots/tubers/bananas and horticulture (in all clusters but Arsi) and perennials (tea, fruit trees and sugarcane, in Nandi), largely at the expense of grains. Due to more favorable market service arrangements for tea, since the 1980s 30–40% of farmers in L_dH_c Nandi South cluster have planted tea; this crowds out dairy, as tea plantations do not offer edible crop residues nor sufficient space for forage. In the Nandi clusters, mechanized land preparation is being replaced by manual work due to declining farm sizes and shift to perennials. In Ethiopia, draft animals are starting to be replaced by equipment such as broad bed makers and combine harvesters, due to scarcity of feed resources for draft animals. Nevertheless, the presence of draft animals explains why only one in three animals in the dairy herd is a dairy cow, compared to two in three in Kenya.

Farmers reported an increase of private business activities and casual labor in agriculture, construction, and transportation services. Around 40% of farmers indicated that they are engaged in off-farm activities, primarily in formal employment, private business, and trade. Households with jobs in the public or civil society sector are generally involved in private business as well, in which they invest their salaries.

Changes in dairy practices—The specialization mentioned above plays out in a number of ‘technology upgrades’ in terms of farming practices. Only some farmers make these changes, and there are large differences between clusters. The highest proportions of farmers who make changes are in H_dH_c Nyandarua and Nandi North clusters and in dairy farms in or close to towns in all clusters:

• Investments in dairy genotypes using AI or improved bulls. This breed-replacement process is ongoing in Ethiopia and mostly completed in Kenya; except for in some remote, barely specialized villages, farmers in Kenya overwhelmingly keep purebred or crossbred Ayrshire, Friesian, Jersey, and Guernsey
• Investments in feeding practices follow a standard pattern over time: (1) grazing and crop residues are supplemented with industrial by-products and mixed rations; (2) grazing land is paddocked; (3) investments are made in production and preservation of planted forages such as oats, maize, and Napier and Rhodes grass to counter forage shortages
• Investments in animal housing in Ethiopia include new barns to house improved breeds; in Kenya, zero-grazing units and feed storage are used when intensifying further
• Investments in animal health care increase; due to the failure of communal cattle dips to control tick-borne diseases, in Kenya many farmers have moved to individual spraying and some vaccination for East Coast Fever; treatment by veterinary workers is increasing, as is self-administration of drugs purchased from agro-veterinary shops, especially de-wormers; in Ethiopia, farmers use government veterinary personnel, who often provide better private service on the side.

3.2.2. Farm–Market Interaction (B)

The data in

Table 4. Support service arrangements and input and output service offer in five clusters.

		Output Service Arrangements with Integrated Input Services (If Any)							Input Service Arrangements (5)
Service Delivery Model:		Local Market	Trader/Restaur. (1),(2)	Self-help group (3)	Cooperative (4)	Processor	Cooperative Company (4)	Processor w/Services	Independ. Suppliers		Dept. of Livestock		Dev. Agencies
Arrangement type:		Spot market	Relational contract	Formal contract	Multilateral contract	Formal contract	Equity participation	Vertical integration	Formal contract		Public services		Various
Prevalence in:
Ethiopia
	Arsi	++	++		+				+/-		++		+
	East Shoa	+	+		+	++	+/-		+		++		+
Kenya
	Nandi South	+/-	++		+/-				+		+/-		+/-
	Nandi North	+/-	+	+	++	+	++	+/-	++		+/-		+/-
	Nyandarua	+/-	+	+	++	+	++	+	+++		+		+/-
Output services offered to farmers
Collection from farm			+	+	+/-	+/-	+/-	+/-
Chilling				+/-	+/-	+/-	+/-	+/-
Transportation MCC to plant					+/-	+	+/-	+
Quality testing at collection					+/-	+	+	+
Input services offered to farmers									Eth	Ken	Eth	Ken
Farm advice							+/-			+/-	+	+/-	+/-
AI							+/-		+/-	+	+	+/-
Veterinary services					+/-		+/-		+/-	+	+	+
Feed, forage, and drugs					+/-		+		+	+	+/-
Linking to input suppliers					+/-		+/-	+/-				+	+/-
Facilitate access to finance					+/-		+	+/-					+/-
Input advancing (on credit)					+/-		+	+/-

+ = commonly provided; number of + denotes relative dominance of service arrangement; +/- = provided by some actors only or to some farmers only.

(1) Direct supply to restaurants primarily by peri-urban farmer.

(2) In Ethiopia: butter traders; in Kenya: private milk collectors buying from farmers and selling to retailers, restaurants, consumers, and MCCs.

(3) In Ethiopia we encountered no farmer groups (less formal than cooperatives) supplying milk.

(4) Some cooperatives also process, which adds additional output services.

(5) In both countries, private companies, public agencies and NGOs/development projects play a role in provision of (subsidized) inputs and services. As these affect other service arrangements, we include them as separate categories.

and

Table 5. Factors affecting choice of service arrangement by farmer category.

Service Arrangement:	Local Market	Trader + Restaur.	Self-help group	Cooperative	Processor	Cooperative company	Processor w/ Serv.
Dominant farm size	Smallholders (peri-urban)	Resource-poor smallholders	Resource-endowed smallholders	Resource-endowed smallholders	Larger farmers, organized smallholders	Resource-endowed smallholders	Larger farmers, organized smallholders
Factors affecting choice
Payment period (days) (1)	direct	negotiable	<45	<45	<45	<45	<45
Farmgate price (US/kg) (2)
- Ethiopia: milk	0.35–0.90	0.55–0.75	-	0.35–0.65	0.35–0.70	-	-
- Ethiopia: butter	3–12	7–13	-	-	-	-	-
- Kenya: milk	0.30–0.45	0.30–0.50	0.28–0.37	0.26–0.34	0.26–0.37	0.26–0.34	0.26–0.37
Milk buyer advances	-	cash	-	(inputs)	-	inputs	inputs
Proximity to services	<1 h	farmgate	------------------- depending on location -------------------------

(1) With the exception of one processor in Kenya, whose terms are 90 days.

(2) Using 2016 prices and exchange rates of ETB 20:USD 1 and KES 100:USD 1; incl. dairy farmer interv. data [42].

reflect upgrading dynamics stemming from the interaction between farming system and market, which become particularly clear when comparing clusters. As input service arrangements are important in more intensive dairy and become increasingly integrated with output service arrangements, Table 4 includes both input and output service arrangements identified. This description follows the value chain upgrading categories of Table 2.

More sophisticated input and output service arrangements, tailored to farmer types—Dominant service arrangements range from local markets and traders in the limited market conditions of L_dL_c Arsi and Nandi South clusters to cooperative companies and processors, with increasingly integrated services in H_dH_c Nyandarua. In L_dH_c East Shoa cluster, processors and cooperatives are replacing the first two output service arrangements, as yet without significant upgrading in input service arrangements. In H_dH_c Nandi North and Nyandarua clusters, service arrangements of cooperative companies (i.e., upgraded cooperative societies) are being upgraded to integrated input and output service packages. Processors here, who source from farmer organizations and larger farms, are experimenting with integrated input and output service arrangements as well, more so in H_dH_c Nyandarua where competition for milk and service provision is fiercer.

Service arrangement use by farmers depends on their market integration and milk sales volumes. Table 5 shows how different service delivery models cater to different farmer categories. Interviews revealed a strong relation between farmers’ choice of service arrangements and farm household resource level, which in turn is related to off-farm activities. For resource-poor farmers, payment conditions are most important. They mainly sell to traders, as they need today’s milk money for today’s food, and they often lack the cash to acquire external inputs and services. Smallholders with more resources tend to sell to cooperatives and processors (sometimes through self-help groups), to benefit from larger two-weekly or monthly payments that can be used for inputs and investments. However, they usually sell at least some milk to traders to benefit from higher prices and to satisfy immediate cash needs. In Kenya, the resource-endowed smallholders selling to cooperatives can benefit from input and service advancing through widespread ‘check-off’ systems, in which costs for inputs and services advanced are deducted from the next milk payment. Medium-scale farms in both countries seem to use any of the output service arrangements and mainly consider price, buyer dependability and transaction costs.

Interviews in both of the countries further indicated that increases in productivity and marketed milk volumes are necessary to be able to pay for the extra inputs and services. Farmers in Ethiopia mentioned a break-even point of 9 L/cow/day.

Chain contracting arrangements and quality assurance—Low levels of trust in the chain form a strong inhibitor to upgrading, especially in Kenya. This is evidenced by significant ‘side-selling’ of milk: farmers and farmer organizations hedge marketing risks by selling to multiple clients. Processors do the same by contracting fixed volumes with suppliers. The result is a supply network rather than a supply chain, with associated high production and transaction costs. Marketing is volume- rather than quality-driven. Marketing relationships are complicated by the stark seasonality of production, with a slump in production during the dry season, and by the seasonality of consumption due to Orthodox Christian fasting seasons in Ethiopia.

Competition in service provision—In Ethiopian clusters, government agencies are the primary input and service providers. Although the main product in L_dH_c Nandi South, Kenya, is fresh milk rather than butter, the output service arrangements are unsophisticated, as in L_dL_c Arsi. Stronger competition leads to more sophisticated arrangements with higher degrees of horizontal and vertical coordination, as observed in H_dH_c Nyandarua cluster. Here, improved service levels were reported in milk contracting, milk collection, value chain financing, feed supply, drug supply, and AI services, but less so in curative health care and hay supply. Use of own bulls rather than AI services is diminishing, but still common in all clusters, pointing to issues with the quality of AI services (proportion of farmers using bulls is lowest in H_dH_c Nyandarua, at around 40%).

Transformation of farmer organizations—The poor track record of cooperatives in both countries in terms of governance, efficiency, and sustainability makes many farmers wary of investing heavily in them; many regard cooperatives primarily as channels for public and NGO subsidies. The more entrepreneurial smallholders in Kenya circumvent these issues by forming less formal ‘self-help’ groups that aggregate milk and supply directly to processors. Cooperative companies, generally initiated with support from development agencies such as Heifer and partners, add a variety of services to these inputs, including access to credit lines (see Table 4). In Ethiopia, such systems are much less developed.

3.2.3. Context Influence on Farm–Market Interaction (C)

This section describes identified upgrading dynamics stemming from interaction with the context. Institutional upgrading (or the absence of it) may have a synergistic, antagonistic, or inconsequential influence on technical and value chain upgrading. The main context factors identified in interviews are presented in

Table 6. Conduciveness of context factors in five study clusters.

	Country:	Ethiopia		Kenya
	Cluster Type:	LdLc	LdHc	LdHc	HdHc	HdHc
Context factors	Cluster Name:	Arsi	East Shoa	Nandi S	Nandi N	Nyandarua
Biophysical
Climate/weather		+++	++	++	++	+++
Absence of disease threat		+	+			++
Infrastructure		+	+	+	++	+++
Enabling environment
Policies promoting dairy				+	++	+++
Policy space for private service prov.		+	+	++++	++++	++++
Public disease prevention services		++	++	+	+	+
Research–extension–farmer linkages		++	++	+	+	+
Enforcement of service quality				+	++	+
Enforcement of milk quality				+	+	+
Access to finance				+	++	++
Chain upgrading facilitators		++	++	++	++	++
Social environment
Dairy history and culture		+++	+++	+++	+++	+++
Dairy seen as business		+	+	++	+++	++++
Milk consumption		+	+	+++	+++	+++
Land availability		+	+			+
Labour availability		+++	++	++	++	++

N.B. The number of +’s indicate how conducive the situation is in comparison with other clusters.

and are described here following the institutional upgrading categories of Table 2. A more elaborate description of policy dynamics is included in Supplementary Material S3.

Impact of role division between private and public actors on service arrangements—Both countries have a turbulent history of public influence on agricultural service provision, contributing to large changes in Kenya and stagnation in Ethiopia. In Ethiopia, public actors play an overriding role in access to inputs, services, and land. In Kenya, 25 years of significant policy changes have affected dairy in diverse ways: very significant cuts in public services in the early 1990s resulted in a collapse of the dairy sector, evidenced by the bankruptcy of many cooperatives and the state processor KCC (1999); market liberalization policy only gradually resulted in private service delivery [48]; and the enabling environment now varies from county to county [49].

In both countries, many interviewees complained about the inconsistency and inadequacy of public services for dairy. Minimization of dairy extension services in Kenya in the 1990s resulted in declining farmer skills and ultimately in declining yields. Public agencies have a (virtual) monopoly on vaccination for notifiable diseases in Kenya and on vaccination, AI, veterinary, and extension services in Ethiopia. The regulatory gaps for private AI, animal health services, and quality assurance of feed and the low policy priority for dairy compared to crops and meat received strong negative feedback. Relatively large positive impact was attributed to development projects.

In both countries, governments use subsidies to promote uptake of more market-oriented practices and to make services more accessible to farmers in remote locations and/or with fewer resources. In Kenya, interviewees mentioned many downsides to subsidized services. In Ethiopia, public monopolies on most inputs and services lead to an insensitivity toward demand, favoritism and lack of a level playing field for private providers. In both countries, subsidies seem to have created dependency on chemical fertilizers, leading to soil fertility issues.

Space for private sector service provision—The above indicates a number of bottlenecks for private service provision, even in Kenya where market liberalization is standing policy. In Ethiopia, regulatory space for private service providers primarily results in private agro-input shops (feed, drugs) and milk/butter trade; in Kenya, it results in agro-input shops and milk trade, as well as AI, veterinary, and advisory services. In both countries various business licenses are required, but monitoring of licenses is lax in Kenya.

Infrastructure development—Infrastructure, in terms of roads and utilities, was improving in all clusters. Market access for remote villages was more restricted by poor roads in Ethiopia than it was for remote villages in Kenya, as was least restricted in H_dH_c Nyandarua, where authorities have invested more in roads. While road upgrading in L_dL_c Arsi did improve access to markets, in L_dH_c East Shoa cluster it was mostly seen as taking away land from farming.

Financial services, factor access and information supply—In Ethiopia, poor access to finance is a significant bottleneck for upgrading of dairy farms and support services; farmers primarily rely on community savings and community credit institutions such as ‘ekub’. This is less of an issue in Kenya, where people who are connected to more formal value chains benefit from chain financing mechanisms, cooperative savings and credit institutions, and easier access to bank loans. Capping of interest rates at 14% per year for agricultural loans was applauded by Kenyan farmers. Access to labor is impeded by the image of dairy as involving much heavy and dirty labor. Access to information is increased by the presence of private advisory service providers next to public ones, and local language radio and TV programs about agriculture are highly appreciated by farmers.

Quality standards for products—In Kenya, demand for dairy products is strong and growing (annual consumption exceeds 110 L/capita [50]). Consumer preference for raw milk gives the informal market a strong advantage. Its market share remains over 70%, despite many decades of formal chain development efforts and presence of product standards [50,51]. In Ethiopia, annual consumption is much lower, at around 20 L/capita, and the informal market trades over 98% of the volume [50]; here, cooperatives and processors find it difficult to deal with seasonality of consumer demand resulting from long fasting seasons (on top of seasonality of production), although interviewees may have been using this as a metaphor for the difficult business climate.

4. Discussion

4.1. Present Upgrading Status of Farming and Clusters

This comparative assessment between clusters clearly draws out important differences in upgrading of farming systems that emanate from farm–market–context interactions. It reveals that all five clusters show clear evidence of technical, value chain and/or institutional upgrading of ‘typical’ semi-subsistence mixed crop–livestock systems to more market-oriented systems. The need for higher returns per hectare requires specialization and commercialization, in order to maintain or increase farm yields and household incomes. Technical, value chain and institutional upgrading are most pronounced in the H_dH_c clusters and least in the L_dL_c cluster, where the market system showed little to no upgrading (see Table 2). Degrees of upgrading are clearly related to secondary drivers that act as accelerators and inhibitors.

The current status of each cluster is the result of diverging pathways along dairy feeding system and cash crop intensification dimensions. These lead to increased market orientation of farmer livelihood strategies, marketed volume, and use of pre- and post-production inputs and services (Figure 3), but for different commodities and to different degrees. More intensive dairy can thus be considered to be one of the high-value ‘cash crop’ options that farmers can specialize in when market and context conditions are right; so are other intensive livestock activities, such as commercial poultry. This makes the Windmill approach, postulated for crop commodities by Leonardo et al. [34], to be applicable to livestock commodities as well. However, ample attention is needed for input service arrangements, which need to be especially elaborate for livestock ventures.

4.2. Cluster Upgrading Pathways Toward the Future

Cluster upgrading directions diverge as a result of different specialization choices. The different clusters react differently to the primary driver of land-use intensification, which requires higher productivity and higher returns per hectare. Choice of either intensive dairy or horticultural and perennial cash crops will be at the expense of the other option (L_dH_c vs. H_dH_c). Most clusters can be expected to move further along the intensification pathway type started, unless actors consciously redirect course:

• H_dH_c—Dairy clusters. Dairy is competitive against other commodities; service arrangements become increasingly sophisticated and competitive; private and/or cooperative actors play a strong role. Continued development of H_dH_c clusters toward dairy seems likely, provided upgrading in farming, market, and context progresses. Further specialization may lead to singular focus on dairy (H_dL_c). This expected further upgrading of the H_dH_c dairy clusters contradicts modeling outcomes of Herrero et al. [4], who only foresaw such upgrading for peri-urban dairy in Kenya, and may warrant review of their modeling assumptions.
• L_dL_c—Grain and fattening cluster. Strong public policy directions and public–private collaboration made grains in L_dL_c Arsi cluster more competitive than dairy. Future development of L_dL_c clusters toward dairy depends on serious value chain and institutional upgrading, if dairy is to effectively compete with cash crops. For the time being, available farmer expertise and presence of improved dairy breeds in L_dL_c Arsi keep the door open for upgrading of dairy, but heifer production and commercial forage production for supply to other dairy clusters seem to be more attractive alternatives. These alternative opportunities are enhanced by (1) the competition for fodder between dairy and draft animals in Ethiopia; and (2) the low capacity of intensifying tropical dairy systems to produce sufficient replacement stock and fodder [52], which results in high prices for dairy heifers and fodder.
• L_dH_c—Perennial and horticultural crop cluster. Severe land scarcity affects these clusters, with specialization toward perennials, horticulture, and intensive livestock. Due to strong path dependency, further upgrading and specialization of L_dH_c clusters around perennials and horticulture are most likely, along with intensive non-dairy livestock-keeping in areas not suitable for perennials and horticulture. It will be interesting to watch whether farmers with a strong ‘cattle identity’ will give up dairy.

Prospects for the remaining L_dH_c East Shoa cluster are still uncertain. It could either move toward intensive dairy, toward horticulture or toward other high-value commodity options. Upgrading prospects for dairy depend on how relative competitiveness of each venture is affected by dynamics in its respective markets (e.g., conduciveness of service arrangements for each option) and context (e.g., spatial planning and enabling policies).

An interesting next step would be to quantify the degree of specialization and intensification of (dairy) farming in clusters, building on recent work in Europe and West Africa [53,54].

4.3. Upgrading Options at Farm Level

To explore upgrading options for dairy farmers in different clusters, we draw attention to path dependency, farmer feasibility space and aspirations. Path dependency [27] as system behavior applies at cluster, value chain, and farm level: past investments in an established commodity favor its current competitiveness. A ‘new’ commodity still needs to build up its capitals and is competing against stakes in the established commodity. This path dependency becomes stronger the more intensive the competitive crop or livestock activity. When dairy is being compared against tea and against barley as an investment choice, investments in technical and value chain upgrading for dairy need to be higher to beat tea than to beat barley, as tea has a higher expected return per hectare. Waithaka et al. [55] suggested that the intensification of farms in Nandi South could increase milk production on purchased feed, but the present study shows that in this L_dH_c cluster, the suite of service arrangements required for entrepreneurial dairy are lacking, whereas they are present for tea. While path dependency is expected to be stronger for H_dH_c and L_dH_c than for L_dL_c type clusters, it can influence upgrading pathways in any cluster. For example, the ongoing reliance on draft oxen rather than on machines in Ethiopia appears to be a significant barrier for transition to market-oriented dairy, as a large proportion of the fodder biomass is fed to oxen and (local) oxen dams, limiting fodder availability for dairy cows.

Differences in farmer livelihood strategies help explain the presence of multiple types of service arrangements coexisting within the same cluster (Table 4). These cater to different farmer groups: the supply conditions of the formal arrangements are suiting resource-endowed farmers with more intensive dairy farming but are unfavorable to resource-poor farmers (Table 5). To them the informal arrangements offer a flexible and convenient market outlet with a competitive milk price, at an input level they can afford (Table 5). For policy makers and development actors who aim to connect more smallholders to (formal) markets, an important consideration should be that farmer livelihood strategies are the result of feasibility space and aspirations, which do not necessarily go hand in hand.

Farmers’ feasibility space expands along with their resource base, access to production factors, presence of service arrangements, and conducive context factors [27]. Resource-endowed farmers can intensify crop or livestock activities; utilize upgraded service arrangements; and access land, labor, credit, and information. In contrast, due to limited feasibility space, resource-poor smallholders are likely to choose autonomy and risk aversion, reducing external input and service use and using informal service arrangements.

Farmer aspirations determine the livelihood strategy choices made within this feasibility space. The less sophisticated informal service arrangements better fit with the livelihood strategies of resource-poor smallholders, for whom dairy likely serves food security, savings, and consumption assets objectives rather than income generation [21]. A growing feasibility space will not necessarily be used to produce more milk (or other produce) for the market, let alone to make the significant changes to farming practices that are required for intensive dairy farming [6]. The effect of farmer aspirations is also apparent in the presence of ‘positive deviants’, those who actually utilize their feasibility space for dairy development. They are recognized by peers as ‘serious farmers’ (Kenya) with ‘good management’ (Ethiopia). These households achieve higher productivity and income levels with intensive dairy farming, utilizing more inputs and services, and marketing through formal channels. They adopt suitable upgrading options, such as investments in zero-grazing units, planted forage, feed rationing, mechanization of milking and forage production, and stronger contracting with milk buyers, which may also involve quality control of milk, inputs, and services [21].

4.4. Sustainability of Intensification Pathways

We now address the question of whether the identified transition pathways do actually contribute to the sustainability challenges mentioned in the introduction.

Alleviation of rural poverty—Social inclusion of smallholders in agricultural markets is a policy priority in both countries. It is enacted through infrastructure development, support to cooperatives, and public facilitation of pre-production inputs and services. To contribute to poverty alleviation, these services need to reach the rural poor, i.e., smallholder dairy farmers, and need to support upgrading of dairy farming. While not intending to evaluate public dairy interventions, this case study yielded the following insights:

(i)

Market access for resource-poor farmers can be positively impacted by policy support instruments and development interventions; these have their own dynamics, which often appear to be at odds with the space for private service provision. Long-term impact assessment is critical, as their effects are often slow and not very noticeable [56].

(ii)

Cooperatives offer no panacea for upgrading. In less sophisticated markets, cooperatives with a basic service offer can stimulate market orientation. In intermediate market conditions, they serve as collection and aggregation centers that are highly valued. In more sophisticated markets, however, in order to stay competitive they have to move beyond being what Royer, et al. [57] call a ‘claim group’ and develop into more efficient service providers.

(iii)

As membership of cooperatives consists of resource-endowed smallholders with a relatively large feasibility space, supporting them through the cooperatives has a large potential to grow agricultural output [58] but excludes resource-poor smallholders.

(iv)

The quality of public services generally is insufficient for dairy farming upgrading, which requires dependable pre-production inputs and services [6]. While in Ethiopia authorities unintentionally hamper dairy farming upgrading by monopolizing key support services, authorities in Kenya at times hinder private service delivery development by subsidizing inputs and services to farmers who have sufficient purchasing power.

The question thus remains: How can authorities effectively support market inclusion of the resource-poor, offering them options to step up or step out, rather than hang in [21]? This study illustrates the urgency of this issue by the observation that in areas such as L_dH_c Nandi South, the size of many farms is close to or already below the 0.4 ha that farmers consider the threshold for a viable livelihood, according to Waithaka et al. [55].

Supply of sufficient and safe food—In terms of quantity and product range, the Kenyan dairy sector is meeting demand [51]. Focus on quantity rather than quality leads, however, to increased concern about safety of milk and dairy products. These need to be addressed through upgrading of quality assurance practices in all three domains. In Ethiopia, the sector cannot meet demand in terms of either quantity or quality, as is evidenced by high prices and growing imports [17,50].

Upgrading should lead to higher marketed milk volumes, higher farmer incomes, and marketing of safe food. This confirms findings of Duncan et al. [28] and Murage et al. [59]. ‘Jumps’ in production are achieved by specialization, which requires investments of different kinds, including management focus. Specific upgrading options are relevant within specific cluster conditions. For example, the hub concept described by Kilelu et al. [11] may work best under smallholder conditions with competitive demand for services and competition for milk; moreover, context conditions for hub success include policy priority for smallholder dairy development, ample space for private service provision, and presence of a third-party innovation intermediary [56].

Making farming climate-smart—Regarding environmental impact, interviewees in both countries showed concern for the imbalanced use of fertilizer, leading to acidification and leaching of soils, and for the injudicious use of agro-chemicals that can affect human health, water quality, and product quality. The results suggest that farmers do worry about increasingly erratic weather—indicating the need for climate adaptation—but did not connect climate change with their own practices. These results show that before dairy sector actors will take action, climate change mitigation does require carefully designed policy regulations that address both farmer and public interests, as was also illustrated by Paul et al. [60].

We conclude that, in both countries, progress is centered around poverty alleviation objectives, which aligns well with current policy interests. Sustainable upgrading pathways require more attention for food safety and climate-smart criteria.

4.5. Upgrading Dynamics as Result of Farm–Market–Context Interactions

This study builds on three approaches for analysis of a farming system and its interaction with the market: the farmers’ perspective of Oosting et al. [3], the market quality perspective of Duncan et al. [28] and the sales arrangement/Windmill perspective of Leonardo et al. [34]. Our exploration of the co-evolution of farming systems and service arrangements offers new insights in three areas.

Firstly, this study sheds light on the reasons particular types of farmers participate in particular chains: upgrading of service arrangements within a dairy cluster offers technical upgrading opportunities and enlarges farmers’ feasibility space, but each individual farmer needs to master the resources required and aspire to upgrade. As farm resource endowments differ, a gradual and incomplete shift of farmers to upgraded chain and farming practices is apparent. This study shows that not only urban farmers but also rural farmers participate in multiple chains as a risk-reduction strategy where service arrangements are insufficiently dependable. The traders’ arrangement connects rural farmers in all clusters to both rural and urban consumers, while in the more dairy-oriented clusters, farmers sell to both traders and processors. This suggests that farmers can be part of both chains for a large part of the transformation trajectory from ‘semi-subsistence with small surplus to local markets’ to ‘commercial supply to wholesale chains’. The transition described by Oosting et al. [3] of ‘rural farmers supplying to rural consumers’ to ‘rural farmers supplying to urban consumers’ can apparently last for decades when market and context conditions are sub-optimal.

Secondly, this study sheds light on dynamics of co-evolution between farming system and service arrangements. It adds five insights to the findings by Duncan et al. [28]: (1) technical upgrading of housing and health care practices accompanies upgrading of breeding and feeding; (2) relations with off-farm activities appear to be complex: while income from off-farm business and employment is important to finance dairy investments and to supplement farm income, the proportion of households engaging in off-farm activities in this study did not change with market quality; further research is warranted into the patterns of such investment and its impact on dairy upgrading; (3) it shows the competition between farming activities in the specialization process: in clusters where dairy support services remain less conducive, farmers specialize into cash crops and short maturity livestock production activities at the expense of dairy; (4) it shows the propelling role of competition between service providers in the co-evolution between farming system and service arrangements; (5) it shows the correlation between farming system upgrading and the activity of innovation intermediaries; various authors [11,26] have shown the important roles of innovation intermediaries in upgrading. While this study identified activities and impact of intermediaries in the various service arrangements—dairy cooperatives, processing companies, public–private collaboration and development agencies—further description goes beyond the scope of this paper.

Lastly, this study sheds light on system behaviors such as system jumps and adaptive cycles [3,27]. We postulate that co-dependencies between farm, market, and context are key to understanding the adaptive cycle dynamics of system upgrading, including system jumps, stagnation, and collapse. Section 4.6 further elaborates on these system dynamics.

4.6. Positive and Negative Co-Dependencies in Relation to System Jumps

The marked differences in upgrading status between clusters can be attributed to co-dependencies between technical, value chain, and institutional upgrading processes. Co-dependencies make upgrading in one domain dependent on that in another. An example of strong co-dependency is when farmers can only adopt a new forage crop with commensurate investments in skills, (imported) seed, and equipment, if service providers simultaneously invest in providing the inputs and services necessary to grow the crop and if policy makers ensure adequate advisory services, as well as regulations for importation and control of seed and equipment. We coin the concept of ‘concurrency’ to describe this mutual dependency in terms of timing of synergistic upgrading in different domains.

Upgrading in all three domains can be expected to occur when ‘all lights are green’, i.e., drivers in all three domains work as accelerators. Positive feedback loops [27] propel upgrading, potentially leading to significant transitions. For example, farmers who consistently supply to formal milk buyers can use their supply records to more easily get credit from financial institutions. This enables investments in higher production capacity, which further improves access to services. This bankability cycle may be initiated by infusion of capital from other income sources, such as employment and/or business, and is more apparent in Kenya than in Ethiopia, where banks rarely provide (scarce) credit to dairy farms due to dairy’s low rate of capital turnover.

Concurrency and positive feedback loops will not occur, however, when one or more drivers ‘throw a spanner in the works’, consecutively inhibiting upgrading in the three domains. In such cases, co-dependencies cause negative feedback loops [27] that lead to stagnation and may be hard to break. For example, the uncertainty about price and payment conditions pushes farmers to lower external input levels, leading to lower production levels and higher seasonality of production. These in turn inhibit processors from offering good payment conditions. Where other livelihood opportunities have a significant competitive advantage, farmers can be expected to turn to those. In their absence, decreasing farm size will lead to stagnation and declining wealth. Where dairy is hard to combine with new livelihood activities, as in the case of tea, dairy may collapse and the farming system will transform to a system without dairy; total disappearance of dairy in the L_dH_c cluster has so far been prevented by the strong ‘cattle identity’ of the Nandi farmers.

Progressive upgrading may lead to transformation of the farming system and/or market system. Farmers in all clusters noticed the final stages of the transformation from grazing land to farmland for crops. The L_dL_c cluster only recently completed this transformation ‘from grazing to grain’, following public promotion of improved grain variety packages in the 2010s. In the meantime, the most upgraded H_dH_c cluster appears to be facing another transformation that will manifest in upgraded feeding strategies: ‘from grazing with crop residues to zero-grazing with planted forage’. However, this is co-dependent on further value chain upgrading that will ensure supply certainty and improved access to and quality of inputs and services.

When a sizable number of upgrades needs to occur concurrently, a system jump can be expected when reaching a certain threshold—or tipping point—of pressure to transform between alternative system states [61]. This study illustrates this for two scale levels: (1) semi-subsistence clusters transforming to more commercial intensive systems (dairy or horticulture) mentioned above; and (2) households shifting their milk supply from traders to wholesale chains. At both levels the jump requires concurrent synergistic upgrading and build-up of resources. In H_dH_c clusters in Kenya, a number of positive dynamics occur that may lead to such a system jump, once the current lock-in of farming and market systems can be overcome. That system lock-in is evident in chain fragmentation, high costs of production and transactions, and disregard for quality assurance of milk and inputs. We speculate that the pressure to upgrade gradually builds up and forces a number of concurrent technical, value chain, and institutional upgrades to suddenly take place. Time will tell whether lock-in will be overcome by a system jump through upgrading, or whether it will persist by protection of vested interests, perpetuating the current situation until a crisis causes system collapse.

5. Conclusions

This comparative case study of five emerging dairy clusters in the East African highlands aimed to explore how interaction of the farming system with market and context shape cluster emergence and transformation from semi-subsistent to market-oriented dairy farming. Key findings of this study add to debates about upgrading in clusters, value chains and farming systems; inclusion of smallholders in markets; system jumps; and sustainable intensification pathways. They include:

• Co-dependencies between technical, value chain, and institutional upgrading processes are key to understanding the adaptive cycle dynamics of farming- and market-system upgrading, including system jumps, stagnation, and collapse. We coin the concept of ‘concurrency’ to describe co-dependency in terms of timing of synergistic upgrading in different domains. When a sizable number of upgrades needs to occur concurrently, a system jump can be expected upon reaching a certain threshold of pressure to transform. The implications for studies of technical upgrading in farming systems are that synergies between internal (farming system) and external (market and context) factors determine upgrading outcomes.
• The upgrading status of dairy clusters results from diverging pathways along two dimensions: feeding system intensification and cash crop intensification. Intensive dairy is competing with other high-value cash crop options—intensive livestock activities, horticulture, and perennials—that farmers specialize in depending on market and context conditions. Clusters can be expected to move further along the intensification pathway started, unless actors consciously influence direction through investments in upgrading conditions. The implications for the debate on cluster upgrading are that (1) transition emerges from synergistic technical, value chain, and institutional upgrading; and (2) evaluation of upgrading options needs to consider notions of path dependency, concurrency, and investments in upgrading conditions.
• Farmers’ feasibility space for participation in transition expands along with their resource base, access to production factors, presence of service arrangements, and conduciveness of context factors. Resource endowment levels help explain why particular farmers participate in particular chains. Transition from ‘semi-subsistent farmers supplying to local markets’ to ‘market-oriented farmers supplying to urban markets’ may take decades when market and context conditions are sub-optimal. This adds to earlier work on inclusiveness of connecting resource-poor farmers to markets.
• The most upgraded H_dH_c cluster appears to be facing another transformation that will manifest in upgraded feeding strategies and further value chain upgrading, which will ensure supply certainty and improved access to and quality of inputs and services. Studies of such real-life system transformation cases will add to understanding of system jumps.

Further research may focus on quantification of the degree and thresholds of specialization and intensification of (dairy) farming in clusters and on the impact of different service arrangements and vertical coordination mechanisms on local economic development.

In both countries dairy development objectives are centered around poverty alleviation, which aligns well with current policy interests. We recommend that policy makers and cluster development planners carefully design sustainable intensification pathways for competitive commodities. Sustainability issues to be considered include: (1) enabling a larger proportion of resource-poor farmers to participate in markets; (2) at the same time, enabling private input and service provision models that can last; and (3) more attention for food safety and climate smartness of agricultural development.

Limitations to this study—The two x three villages sampling scheme used appears to sufficiently capture variation within clusters to assess upgrading dynamics and transitions. While the small number of one x three study villages in East Shoa cluster may insufficiently capture variation in the zone, the study area can be considered representative for the peri-urban half of the zone. The three x three scheme used in Nyandarua did not yield significantly more insight than the two x three scheme used elsewhere.

The sub-regional administrative units taken as starting points for cluster boundaries allow a researchable unit in which farm, market and context can show sufficient homogeneity and variation. However, clusters do not necessarily coincide with such units. Nandi County in Kenya shows such distinct differences that we can speak of two clusters, each appearing to be part of multi-county clusters with Eldoret and Kisumu as centers. Further research will benefit from clearer delineation of clusters. This will also improve sampling of study sites.

The retrospective interview tools, which explored timelines and past changes in farming practices, did provide considerable insight in developments since the 1980s. Nevertheless, overcoming the bias inherent in a snapshot approach when looking at time-based processes may only be possible through longitudinal or historic research.

While this study analyzed interaction between two systems—farming and market—the farming system was analyzed in more detail. Additional analysis of the market system may add valuable insights, as suggested by Reardon [62], although it risks making the analysis too complex. Using a food systems approach may be useful.

Additional studies may explore the impact of different service arrangements and vertical coordination mechanisms on local economic development. Out-of-cluster service providers such as processors and input suppliers may play a key role in upgrading dynamics, but may also capture a significant part of the benefits of transition.