The Future of Farming: The (Non)-Sense of Big Data Predictive Tools for Sustainable EU Agriculture

Abstract

The agricultural sector is one of the key sectors that need to be transformed in order to mitigate climate change. The use of predictive models supported by big data (“big data predictive tools”) has already been named in the literature as one key possibility to facilitate this change. This contribution maps out the possibilities and potential harms of big data predictive tools for sustainable agricultural use and analyses the role that regulation can play to address these challenges, answering the following question: how can the EU Common Agricultural Policy (CAP) and the European Green Deal address potential harms of big data predictive tools for sustainable agriculture while safeguarding its possibilities. Based on a combination of doctrinal legal research and a review of secondary sources, this contribution concludes that in theory, both instruments recognize the possibilities of big data predictive tools for agriculture and emphasize the necessity of environmental sustainability in this regard. However, some of the most promising and essential elements of achieving sustainable digitalisation in agriculture, risk not being substantiated because of a watered-down CAP, significant focus on larger farms and strong member state margin of appreciation. Although at first sight the CAP and Green Deal seem aligned, it can be concluded that the depth has yet to be proven. Whether this depth can be substantiated will also determine the extent to which digital technologies, such as big data predictive tools, will help in enforcing a sustainable agriculture or risk intensifying unsustainable practices in the EU.

1. Introduction

Climate change is one of the greatest challenges to mankind in the 21st century. The United Nations (UN) 2020 internet governance forum recognizes the importance of leveraging artificial intelligence (AI) and big data for environmental sustainability and climate action [1]. At the same time, the UN acknowledges that there is an urgent need to identify and address the cases in which these technologies may have adverse environmental impacts [1]. The need for a transition towards sustainable and climate-resilient food systems was more recently recognized and underscored at COP26 [2].

Agriculture is currently responsible for 10% of total greenhouse gas emissions in the European Union (EU) [3]. However, at the same time, agriculture plays a role in climate change mitigation as well. The crops, hedgerows, and trees found on farmland sequester carbon, while properly managed soils provide carbon storage [4]. This duality characterizes the agricultural sector and clearly indicates its vital importance in environmental and climate policy.

Therefore, it is of utmost importance that the EU directs the future of agriculture in a sustainable way. The prospects of digitalisation enhancing the EU’s agriculture are also recognized in the Common Agricultural Policy for 2023–2027 (CAP) (Article (23), Article 6 §1b and Article 6§2) [5]. Sustainable farming has been highlighted in the EU Green Deal as an essential objective in future European environmental policy [6]. The Farm to Fork Strategy recognizes even more clearly the link between digitalisation and achieving climate and environmental goals [7]. As such, it serves as a good basis for aligning the digital and green transitions within the agricultural sector [7].

One key element of these transitions is the use of predictive models supported by big data (“big data predictive tools”) [8,9,10,11,12,13,14]. Such tools enable the prediction of pest and disease outbreaks and offer recommendations for better seed placement in fields and selection of the best plant varieties [15]. In addition, they allow for enhanced monitoring of agricultural practices, including impacts on ecosystems and biodiversity [15]. Big data predictive tools promise to help farmers make well-informed decisions that improve the quality and quantity of their production, with less labour and less impact on the environment [15]. These models could stimulate a more precise and sustainable pesticide use, while still ensuring yield. Moreover, big data predictions can be used to adapt farming methods to the effects of climate change (weather conditions, pests, soil condition) [15].

However, big data predictive tools can also be harmful for the environment unless they are used in a way that, while often requiring a lot of energy to run, save more energy by providing more efficient solutions [16]. Therefore, it is necessary to merge two pivotal debates, one on digital technology and one on sustainable development [16].

The starting point of this contribution is the prediction that big data predictive tools will play an increasingly important role in EU agriculture. However, legal research into the advantages and possible disadvantages of this technology is currently lacking.

This contribution aims to overcome the limitations of the recent literature about the application of big data in agriculture and will map out the possibilities and potential harms of big data predictive tools for sustainable agricultural use. The central question underpinning the present contribution can be formulated as follows: “How can EU regulatory measures of the CAP and those included in the Green Deal address potential harms of big data predictive tools for sustainable agriculture while safeguarding its possibilities”. The objective of this contribution is to shed some much-needed light on big data predictive tools in a sustainable agricultural context and offer guidance considering the 2023–2027 CAP and implementation of the Green Deal.

2. Methodology

The main research question implies both descriptive and evaluative questions. Different regulatory possibilities will be identified to address the possible harms and benefits of big data predictive tools to a more sustainable agriculture. Subsequently, these possibilities will be evaluated.

The methodology to answer the research question will entail a combination of doctrinal legal research and a review of secondary sources. An academic literature review will give insight into big data predictive tools and their specific application in an agricultural context. Through doctrinal legal research, relevant legal primary and secondary sources will be identified. These primary and secondary sources allow a thorough review of the legal framework. Doctrinal legal research can encompass an analysis of case law, regulations, policy documents, explanatory reports and other legal sources. This methodology looks at the law within itself and focuses on identifying the current state of the law to enable further evaluation of the law through a review of secondary literature. The review of secondary sources will take the analysis further to provide an evaluation of the identified legal body.

Firstly, general literature on big data predictive tools, doctrine and EU publications on agricultural policy and precision agriculture will be consulted to introduce big data predictive tools and how these tools can be used in the agricultural context. Subsequently, the possible benefits and harms of these predictive tools will be reviewed.

Secondly, an overview of the relevant regulatory framework of sustainable agriculture and big data tools in the EU will be supported by an in-depth analysis of the European framework of primary and secondary legal sources. This analysis can contain regulations, directives, case law and policy documents. Legal doctrine will be consulted. In this regard, the CAP and the Green Deal will be reviewed. These two instruments are vital pillars of the regulatory framework for the governance of agriculture in the EU. The CAP is a pivotal instrument that has been an important column supporting EU agriculture policy. The Green Deal is the most recent attempt at significantly steering EU policy and agriculture into a sustainable direction. This contribution is limited to an assessment of the regulatory framework in the EU. This limitation is based on the fact that the EU is the driving force for environmental policy and dictates in a substantive way national policy and legislation. National member state law will not be taken into account as it will be mainly based on EU legislation.

Lastly, this contribution will offer an evaluation of the relevant European regulatory framework. The evaluation will be mainly based on a literature review. This refers to the collection of scholarly publications that allow an evaluation of the identified legal instruments, de lege lata. Subsequently, this leads to a formulation of how the law should be, de lege ferenda.

3. Big Data Predictive Tools for Sustainable Agriculture?

When applied in the agricultural sector, big data prediction models can help to predict and manage performance under current conditions, but more importantly, under future conditions that cannot be observed in historical data [17]. Researchers have already developed promising integrative and multi-scale big data models to predict how different crop varieties will perform under untested environmental conditions [18]. Both agricultural science and economic-behavioural sciences emphasize that improved acquisition and use of data is critical for agricultural research and its successful application [17]. Better data are needed to further improve crop and livestock models in ways that are useful for both on-farm management decision-making, as well as evaluating productivity and sustainability [17].

Private data are being developed by a growing array of management advisory and technology companies. Data generated by individual producers or by private firms selling data or advisory services are not public and thus not findable or accessible, often even by farmers themselves [17]. There are no established data standards being used, and thus data are not interoperable even when findable and accessible [17]. There are also many limitations of currently produced public data. Public data such as weather, price, and crop yield data, can be open access or available for fees. However, many of the data related to agricultural production are collected for various government administrative purposes and are not intended to be used for research or for private decision-making [17].

Understanding farmers’ decision-making behaviour remains another complicating factor. To varying degrees, new technologies such as mobile sensors on farm machinery are enabling precision management, but it is unclear what criteria are being used or should be used for management algorithms [17]. In addition, the concrete economic and environmental benefits of these technologies remain to be demonstrated [19]. Notwithstanding that recent technological developments are promising, it is still unclear whether agricultural prediction models work well enough to be used for on-farm management prescriptions or analysis of climate change impacts [17]. Research on next-generation prediction models concluded that a major limitation is the difficulty faced by potential users in obtaining and interpreting model outputs [17,20]. Even in technologically advanced countries, most farmers cannot effectively utilize the data becoming available from new digital technologies, in part because useful predictive models are not readily available. These tools have not yet been integrated into mainstream agricultural management. In general, adoption of technological innovations is mainly dependent on the characteristics of the innovation, such as cost and complexity, the innovator and his or her socio-economic background entailing preferences and educational level of the farmer and the perceived usefulness and ease of use [21]. This has been confirmed for agricultural innovations [15]. In addition, the adoption of innovation in agriculture is specifically highly dependent on the knowledge support systems in place [22,23,24].

Data-driven innovations have already revolutionized several sectors of the economy [25]. Agriculture seems to have started falling behind other major industries in the use of digital technology [26,27]. This can be clarified by the fact that EU agriculture involves highly diverse, complex systems operated by relatively small businesses. These factors complicate the use of data science technologies that depend on standardized big data and make it more difficult to exploit scale economies. Nonetheless, there is a high potential return on public investment in better data and data infrastructure, in computational methods in the agricultural sciences, and in technologies for farm decision-making [17,27]. Small and medium-sized enterprises must also be able to benefit from these technological advancements. Special attention should be paid to the needs of these farms [28]. The promise that a data-driven revolution in agriculture may provide benefits is contributing to a growing interest in the application of ICT in agriculture. Data collection, data modelling and analysis, and data sharing have become core challenges, an opportunity for innovation and a growth area for commercial development [26].

However, whereas there are clear potential benefits of the increased deployment of big data predictive tools, their potential risks should be adequately researched. The European Policy Centre warned that increased emphasis on digitalisation within agriculture should be steered adequately to prevent the increased stimulation of intensive farming and the optimization of unsustainable farming practices [15]. A major legal challenge associated with big data predictive tools largely stems from the way of processing large amounts of information accumulated through a variety of technical means, and the use of decision algorithms [15].

The existing body of literature recognizes big data predictive tools as a part of the future of agriculture [17,26,29,30]. As is often the case with new technologies, big data predictive analysis promises great opportunities that are inevitably accompanied by daring challenges. However, research often consists of economic-behavioural analysis and agricultural science analysis [17,31]. Research from a legal point of view is lacking, however. Moreover, most of the existing scholarship is not focused on European agriculture. The creation of an internal market that aims to cover a heterogenous European agricultural sector is deeply embedded in EU agricultural policy and legislation. Due to these specific characteristics of EU agriculture and, therefore, agricultural policy and legislation, it is valuable to conduct research that focusses on the EU. Additionally, few publications discuss environmental sustainability regarding big data predictive tools. Since environmental sustainability will be at the heart of future EU agricultural policy, it is necessary to explore this technology in the context of sustainable agriculture.

3.1. Big Data Analysis and the Possible Benefits and Harms to Sustainable Agriculture

With a global population expected to reach 8.5 billion in 2030 and 9.7 billion in 2050 [32], the challenge of how to increase agricultural production to achieve food security becomes more pressing every day [12]. The impact of climate change only amplifies this challenge and emphasizes the pressing need for a sustainable agriculture [12].

Precision farming is often regarded as one key approach for linking environmental and economic interests in the agricultural sector [33]. Precision agriculture is the scientific domain that deals with management of spatial and temporal variability to improve economic returns and reduce environmental impact [34]. This entails optimisation processes of production practices through a variety of technologies, such as site-specific management through data collection from the field, data analyses and evaluation, and decision support [33]. Smart farming entails the use of information and communication technology in the farm management cycle [35].

It is predicted that the next revolution in agriculture will be driven by sustainable precision and smart agriculture and the leveraging of technologies such as GPS, satellite imagery, Internet of things (IoT), cloud computing and artificial intelligence [12]. Big data analytics and decision-making will encompass this development [35].

With the ever-increasing amount of digital connections, both in terms of number of connections and time spent, the volume, velocity and variety of big data is only going to keep growing [35]. Investment in developing efficient processes and infrastructure is needed to support these technologies and ensure the efficacy of the data generated [36]. Vast amounts of data are collected with proximal, aerial or satellite-based sensors, in situ sensors (soil moisture sensors), on-farm weather stations and instrumented farm equipment. This qualifies as big data since it entails information assets that are characterised by high volume, high velocity, and high variety and that require specific technology and analytical methods for its transformation into value [37].

Predictive big data analytics is a technique used in big data analytics that can predict future outcomes based on gathered data [26]. Big data recommendation tools can be considered as intelligent application systems to help users with their decision-making [38]. In the agricultural sector, big data recommendation tools use algorithms to analyse data and make recommendations on crop management [38].

Agriculture is increasingly knowledge-intensive [38]. Knowledge derived from combining data from various sources can be used to derive valuable actionable insights. At the farm level, the farmers of today must deal with a myriad of data to be able to make livelihood-based decisions on a regular basis [37]. Data on soil health, weather, irrigation, markets, early-warning systems, diseases and pests, finance/loan availability, as well as government-related information/subsidies all come into play in making a decision at the farm level [37]. At the province or district level, the policymaker has to have real-time or near real-time information on market prices, projected yield of a particular crop at the end of the harvest season, beneficiaries of government schemes/subsidies, efficacy of pre-emptive actions to protect against diseases and pests, disaster mitigation and much more. At the national level, quality data would help design effective policies to assist smallholder farmers, monitor and remove inefficiencies in the value chains, ensure consumers about quality produce, eradicate hunger and malnutrition, and ensure food security in the country [37]. At the global level, a coalition including FAO and several national governments has launched significant support to help developing countries gather data on small-scale farmers to help fight hunger and promote rural development [39].

Agriculture, like other fields, will have the unprecedented capability to extract intelligence and make evidence-based decisions firmly grounded on real-time, reliable data and effective analytics. The introduction of new actors in the value chain and investment in developing the capacities of existing actors will be some of the key challenges that governments will have to address [37]. Newer ways of harvesting data will lead to granular insights that previously were impossible to achieve. The FAO urges that “we should not miss the opportunity to harness actionable intelligence from big data to achieve the Sustainable Development Goals. The cost of inaction would be far greater if we do not put in place an organizational data ecosystem to utilize the wealth of data that we have and to derive valuable insights from it” [37].

An intelligent and widespread use of data coming from sensors and Earth observations can substantively change agriculture. The smart use of big data in agriculture can increase productivity, food security and farmer incomes [40]. Better and intelligent use of available national and local datasets and data coming from sensors and Earth observations can lead to a more efficient use of natural resources, such as water and sunlight, in agricultural practices [40]. Big data tools can offer farmers access to data in real time to adapt their agricultural activities. For example, big data tools can provide information on their farm machinery and historic weather patterns, topography, and crop performance [40]. In this regard, the open data from the European Union’s Earth observation program, the Copernicus Program, can be of value [40] In addition, some specific EU big data projects need to be mentioned. AgroIT is a project that researches the creation of an open standard IT system that seeks to make agriculture more efficient. It built several tech applications on farms. Some of these applications collect data from various types of agricultural sensors and send them to a cloud environment for processing. Other applications access those data from the cloud environment and make them available to farmers on standard smart phones. Farmers use these applications to monitor their farms and plan their work more efficiently [41]. FOODIE is a research project dedicated to the use and promotion of open data for agricultural applications. FOODIE aims at enabling in an easy manner the (re)use of open data in the agricultural domain to create new applications that provide added value to different stakeholder groups [42]. The EUXDAT is an initiative that aims to showcase solutions through three pilot applications focused on sustainable agriculture and development. These will highlight how farmers and decision-makers can use real-time actionable intelligence by combining and analysing Earth observation data from satellites, meteorological information from robotic sensors in fields, and images from unmanned aerial vehicles. The project was finalized in October 2020 and addressed the relevance of big data to the Farm to Fork Strategy and a Green CAP [8].

The European Commission’s High-Level Expert Group on Artificial Intelligence recognizes environmental well-being as a requirement for trustworthy AI. The potential of AI systems to help the environment is highlighted [43]. However, the Guidelines stress that it must be ensured that this occurs in the most environmentally friendly way possible [43]. This encourages an approach to big data predictive tools’ benefits and harms in light of this recognized sustainability and ecological responsibility [8]. Big data is the backbone of AI-based applications which can be applied in agriculture for yield prediction algorithms based on weather and historical yield data and image recognition algorithms to detect pests and diseases in plants [44]. AI can enable farmers to analyse decades of weather and crop data and look for patterns to predict crop yields and monitor water and air conditions to help predict farming problems in specific regions. To learn new concepts and tasks and to reason and draw relevant conclusions about the world, AI has to have access to big data [44].

3.2. Big Data for Sustainable Agriculture

The data revolution in farming has given rise to smart farming management information systems. Sources of specific problems can be identified and predicted in advance through the combination of aggregated data from agricultural holdings with relevant datasets comprising data on weather, soil conditions, soil moisture and cropping intensity [45]. Predictive tools can be developed to help a farmer with production decisions, soil management and production insurance options [45]. Big data predictive tools promise to help farmers make well-informed decisions that improve the quality and quantity of their production, with less labour and less impact on the environment.

Big data analysis could stimulate more precise and sustainable pesticide and fertilizer use, while still ensuring yield [46]. In this regard, big data predictive tools could be an important instrument to facilitate a sustainable increase in productivity and income for farmers and the reduction of greenhouse gas emissions. Predictive models supported by big data enable crop prediction and yield prediction [38]. They can be used to offer recommendations for better seed placement in fields and selection of the best plant varieties and predict pest and disease outbreaks [46].

As the world already grapples with climate impacts, it becomes clear that an important policy reality should be addressed: adaptation is necessary and urgent, including a need for emergency responses to extreme climate impacts as well [46]. Moreover, some of the most important climate adaptation policies could also be among the most important development policies [46]. For example, one way to cut the impacts of climate change on agriculture is to adopt early warning systems and forecasting systems, so that farmers can adjust seeds, cropping methods, and planting times to reduce harmful impacts from weather events. Implementing those systems makes sense even without the pressing urgency of climate change [46].

Big data predictive tools could fulfil an important role in adapting and building resilience to climate change. Agriculture is dependent on a predictable and consistent climate. Farmers need to be able to plan in advance and make both short- and long-term decisions to harvest the best possible yield. Water shortages, droughts, heat waves, an increased and changing occurrence of weeds, and diseases are examples of climate change consequences that will increasingly impact agriculture and food production [47]. Big data predictive tools can enable farmers to adapt their farming methods to the effects of climate change and anticipate consequences such as weather conditions, pests and soil condition for both the short term and long term [46]. Specifically, these tools could predict plagues and disease outbreaks, offer recommendations for better seed placement in fields and selection of the best plant varieties [46]. Predictive weather tools could allow farmers to prepare for drought, heat, heavy rainfall and floods [48].

To realize these aforementioned opportunities, there is an important legal side that needs to be addressed. Looking at these opportunities through a legal lens highlights several needs. The sustainable development and use of big data predictive tools in agriculture requires support in the relevant EU legal instruments and efficient implementation by the member states. These opportunities depend upon equity and a fair, non-discriminatory distribution of benefits and access [30]. This should be safeguarded by EU legislation. In this regard, the inclusion of stakeholders is necessary. Understanding farmers’ decision-making behavior is an important factor. Small and medium-sized farms must be able to benefit from these technological advancements, and special attention should be paid to the inclusion of these farms [17,25]. Shared standards and criteria on data collection and data management are also an important element in guaranteeing quality data, quality predictions and the effective realisation of the possibilities of big data predictive tools for sustainable agriculture [17,25]. The realisation of the opportunities is also highly dependent on the extent to which users can interpret the model outputs. Therefore, it is essential that this is recognized by EU legislators and that knowledge support systems are established [49].

In addition, it is crucial that potentially harmful effects of big data predictive tools to sustainable agriculture are also identified and considered.

4. Results and Discussion

4.1. Big Data, Big Challenges

Big data shows great potential in the field of sustainable agriculture. However, it is questioned whether big data can capture all the complexities of agricultural systems [49]. Agricultural management can require considering very specific agricultural profiles and circumstances. Therefore, big data predictive tools cannot always be used across farms and farming conditions [49].

This harm can be tempered by investing in the creation of better data, data infrastructure, accessibility and information-sharing and guidance. The lack of criteria and standards on data quality, collection and management is a legal barrier that enables this harm. Therefore, it is important that EU legislators develop standards on data quality, data infrastructure, data security and data management and a shared and accessible data space. Both large farms and smallholder farms, governments, NGOs, and agro-industry should be included and consulted. Different agricultural and climate and environmental profiles for the member states need to be represented. Farmers should be able to access guidance on how to use and interpret big data analytics output, taking into account their specific profiles.

In its 2017 report, the FAO stressed that agricultural chemicals have had “catastrophic impacts on the environment, human health and society as a whole” [50]. Regardless of how efficient industrial agricultural practices may become under new tools, they often continue to rely on damaging chemical inputs [49]. In big data analytics, the quality and quantity of the data are essential. In this regard, more data are not always better. In many instances, aimlessly gathering a broad set of data provides little value, and we will find ourselves, as stated by the FAO, “drowning in data but thirsty for knowledge” [37] Therefore, the need for good quality data needs to be underscored [37]. Otherwise, the FAO warns, we would fall into a garbage-in-garbage-out scenario [37]. When the data input is based on intensive agricultural methods and pesticide and fertilizer use, this can lead to overpredictions on this matter. Again, this harm is enabled by the legal barrier that rises from the lack of standardization on, for example, data quality and data management criteria and guidelines.

Whether big data is going to serve productivity and sustainability, or productivity at the expense of sustainability, is an essential question that requires careful thought about which technologies to fund, how to regulate them, and for whom [51]. It is not yet clear whether big data tools, and digitized agriculture in general, are going to mostly favour large-scale, intensive industrial farming [51,52].

This technology could disadvantage smallholder farmers and lead to a “digital divide” [30,53,54]. However, it also can be argued that smallholder’s disadvantages and the information asymmetry can be reduced using these applications [55]. Big data predictive tools can restrict the farmers’ agency in farm management decisions. Monsanto is currently the most prominent biotech agribusiness to buy into big data. Sensors on tractors are able to monitor or dictate every decision a farmer makes, which enables Monsanto to aggregate large quantities of valuable, previously proprietary farming data and unique insights on a field-by-field basis [53]. Dependence on large agri-food companies, due to the concentration of big data technology in their hands, can significantly limit sustainable agriculture [38]. Agro-companies are developing big data tools to assist in crop and soil management. These services are tied to the products produced by these companies, such as their chemicals. Monsanto collects real-time data from a farmer’s fields using ground sensors and then identifies which of its hybrid seeds “best match” to the field conditions [56]. The algorithm also recommends Monsanto’s chemical products to be bought as part of the advice for the complete growing season [57]. This can maintain an excessive use of pesticides and fertilizers by farmers. These paid services provided by the agro-industry collect large volumes of data. However, these data are not publicly available and can only be obtained through paid subscriptions [18]. Therefore, it is important to determine carefully which data can be collected to attain environmental goals rather than merely commercial objectives. Determining which data can remain proprietary versus those which ought to be opened to meet environmental challenges will require careful decision-making in the public and private sector [51]. In this regard, there are pressing key questions and issues that need to be addressed in the further development of digital technology and big data in agriculture, specifically around data ownership, trust, equity, distribution of benefits, and access [30]. These issues need to be addressed quickly, as otherwise the decisions would be left to the large agricultural companies, which are already investing in digitalization, to increase their market power based on their data [51]. generated by private firms selling data or advisory services are not public and not findable or accessible, often even by farmers themselves. There are no established data standards being used, and thus data are not interoperable even when findable and accessible [17]. This forms a major legal barrier. This can be overcome if EU legislators create data standards and a shared, accessible data platform. This will enable access to quality and usable data and predictions serving sustainability and agency for the farmers.

The aforementioned possibilities and harms of big data predictive tools present several important legal barriers. The lack of accessible, interoperable, quality, unbiased data, the exclusion of farmers and smallholder farmers in particular, the lack of data standards and data management criteria, and the lack of guidance and education on the use and interpretation of big data predictive tools are the most prominent practical and legal barriers that hinder the realisation of the possibilities of big data predictive tools for sustainable agriculture. The next chapter will identify the relevant legal instruments and review to what extent the identified opportunities and harms and the uncovered legal barriers are addressed.

4.2. Do the CAP and Green Deal Capture Predictive Tools?

4.2.1. The CAP

Since 1962, the EU has provided a common agricultural policy for all member states. In the words of the European Commission, the CAP is a partnership between agriculture and society on the one hand, and between the EU and EU farmers on the other. This common policy aims to support European farmers financially, improve agricultural productivity and keep the rural economy viable [58]. In achieving these goals, the protection of the climate and the promotion of sustainable management of natural resources must be ensured [58]. The CAP has its legal basis in the Treaty on the Functioning of the European Union (TFEU), which stipulates that the internal market also includes agriculture and trade in agricultural products and that the EU should define and implement a common agricultural policy [59].

The CAP has a direct impact on the environment through the financing of agricultural activities [60]. On 2 December 2021, the agreement on the reform of the CAP was formally adopted. The new legislation, which is due to take effect in 2023, is supposed to “pave the way for a fairer, greener and more performance-based CAP” [61].

A smarter, modernised and more sustainable agricultural policy needs to embrace research and innovation in order to serve the multi-functionality of agriculture, investing in technological development and digitalisation, as well as improving the access to impartial, sound, relevant and new knowledge Article (23) [5]. Therefore, the CAP identifies the modernisation of agriculture by fostering and sharing of knowledge, innovation and digitalisation as a cross-cutting objective Annex I, [5]. Specifically, the future CAP will pursue an increase in competitiveness, including greater focus on research, technology and digitalisation, contribute to climate change mitigation and adaptation, as well as sustainable energy, and foster sustainable development Article 6, [5]. The member states are expected to describe the elements that ensure the modernisation of agricultural policy in their national strategic plans. This should, in particular, include their national strategies for the development of digital technologies in agriculture and for the use of these technologies to improve effectiveness and efficiency Article 114, [5].

In assessing these national strategic plans, the share of farmers benefitting from support for digital farming technology through CAP and the share of farmers receiving investment support to restructure and modernise, including to improve resource efficiency, are seen as indicators to assess progress on implementing these CAP objectives Annex I R3, Annex 14 R9, [5].

While the general direction is dictated by the Commission, the new CAP mainly puts the steering wheel into the hands of the Member States. Although the CAP offers several tools that could support climate smart digitalisation, such as the eco schemes, farm advisory services, AKIS and rural development measures, the national administrations will have the flexibility to decide on the ‘how’ of it all.

In order to stimulate the development of big data analytics and digitalisation in agriculture across the EU, public funding is essential. Financial resources could be an important incentive for member states and farmers to make their land and environmental data available and implement agricultural technologies in an environmentally sustainable way [57]. These financial resources should not be the sole responsibility of the member states [57]. Big data predictive tools can contribute to the development of sustainable agriculture. However, digitalisation of agriculture and climate change mitigation and adaptation are mainly included in the CAP through more general objectives or voluntary measures [57].

The use of big data predictive tools in agriculture could be stimulated under eco-schemes or rural development payments. However, since these both rely strongly on the initiative of the member states and farmers, it is uncertain whether big data predictive tools will be supported at all, let alone in a sustainable context.

In addition, the CAP primarily supports the largest (most hectares) and most mechanized farms [45]. This does not meet the needs of sustainable smart farming. Focusing on the largest farms contributes to the uneven adoption of digital products and services in EU agriculture [45]. This maintains inequity and inequality as an important legal barrier to the sustainable adoption of big data predictive tools. Smallholder farms risk being excluded and falling behind the larger farms.

The current policy framework also lacks means for the combination of large amounts of public data on agriculture with private data collected on the farms. The most important databanks for the CAP function independently from the growing volumes of smart farming data originated at the farm level. The smart farming data chain is currently mainly shaped by private contractual agreements between farmers and agricultural technology providers. These contracts often neglect potentially negative and unsustainable effects of the established data flows [45]. Although the EU industry-level code of conduct on agricultural data sharing promotes transparency and ethical business practices in agricultural data harvesting, collection and use, it does not address broader needs for developing sustainability in smart farming [45]. In this regard, the ambition of the Farm to Fork Strategy to create an accessible and interoperable shared data space can address this barrier. However, the lack of provisions that encourage the industry to tackle unsustainable effects of their data flows remains an important legal barrier.

The CAP should include more specific measures directed towards supporting digitalisation and data production in farms of various types and sizes, with particular attention to the support of smaller farms [45]. By combining publicly owned databanks along with aggregated and anonymized smart farming data collected from farmers or purchased from agricultural technology providers, the smart farming databanks and insights could be expanded.

The CAP mainly sets general targets and does not refer to the specific targets of the Farm to Fork strategy. If the CAP could offer more specific sustainable requirements for implementing (climate smart) digital technologies in agriculture, including leveraging big data predictive tools, the realization of the CAP’s objectives to develop resilient and sustainable farming systems in the EU would be more certain and effective [45].

Although the CAP does include environment and climate-related measures and provisions on digitalisation, it seems inadequate to introduce and stimulate a widespread sustainable adoption of digital agriculture technologies, such as big data predictive tools. The rather unspecific objectives and requirements, the continued focus on larger agricultural holdings, the substantial flexibility for the member states, and implementation and enforcement issues suggest that the possibilities of big data predictive tools will not fully reach their potential and that the harms risk not being addressed.

However, the CAP puts forward many objectives that scope a much wider policy than environmental sustainability. As such, it cannot be considered an environmental instrument and is not linked to the EU environmental policy. The CAP could change course and adopt a more environment-based approach. This would entail rewarding environmental services delivered rather than reserving finances based on hectares and offering more incentives for farmers to mitigate and adapt to climate change. Additionally, it should support technologies such as big data predictive tools through farm advisory services, supply chain development and infrastructural investments with a clear focus on environmental sustainability. In adopting a more environment-focused approach including specific requirements for climate change mitigation and adaptation technologies, the CAP could play an important role in keeping farmers in business while mainstreaming the sustainable use of agricultural digital technologies. However, this requires a fundamental change. With the CAP set for 2023–2027, a full and swift change has become highly unlikely. Since the road to this CAP was already difficult and full of compromise, the CAP might prove not to be the appropriate catalyst for the transition of EU agriculture needed to deal with the environmental crisis and consequences of climate change on agriculture.

4.2.2. Green Deal

The Green deal sets direction to a resilient and sustainable agriculture. Concerning agriculture, this course of action is reflected and substantialized in the Farm to Fork strategy. The Farm to Fork Strategy is the first attempt at a European-wide approach to food systems of this scale [62]. Further, in dealing with an EU-wide approach, the importance of context-specific solutions is clear in the differing needs, legislative set-ups, and ecoregion types across the member states [62]. Agriculture and nature exist in a complex and dynamic interdependent system that cannot be understood or protected by looking at mere components [62]. This complexity must be reflected in policy.

The Farm to Fork Strategy states that farmers need to make “the best of use technological and digital solutions to deliver better climate and environmental results, increase climate resilience and reduce and optimize the use of pesticides and fertilizers” [63]. It can be pointed out that the Farm to Fork Strategy seems to link the use of digital technologies to delivering sustainable agriculture goals. In other words, farmers are called upon to make use of technologies to deliver better environmental results. Therefore, it can be questioned whether big data predictive tools that are biased to intensive farming methods can be considered "the best of use". This could offer a promising benchmark to look at the use of big data predictive tools in agriculture and criticize unsustainable effects.

The Farm to Fork Strategy strongly recognizes knowledge transfer and advice as key factors in achieving a sustainable agriculture, 3.2 [63]. Within this context, it also states that farmers have a particular need for objective, tailored advisory services on sustainable management choices. Big data predictive tools could offer such service. However, following the Farm to Fork strategy, it needs to support environmental sustainability, which would require tackling risks such as bias. Provisions on standardization and data management are, in this regard, important factors lacking in the Farm to Fork strategy.

The development of Big Data analytics platforms is challenging. In this regard, lack of rural connectivity is a barrier that inhibits digital agriculture. Across many rural areas of Europe, there is a lack of broadband. The aim of the Commission to accelerate the roll-out of fast broadband internet in rural areas is promising and would support the uptake of digital technologies such as big data predictive tools, 3.1 [63]. This commitment is, however, formulated broadly without any specific targets. Therefore, it remains to be seen to what extent this aim will be substantiated within this short time frame.

The fact that the Farm to Fork Strategy emphasizes the importance of a common European agriculture data space can be welcomed. The realization of the possibilities and mitigation of the harms is dependent on the availability and quality of agricultural data. It is, therefore, important that the EU will be involved. A shared EU data space will support the sustainability of EU agriculture through the processing and analysis of production, land use, environmental and other data, allowing precise and tailored applications at the farm level, 3.2 [63]. If the Commission succeeds in developing and supporting a European data space that guarantees both availability and data quality, this will benefit the sustainable use of big data predictive tools in agriculture. A platform on which both companies and farmers share and access data can help prevent data concentration and control by large companies. This would help farmers to maintain their independence [57]. A shared data space would mitigate some of the most important legal barriers regarding accessibility and equity. However, the Farm to Fork Strategy does not mention the need to create data standards and management criteria. This is important to lift the legal barrier of bad data quality and interoperability.

The overarching aim of the Biodiversity Strategy 2030 to bring nature back into our lives must, therefore, include bringing nature back into agriculture [62]. In other words, agricultural practices should seek harmony with local ecosystems, and production systems should promote biodiversity growth in the region rather than depleting it [62]. However, big data predictive tools risk failing to capture all of these complexities [49]. Agricultural management involves very specific agricultural profiles and circumstances. Therefore, it should be carefully assessed whether predictive tools can be used across farms and farming conditions [49]. In order to cope with this risk, and to align with context-specificity highlighted both in the Farm to Fork Strategy and the Biodiversity Strategy 2030, this assessment should also explicitly consider the local environmental conditions and ecosystems.

Packed with ambitious targets, the Farm to Fork Strategy aims to create a “fair, healthy and environmentally friendly” food system. However, it has been pointed out that there is a gap between the Farm to Fork ambitions and the reality on the ground. In this regard, it is can be questioned whether the strategy focuses too much on the farmers, rather than the entire chain [64]. Concerning big data predictive tools, this is especially relevant because the risk of bias to intensive agriculture and concentration of big data technology in the industry’s hands are problems situated at industry level rather than farm level. The creation of data standards for quality and management by EU legislators could offer help. However, this is lacking in the Farm to Fork Strategy and, therefore, remains a legal barrier.

Although healthier and affordable food as well as better incomes for farmers are objectives, the 2030 quantitative objectives only target agriculture with the reduction of fertilizer use by 20%, the reduction of pesticides and antimicrobials by 50%, 25% of agricultural land to be used for organic farming and 10% in high-diversity landscapes features [63]. The policy and financial means to achieve this highly improved ambition remain questionable [65]. The Farm to Fork Strategy paints a picture of a future where digital technology and agriculture are intertwined [66]. It is yet to see whether the EU legislators will have the necessary courage to allow and stimulate new technologies [66]. Farmers need the right tools to shift to more sustainable practices while maintaining their economic stability. This is crucial both for farmers and for the environment, the climate and our rural areas. Consequently, there is a need for strong funding methods to promote innovation and digitization [66]. It is questioned whether sufficient tools are ensured for farmers to make the necessary investments and changes for a digital transition [64].

As stated by Norbert Lins, chair of the European Parliament’s Agriculture and Rural Development Committee, the “Farm to Fork strategy may be a silver bullet, but it will not be implemented by magic" [66].

The Farm to Fork Strategy points to research and innovation as key drivers in accelerating the transition to sustainable, healthy and inclusive food systems [63]. To achieve these objectives, digital technologies are presented as a steppingstone. The Farm to Fork Strategy calls upon the Commission to work with member states on strengthening the role of the European Innovation Partnership for agricultural productivity and sustainability in the CAP strategic plans. This is seen as an essential step to accelerate innovation and knowledge transfer. In addition, the Farm to Fork Strategy states that member states will need to scale up support plans for AKIS and strengthen resources in their CAP strategic plans to develop and maintain appropriate advisory services needed to achieve the Green Deal objectives and targets [63]. The Farm to Fork Strategy considers research, innovation and knowledge transfer as pivotal to establish a green and digital transformation of EU agriculture. In relation to financial investments, the Farm to Fork strategy also refers to the CAP. It requires the CAP to increasingly facilitate investment support to accelerate the green and digital transformation of agricultural holdings [63]. This is important, considering that the Farm to Fork Strategy has been criticized as falling short in allocating financial means for the sustainable and digital transition of agriculture [64].

It is remarkable that the Farm to Fork Strategy refers these essential elements, at least in part, to the CAP. This points to how the CAP must be interpreted within the Green Deal framework, and to what extent it can be considered a Green Deal instrument. The Commission expressed that it remains fully committed to a reformed CAP delivering the Green Deal objectives. As the Executive Vice-President Frans Timmermans and Commissioner for Agriculture Janusz Wojciechowski will engage in support of finalizing the negotiations, the Commission set up a dedicated task force to ensure a whole-of-government approach and provide all necessary support for an ambitious outcome [67]. This is a sign that the CAP is a Green Deal policy [67].

It should be pointed out that by referring to the CAP, the extent to which these goals will be achieved becomes dependent on the success of the CAP. This can be uncertain and differentiated since the future CAP places significant power with the member states to implement the CAP based on local needs. Member states have already started to devise their National Strategic Plans to implement the future CAP. Coordinating and aligning the 27 National Strategic Plans with the Green Deal objectives remains a considerable challenge [65].

With the Farm to Fork Strategy, the Green Deal aims to build a sustainable food system, safeguarding a new and better balance of nature, food systems and biodiversity, protecting people’s health and well-being, and at the same time increasing the EU’s competitiveness and resilience [63]. The CAP seeks to increase competitiveness, including a greater focus on research, technology and digitalisation, Article 6(b) [5], contribute to climate change mitigation and adaptation, Article 6(d) [5], as well as sustainable energy, and foster sustainable development. An efficient sustainable food system needs policy coherence across these different objectives. Fragmented governance could lead to delays in policy implementation and threaten progress [62]. In addition, a sustainable food system is one that reframes efficiency so that it means food systems deliver profits, healthy diets, and a healthy planet, rather than trade, yield (increasing), and price (decreasing) policies [68]. With a multitude of objectives, it is uncertain whether the CAP corresponds with the Green Deal, which puts sustainability at the forefront of future EU agricultural policy with a Farm to Fork Strategy that highlights the link between digitalisation and achieving climate and environmental goals.

5. Conclusions

The data revolution in farming has given rise to a significant opportunity for big data predictive tools to tackle the environmental and climate crises in agriculture. Predictive tools could potentially be deployed to help farmers with production decisions, soil management and production insurance options [15]. Big data predictive tools promise to help farmers make well-informed decisions that improve the quality and quantity of their production, with less labour and less impact on the climate. These models could stimulate a more precise and decreased use of pesticides and fertilizers, while still ensuring yield [46]. In this regard, big data predictive tools could be important instruments to facilitate the sustainable increase in productivity and income for farmers and the reduction of greenhouse gas emissions. Big data predictive tools could fulfil an important role in adapting and building resilience to climate change as well. They can enable farmers to adapt their farming methods to the effects of climate change and anticipate consequences such as drought, heat, extreme weather, floods, diseases and plagues.

However, this contribution showed that big data predictive tools, as such, do not guarantee a more sustainable agriculture. In this regard, some specific risks were highlighted. It is questionable whether big data can capture all the complexities of agricultural systems [49]. In addition, it is important to point out that regardless of how efficient industrial agricultural practices may become under new technologies, they often continue to rely on damaging and excessive chemical inputs [49]. When the data input is based on intensive agricultural methods and pesticide and fertilizer use, this could lead to unsustainable recommendations on this matter. To avoid this scenario, the need for good quality data needs to be underscored. Otherwise, big data predictive tools, and digitized agriculture in general, might very well merely favour large-scale, intensive industrial farming [49,69].

In addition, this technology could disadvantage smallholder farmers, lead to a digital divide, and further a power asymmetry [53] Big agro-chemical companies are already investing in big data analysis to increase their market power [53].

Dependence on large agri-food companies, due to the concentration of big data technology in their hands, can significantly limit the benefits of this technology for sustainable agriculture [38].

Tackling these risks of big data predictive tools while still leaving room for the advantages to flourish will require careful decision-making in both the public and private sectors [51]. Agriculture and nature exist in a complex and dynamic interdependent system. Neither agriculture nor nature can be understood or protected by looking at mere components [62]. Smart agriculture technologies such as big data predictive tools have the potential both to support or damage this complex dynamic. Here, the importance of steering this technology within a strong policy framework comes to play.

To answer the research question on how the EU CAP and Green Deal address potential harms of big data predictive tools while safeguarding their possibilities, one can conclude that, at least in theory, both the Green Deal [63] and the CAP, Article (23), Article 6, Article 114, Annex I [5] recognize the possibilities of digital technologies such as big data predictive tools for agriculture and emphasize the necessity of environmental sustainability in this regard [70]. However, some of the most promising and essential elements of achieving sustainable digitalisation in agriculture risk not being substantiated because of a watered-down CAP, significant focus on larger farms, and strong member state margin of appreciation. Although at first sight, the CAP and Green Deal might seem aligned, and the intention might well be there, it can be concluded that their depth has yet to be proven [62]. Whether this depth can be substantiated will also determine the extent to which digital technologies, such as big data predictive tools, will help enforce a sustainable agriculture or merely intensify unsustainable practices in the EU.

By tackling the barriers and weak links concerning data ownership, data security, data quality and standards, equity, distribution of benefits and access, EU legislators can address the harms while safeguarding the possibilities of big data predictive tools for sustainable agriculture. In doing so, the use of big data analysis in agriculture should be aligned and linked with the EU environmental targets and climate protection.

These findings have several important implications. The CAP has put the steering wheel mainly into the hands of the national administrations, and most the national strategic plans are submitted for the Commission to assess. These findings can offer a benchmark and raise public awareness to increase pressure on national administrations and the Commission to take the Green Deal’s ambitions to heart, and with it, the sustainable digitalisation of agriculture. The Green Deal sets the EU agricultural policy for years to come. Pointing out these findings early on can be helpful for future legal initiatives within this framework.

However, several issues remain unresolved. This implies the great importance of continuing research on this topic. Further research should follow up on the national strategic plans and the Commission’s assessment and review to what extent sustainable digitalisation is stimulated. Research mapping out the risks and possibilities of big data analytics for sustainable agriculture in the context of the EU data strategy is also required. Further, it should be highlighted that the extent to which big data analytics can facilitate an appropriate response to agricultural sustainability standards, such as integrated pest management, offers fertile ground for further research, in particular considering the revision of the Sustainable Pesticides Directive [62].

The findings presented in this contribution are of vital and timely importance. Big data predictive tools are a double-edged sword that can either be an ally or an enemy in fighting the environmental crisis and climate change. Agriculture and food production are responsible for significant greenhouse gas emissions [3,71,72]. At the same time, the agricultural sector is increasingly suffering from the adverse impact of climate change [4,73]. Substantial damage due to climate change will increasingly occur and will cause reduced yields as a result of drought, heat, extreme weather, floods, increases in disease and plagues. We need to make the most of this timely opportunity to realize a more interconnected approach to agriculture, digitalisation and environmental sustainability. Courageous and ambitious EU policy and member state implementation are needed now, to provide food security for many generations to come. With the CAP putting the member states at the forefront and the Green Deal as a vague policy backbone, the course is set until 2027.