A Multicriteria Analysis Approach for Evaluating the Performance of Agriculture Decision Support Systems for Sustainable Agribusiness

Abstract

Agriculture decision support systems (DSSs) play an important role in facilitating evidence-based agricultural decision-making for improving agribusiness productivity. Evaluating and selecting the most appropriate agriculture DSS for sustainable agribusiness is, however, challenging due to the existence of production and marketing alternatives, a variety of objective functions from economic to lifestyle to long-term sustainability, and the subjectiveness and imprecision involved in the evaluation process. To help decision makers effectively deal with these issues, this paper presents a multicriteria analysis approach for evaluating and selecting the most appropriate agriculture DSS for sustainable agribusiness. The subjective assessments of decision makers in the evaluation process are formulated using linguistic variables approximated by fuzzy numbers. The concept based on the positive and the negative ideal solutions is applied for producing a performance index value for every agriculture DSS alternative across all evaluation criteria based on which the most appropriate agriculture DSS is. An empirical study is presented for demonstrating the step-wise process for evaluating and selecting the most appropriate agriculture DSS for sustainable agribusiness. The outcome from the performance evaluation process allows agribusinesses to effectively adopt appropriate agriculture DSSs for achieving competitive advantages.

1. Introduction

Agriculture development plays an important role in the global economy. According to the World Bank, the agricultural sector can boost shared prosperity and feed a projected 9.7 billion people by 2050 [1]. The global agriculture markets focus on the development of agricultural activities to reduce crop yield cycles, high-quality fertilizers production, and effective management of supply chain management to reduce food wastage [2]. With the unpredictable changes in climate and growing concerns with the impact of farming, agribusinesses are facing complex challenges in managing their farming activities. In particular, they have to make decisions with respect to the inputs of fertilizers and pesticides, and the limited supply of natural resources like water, soil, and energy with the broader objective to maintain crop production in improving their profitability, while reducing negative environmental impacts as much as possible. These factors have motivated a search for ways whereby knowledge can be incorporated into a decision-making system (DSS) to help agribusinesses make effective agricultural management decisions [2,3].

The application of DSS for solving general decision-making problems has become increasingly popular due to its flexibility and adaptability for tackling various decision situations effectively and efficiently [4]. However, it is important for the decision maker to select the right DSS for development and implementation. Previous studies have been conducted on multicriteria decision-making approaches for the selection of information systems, management information systems, and, in particular, DSSs. For example, Akoka [5] developed a framework for dealing with the DSS evaluation problem. The framework could be used as a guide to determine the best approach for evaluation that is suitable to the characteristics of the DSS concerned. Evans and Riha [6] presented an approach to develop guidance for evaluating DSSs. This approach was based on assessing the realistic (objective) value of information, as opposed to current approaches that are based on the perceived (subjective) value of information.

Phillips-Wren et al. [7] presented a framework for evaluating DSSs that combined outcome- and process-oriented evaluation measures. By using the Analytic Hierarchy Process (AHP) approach, the authors found that the real-time DSS could offer a significant improvement in terms of process-related characteristics. Phillips-Wren et al. [8] proposed a framework that was implemented with the AHP to measure the overall decision value of a DSS and determine the precise contributions of individual characteristics to the value. Gerogiannis et al. [9] presented an integrated approach that combined the Technique for Order Preference by Similarity to Ideal Solution with intuitionistic fuzzy group decision-making for evaluating DSSs. The approach enabled the decision makers to distinguish between benefit criteria and cost criteria. It also considered the vagueness and imprecision of decision makers’ assessments when evaluating the available alternatives. Gerogiannis et al. [10] developed a hybrid approach that exploited the benefits of the group AHP along with the simplicity of the scoring model for justifying the final selection. The proposed approach permitted consideration of more criteria than what a typical AHP approach can handle, and took into account experts’ opinions for the comparison of the information systems as well as users’ opinions on specific preferences/needs. These studies show that the existing approaches are capable of adequately dealing with the evaluation and selection of information systems, management information systems, and DSSs in different contexts.

The potential benefits of DSSs have led agribusinesses to focus their attention on the application of agriculture DSSs which are capable of deploying information and communication technologies to gather the best available farming data in building knowledge repositories to facilitate sustainable agribusiness [3]. Agriculture DSSs can support agribusinesses in making effective decisions on crop productivity by incorporating an enormous amount of information including inputs on climate, water, fertilizer, landscape, human, and economic resources [11]. Such agriculture DSSs play an important role in enhancing agricultural efficiency [3].

Despite the significant interest of agribusinesses in adopting an agriculture DSS for sustainable agribusiness with an understanding of the various benefits of agriculture DSS, the uptake has been poor [2,3,11,12]. Moreover, agribusinesses found that their adopted agriculture DSSs failed to fulfil the intended expectations of their business needs; one significant reason was attributed to the inappropriate choice of agriculture DSS [3,11].

Each agribusiness needs to customize its agriculture DSS by taking into account various concerns relating to its business needs. The range of concerns include the availability of numerous production and marketing alternatives available to agribusinesses, a variety of objective functions for agribusinesses from economic to long-term sustainability, and the subjectiveness and imprecision involved in the decision-making process. Nevertheless, the numerous criteria selections can engender difficulties for agribusinesses in their evaluation and selection of the most appropriate agriculture DSS for sustainable agribusiness. As a result, the development of an effective approach for comprehensively evaluating the overall performance of agriculture DSS is critically important [2,13].

Much research has been done on the development of appropriate approaches for evaluating the performance of agriculture DSSs [2,3,14,15]. Maynard et al. [14], for example, developed a multiple constituency approach for evaluating the performance of agriculture DSSs. Hochman and Carberry [15] conducted participatory action research for exploring the effectiveness of existing agriculture DSSs. Rose et al. [2] proposed a multicriteria checklist for assessing the performance of agriculture DSSs. Mir and Padma [3] developed a generic multicriteria framework for evaluating agriculture DSSs. The above research has demonstrated the practical benefits of the development of a multicriteria approach for evaluating the performance of agriculture DSSs. These studies, however, are not totally satisfactory due to (a) the inadequacy of handling the subjectiveness in the evaluation process, and (b) the cognitively demanding process on the decision maker. It is thus desirable to develop a simple [16], practical, and effective [3,13,17] approach capable of addressing the above shortcomings.

This study attempts to fill in the research gap by designing a multicriteria analysis approach for evaluating the performance of agriculture DSSs for sustainable agribusiness. Based on the technology–organization–environment (TOE) framework [18] and the triple bottom line principle of sustainability [19], this paper explores technology, organization, environment, economic, and social evaluation criteria for evaluating the performance of agriculture DSSs for sustainable agribusiness. The subjective assessments that are selected during the evaluation and selection process are represented by linguistic variables approximated by fuzzy numbers. This study is the pioneer that integrates the TOE framework with the triple bottom line principle of sustainability for evaluating the performance of agriculture DSSs for sustainable agribusiness. The outcome from the performance evaluation process allows agribusinesses to effectively adopt appropriate agriculture DSSs for achieving sustainable competitive advantages.

In what follows, Section 2 reviews the agriculture DSSs assessment criteria. Section 3 proposes a multicriteria analysis approach. Section 4 presents an empirical study for demonstrating the applicability of the approach, followed by the discussion in Section 5, and the conclusion of this paper in Section 6.

2. Agriculture DSSs Evaluation Criteria

Evaluating the performance of agriculture DSSs is complex and challenging, mainly because of the existence of many conflicting and incommensurable criteria, the multidimensional nature of the decision-making process, and the fuzzy data generated from imprecise judgments of decision makers in the evaluation process using qualitative performance ratings. To effectively solve this problem, an overall evaluation of the performance of agriculture DSSs for sustainable agribusiness is desirable [2,3].

Previous research has explored various factors of the poor uptake or non-implementation of agriculture DSSs [3,11,13,15,20,21,22]. Such factors can be categorized into technology, organization, and environment dimensions (see

Table 1. Critical performance evaluation factors on sustainable agriculture decision support systems (DSSs).

Type	Factors	Source
Technology	performance; ease of use; cost; benefits; relevance; real time information updates; data accuracy	[12,15,22,23]
Organization	farming type; farming scale; culture; budget; owner characteristics	[2,15,16,20]
Environment	peer recommendation; incentive	[2,3]

). The technological context encompasses all technologies that are relevant to the respective agribusiness [12,15,22,23]. The organizational context addresses the characteristics and resources of the agribusiness, including its formal and informal linking structures between decision makers, communication processes, and the number of slack resources [2,15,16,20]. The environmental context characterizes the external context in which an agribusiness conducts its operational and economic activities, including peer recommendation and incentives [2,3].

Nowadays, agribusiness has to focus on sustainable agriculture management which plays an important role in improving the profitability of their business. Sustainable agriculture is related to the triple bottom line approach [19]. In particular, the sustainability of the agribusiness depends upon its capacity to satisfy the human demands, protect the natural environment, and provide employment and economic growth [24,25]. Environmental, social, and economic sustainability have a positive impact on the competitiveness of the agribusiness. Therefore, our multicriteria analysis approach for evaluating the performance of agriculture DSSs incorporates the three dimensions of the triple bottom line principle of sustainability (economic, environmental, and social) and TOE. As such, the criteria for evaluating the performance of agriculture DSSs for sustainable agribusiness can be classified into five perspectives: (a) technology dimension; (b) organization dimension; (c) environment dimension; (d) economic dimension; and (e) social dimension. Figure 1 shows the hierarchical structure of criteria and sub-criteria for the evaluation of agriculture DSSs.

The technology dimension (C₁) refers to specific characteristics of an agriculture DSS for supporting decision-making in agricultural management [23,26]. Prior research tends to focus on the technology and implementation of the agriculture DSS. For instance, Hochman and Carberry [15] and Matthews et al. [27] looked at how agriculture DSS developers could input more critical farming data into the DSS so that decision makers in agribusinesses could convert the data to knowledge in responding to opportunities and challenges. Han et al. [22] contended in their empirical study that data input such as historical weather data, soil type, fertilizer application, and irrigation method had an impact on agribusinesses’ decisions on their agricultural operations. On the other hand, Rose et al. [2] investigated implementation issues in agriculture DSSs. Their empirical studies contended that ease of use is an important technology factor. When farmers face difficulties in using the agriculture DSS, they tend to abort the implementation. The criteria for the technology dimension are measured by performance expectancy (C₁₁), ease of use (C₁₂), cost (C₁₃), and relevance (C₁₄). The performance expectancy (C₁₁) is related to the degree to which the end-users believe that using an agriculture DSS will help them improve their decision-making. The ease of use (C₁₂) reflects on how easy the user interface is to navigate. The cost (C₁₃) is related to the initial and on-going cost for using an agriculture DSS. The relevance (C₁₄) concerns the usefulness of an agriculture DSS to an individual agribusiness.

The organization dimension (C₂) is concerned with the characteristics of an agribusiness for the adoption of an agriculture DSS [2,26,28]. Prior research tends to focus on the behavior of various stakeholders who are involved in the adoption of the agriculture DSS. For instance, Hochman and Carberry [11] and Rose et al. [2] emphasized the participation and commitment of stakeholders in the designing and delivery of the agriculture DSS. Furthermore, implementation problems can resolve more effectively when decision makers in agribusinesses and DSS developers share similar goals throughout the agriculture DSS innovation [11,12,19]. Along the same vein, Rose et al. [2] asserted that decision makers in agribusinesses must have confidence in the adopted agriculture DSSs, as they are relying on them to make informed decisions. Moreover, they emphasize that factors such as farming type, level of marketing, lack of budget, and peer recommendation can influence the adoption of the agriculture DSS. The criteria for the organization dimension are measured by the scale of farming (C₂₁), culture (C₂₂), and owner characteristics (C₂₃). The scale of farming (C₂₁) is related to the size of the farming organization. The probability of adopting an agriculture DSS is higher in larger agribusinesses [2]. The culture (C₂₂) focuses on the attitudes and behavior of workers for the adoption of an agriculture DSS. The owner characteristics (C₂₃) concerns the degree to which the owner of an agribusiness is likely to accept a certain agriculture DSS.

The environment dimension (C₃) emphasizes the external influences on decision makers in agribusinesses in their adoption of agriculture DSSs [2,3]. It is reflected from two perspectives, including the agribusiness environment and the natural environment. The criteria for the environment dimension are peer recommendation (C₃₁), incentive (C₃₂), environmental competency (C₃₃), and energy consumption (C₃₄). The peer recommendation (C₃₁) is related to the degree to which end-users know an agriculture DSS from significant others. The incentive (C₃₂) refers to the financial incentives that governments provide for the agribusinesses to adopt agriculture DSSs for assisting their decision-making. The environmental competency (C₃₃) measures agribusinesses’ competency in their planning and executing of the environmental policies [29]. The energy consumption (C₃₄) concerns the energy consumption of the use of agribusiness DSSs in agribusiness.

The economic dimension (C₄) emphasizes agribusiness activities that generate profits while reducing operating costs and improving product quality [30]. Prior research tends to focus on profit maximization of products, and flexibility and cost minimization of operational processes [29]. Munonye and Esiobu [31] asserted that financial resources were a crucial enabler for agribusinesses to achieve sustainability. In another context, Guo et al. [32] found that cost and quality were important criteria for evaluating and selecting green suppliers for the apparel supply chain. Similarly, Luthra et al. [33] emphasized that product pricing and product quality played an important role in assisting organizations to decide when selecting and evaluating their sustainable suppliers. The criteria for the economic dimension are therefore measured by productivity (C₄₁), profitability (C₄₂), and quality (C₄₃). The productivity (C₄₁) measures the quantity of output produced with a given quantity of inputs. Improving productivity with the use of agriculture DSSs contributes to profitability and competitiveness because it allows farmers to produce more output using fewer inputs [2]. The profitability (C₄₂) reflects the potential profit and financial gain of agribusinesses in using the agriculture DSS. The quality (C₄₃) concerns the degree to which the product produced in agribusiness is of high quality.

The social dimension (C₅) emphasizes how agribusiness delivers health and well-being to their employees and communities. For instance, Phochanikorn and Tan [29] focused on the behavior of employees and their working environment, while Harris and Heyer [34] asserted that agribusinesses had responsibilities to provide a sufficient food supply to people within their communities. In addition to food availability, agribusiness should supply plenty of alternative commodity bundles for the people. The criteria for the social dimension are therefore measured by human rights (C₅₁), social contribution (C₅₂), and stakeholder engagement (C₅₃). The human rights (C₅₁) is related to the nondiscrimination and freedom of association culture within the agribusiness. The social contribution (C₅₂) focuses on the social impact of agricultural practices, for example, facilitating food recycling by reducing the volumes of organic food waste through compost [34]. The stakeholder engagement (C₅₃) concerns the degree to which the stakeholders are satisfied with the performance of the agribusiness when using a certain agriculture DSS.

With the identified evaluation criteria and sub-criteria as above, every available agriculture DSS has to be comprehensively evaluated for determining their overall performance across all the evaluation criteria, so that the most appropriate agriculture DSS can be selected in a given situation for sustainable agribusiness. To effectively solve this problem, the next section presents a multicriteria analysis approach for evaluating the performance of agriculture DSSs.

3. A Multicriteria Analysis Approach

Multicriteria analysis approaches are proven to be effective in tackling problems involving evaluating and selecting alternatives from a finite number of alternatives with respect to multiple, often conflicting criteria [35,36,37]. The multidimensional nature of the agriculture DSS evaluation process justifies the use of the multicriteria analysis methodology for dealing with the agriculture DSS evaluation and selection problems.

The agriculture DSS evaluation and selection problem can be described as the existence of (a) alternative agriculture DSSs for evaluation and selection A_i (i = 1, 2, …, n) and (b) multiple evaluation criteria C_j (j = 1, 2, …, m) and their associated sub-criteria C_jk (k = 1, 2, …, p_j) as shown in Figure 1. The evaluation process involves in (a) determining the performance ratings of each agriculture DSS with respect to the criteria as x_ij (i = 1, 2, …, n, j = 1, 2, …, m), (b) examining the relative importance of the criteria as criteria weights W = (w₁, w₂, …, w_j) and sub-criteria weights W_j = (w_j1, w_j2, …, w_jk), and (c) determining the overall performance of individual agriculture DSSs from where the selection decision can be made.

To adequately model the subjectiveness and imprecision of the agriculture DSS evaluation and selection process, linguistic variables approximated by triangular fuzzy numbers [38] are used for representing the decision maker’s subjective assessments of criteria weightings and alternative performance ratings. Triangular fuzzy numbers are usually denoted as (a, b, c) in which b is used to represent the most possible assessment value, and a and c are used to represent the lower and upper bounds used to reflect the fuzziness of the assessment [16,39,40].

Table 2. Linguistic variables and their corresponding triangular fuzzy numbers.

Performance		Importance
Linguistic Variable	Fuzzy Numbers	Linguistic Variable	Fuzzy Numbers
Very Poor (VP)	(0.0, 0.0, 0.3)	Very Low (VL)	(0.0, 0.0, 0.3)
Poor (P)	(0.1, 0.3, 0.5)	Low (L)	(0.1, 0.3, 0.5)
Fair (F)	(0.3, 0.5, 0.7)	Medium (M)	(0.3, 0.5, 0.7)
Good (G)	(0.5, 0.7, 0.9)	High (H)	(0.5, 0.7, 0.9)
Very Good (VG)	(0.7, 1.0, 1.0)	Very High (VH)	(0.7, 1.0, 1.0)

shows the approximate distribution of Performance and Importance variables for measuring the alternative performance rating and criteria weightings respectively in the agriculture DSS evaluation and selection process.

Using the linguistic variables defined as in Table 2, the fuzzy decision matrix for representing the Performance variable in the agriculture DSS evaluation and selection problem can be determined as in Equation (1) in which x_ij represents the decision maker’s assessment of the performance rating of Ai in regards to C_j. C_j can be either given by the decision maker using linguistic variables or aggregated from a lower-level decision matrix with the use of associated sub-criteria.

$$X=\left[\begin{array}{llll}{x}_{11}& x_{12}& \dots & x_{1m}\\ x_{21}& x_{22}& \dots & x_{2m}\\ \dots & \dots & \dots & \dots \\ x_{n1}& x_{n2}& \dots & x_{nm}\end{array}\right]$$

(1)

If there is sub-criteria C_jk for C_j, a lower-level fuzzy decision matrix can be formulated in Equation (2) in which y_jk is the decision maker’s assessment of the performance rating of A_i in regards to C_jk.

$$Y_{C_j}=\left[\begin{array}{llll}{y}_{11}& y_{21}& \dots & y_{n1}\\ y_{12}& y_{22}& \dots & y_{n2}\\ \dots & \dots & \dots & \dots \\ y_{1p_j}& y_{2p_j}& \dots & y_{n{p}_j}\end{array}\right]$$

(2)

Weighting vectors for C_j and C_jk can then be given in Equations (3) and (4) by the decision maker using the linguistic variable Importance as shown in Table 2.

$$W=\left(w_1, w_2, \cdots , w_j\right)$$

(3)

$$W_j=\left(w_{j1}, w_{j2}, \cdots , w_{jk}\right)$$

(4)

With the formulation of the lower-level fuzzy decision matrix in Equation (2), and the weight vector in Equation (4) for C_j and C_jk, the decision vector across all the alternatives with respect to C_j can be determined, where W_j is the weighting vector for the sub-criteria and $Y_{Cj}$ is the assessments of the performance rating of Ai in regards to C_jk.

$$\left(x_{1j},x_{2j},\cdots ,x_{nj}\right)=\frac{W_j{Y}_{Cj}}{{{\displaystyle \sum }}_{k=1}^{p_j}{w}_{jk}}$$

(5)

With the agriculture DSS evaluation and selection problem described as above, the overall objective for solving the agriculture DSS evaluation and selection problem is to rank all the alternative agriculture DSSs by determining a performance rating with respect to all criteria and sub-criteria. To identify the overall performance of each agriculture DSS across all evaluation criteria and sub-criteria, the overall weighted performance matrix of all agriculture DSSs needs to be calculated by multiplying the criteria weights w_j and the alternative performance rating x_ij as follows:

$$Z=\left[\begin{array}{cccc}{w}_1{x}_{11}& w_2{x}_{12}& \dots & w_m{x}_{1m}\\ w_1{x}_{21}& w_2{x}_{22}& \dots & w_m{x}_{2m}\\ \dots & \dots & \dots & \dots \\ w_1{x}_{n1}& w_2{x}_{n2}& \dots & w_m{x}_{nm}\end{array}\right]$$

(6)

It is important to consider the decision maker’s confidence toward his/her assessments, and thus the concept based on λ (0 ≤ λ ≤ 1) is introduced. It is common that λ = 1, 0.5, or 0 are used to indicate that the decision maker has an optimistic, moderate, or pessimistic view, respectively, in the evaluation process. Based on this concept, the decision maker’s assessments in regards to his/her confidence level is defined as

$$Z_{ij}^{\mathsf{\lambda }}=(a_1+\mathsf{\lambda }\left(a_2-a_1\right), a_2,a_3-\mathsf{\lambda }\left(a_3-a_2)\right)$$

(7)

where a₂ is used to represent the most possible value of the variable, and a₁ and a₃ are used to reflect the fuzziness of the variable [38].

To avoid the complex and unreliable process of comparing fuzzy utilities often required in the fuzzy multicriteria analysis [16,35], the defuzzification method, determined based on the geometric center of a fuzzy number, is applied to the weighted fuzzy performance matrix in Equation (8).

$$r_{ij}=\frac{{{\displaystyle \int }}_{s_{ij}}^{}x{\mathsf{\mu }}_{Z_{ij}^{\mathsf{\lambda }}}\left(x\right)dx}{{{\displaystyle \int }}_{s_{ij}}^{}x{\mathsf{\mu }}_{Z_{ij}^{\mathsf{\lambda }}}\left(x\right)dx}$$

(8)

where S_ij is the support of fuzzy number w_jx_ij, $s_{ij}=\{x, {\mu }_{z_{ij}^{\lambda }}>0, x\in R\}$ and x > 0.

A weighted performance matrix in a crisp value format can then be obtained as

$$R^{\mathsf{\lambda }}=\left[\begin{array}{cccc}{r}_{11}^{\mathsf{\lambda }}& r_{12}^{\mathsf{\lambda }}& \dots & r_{1m}^{\mathsf{\lambda }}\\ r_{21}^{\mathsf{\lambda }}& r_{22}^{\mathsf{\lambda }}& \dots & r_{2m}^{\mathsf{\lambda }}\\ \dots & \dots & \dots & \dots \\ r_{n1}^{\mathsf{\lambda }}& r_{n2}^{\mathsf{\lambda }}& \dots & r_{nm}^{\mathsf{\lambda }}\end{array}\right]$$

(9)

This is followed by applying the concept based on the positive-ideal solution and the negative-ideal solution as in Equations (10) and (11), respectively. $A_{}^{\mathsf{\lambda }+}$ and $A_{}^{\mathsf{\lambda }-}$ represent the best possible and the worst possible results among the agriculture DSSs, respectively, across all criteria.

$$A^{\mathsf{\lambda }+}=\left(r_1^{\mathsf{\lambda }+},r_2^{\mathsf{\lambda }+},\dots ,r_m^{\mathsf{\lambda }+}\right)$$

(10)

$$A^{\mathsf{\lambda }-}=\left(r_1^{\mathsf{\lambda }-},r_2^{\mathsf{\lambda }-},\dots ,r_m^{\mathsf{\lambda }-}\right)$$

(11)

$$a_i^{\mathsf{\lambda }+}=\sup (r_{ij}^{\mathsf{\lambda }},r_{2j}^{\mathsf{\lambda }},\dots ,r_{nj}^{\mathsf{\lambda }})$$

(12)

$$a_i^{\mathsf{\lambda }-}=\inf (r_{ij}^{\mathsf{\lambda }},r_{2j}^{\mathsf{\lambda }},\dots ,r_{nj}^{\mathsf{\lambda }})$$

(13)

where $a_i^{\mathsf{\lambda }+}$ and $a_i^{\mathsf{\lambda }-}$ are the maximum and minimum numbers of $r_{ij}^{\mathsf{\lambda }}$, respectively.

The Hamming distance be calculated using Equations (14) and (15) where $S_i^{\mathsf{\lambda }+}$ represents the distance between alternative Ai and the positive-ideal solution, and $S_i^{\mathsf{\lambda }-}$ represents the distance between alternative Ai and the negative solution.

$$S_i^{\mathsf{\lambda }+}={\displaystyle \sum }_{j=1}^m\left(a_j^{\mathsf{\lambda }+}-r_{ij}^{\mathsf{\lambda }}\right)$$

(14)

$$S_i^{\mathsf{\lambda }-}={\displaystyle \sum }_{j=1}^m\left(r_{ij}^{\mathsf{\lambda }}-a_j^{\mathsf{\lambda }-}\right)$$

(15)

As a result, an overall performance index (P_i) for alternative Ai across all criteria can be determined by Equation (16). The larger the performance index, the more preferred the alternative.

$$P_i=\frac{S_i^{\mathsf{\lambda }-}}{S_i^{\mathsf{\lambda }+}+S_i^{\mathsf{\lambda }-}}$$

(16)

The approach presented above can be summarized as follows:

Step 1. Obtain the decision maker’s assessment of the performance rating of alternative A_i with respect to sub-criteria C_jk, as expressed in Equations (1) and (2).

Step 2. Obtain the weighting vectors for the evaluation criteria C_j and sub-criteria C_jk as in Equations (3) and (4), respectively.

Step 3. Determine the overall performance of each alternative by multiplying the criteria weights w_j and the alternative performance rating x_ij, as in Equations (5) and (6). Then, calculate the decision maker’s assessments in regard to his/her confidence level by using Equation (7).

Step 4. Determine the weighted performance matrix by Equations (8) and (9).

Step 5. Calculate the Hamming distance between alternative Ai and the positive-ideal solution $S_i^{\mathsf{\lambda }+}$ and between Ai and the negative solution $S_i^{\mathsf{\lambda }-}$ by using Equations (10)–(15).

Step 6. Obtain an overall performance index (P_i) for alternative Ai across all criteria by Equation (16).

4. A Case Study

Australia invests heavily in the agriculture industry. According to [41], 371 million hectares of Australia’s land is managed by 85,000 agribusinesses, with 25 to 30 million hectares used for broadacre crops, hay and silage, and horticulture, and 312 million hectares used for grazing. However, agricultural productivity is threatened by the increasing degradation of soil quality, the rising cost of nutrient inputs, and competing land uses. Agribusinesses have been using agriculture DSSs to enhance the organizational decision process, thus improving agricultural performance [2,3,12].

An exploratory qualitative design was used to demonstrate the applicability and effectiveness of the proposed multicriteria analysis approach discussed in the previous section. The primary data was collected from an agribusiness in Australia. The agribusiness had different lines of operations and had decided to adopt an agriculture DSS that aimed to enhance the organizational decision process. Nevertheless, it was faced with the challenges in evaluating and selecting the most appropriate agriculture DSS out of four alternatives due to the existence of multiple and conflicting evaluation criteria, and the imprecise judgments of decision makers. These issues need to be adequately addressed for informing effective decisions. A focus group was held with six decision makers in the agribusiness including farmers and farmers’ advisors for identifying important criteria for evaluating agriculture DSSs.

Four alternative agriculture DSSs under consideration and evaluation for sustainable agribusiness include GrazPlan (A₁), CropSyst (A₂), DSSAT (A₃), and Yield Prophet (A₄). GrazPlan was particularly designed to help farmers make decisions about their farm management, principally in grazing [42]. CropSyst is a simulation-based DSS that was developed for examining the effect of cropping systems management on productivity and the environment [43]. DSSAT was designed to help farmers make decisions on crop management with the incorporation of over 42 crops in the simulation model [44]. Yield Prophet aims to help validate key decisions made in farm operations [45].

A detailed interview protocol was developed to guide the interviews with the focus group [46]. Interview questions were broadly designed around the identification of the technology, organization, environmental, economic, and social evaluation criteria based upon the hierarchical structure of agriculture DSSs performance evaluation as shown in Figure 1. A Delphi process was followed to reach a consensus among several decision makers in the organization about the performance ratings of individual agriculture DSSs and the criteria weights used for evaluating the performance of agriculture DSSs. Such a consensus process facilitated the acceptance of the decision-making outcome among the key stakeholders in the agriculture DSS evaluation and selection process.

With prior permission from participants, the discussions were tape-recorded and transcribed for further analysis to improve data validation. Data validation was carried out by confirming the issues captured in the transcripts with individual participants and cross-checking the notes of observations by the researchers.

The interview data were analyzed using content analysis in which researchers coded the text of the interview transcripts line by line, categorizing relevant data into themes based on the hierarchical structure of agriculture DSSs performance evaluation [47]. Based upon the hierarchical structure of agriculture DSSs performance evaluation as in Figure 1, interview data were coded to establish key dimensions and checked for inconsistencies.

The proposed multicriteria analysis approach presented in Section 3 was used for evaluating and selecting agriculture DSS. The steps followed are illustrated in the following:

Step 1: With the determination of the criteria as shown in Figure 1, the performance of each agriculture DSS with respect to each criterion is determined by using the linguistic variables presented in Table 2.

Table 3. Performance assessment results for each sub-criterion.

Criteria	Alternatives
	A1	A2	A3	A4
Technology (C1)
C11	VG	F	P	F
C12	P	G	VG	P
C13	G	F	G	F
C14	G	G	P	F
Organization (C2)
C21	F	VG	P	P
C22	F	G	G	G
C23	VG	F	F	F
Environment (C3)
C31	G	P	G	P
C32	G	VP	G	VG
C33	F	F	G	VG
C34	G	F	G	G
Economic (C4)				G
C41	G	F	P	F
C42	VG	VP	P	VG
C43	G	F	VP	F
Social (C5)
C51	VG	F	F	G
C52	G	VP	F	P
C53	G	P	VP	VP

shows the assessment results of four alternative agriculture DSSs with respect to each sub-criterion.

Step 2: The relative importance of the agriculture DSSs is identified by applying the linguistic variables in Table 2.

Table 4. Criteria and sub-criteria weights for agriculture DSSs performance evaluation.

	Fuzzy Criteria Weights
W	((0.5, 0.7, 0.9), (0.5, 0.7, 0.9), (0.5, 0.7, 0.9), (0.7, 1.0, 1.0), (0.7, 1.0, 1.0))
W1	((0.7, 1.0, 1.0), (0.7, 1.0, 1.0), (0.5, 0.7, 0.9), (0.5, 0.7, 0.9))
W2	((0.5, 0.7, 0.9), (0.3, 0.5, 0.7), (0.7, 1.0, 1.0))
W3	((0.5, 0.7, 0.9), (0.7, 1.0, 1.0), (0.5, 0.7, 0.9), (0.3, 0.5, 0.7))
W4	((0.7, 1.0, 1.0), (0.5, 0.7, 0.9), (0.7, 1.0, 1.0))
W5	((0.7, 1.0, 1.0), (0.7, 1.0, 1.0), (0.3, 0.5, 0.7))

shows the criteria and its associated sub-criteria weights for the agriculture DSSs evaluation and selection problem.

Step 3: To construct the fuzzy performance matrix for all agriculture DSSs with regard to multiple evaluation as in (1), a lower-level fuzzy performance matrix of all alternatives with respect to sub-criteria determined from Table 3 is calculated with the use of Equations (5) and (6). Then, apply Equation (6) and assign λ = 0.5 to indicate that the decision maker has a moderate view on his/her confidence in the evaluation process.

Table 5. Fuzzy decision matrix for agriculture DSSs performance evaluation.

Criteria	Alternatives
	A1	A2	A3	A4
Technology (C1)
C11	(0.392, 0.471, 0.603)	(0.137, 0.232, 0.307)	(0.213, 0.377, 0.462)	(0.318, 0.425, 0.511)
C12	(0.336, 0.524, 0.789)	(0.086, 0.122, 0.186)	(0.073, 0.127, 0.219)	(0.336, 0.524, 0.789)
C13	(0.324, 0.425, 0.573)	(0.116, 0.187, 0.253)	(0.139, 0.262, 0.371)	(0.226, 0.325, 0.426)
C14	(0.126, 0.239, 0.487)	(0.214, 0.286, 0.318)	(0.148, 0.292, 0.414)	(0.164, 0.392, 0.438)
Organization (C2)
C21	(0.369, 0.484, 0.721)	(0.038, 0.127, 0.205)	(0.069, 0.216, 0.275)	(0.067, 0.126, 0.218)
C22	(0.172, 0.238, 0.492)	(0.089, 0.157, 0.247	(0.139, 0.275, 0.316)	(0.093, 0.179, 0.316)
C23	(0.211, 0.367, 0.582)	(0.077, 0.114, 0.216)	(0.174, 0.313, 0.398)	(0.064, 0.209, 0.337)
Environment (C3)
C31	(0.105, 0.246, 0.472)	(0.068, 0.128, 0.172)	(0.179, 0.248, 0.406)	(0.276, 0.335, 0.395)
C32	(0.226, 0.371, 0.596)	(0.116, 0.177, 0.193)	(0.148, 0.292, 0.414)	(0.216, 0.309, 0.497)
C33	(0.171, 0.257, 0.592)	(0.082, 0.178, 0.214)	(0.126, 0.284, 0.426)	(0.076, 0.109, 0.318)
C34	(0.274, 0.371, 0.624)	(0.091, 0.129, 0.217)	(0.126, 0.239, 0.487)	(0.076, 0.127, 0.342)
Economic (C4)
C41	(0.172, 0.357, 0.626)	(0.081, 0.148, 0.317)	(0.125, 0.237, 0.499)	(0.116, 0.271, 0.418)
C42	(0.127, 0.318, 0.519)	(0.066, 0.147, 0.263)	(0.114, 0.221, 0.427)	(0.136, 0.237, 0.429)
C43	(0.146, 0.237, 0.481)	(0.037, 0.109, 0.214)	(0.126, 0.258, 0.517)	(0.157, 0.241, 0.384)
Social (C5)
C51	(0.204, 0.293, 0.472)	(0.074, 0.158, 0.206)	(0.092, 0.247, 0.503)	(0.117, 0.246, 0.374)
C52	(0.254, 0.479, 0.571)	(0.113, 0.168, 0.196)	(0.217, 0.336, 0.497)	(0.125, 0.283, 0.358)
C53	(0.117, 0.316, 0.516)	(0.087, 0.109, 0.163)	(0.163, 0.246, 0.514)	(0.093, 0.186, 0.226)

shows the aggregated fuzzy performance matrix of alternatives with respect to the agriculture DSSs evaluation criteria.

Step 4: The weighted performance matrix as shown in

Table 6. Weighted performance matrix.

Criteria	Alternatives
	A1	A2	A3	A4
Technology (C1)
C11	(0.328, 0.417, 0.514)	(0.247, 0.361, 0.484)	(0.233, 0.446, 0.752)	(0.339, 0.417, 0.509)
C12	(0.336, 0.524, 0.789)	(0.084, 0.120, 0.195)	(0.046, 0.099, 0.186)	(0.336, 0.524, 0.789)
C13	(0.214, 0.293, 0.374)	(0.103, 0.128, 0.248)	(0.172, 0.247, 0.335)	(0.084, 0.163, 0.339)
C14	(0.235, 0.406, 0.483)	(0.082, 0.156, 0.263)	(0.182, 0.278, 0.316)	(0.174, 0.226, 0.327)
Organization (C2)
C21	(0.137, 0.319, 0.524)	(0.069, 0.148, 0.170)	(0.068, 0.228, 0.172)	(0.117, 0.273, 0.418)
C22	(0.121, 0.296, 0.507)	(0.072, 0.205, 0.236)	(0.116, 0.177, 0.193)	(0.106, 0.226, 0.407)
C23	(0.126, 0.288, 0.492)	(0.086, 0.101, 0.234)	(0.074, 0.136, 0.198)	(0.154, 0.241, 0.379)
Environment (C3)
C31	(0.105, 0.227, 0.472)	(0.087, 0.115, 0.177)	(0.125, 0.263, 0.438)	(0.073, 0.179, 0.317)
C32	(0.139, 0.249, 0.418)	(0.091, 0.135, 0.326)	(0.159, 0.226, 0.291)	(0.094, 0.148, 0.325)
C33	(0.217, 0.416, 0.483)	(0.066, 0.147, 0.267)	(0.174, 0.289, 0.312)	(0.181, 0.218, 0.370)
C34	(0.114, 0.246, 0.378)	(0.037, 0.109, 0.174)	(0.205, 0.302, 0.357)	(0.087, 0.236, 0.379)
Economic (C4)
C41	(0.114, 0.271, 0.418)	(0.068, 0.218, 0.272)	(0.117, 0.246, 0.382)	(0.076, 0.102, 0.128)
C42	(0.106, 0.226, 0.417)	(0.126, 0.173, 0.190)	(0.126, 0.282, 0.357)	(0.082, 0.110, 0.146)
C43	(0.154, 0.241, 0.379)	(0.074, 0.123, 0.168)	(0.096, 0.183, 0.212)	(0.211, 0.247, 0.316)
Social (C5)
C51	(0.155, 0.254, 0.348)	(0.092, 0.109, 0.169)	(0.0931, 0.246, 0.542)	(0.092, 0.180, 0.313)
C52	(0.126, 0.228, 0.495)	(0.107, 0.128, 0.293)	(0.114, 0.206, 0.271)	(0.079, 0.142, 0.258)
C53	(0.337, 0.465, 0.681)	(0.185, 0.276, 0.392)	(0.103, 0.182, 0.209)	(0.064, 0.137, 0.367)

can be determined with the use of Equations (8) and (9).

Step 5: Based on Equations (10)–(15), the Hamming distance between alternative Ai and the positive-ideal solution $S_i^{\mathsf{\lambda }+}$, and between Ai and the negative solution $S_i^{\mathsf{\lambda }-}$, can be calculated [48].

Table 7. The Hamming distance between alternatives and ideal solutions.

Ai	${\mathit{S}}_{\mathit{i}}^{\mathsf{\lambda }+}$	${\mathit{S}}_{\mathit{i}}^{\mathsf{\lambda }-}$
A1	49.47	73.31
A2	39.73	75.21
A3	50.26	88.32
A4	58.24	91.48

shows the results.

Step 6. The overall performance index (P_i) for alternative Ai across all criteria can then be calculated by using Equation (16).

Table 8. Performance index and ranking for agriculture DSSs performance evaluation.

Criteria	Alternatives
	A1	A2	A3	A4
Technology (C1)
Index	0.709	0.638	0.563	0.674
Ranking	1	3	4	2
Organization (C2)
Index	0.612	0.516	0.583	0.697
Ranking	2	4	3	1
Environment (C3)
Index	0.715	0.549	0.648	0.671
Ranking	1	4	3	2
Economic (C4)
Index	0.624	0.551	0.588	0.704
Ranking	2	4	3	1
Social (C5)
Index	0.701	0.586	0.537	0.628
Ranking	1	3	4	2
Overall Index	0.675	0.528	0.569	0.637
Ranking	1	4	3	2

shows the performance index and ranking for agriculture DSSs performance evaluation.

From Table 8, the agriculture DSS A₁ is the preferred choice for the agribusiness as it has the highest performance index of 0.675. Figure 2 shows that agriculture DSS A₁ does not have competitive advantages in all evaluation criteria. This is because evaluation criteria for Organization (C₂) and Economic (C₄) are ranked second behind agriculture DSS A₄, with an overall index of 0.612 and 0.624, respectively. Agriculture DSS A₄ requires improvement in technology, environment, and social criteria. On the other hand, the other two agriculture DSSs, A₂ and A₃, need to improve in all performance-related areas.

To demonstrate the effectiveness of the proposed multicriteria analysis approach, a comparative study on the relative performance of the proposed approach with comparable multicriteria decision-making approaches was conducted. Five other methods [49,50,51,52,53] were used in the comparative study here as examples to show the performance of the multicriteria analysis approach developed. The results show that the proposed multicriteria analysis approach produces consistent ordering results compared with most of the existing multicriteria decision-making approaches. The advantage of the proposed approach over the other approaches is its simplicity in concept and efficiency in computation.

Table 9. Results of the comparative study.

Approaches	Ordering
Awasthi et al. [49]	A1 > A4 > A3 > A2
Kaya and Kahraman [50]	A4 > A1 > A3 > A2
Yeh and Xu [51]	A1 > A4 > A3 > A2
Guo and Zhao [52]	A1 > A4 > A2 > A3
Karimi et al. [53]	A1 > A4 > A3 > A2
The proposed approach	A1 > A4 > A3 > A2

shows the results of the comparative study.

5. Discussion

Agriculture DSSs are an important part of the quest for evidence-based decision-making in agriculture for agribusinesses to improve productivity and environmental outputs, which in turns brings economic benefits [2,3]. They are designed to help decision makers in agribusinesses make effective decisions by leading them through clear decision stages and presenting the likelihood of various outcomes resulting from different options [8,9].

Evaluating and selecting the most appropriate agriculture DSS is, however, challenging due to the existence of production and marketing alternatives available to decision makers, a variety of objective functions for decision makers from economic, to lifestyle, to long-term sustainability, and the subjectiveness and imprecision involved in the evaluation process [17,27]. To ensure effective decision outcomes, it is important to adequately handle multiple evaluation criteria with the presence of subjective and imprecise assessments in the human decision-making process.

This study proposes a multicriteria analysis approach for evaluating the performance of agriculture DSSs for sustainable agribusiness with the incorporation of TOE and the triple bottom line principle of sustainability. Based on the feedback from the participants in the case study, the approach is considered effective and efficient, due to the simple computation process involved for capturing the comprehensibility of the concepts. An interactive DSS interface with the incorporation of the proposal algorithm for evaluating the performance of agriculture DSSs can be developed for users’ ease of use.

To show the reliability of the proposed multicriteria analysis approach, five other approaches [49,50,51,52,53] were used in the comparative study. The results show that the proposed multicriteria analysis approach has generated consistent ordering results compared with most of the existing multicriteria decision-making approaches.

The novelty of this paper lies in the formulation of a multicriteria analysis approach for evaluating the performance of agriculture DSSs for effective agricultural management. The application of such an approach facilitates the evaluation and selection of appropriate agriculture DSSs in farming organizations for achieving competitive advantages.

This study makes major contributions to agriculture DSS evaluation and selection. Theoretically, this study contributes to the methodology development in the technology adoption and evaluation studies by providing a novel methodology for effectively evaluating the agriculture DSS for effective agricultural management in agribusinesses. This study has also extended and demonstrated the applicability of the TOE framework in the agriculture DSS evaluation context, which has not been done in prior research. Practically, the proposed multicriteria analysis approach can be used as a decision-making tool in agribusinesses for providing decision makers with useful and strategic information concerning the performance of agriculture DSSs in a given situation.

There are two main limitations of this study which suggest future research. First, this study ignores specific characteristics of individual agriculture DSSs when evaluating the performance of agriculture DSSs. Each agriculture DSS has its unique features, such as different degrees of ease of use, reliability, and adaptability [12,16]. Future research can be carried out to better address this question by considering the specific characteristics of individual agriculture DSSs to address the performance evaluation problem. Second, this study only approaches the agriculture DSSs evaluation criteria from the TOE and the triple bottom line principle of sustainability perspectives. Other criteria, for example, human education and knowledge, may influence the implementation of agriculture DSSs. It is therefore important to consider other criteria in future studies for evaluating the performance of agriculture DSSs for sustainable agribusiness.

6. Conclusions

This paper has presented a multicriteria analysis approach for effectively evaluating the performance of agriculture DSSs for sustainable agribusinesses. With the case study, the proposed multicriteria decision-making approach has demonstrated a number of advantages for adequately handling the multiple and conflicting evaluation criteria, the subjectiveness in the evaluation process, and the demanding process on the decision maker in the agriculture DSSs performance evaluation problem. The approach is found to be effective and efficient, due to the simple computation process involved for capturing the comprehensibility of the concepts.