Application of Comprehensive Evaluation in New-Product-Development Evaluation: The Case of Landscape-Architectural Outdoor Wooden Furnishing

Abstract

Successful new product development (NPD) is critical for modern outdoor wooden furnishing (OWF) manufacturing companies to achieve competitive success, since current users have the serious requirements of aesthetics, materials and environmental quality. Identifying the competitive performance of a product in development is an effective means to minimize the risk of failure. However, the literature reviews of the comprehensive evaluation (CE) model for OWF NPD are very rare. In this study, the CE method that applies three steps, which include constructing evaluation criteria, establishing a trapezoidal fuzzy analytic hierarchy process (AHP) and proposing a CE model is applied to assess the performance of a product in development and to minimize the risk of product failure in the market. The study aims to propose a CE approach for OWF NDP, which utilizes multiple methods that incorporate a literature review, questionnaire, Delphi method and fuzzy trapezoidal AHP. Finally, an integrated CE model is proposed to measure the competitive performance of NPD. A case study of a series of OWF in Harbin Pingfang Park, China is presented to illustrate the feasibility of the model. The result demonstrates that the proposed method predicts the performance of a product in development objectively and comprehensively. This evaluation method, being an assessment tool, can help designers and decision makers make better decisions and will predict the competitive performance of a product so as to reduce the risk of economic losses, not merely depending on previous experience and personal expectation.

1. Introduction

In order to gain competitive advantage, evaluations for the performance of products in the development process are becoming crucial for the success of manufacturing firms in design and construction. Although some techniques and management strategies have been developed to make the NPD process more effective [1,2,3], product development projects are still prone to failure. Wang stated that new product failure rates could be as low as 33.3% or as high as 90% of new grocery products and the average failure rate of new products also reached up to 41% [4]. Cormican and O’Sullivan believed that product innovation has low success rates and many projects are terminated midway in development cycle due to risk and expense [5]. Several researchers also indicated that a high proportion of new product ideas fail commercially when introduced [6,7,8]. If unsuccessful products were brought to market, the company’s brand would be greatly affected and great economic losses would also be caused. These problems above would be solved or be kept to a minimum if more attention was paid to evaluation in the product-development process. An appropriate project-management approach is one of the key success factors of NPD [9,10]. Hence, measuring the success of a new product in the development process has become a critical issue. However, to perceive and to measure effectively the capability of NPD is a significantly challenging task for business managers [4].

NPD is a complex and risky process from idea creation to product introduction. Companies must be able to manufacture the right products at the right time and at the suitable price [11]. Otherwise, their competitive advantage can quickly be lost [11]. Therefore, sensible decisions should be made according to a comprehensive assessment of various product attributes. Managers must ensure that this process is well-managed and successful [5]. Fitzsimmons pointed out that product design is a strategic activity and an organization has to compete to ensure its survival [12]. Surely, successful product development is closely correlated with corporate success and has become more important in highly competitive environments [4,13,14]. Previous studies focused more on customer satisfaction evaluation, the value of customers, eco-innovation, product innovation management and strategies in the NPD process [5,15,16,17,18,19,20]. However, there are few studies on the CE mechanism of a product in OWF NPD process.

The objective of this study is to apply the CE method to assess the performance of OWF to minimize the risk of product failure in the market. Nevertheless, this process does not merely depend on previous experience and personal expectation. In this study, due to the subjective uncertainties of the evaluations in NPD process, the CE method was adopted. CE utilizes fuzzy set theory to effectively handle imprecise and vague situations [3,20]. The evaluation results, being the foundation of judgment, can help designers and decision makers make better decisions and help them predict the competitive performance of this product, so as to avoid greater economic losses.

2. Literature Review

2.1. Evaluation Methods of NPD

In order to minimize failure rates, a number of evaluations have been widely implemented in the NPD process. The existing research methods include questionnaires, surveys, AHP, ergonomic methods, management approaches, etc. (

Table 1. Literature review of existing evaluation methods of NPD.

Reference	Research Topic	Research Method	Main Contribution
Ye and Li [21]	Emotional design elements of furniture	Questionnaire and statistical analysis	Discuss the difference and relationship between positive and negative emotion factors.
Fabisiak et al. [22]	Explore seniors’ preferences in relation to characteristics of sitting furniture	Surveys	Propose the characteristics of chair preferred by seniors, such as armchairs with a high backrest reaching above the head, chairs with armrests and an upholstered backrest and seat.
Li et al. [18]	Propose a customer satisfaction (CS) evaluation method for customized product development	Entropy weight and analytic hierarchy process (AHP).	Classify customer requirements into four categories and develop a customer-satisfaction evaluation model.
Gonzalez-Garcia et al. [17]	Conduct an environmental analysis of a childhood set based on wooden materials	Eco-design or design for life-cycle assessment (LCA) and environment (DfE)	Identify the key environmental issues throughout the life cycle of the childhood set and propose improvement alternatives
Wu et al. [23]	Evaluate customized furniture design in a virtual environment	Ergonomic evaluation method	Propose a method to help customized furniture design including gathering users’ information and modify a furniture model
Žužek et al. [11]	Propose a new product-development model specifically appropriate for small- and medium-sized enterprises (SMEs)	Agility to the plan-driven concurrent product development approach	Improve communication, faster detection of discrepancies, more effective problem solving, and greater flexibility in SMEs
Villamil and Hallstedt [24]	Understand how sustainability can be integrated in the company portfolio development	Semi-structured interview and literature review	Develop a method to guide manufacturing companies in sustainability integration and implementation in product portfolios
Ding et al. [25]	Propose a product-color emotional-design method	A product-color emotional-design method based on a convolutional neural network and a search neural network	Provide accurate product-color design solutions that can be used to develop practical large-scale applications of product-color emotional-design theory and methods
Silva et al. [3]	Explore new tools for improving innovation effects in the concept phase of NPD	Design thinking and TRIZ	Propose a framework to improve innovation effectiveness during NPD processes in industry
Diaz et al. [20]	Investigate the implications of R-strategy adoption for decision making in SPD	Empirical approach, combining a literature review and in-depth semi-structured interviews	Reveal new directions to adjust the contextual factors of SPD to further align existing processes with widely expanding CE organizational cultures

).

These existing studies generally involve both qualitative and quantitative factors. The subjective evaluation of performance criteria as uncertain and fuzzy problems cannot be effectively handled. In order to overcome this drawback, comprehensive evaluation can be adopted as an effective means.

2.2. Comprehensive Evaluation (CE) Method

CE methods have been rapidly and widely used in many fields. There are many comprehensive evaluation methods based on different theories, such as fuzzy set theory [26], systems engineering theory [27] and artificial intelligence theory [28]. These comprehensive evaluation methods based on different theories have their advantages and disadvantages, as well as different decision-making processes and application objects. Zhou, R and Chan, A.H.S. presented a fuzzy comprehensive method with AHP to determine product usability [29]. Zhang et al. adopted a comprehensive evaluation of a VR mine-safety training system with the AHP and fuzzy logic technology [30]. He et al. established a method to evaluate the perceived quality of products by combining text mining with a fuzzy comprehensive evaluation [31]. Wang et al. proposed a fuzzy multiple-criteria decision-making model combining FANP and the technique of order preference for site selection in the renewable energy sector [32]. Huang et al. combined fuzzy sets with genetic algorithms and neural networks to form an integrated approach for the generation and evaluation of design concepts [33]. Zhai et al. proposed a method based on grey relation analysis and rough set theory to evaluate design concepts [34]. Chou offered a fuzzy linguistic evaluation technique to assess alternative ideas and to determine promising product ideas. Obviously, fuzzy comprehensive evaluation has received the most attention in dealing with evaluations in product studies. However, there are few studies on the NPD process, especially for the field of OWF. In this study, this research and exploration is carried out.

Considering the indicator system of products in NPD of outdoor wooden furnishing and the uncertainty of subjective and qualitative evaluation about the indicators, the CE methods with fuzzy set theory was employed to assess products in this research. Its theoretical basis is fuzzy set theory, which is designed to model the concepts of uncertainty and vagueness [26]. Fuzzy set theory was initially applied to economic problems, in which the object to be evaluated was affected by multiple factors and the relations among these factors were not clear. It provides formalized tools for dealing with imprecise problems. In fuzzy set theory, fuzzy numbers are used to represent uncertain parameters. For a normalized fuzzy number N, there is a continuous membership function that describes to which grade a variable belongs. Then, it determines that x is “not included” and when x is a “fully included” crisp number in the theory set [35]. Compared to traditional evaluation methods, the strength of the fuzzy method is that it can capture the subjective uncertainties and fuzziness in human judgments and develop an integrated model that combines vague judgments.

2.3. Theoretical Framework

In this study, researchers propose a theoretical framework with three main steps (Figure 1). The first step is index construction, where the parameters of evaluation criteria are decided. Within this step, the research group use a literature review, expert questionnaire, and the Delphi method to identify these criteria, respectively. In the second step, the research group builds the trapezoidal AHP to obtain the weights of criteria and sub-criteria. In the third step, the CE model is based on the weights of evaluation criteria and sub-criteria, and calculates the evaluation value of the studied outdoor wooden furnishing in Harbin Pingfang Park.

3. Research Methodology

3.1. Step 1: Constructing the Evaluation Criteria

A case study of a series of landscape architectural OWF that includes the different models of seating and tables in Harbin Pingfang Park, China is presented to illustrate the feasibility of the model. Harbin Pingfang Park has 6 scenic areas and 15 scenic spots. OWF products would be needed to provide a relaxed leisure and rest space for local people. The reasonable decision group is very important for reliable evaluation results. The decision makers’ knowledge coverage and academic views should be considered in order to obtain representative views [28]. In addition, the rate of decision makers should be considered. In this study, the research group involves experts in the fields of industrial manufacturing, marketing, environmental design, and landscape design to complete the step of expert questionnaire. The expert decision group was divided into two parts. One group consisted of 50 decision makers including 15 landscape-architectural outdoor furnishing manufacturers, 10 landscape-architectural outdoor furnishing design experts, 15 outdoor fitness equipment retailers and 10 landscape-architectural OWF developers. They were asked to make pair-wise comparisons according to the 9-scale comparative ratings. The other group was composed of 10 decision makers including 3 landscape-architectural outdoor furnishing manufacturers, 2 landscape-architectural OWF design experts, 3 landscape-architectural OWF retailers and 2 landscape-architectural OWF developers. Through the expert questionnaire, researchers identified 4 parameters of evaluation criteria, plus the criteria summarized from the literature review [12,13,14,15]; the research group finally decided 6 primary parameters of evaluation criteria and 26 sub-criteria (Figure 2).

Six criteria (aesthetics, functionality, economy, material, technics and structure, and environmental quality) and 26 sub-criteria shown in Figure 2 were proposed for the evaluation of landscape-architectural OWF in development process. All the experts were asked to use the evaluation grades (VG, G, M, P, VP) to make the evaluation for these parameters of evaluation criteria in order to guarantee they are rigid and reliable [36]. The primary criteria and sub-criteria used in the hierarchy system were also assessed and examined through surveys, questionnaires and discussions by using Delphi approach introduced by the RAND Corporation in the 1950s [37].

3.2. Step 2: Establishing a Trapezoidal Fuzzy AHP

Acquiring the weights of criteria is a very important step. Analytic hierarchy process (AHP) established by Satty [27] is usually applied to solve this problem. However, the conventional AHP is inadequate to deal with the assessment accurately [28]. The fuzzy AHP method [37], an effective method of solving multi-criteria decision analysis (MCDA) problem, has been widely used to determine the weights of criteria [28]. In this CE method, a trapezoidal fuzzy AHP was adopted to determine the weights of criteria. The trapezoidal fuzzy number can be defined as $\tilde{A}$ = (a, b, c, d), where $a\le b\le c\le d$; the trapezoidal-type membership function can be given by Equation (1).

$${\mu }_{\tilde{A}}\{\begin{array}{l}0,\hspace{1em}\hspace{1em}\hspace{1em} x<a\\ \frac{\left(x-a\right)}{\left(b-a\right)}, a\le x\le b\\ 1,\hspace{1em}\hspace{1em}b\le x\le c\hspace{1em}\\ \frac{\left(x-d\right)}{\left(c-d\right)}, c\le x\le d\\ 0,\hspace{1em}\hspace{1em}\hspace{1em} d<x\end{array}$$

(1)

where [b, c] is called a mode interval of $\tilde{A}$, and a and b are called lower and upper limits of $\tilde{A}$, respectively [38]. When b = c, the triangular fuzzy number is denoted as $\tilde{A}$ = (a, b, d), where $a\le b\le d$, showing the triangular-type membership function [33]. Therefore, compared to a triangular fuzzy number, a trapezoidal fuzzy number can handle more situations [38] when facing a number of uncertainties.

In order to obtain crisp values from fuzzy numbers, a defuzzification process is needed. If the trapezoidal fuzzy number is $\tilde{A}$ = (a, b, c, d), the matching defuzzified crisp value N can be calculated by Equation (2) [39].

$$N=\frac{\left(b+c\right)}{2}+\frac{\left[\left(d-c\right)-b-a\right]}{6}=\frac{\left(a+2b+2c+d\right)}{6}$$

(2)

3.3. Step 3: Proposing CE model

The key of CE is that the contributions of multiple related factors are comprehensively considered according to weight factors and fuzziness is decreased by using membership functions [40]. The principle of fuzzy comprehensive arithmetic can be defined as follows:

$$B=A\times R$$

(3)

where A is a weight vector, R is a fuzzy evaluation matrix, and B is a comprehensive grading vector.

4. Data Analysis

4.1. Collecting the Initial Data

Two questionnaires based on the evaluation index system (Figure 2) were conducted in order to acquire the initial data. First, each member from 50 experts was asked to make pair-wise comparisons by comparing the elements at a given level based on the evaluation index system (Figure 2) and the nine-scale comparative ratings described in

Table 2. The nine-point scale of relative importance used in the pair-wise comparison of AHP.

Scale of Relative Importance	Trapezoidal Fuzzy Number	Linguistic Variable
1	(1, 1, 1, 1)	Equally important
3	(2, 5/2, 7/2, 4)	Weakly important
5	(4, 9/2,11/2, 6)	Essentially important
7	(6,13/2,15/2,8)	Very strongly important
9	(8, 17/2, 9, 9)	Absolutely important
2, 4, 6, 8 are intermediate scales	(x − 1, x − 1/2, x + 1/2, x + 1)

. The nine-point ratio scale proposed by Saaty was used [41]. It shows the preferences between options as “equally important”, “weakly important”, “essentially important”, “very strongly important” and “absolutely important”. The pair-wise comparison matrices of the criteria and sub-criteria were carried out. Second, the other 10 experts were asked to make the evaluations for different models of wooden outdoor furnishings in Harbin Pingfang Park using the evaluation grades (VG, G, M, P, VP). The grades include five levels: very good (VG), good (G), medium (M), poor (P) and very poor (VP). Each member from the expert group was required to make judgments based on their knowledge and practical experience for the bottom level (sub-criteria) in this hierarchy. They were asked to fill a “√” in the evaluation table shown in Appendix A. Each linguistic variable was denoted by the corresponding trapezoidal fuzzy numbers listed in Table 2 [42]. The results are summarized in

Table 3. The summarization of the evaluations.

Sub-Criteria	VG	G	M	P	VP
C11	4	2	2	2	0
C12	3	3	2	2	0
C13	2	4	2	2	0
C14	3	2	4	1	0
C21	1	3	2	4	0
C22	3	2	3	2	0
C23	2	4	1	3	0
C24	4	1	1	4	0
C31	1	2	2	5	0
C32	4	3	2	1	0
C33	3	3	2	2	0
C34	2	4	2	2	0
C35	3	2	4	1	0
C41	5	2	2	1	0
C42	3	2	4	1	0
C43	2	3	3	2	0
C44	1	1	5	3	0
C45	2	5	1	2	0
C51	1	2	3	4	0
C52	3	1	5	1	0
C53	2	3	2	3	0
C54	1	4	2	3	0
C61	2	3	3	2	0
C62	3	4	2	1	0
C63	2	2	5	1	0
C64	1	5	1	3	0

.

4.2. Calculating the Weighs

4.2.1. Establishing the Pair-Wise Comparison Matrix

In this study, the trapezoidal fuzzy numbers were employed to make the pair-wise comparisons; the detailed results of this pair-wise comparison analysis are shown in Appendix B. According to the pair-wise comparison, the judgment matrix for evaluation was established as follows:

$$\tilde{A}=\left[\begin{array}{cccc}{\tilde{a}}_{11}& {\tilde{a}}_{12}& \cdots & {\tilde{a}}_{1n}\\ {\tilde{a}}_{21}& {\tilde{a}}_{22}& \cdots & {\tilde{a}}_{2n}\\ ⋮& ⋮& \cdots & ⋮\\ {\tilde{a}}_{n1}& {\tilde{a}}_{n2}& \cdots & {\tilde{a}}_{nn}\end{array}\right].$$

(4)

where ${\tilde{a}}_{ij}$ is the scale of $T_i$ comparing with $T_j$; while the scale is ${({\tilde{a}}_{ij})}^{-1}$ when $T_j$ comparing with $T_i$.

$${\tilde{a}}_{ij}=\left(l_{ij},m_{ij},n_{ij},s_{ij}\right)$$

(5)

$${\tilde{a}}_{ji}={({\tilde{a}}_{ij})}^{-1}=\left(s_{ij}{}^{-1},n_{ij}{}^{-1},m_{ij}{}^{-1},l_{ij}{}^{-1}\right)$$

(6)

4.2.2. Checking Consistency

The defuzzification of fuzzy numbers should be done by Equation (3). Then, the fuzzy comparison matrix $\tilde{A}$ can be converted to:

$$A=\left[\begin{array}{cccc}{a}_{11}& a_{12}& \cdots & a_{1n}\\ a_{21}& a_{22}& \cdots & a_{2n}\\ ⋮& ⋮& \cdots & ⋮\\ a_{n1}& a_{n2}& \cdots & a_{nn}\end{array}\right]$$

(7)

After the defuzzification of each pair-wise matrix, the consistency can be checked by the following:

(1) Calculating the largest eigenvalue of the matrix. The largest eigenvalue can be calculated as follows:

$${\lambda }_{max}={\displaystyle \sum }_{i=1}^n\frac{{\left(AW\right)}_i}{n{W}_i}$$

(8)

where W is the principal eigenvector of the matrix.

(2) Consistency check

The consistency of the pair-wise comparison is checked by the consistency ratio (CR) defined as follows:

$$CR=\frac{CI}{RI}$$

(9)

$$CI=\frac{{\lambda }_{max}-n}{n-1}$$

(10)

Here, CI is the consistency index. RI is the average index from the table of random consistency indices shown in

Table 4. The random consistency index (RI).

Size (n)	1	2	3	4	5	6	7	8	9	10
RI	0.00	0.00	0.58	0.90	1.12	1.24	1.32	1.41	1.45	1.49

. If CR < 0.10, the results calculated are reliable, representing a high level of consistency. On the contrary, if CR > 0.10, the results of the pair-wise comparison are inconsistent and should be revised [43].

In this study, the values of the consistency were checked by Equations (8)–(10) and were all less 0.10, so each comparison matrix was consistent. This shows that the judgment matrix has good consistency.

4.2.3. Calculating the Weight

According to the pair-wise comparison matrix A, the relative importance weights are calculated as follows [44]:

$${\mathit{\alpha }}_{\mathit{i}}\mathit{=}{\mathit{\left(}{\displaystyle \mathit{\prod }}_{\mathit{j}\mathit{=}\mathit{1}}^{\mathit{s}}{\mathit{l}}_{\mathit{i}\mathit{j}}\mathit{\right)}}^{\mathit{1}\mathit{/}\mathit{s}}\mathit{,}\mathit{\alpha }\mathit{=}{\displaystyle \mathit{\sum }}_{\mathit{i}\mathit{=}\mathit{1}}^{\mathit{s}}{\mathit{\alpha }}_{\mathit{i}}$$

(11)

$${\mathit{\beta }}_{\mathit{i}}\mathit{=}{\mathit{\left(}{\displaystyle \mathit{\prod }}_{\mathit{j}\mathit{=}\mathit{1}}^{\mathit{s}}{\mathit{m}}_{\mathit{i}\mathit{j}}\mathit{\right)}}^{\mathit{1}\mathit{/}\mathit{s}}\mathit{,}\mathit{\beta }\mathit{=}{\displaystyle \mathit{\sum }}_{\mathit{i}\mathit{=}\mathit{1}}^{\mathit{s}}{\mathit{\beta }}_{\mathit{i}}$$

(12)

$${\mathit{\gamma }}_{\mathit{i}}\mathit{=}{\mathit{\left(}{\displaystyle \mathit{\prod }}_{\mathit{j}\mathit{=}\mathit{1}}^{\mathit{s}}{\mathit{n}}_{\mathit{i}\mathit{j}}\mathit{\right)}}^{\mathit{1}\mathit{/}\mathit{s}}\mathit{,}\mathit{\gamma }\mathit{=}{\displaystyle \mathit{\sum }}_{\mathit{i}\mathit{=}\mathit{1}}^{\mathit{s}}{\mathit{\gamma }}_{\mathit{i}}$$

(13)

$${\mathit{\delta }}_{\mathit{i}}\mathit{=}{\mathit{\left(}{\displaystyle \mathit{\prod }}_{\mathit{j}\mathit{=}\mathit{1}}^{\mathit{s}}{\mathit{s}}_{\mathit{i}\mathit{j}}\mathit{\right)}}^{\mathit{1}\mathit{/}\mathit{s}}\mathit{,}\mathit{\delta }\mathit{=}{\displaystyle \mathit{\sum }}_{\mathit{i}\mathit{=}\mathit{1}}^{\mathit{s}}{\mathit{\delta }}_{\mathit{i}}$$

(14)

Then, the weights can be gained as follows:

$${\tilde{w}}_i=\left({\alpha }_i{\delta }^{-1},{\beta }_i{\gamma }^{-1},{\gamma }_i{\beta }^{-1},{\delta }_i{\alpha }^{-1}\right)\hspace{1em}i\in \{1,2,\cdots ,s\}$$

(15)

Therefore, the fuzzy weight vector $\tilde{W}$ can be constructed as follows:

$$\tilde{W}=\left({\tilde{w}}_1,\hspace{0.17em}{\tilde{w}}_2,\hspace{0.17em}\cdots \hspace{0.17em},\hspace{0.17em}{\tilde{w}}_{n\hspace{0.17em}}\right)$$

(16)

According to the pair-wise comparison results and Equations (11)–(16), the fuzzy weight vectors of the criteria and sub-criteria can be calculated. In addition, the vectors were defuzzified using Equation (2) in order to clearly compare the weights. Finally, the matching crisp weight values were obtained, as in

Table 5. The weights of the criteria and sub-criteria.

Criteria and Sub-Criteria	Fuzzy Weight Vector	Defuzzified Weights
C1	(0.036, 0.044, 0.069, 0.091)	0.055
C2	(0.078, 0.106, 0.188, 0.256)	0.144
C3	(0.282, 0.357, 0.549, 0.684)	0.435
C4	(0.036, 0.044, 0.069, 0.091)	0.055
C5	(0.048, 0.066, 0.118, 0.164)	0.091
C6	(0.125, 0.168, 0.284, 0.371)	0.220
C11	(0.189, 0.289, 0.560, 0.776)	0.397
C12	(0.171, 0.240, 0.451, 0.652)	0.329
C13	(0.077, 0.108, 0.202, 0.295)	0.148
C14	(0.065, 0.087, 0.168, 0.266)	0.126
C21	(0.258, 0.357, 0.614, 0.804)	0.461
C22	(0.153, 0.212, 0.386, 0.543)	0.291
C23	(0.098, 0.129, 0.229, 0.328)	0.175
C24	(0.046, 0.058, 0.093, 0.125)	0.073
C31	(0.049, 0.061, 0.097, 0.129)	0.075
C32	(0.154, 0.201, 0.340, 0.487)	0.263
C33	(0.161, 0.259, 0.491, 0.664)	0.355
C34	(0.045, 0.056, 0.086, 0.112)	0.067
C35	(0.134, 0.179, 0.310, 0.449)	0.239
C41	(0.290, 0.369, 0.584, 0.732)	0.459
C42	(0.067, 0.084, 0.138, 0.185)	0.109
C43	(0.036, 0.044, 0.071, 0.094)	0.056
C44	(0.095, 0.123, 0.210, 0.285)	0.164
C45	(0.118, 0.160, 0.282, 0.355)	0.212
C51	(0.050, 0.060, 0.094, 0.125)	0.078
C52	(0.136, 0.171, 0.257, 0.315)	0.210
C53	(0.071, 0.092, 0.146, 0.187)	0.118
C54	(0.420, 0.505, 0.712, 0.848)	0.595
C61	(0.260, 0.356, 0.599, 0.773)	0.457
C62	(0.184, 0.240, 0.410, 0.561)	0.318
C63	(0.047, 0.058, 0.091, 0.120)	0.072
C64	(0.092, 0.118, 0.197, 0.265)	0.153

.

4.3. Building the Fuzzy CE

4.3.1. Determining the Grading Level Set

Assuming that the number of comments is m, the grading level set of comments of the evaluation object is:

$$V=\left(v_1,v_2,\cdots ,v_m\right)$$

(17)

where $v_j\left(j=1,2,\cdots ,m\right)$ represents the level of evaluation grades from high to low. In this study, $V=\left(0.9,0.7,0.5,0.3,0.1\right)$ indicates that the evaluation attitudes are ranked from very good (VG), good (G), medium (M), poor (P) to very poor (VP).

4.3.2. Establishing the Fuzzy Evaluation Matrix

Based on the summarization of the evaluations from experts, a certain criterion fuzzy evaluation matrix can be defined as follows:

$$R_k=\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{1j}& r_{1n}\\ r_{21}& r_{22}& r_{2j}& r_{2n}\\ ⋮& ⋮& ⋮& ⋮\\ \cdots & \cdots & \cdots & \cdots \\ r_{i1}& r_{i2}& r_{ij}& r_{in}\\ ⋮& ⋮& ⋮& ⋮\\ r_{m1}& r_{m2}& r_{mj}& r_{mn}\end{array}\right]$$

(18)

4.3.3. Calculating the Fuzzy Comprehensive Evaluation Value

According to the pair-wise comparison result and the defuzzification, the weight coefficients of the sub-criteria within the same above criterion can be obtained.

$${\beta }_k=\left({\beta }_1,{\beta }_2,\cdots ,{\beta }_n\right),k=1,2,\cdots ,s$$

(19)

Then a certain criterion’s fuzzy comprehensive evaluation can be defined as follows:

$$U_k={\beta }_k\hspace{0.17em}•\hspace{0.17em}{R}_k•\hspace{0.17em}{V}^T$$

(20)

Consequently, an integrated evaluation value for a set of products is obtained as follows:

$$U=\left({\alpha }_1,{\alpha }_2,\cdots ,{\alpha }_s\right)\hspace{0.17em}•\hspace{0.17em}\left[\begin{array}{c}{U}_1\\ U_2\\ ⋮\\ U_s\end{array}\right]$$

(21)

where ${\alpha }_i\left(i=1,2,\cdots ,s\right)$ are the weights of the criteria in the same level.

Here, in order to illustrate the use of this model, the criterion (aesthetics C₁) is exemplified. Firstly, the fuzzy evaluation matrix of aesthetics C₁ for the kitchen cabinet in the case is constructed by Equation (18) on the basis of the summarization of the evaluations showed in Table 3.

$$R_1=\left[\begin{array}{ccccc}0.4& 0.2& 0.2& 0.2& 0\\ 0.3& 0.3& 0.2& 0.2& 0\\ 0.2& 0.4& 0.2& 0.2& 0\\ 0.3& 0.2& 0.4& 0.1& 0\end{array}\right]$$

(22)

The fuzzy comprehensive evaluation value U₁ of aesthetics C₁ can be obtained using Equations (19) and (20).

$$\begin{array}{l}{U}_1={\beta }_1•R_1•V^T\hspace{0.17em}\\ =\left(0.397\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0.329\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0.148\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0.126\right)\hspace{0.17em}•\hspace{0.17em}\left[\begin{array}{ccccc}0.4& 0.2& 0.2& 0.2& 0\\ 0.3& 0.3& 0.2& 0.2& 0\\ 0.2& 0.4& 0.2& 0.2& 0\\ 0.3& 0.2& 0.4& 0.1& 0\end{array}\right]\hspace{0.17em}•\hspace{0.17em}\left[\begin{array}{c}0.9\\ 0.7\\ 0.5\\ 0.3\\ 0.1\end{array}\right]=0.64498\end{array}$$

By the same manner, then the fuzzy comprehensive evaluation values U₂, U₃, U₄, U₅, and U₆ for C₂, C₃, C₄, C₅, and C₆ yields 0.56894, 0.64180, 0.65152, 0.57084, and 0.62238, respectively.

Finally, the integrated evaluation value U of the kitchen cabinet assessed in this study can be obtained by Equation (21).

$$\begin{array}{l}U=\left({\alpha }_1,{\alpha }_2,{\alpha }_3,{\alpha }_4,{\alpha }_5,{\alpha }_6\right)\hspace{0.17em}•\hspace{0.17em}\left[\begin{array}{c}{U}_1\\ U_2\\ U_3\\ U_4\\ U_5\\ U_6\end{array}\right]\\ =\left(0.055\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0.144\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0.435\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0.055\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0.091\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0.220\right)\hspace{0.17em}•\hspace{0.17em}\left[\begin{array}{c}0.64498\\ 0.56894\\ 0.64180\\ 0.65152\\ 0.57084\\ 0.62238\end{array}\right]=0.621\end{array}$$

5. Results

The overall evaluation value of 0.621 indicates that the landscape architectural OWF in Harbin Pingfang Park is rated in a range between good (G) and medium (M) according to the grading level set V = (0.9, 0.7, 0.5, 0.3, 0.1) = (VG, G, M, P, VP). Consequently, the result reflects the fact that the competitive performance of this outdoor wooden equipment is not strong. In addition, this work also reveals the competitive performance of each criterion. In terms of criteria, the functionality (C₂), technics and structure (C₅) and environmental quality (C₆) are less powerful and their fuzzy comprehensive evaluation values are (U₂) 0.56894, (U₅) 0.57084 and (U₆) 0.62238, respectively. On the contrary, these three criteria hold more weight. The result demonstrates that these OWF products were not designed for the three attributes above.

6. Discussion

Predicting the competitive performance of a product in the development process is an effective measure to minimize the risk of failure and also helps companies take some appropriate measures to reduce economic losses. Based on the research finding (0.621) from the case study, the integrated evaluation did not receive high remarks and this company can take some measures to further enhance the competitiveness of different models of outdoor wooden site furnishings in Harbin Pingfang Park before it is put into production or to market. Outdoor furnishings are an important part of urban landscapes, which directly affects the quality of urban space planning, affecting people’s comfort and happiness. Through analyzing the calculation results, the overall design of these OWF products in Harbin Pingfang Park need to be strengthened, especially for the three criteria (functionality (C₂) technics and structure (C₅) and environmental quality (C₆) with lower evaluation values. In fact, as is shown in Table 5, functionality (C₂), technics and structure (C₅) and environmental quality (C₆) have higher weights. However, their fuzzy comprehensive evaluation values are lower than other criteria. The result indicates that designers need to further study these three criteria and address their performance level for the Harbin Pingfang Park project. To enhance the functionality of OWF, more detailed and reasonable design should be made rather than paying more attention to aesthetics. To ensure the technics and structure, designers in the landscape design firm should cooperate closely with structure engineers and professional OWF manufacturers. In this way, the design and structure can be well-integrate. Aimed at improving the environmental quality of OWF in landscape projects, ecological-friendly and low-impact materials should be selected. Wood and metal parts should be non-toxic and harmless to the environment. In landscape architectural projects that emphasize ecological protection and sustainability, the form and scale of OWF should be weighed to avoid occupying unnecessary environmental space and disturbing environmental habitats. For further improvements, the sub-criteria shown in Figure 2 can be considered as important implications. In addition, designers could consider redesigning the product if the evaluation result is very poor. Successful products in the development cycle may be classified according to the most outstanding trait (e.g., economy, functionality or environmental quality). This can help companies to provide specialized products for each individual market segment. In this situation, different customers’ requirements can be better and quickly satisfied. In addition, it can also make product management more systematic. Furthermore, the decision-making approach can be extended to other real-world applications of NPD or risk management for a broader population of innovating firms or manufacturing companies.

7. Conclusions

In this study, the CE approach was applied to assess the performance of a product in development. The major steps involve:

• Constructing the evaluation index system;
• Establishing trapezoidal fuzzy AHP;
• Proposing the CE model.

This evaluation approach shows two advantages: first, the CE method is a multi-factor fuzzy comprehensive evaluation and conducts a thorough evaluation taking into account multiple perspectives (i.e., aesthetics, functionality, economy, material, technics/structure and environmental quality). Therefore, evaluation results are more objective compared with the other methods. On the other hand, trapezoidal fuzzy numbers are employed to make pair-wise comparisons in fuzzy AHP. This can better handle the subjective evaluation of experts and result in more accurate and effective weights of criteria and sub-criteria. Therefore, this research can provide a systematic and scientific method for designers or decision makers to identify product performance and to minimize failure rates.

A case study involving real-world landscape architectural OWF products in Harbin Pingfang Park illustrates the feasibility of the proposed model. The result demonstrates that the model can reflect the competitive performance of products. A manufacturer could redesign or address the weakness of a product to develop a more competitive one. This case shows that the CE method is suitable for evaluating the performance of a product during the development process and can reduce the risk of market failure.

Although the proposed approach is feasible, there are still some limitations. First, since members of the expert group are from China, the weights and the evaluation results in the case are acquired in a Chinese context. Hence, the results may have less application outside China. In addition, experts’ assessments heavily rely on their experience and knowledge; experts may display individual subjectivity. Second, the case study is only limited to landscape-architectural OWF. Thirdly, the evaluation index system and the parameters of the criteria included can only be applied to landscape-architectural OWF related to rest and sports; there could be more comprehensive criteria and sub-criteria but these require more complex surveys and calculations. Therefore, it is necessary to develop a computational model and related analytical system to facilitate the better processing of evaluation-index system building. These possibilities and assumptions are directions for future studies.