A Hybrid Fuzzy MCDM Approach to Open Innovation Partner Evaluation
Abstract
:1. Introduction
2. The Problem of OI Partner Selection
3. Materials and Methods
- (i)
- For the first time, a comprehensive framework of OI partner performance indicators is provided, consisting of five critical dimensions and twenty-seven indicators, encompassing all relevant technological, operational, and strategic evaluation aspects.
- (ii)
- The novel hybrid IT2F MCDM model combining IT2FD, IT2F AHP, and IT2F PROMETHEE is established, which provides a contingent approach to identifying OI partner evaluation criteria considering the nature of the company’s innovation processes, contextual conditions, and innovation strategy; and yields precise multi-criteria alternatives evaluation under a high uncertainty level.
3.1. Multi-Criteria Decision-Making and Interval Type-2 Fuzzy Sets
3.2. Interval Type-2 Fuzzy Sets
- The meanings of the used linguistic terms and the consequences of the rules can be uncertain;
- Consequents may have a histogram of the values associated with them, especially when knowledge is extracted from a group of experts who do not have a unified attitude;
- The measurements that activate type-1 fuzzy logic may be uncertain;
- The data used to tune the parameters of the type-1 fuzzy logic system may be noisy.
- —reference points of IT2F number
- —value of in upper trapezoidal membership function.
- —value of in lower trapezoidal membership function.
3.3. Interval Type-2 Delphi Model
3.4. Interval Type-2 AHP Model
3.5. Interval Type-2 PROMETHEE Model
- —indifference—
- —weak preference—
- —strong preference—
- —strict preference—
- Usual Criterion
- U-shape Criterion
- V-shape Criterion
- Level Criterion
- V-shape with indifference Criterion
- Gaussian Criterion
4. Results
4.1. Establishing the Evaluation Criteria List by IT2FD
4.2. Criteria Weighting by the IT2F AHP
4.3. OI Partner Evaluation by IT2F PROMETHEE
5. Discussion
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Zhang, X.; Chu, Z.; Ren, L.; Xing, J. Open Innovation and Sustainable Competitive Advantage: The Role of Organizational Learning. Technol. Forecast. Soc. Chang. 2023, 186, 122114. [Google Scholar] [CrossRef]
- Barrett, G.; Dooley, L.; Bogue, J. Open Innovation within High-Tech SMEs: A Study of the Entrepreneurial Founder’s Influence on Open Innovation Practices. Technovation 2021, 103, 102232. [Google Scholar] [CrossRef]
- Kwon, S.; Motohashi, K. How Institutional Arrangements in the National Innovation System Affect Industrial Competitiveness: A Study of Japan and the US with Multiagent Simulation. Technol. Forecast. Soc. Chang. 2017, 115, 221–235. [Google Scholar] [CrossRef]
- Tidd, J.; Bessant, J.R. Managing Innovation: Integrating Technological, Market and Organizational Change; John Wiley & Sons: Hoboken, NJ, USA, 2013. [Google Scholar]
- Hung, K.P.; Chiang, Y.H. Open Innovation Proclivity, Entrepreneurial Orientation, and Perceived Firm Performance. Int. J. Technol. Manag. 2010, 52, 257–274. [Google Scholar] [CrossRef]
- Chesbrough, H.; Bogers, M. Explicating Open Innovation: Clarifying an Emerging Paradigm for Understanding Innovation. In New Frontiers in Open Innovation; Oxford University Press, Forthcoming: Oxford, UK, 2014. [Google Scholar]
- Becker, W.; Dietz, J. R&D Cooperation and Innovation Activities of Firms: Evidence for the German Manufacturing Industry. Res. Policy 2004, 33, 209–223. [Google Scholar]
- Dubouloz, S.; Bocquet, R.; Equey Balzli, C.; Gardet, E.; Gandia, R. SMEs’ Open Innovation: Applying a Barrier Approach. Calif. Manag. Rev. 2021, 64, 113–137. [Google Scholar] [CrossRef]
- Bogers, M.; Chesbrough, H.; Moedas, C. Open Innovation: Research, Practices, and Policies. Calif. Manag. Rev. 2018, 60, 5–16. [Google Scholar] [CrossRef]
- Hizam-Hanafiah, M.; Soomro, M.A. The Situation of Technology Companies in Industry 4.0 and the Open Innovation. J. Open Innov. Technol. Mark. Complex. 2021, 7, 34. [Google Scholar] [CrossRef]
- Shmeleva, N.; Gamidullaeva, L.; Tolstykh, T.; Lazarenko, D. Challenges and Opportunities for Technology Transfer Networks in the Context of Open Innovation: Russian Experience. J. Open Innov. Technol. Mark. Complex. 2021, 7, 197. [Google Scholar] [CrossRef]
- Carmona-Lavado, A.; Cuevas-Rodríguez, G.; Cabello-Medina, C.; Fedriani, E.M. Does Open Innovation Always Work? The Role of Complementary Assets. Technol. Forecast. Soc. Chang. 2021, 162, 120316. [Google Scholar] [CrossRef]
- Wang, X.; Wang, Z.; Huang, Y.; Liu, Y.; Zhang, J.; Heng, X.; Zhu, D. Identifying R&D Partners through Subject-Action-Object Semantic Analysis in a Problem & Solution Pattern. Technol. Anal. Strateg. Manag. 2017, 29, 1167–1180. [Google Scholar]
- Holle, M.; Elsesser, L.; Schuhmacher, M.; Lindemann, U. How to Motivate External Open Innovation-Partners: Identifying Suitable Measures. In Proceedings of the 2016 Portland International Conference on Management of Engineering and Technology (PICMET), Honolulu, HI, USA, 4–8 September 2016. [Google Scholar]
- Du, J.; Leten, B.; Vanhaverbeke, W. Managing Open Innovation Projects with Science-based and Market-based Partners. Res. Policy 2014, 43, 828–840. [Google Scholar] [CrossRef]
- Santos, R.S.; Soares, J.; Marques, P.C.; Navas, H.V.; Martins, J.M. Integrating Business, Social, and Environmental Goals in Open Innovation through Partner Selection. Sustainability 2021, 13, 12870. [Google Scholar] [CrossRef]
- Wang, G.; Tian, X.; Hu, Y.; Evans, R.D.; Tian, M.; Wang, R. Manufacturing Process Innovation-Oriented Knowledge Evaluation Using MCDM and Fuzzy Linguistic Computing in an Open Innovation Environment. Sustainability 2017, 9, 1630. [Google Scholar] [CrossRef] [Green Version]
- Nguyen, P.H.; Tran, L.C.; Nguyen, H.B.D.; Ho, T.P.T.; Duong, Q.A.; Tran, T.N. Unlocking the Potential of Open Innovation through Understanding the Interrelationship among Key Determinants of FDI Attractiveness. J. Open Innov. Technol. Mark. Complex. 2023, 9, 100021. [Google Scholar] [CrossRef]
- Shahin, A.; Mahdian, E. Ranking the Indicators of Open Innovation Adoption based on NPD Factors. Int. J. Bus. Innov. Res. 2023, 22, 408–428. [Google Scholar] [CrossRef]
- Ar, İ.M.; Peker, İ.; Birdoğan, B.A.K.İ. Determining the Appropriate Open Innovation Model for Logistics Firms Using An Integrated Fuzzy AHP-VIKOR Approach. Int. J. Econ. Admin. Stud. 2023, 2023, 135–148. [Google Scholar] [CrossRef] [Green Version]
- Yildirim, E.; Ar, I.M.; Dabić, M.; Baki, B.; Peker, I. A Multi-Stage Decision Making Model for Determining a Suitable Innovation Structure Using an Open Innovation Approach. J. Bus. Res. 2022, 147, 379–391. [Google Scholar] [CrossRef]
- Vakil Alroaia, Y. Open Innovation and SMEs: Providing a Model for Business Development (An Application on Iranian Industrial Park). J. Appl. Res. Ind. Eng. 2023, 10, 125–140. [Google Scholar]
- Hakaki, A.; Nikabadi, M.S.; Heidarloo, M.A. An Optimized Model for Open Innovation Success in Manufacturing SMES. RAIRO Oper. Res. 2021, 55, 3339–3357. [Google Scholar] [CrossRef]
- Mubarak, M.F.; Petraite, M.; Rasli, A.; Shabbir, M. Capability Framework to Support Supply Chain Open Innovation Networks. In Blockchain Driven Supply Chain Management: A Multi-Dimensional Perspective; Springer Nature: Singapore, 2023; pp. 119–134. [Google Scholar]
- Vesic Vasovic, J.; Puzovic, S.; Paunovic, V. Selection of Partners in Collaborative Technological R&D Projects: An Approach to Criteria Prioritization. In Operational Research in the Era of Digital Transformation and Business Analytics. BALCOR 2020. Springer Proceedings in Business and Economics; Matsatsinis, N., Kitsios, F., Madas, M., Kamariotou, M., Eds.; Springer: Cham, Switzerland, 2023; pp. 77–87. [Google Scholar]
- Aleksić, A.; Tadić, D. Industrial and Management Applications of Type-2 Multi-Attribute Decision-Making Techniques Extended with Type-2 Fuzzy Sets from 2013 to 2022. Mathematics 2023, 11, 2249. [Google Scholar] [CrossRef]
- Yoon, B.; Song, B. A Systematic Approach of Partner Selection for Open Innovation. Ind. Manag. Data Syst. 2014, 114, 1068–1093. [Google Scholar] [CrossRef]
- Manotungvorapun, N.; Gerdsri, N. Complementarity vs. Compatibility: What Really Matters for Partner Selection in Open Innovation? Int. J. Transit. Innov. Syst. 2016, 5, 122–139. [Google Scholar] [CrossRef]
- Park, I.; Yoon, B. Identifying Potential Partnership for Open Innovation by Using Bibliographic Coupling and Keyword Vector Mapping. Int. J. Comput. Electr. Autom. Contr. Inf. Eng. 2013, 7, 206–211. [Google Scholar]
- Wang, M.Y. Exploring Potential R&D Collaborators with Complementary Technologies: The Case of Biosensors. Technol. Forecast. Soc. Chang. 2012, 79, 862–874. [Google Scholar]
- Jeon, J.; Lee, C.; Park, Y. How to Use Patent Information to Search Potential Technology Partners in Open Innovation. J. Intellect. Prop. Rights 2011, 16, 385–393. [Google Scholar]
- Angue, K.; Ayerbe, C.; Mitkova, L. A Method Using Two Dimensions of the Patent Classification for Measuring the Technological Proximity: An Application in Identifying a Potential R&D Partner in Biotechnology. J. Technol. Transf. 2014, 39, 716–747. [Google Scholar]
- Ades, C.; Figlioli, A.; Sbragia, R.; Porto, G.; Ary Plonski, G.; Celadon, K. Implementing Open Innovation: The Case of Natura, IBM and Siemens. J. Technol. Manag. Innov. 2013, 8, 57. [Google Scholar] [CrossRef]
- Paixao Garcez, M.; Sbragia, R. The Selection of Partners in Technological Alliances Projects. J. Technol. Manag. Innov. 2013, 8, 49. [Google Scholar] [CrossRef]
- Emden, Z.; Calantone, R.J.; Droge, C. Collaborating for New Product Development: Selecting the Partner with Maximum Potential to Create Value. J. Prod. Innov. Manag. 2006, 23, 330–341. [Google Scholar] [CrossRef]
- Büyüközkan, G.; Feyzioğlu, O.; Nebol, E. Selection of the Strategic Alliance Partner in Logistics Value Chain. Int. J. Prod. Econ. 2008, 113, 148–158. [Google Scholar] [CrossRef]
- Holmberg, S.R.; Cummings, J.L. Building Successful Strategic Alliances: Strategic Process and Analytical Tool for Selecting Partner Industries and Firms. Long Range Plan. 2009, 42, 164–193. [Google Scholar] [CrossRef]
- Arranz, N.; de Arroyabe, J.C.F. The Choice of Partners in R&D Cooperation: An Empirical Analysis of Spanish Firms. Technovation 2008, 28, 88–100. [Google Scholar]
- Shah, R.H.; Swaminathan, V. Factors Influencing Partner Selection in Strategic Alliances: The Moderating Role of Alliance Context. Strateg. Manag. J. 2008, 29, 471–494. [Google Scholar] [CrossRef]
- Sarkar, M.B.; Echambadi, R.; Cavusgil, S.T.; Aulakh, P.S. The Influence of Complementarity, Compatibility, and Relationship Capital on Alliance Performance. J. Acad. Mark. Sci. 2001, 29, 358–373. [Google Scholar] [CrossRef]
- Tian, Q.F.; Zhang, T.; Zhang, S.; Miao, D.D. Research on Elements Fusion Mechanism of Dual-use Science and Technology Collaborative Innovation. J. Sci. Technol. Policy Manag. 2020, 37, 136–145. [Google Scholar]
- Harrigan, K.R. Joint Ventures and Competitive Strategy. Strateg. Manag. J. 1988, 9, 141–158. [Google Scholar] [CrossRef]
- Guertler, M.R.; Haymerle, R.; Endres, F.; Lindemann, U. Identifying Open Innovation Partners: A Case Study in Plant Manufacturing. In Proceedings of the International Society for Professional Innovation Management (ISPIM) Innovation Summit, Brisbane, Australia, 1–18 December 2015. [Google Scholar]
- Yin, J.; Tan, Q.M. A Research on the Evaluation of the Degree of Civil-military Integration and Its Optimization Measures. Sci. Res. Manag. 2020, 41, 90–97. [Google Scholar]
- Geringer, J.M. Strategic Determinants of Partner Selection Criteria in International Joint Ventures. J. Int. Bus. Stud. 1991, 22, 41–62. [Google Scholar] [CrossRef]
- Chen, M.K.; Wang, S.C. The Use of a Hybrid Fuzzy-Delphi-AHP Approach to Develop Global Business Intelligence for Information Service Firms. Expert Syst. Appl. 2010, 37, 7394–7407. [Google Scholar] [CrossRef]
- Solesvik, M.; Westhead, P. Partner Selection for Strategic Alliances: Case Study Insights from the Maritime Industry. Ind. Manag. Data Syst. 2010, 110, 841–860. [Google Scholar] [CrossRef]
- Kim, Y.; Lee, K. Technological Collaboration in the Korean Electronic Parts Industry: Patterns and Key Success Factors. R&D Manag. 2003, 33, 59–77. [Google Scholar]
- Dacin, M.T.; Hitt, M.A.; Levitas, E. Selecting Partners for Successful International Alliances: Examination of US and Korean Firms. J. World Bus. 1997, 32, 3–16. [Google Scholar] [CrossRef]
- Brouthers, K.D.; Brouthers, L.E.; Wilkinson, T.J. Strategic Alliances: Choose Your Partners. Long Range Plan. 1995, 28, 2–25. [Google Scholar] [CrossRef]
- Zadeh, L.A. Fuzzy Sets. Inf. Control 1965, 8, 338–353. [Google Scholar] [CrossRef] [Green Version]
- Liang, G.S. Fuzzy MCDM Based on Ideal and Anti-ideal Concepts. Eur. J. Oper. Res. 1999, 112, 682–691. [Google Scholar] [CrossRef]
- Puzovic, S.; Vasovic Vasovic, J.; Radojicic, M.; Paunovic, V. An Integrated MCDM Approach to PLM Software Selection. Acta Polytech. Hung. 2019, 16, 45–65. [Google Scholar]
- Aleksic, A.; Runic Ristic, M.; Komatina, N.; Tadic, D. Advanced Risk Assessment in Reverse Supply Chain Processes: A Case Study in Republic of Serbia. Adv. Prod. Eng. Manag. 2019, 14, 421–434. [Google Scholar] [CrossRef]
- Mathew, M.; Chakrabortty, R.K.; Ryan, M.J. Selection of an Optimal Maintenance Strategy under Uncertain Conditions: An Interval Type-2 Fuzzy AHP-TOPSIS Method. IEEE Trans. Eng. Manag. 2020, 69, 1121–1134. [Google Scholar] [CrossRef]
- Ecer, F. Multi-Criteria Decision Making for Green Supplier Selection Using Interval Type-2 Fuzzy AHP: A Case Study of a Home Appliance Manufacturer. Oper. Res. 2022, 22, 199–233. [Google Scholar] [CrossRef]
- Jovcic, S.; Průsa, P.; Dobrodolac, M.; Svadlenka, L. A Proposal for a Decision-Making Tool in Third-Party Logistics (3PL) Provider Selection based on Multi-Criteria Analysis and the Fuzzy Approach. Sustainability 2019, 11, 4236. [Google Scholar] [CrossRef] [Green Version]
- Puška, A.; Stojanović, I. Fuzzy Multi-Criteria Analyses on Green Supplier Selection in an Agri-Food Company. J. Intell. Manag. Decis. 2022, 1, 2–16. [Google Scholar] [CrossRef]
- Su, J.; Xu, B.; Li, L.; Wang, D.; Zhang, F. A Green Supply Chain Member Selection Method Considering Green Innovation Capability in a Hesitant Fuzzy Environment. Axioms 2023, 12, 188. [Google Scholar] [CrossRef]
- Petrovic, T.; Vesic Vasovic, J.; Komatina, N.; Tadic, D.; Klipa, Đ.; Đuric, G. A Two-Stage Model based on EFQM, FBWM, and FMOORA for Business Excellence Evaluation in the Process of Manufacturing. Axioms 2022, 11, 704. [Google Scholar] [CrossRef]
- Celik, E.; Gul, M.; Aydin, N.; Gumus, A.T.; Guneri, A.F. A Comprehensive Review of Multi Criteria Decision Making Approaches Based on Interval Type-2 Fuzzy Sets. Knowl. Based Syst. 2015, 85, 329–341. [Google Scholar] [CrossRef]
- Shringi, A.; Arashpour, M.; Golafshani, E.M.; Dwyer, T.; Kalutara, P. Enhancing Safety Training Performance Using Extended Reality: A Hybrid Delphi–AHP Multi-Attribute Analysis in a Type-2 Fuzzy Environment. Buildings 2023, 13, 625. [Google Scholar] [CrossRef]
- Mendel, J.M.; John, R.I.; Liu, F. Interval Type-2 Fuzzy Logic Systems Made Simple. IEEE Trans. Fuzzy Syst. 2006, 14, 808–821. [Google Scholar] [CrossRef] [Green Version]
- Zadeh, L.A. The Concept of a Linguistic Variable and Its Application to Approximate Reasoning: I. Inf. Sci. 1975, 8, 199–249. [Google Scholar] [CrossRef]
- Gölcük, İ. An Interval Type-2 Fuzzy Reasoning Model for Digital Transformation Project Risk Assessment. Expert Syst. Appl. 2020, 159, 113579. [Google Scholar] [CrossRef]
- Wu, Y.; Zhang, T.; Yi, L. An Internal Type-2 Trapezoidal Fuzzy Sets-PROMETHEE-II based Investment Decision Framework of Compressed Air Energy Storage Project in China under the Perspective of Diffewt Investors. J. Energy Storage 2020, 30, 101548. [Google Scholar] [CrossRef]
- Baykasoğlu, A.; Gölcük, İ. Development of an Interval Type-2 Fuzzy Sets based Hierarchical MADM Model by Combining DEMATEL and TOPSIS. Expert Syst. Appl. 2017, 70, 37–51. [Google Scholar] [CrossRef]
- Boral, S.; Chakraborty, S. Failure Analysis of CNC Machines Due to Human Errors: An Integrated IT2F-MCDM-based FMEA Approach. Eng. Fail. Anal. 2021, 130, 105768. [Google Scholar] [CrossRef]
- Bera, A.K.; Jana, D.K.; Banerjee, D.; Nandy, T. Supplier Selection Using Extended IT2 Fuzzy TOPSIS and IT2 Fuzzy MOORA Considering Subjective and Objective Factors. Soft Comput. 2020, 24, 8899–8915. [Google Scholar] [CrossRef]
- Karagöz, S.; Deveci, M.; Simic, V.; Aydin, N. Interval Type-2 Fuzzy ARAS Method for Recycling Facility Location Problems. Appl. Soft Comput. 2021, 102, 107107. [Google Scholar]
- Wei, Q.; Zhou, C.; Liu, Q.; Zhou, W.; Huang, J. A Barrier Evaluation Framework for Forest Carbon Sink Project Implementation in China Using an Integrated BWM-IT2F-PROMETHEE II method. Expert Syst. Appl. 2023, 2023, 120612. [Google Scholar]
- Mendel, J.M.; John, R.B. Type-2 Fuzzy Sets Made Simple. IEEE Trans. Fuzzy Syst. 2002, 10, 117–127. [Google Scholar] [CrossRef]
- Mabrouk, N. Green Supplier Selection Using Fuzzy Delphi Method for Developing Sustainable Supply Chain. Decis. Sci. Lett. 2021, 10, 63–70. [Google Scholar] [CrossRef]
- Deveci, M.; Simic, V.; Karagoz, S.; Antucheviciene, J. An Interval Type-2 Fuzzy Sets based Delphi Approach to Evaluate Site Selection Indicators of Sustainable Vehicle Shredding Facilities. Appl. Soft Comput. 2022, 118, 108465. [Google Scholar]
- Aleksić, A.; Nestić, S.; Huber, M.; Ljepava, N. The Assessment of the Key Competences for Lifelong Learning—The Fuzzy Model Approach for Sustainable Education. Sustainability 2022, 14, 2686. [Google Scholar] [CrossRef]
- Ayyildiz, E.; Taskin Gumus, A.; Erkan, M. Individual Credit Ranking by an Integrated Interval Type-2 Trapezoidal Fuzzy ELECTRE Methodology. Soft Comput. 2020, 24, 16149–16163. [Google Scholar] [CrossRef]
- Gupta, R.; Sachdeva, A.; Bhardwaj, A. Selection of 3pl Service Provider Using Integrated Fuzzy Delphi and Fuzzy TOPSIS. In Proceedings of the World Congress on Engineering and Computer Science, San Francisco, CA, USA, 20–22 October 2010. [Google Scholar]
- Çolak, M.; Kaya, İ. Prioritization of Renewable Energy Alternatives by Using an Integrated Fuzzy MCDM Model: A Real Case Application for Turkey. Renew. Sustain. Energy Rev. 2017, 80, 840–853. [Google Scholar] [CrossRef]
- Saaty, T.L. The Analytic Hierarchy Process; McGraw-Hill: New York, NY, USA, 1980. [Google Scholar]
- Wu, Y.; Xu, C.; Ke, Y.; Tao, Y.; Li, X. Portfolio Optimization of Renewable Energy Projects Under Type-2 Fuzzy Environment with Sustainability Perspective. Comput. Ind. Eng. 2019, 133, 69–82. [Google Scholar]
- Hoseini, S.A.; Hashemkhani Zolfani, S.; Skačkauskas, P.; Fallahpour, A.; Saberi, S. A Combined Interval Type-2 Fuzzy MCDM Framework for the Resilient Supplier Selection Problem. Mathematics 2021, 10, 44. [Google Scholar] [CrossRef]
- Kaya, S.K.; Aycin, E. An Integrated Interval Type 2 Fuzzy AHP and COPRAS-G Methodologies for Supplier Selection in the Era of Industry 4.0. Neural. Comput. Appl. 2021, 33, 10515–10535. [Google Scholar] [CrossRef]
- Saaty, T.L.; Vargas, L.G. Comparison of Eigenvalue, Logarithmic Least Squares and Least Squares Methods in Estimating Ratios. Math. Model. 1984, 5, 309–324. [Google Scholar] [CrossRef] [Green Version]
- Buckley, J.J. Fuzzy Hierarchical Analysis. Fuzzy Sets Syst. 1985, 17, 233–247. [Google Scholar] [CrossRef]
- Crawford, G.; Williams, C. A Note on the Analysis of Subjective Judgment Matrices. J. Math. Psychol. 1985, 29, 387–405. [Google Scholar] [CrossRef]
- Chang, D.Y. Applications of the Extent Analysis Method on Fuzzy AHP. Eur. J. Oper. Res. 1996, 95, 649–655. [Google Scholar] [CrossRef]
- Bryson, N. A Goal Programming Method for Generating Priority Vectors. J. Oper. Res. Soc. 1995, 46, 641–648. [Google Scholar] [CrossRef]
- Mikhailov, L. A Fuzzy Programming Method for Deriving Priorities in the Analytic Hierarchy Process. J. Oper. Res. Soc. 2000, 51, 341–349. [Google Scholar] [CrossRef]
- Chandran, B.; Golden, B.; Wasil, E. Linear Programming Models for Estimating Weights in the analytic Hierarchy Process. Comput. Oper. Res. 2005, 32, 2235–2254. [Google Scholar] [CrossRef]
- Xu, Z.; Da, Q. A Least Deviation Method to Obtain a Priority Vector of a Fuzzy Preference Relation. Eur. J. Oper. Res. 2005, 164, 206–216. [Google Scholar] [CrossRef]
- Ye-jun, X.U.; Qing-li, D.A. Weighted Least-Square Method and its Improvement for Priority of Incomplete Complementary Judgement Matrix. Syst. Eng. Electron. 2008, 7, 1273–1276. [Google Scholar]
- Brans, J.P.; Vincke, P. Note—A Preference Ranking Organisation Method: (The PROMETHEE Method for Multiple Criteria Decision-Making). Manag. Sci. 1985, 31, 647–656. [Google Scholar] [CrossRef] [Green Version]
- Chen, T.Y. A PROMETHEE-based Outranking Method for Multiple Criteria Decision Analysis with Interval Type-2 Fuzzy Sets. Soft Comput. 2014, 18, 923–940. [Google Scholar]
- Wu, Y.; Liu, F.; Huang, Y.; Xu, C.; Zhang, B.; Ke, Y.; Jia, W. A Two-Stage Decision Framework for inland Nuclear Power Plant Site Selection based on GIS and Type-2 Fuzzy PROMETHEE II: Case Study in China. Energy Sci. Eng. 2020, 8, 1941–1961. [Google Scholar] [CrossRef]
MCDM Methods | Research Focus | Studies |
---|---|---|
Fuzzy AHP | Collaboration network partner selection with integration business, social, and environmental goals | [16] |
Fuzzy AHP | Evaluation of process of innovation-oriented knowledge under the open innovation paradigm | [17] |
Fuzzy Delphi, Fuzzy DEMATEL, DANP | Prioritizing and analyzing interrelationships among factors affecting Foreign Direct Investment attractiveness and open innovation | [18] |
Fuzzy TOPSIS | Ranking the indicators of open innovation adoption based on new product development factors | [19] |
Fuzzy AHP, Fuzzy VIKOR | Determination of an appropriate open innovation model for logistics firms | [20] |
Delphi, Fuzzy ANP | End-to-end analysis of an open innovation setup for determining a suitable innovation structure | [21] |
ANP, PROMETHEE | Ranking the moderating factors that have contributed to the degree of small and mid-size enterprises’ participation in open innovation activities | [22] |
DEMATEL | Determination the best ranking of effective factors in open innovation success in manufacturing enterprises | [23] |
AHP ISM | Investigation mechanisms for improving supply chain open innovation networks | [24] |
IT2F AHP | Supporting the effective selection of partners for collaborative technological R&D projects | [25] |
Criteria | Studies | |
---|---|---|
Technological competencies | Technological innovation level | [41,42] |
Technological complementarity | [34] | |
Product experience | [43] | |
Number of patents held | [27] | |
Expected capabilities of abstraction | [43] | |
Technology transfer capability | [44] | |
Resource complementarity | Overlapping knowledge base | [35] |
Product-specific knowledge | [45] | |
Market knowledge complementarity | [35,45] | |
Expected knowledge maturity | [43] | |
Past experiences | [34] | |
Financial assets | [45] | |
Financial terms | Expected debt ratio and refund ability | [46] |
Financial resources demand of the project | [34] | |
Return of investment | [46] | |
Cooperative capability | Collaborative behavior | [28] |
Mutual trust and commitment | [35,39,45,47,48] | |
Management and organizational culture | [34,35,42,46,49,50] | |
Previous relationship | [28] | |
Propensity to change | [35] | |
Geographical proximity | [28,43] | |
Symmetry of scale and scope | [45,46] | |
Strategic alignments | Compatibility of corporation strategies | [46] |
Convergence of expectations between partners | [34] | |
Motivation and goal correspondence | [35,50] | |
Strategic objectives of intellectual property management | [28] | |
Market complementarity | [34] |
Experience | Qualification | Designation | IP Phase | Linguistic Variables | IT2F Numbers |
---|---|---|---|---|---|
≤5 | Under graduate | Up to executive | Launch and market penetration | Low | (0, 0.1, 0.15, 0.3; 1, 1) (0.05, 0.1, 0.15, 0.2; 0.9, 0.9) |
5–10 | Graduate | Executive to Specialist | Idea generation | Medium | (0.3, 0.5, 0.55, 0.7; 1, 1) (0.4, 0.5, 0.55, 0.6; 0.9, 0.9) |
10–15 | Master graduation | Specialist to Manager | Concept development | High | (0.7, 0.85, 0.9, 1; 1, 1) (0.8, 0.85, 0.9, 0.95; 0.9, 0.9) |
≥15 | Post graduate | Manager to GM | Product development | Very high | (0.9, 1, 1, 1; 1, 1) (0.95, 1, 1, 1; 0.9, 0.9) |
Linguistic Variables | IT2F Numbers |
---|---|
Very Low (VL) | (0, 0, 0, 0.01; 1, 1) (0, 0, 0, 0.05; 0.9, 0.9) |
Low (L) | (0, 0.1, 0.15, 0.3; 1, 1) (0.05, 0.1, 0.15, 0.2; 0.9, 0.9) |
Medium Low (ML) | (0.1, 0.3, 0.35, 0.5; 1, 1) (0.2, 0.3, 0.35, 0.4; 0.9, 0.9) |
Medium (M) | (0.3, 0.5, 0.55, 0.7; 1, 1) (0.4, 0.5, 0.55, 0.6; 0.9, 0.9) |
Medium High (MH) | (0.5, 0.7, 0.75, 0.9; 1, 1) (0.6, 0.7, 0.75, 0.8; 0.9, 0.9) |
High (H) | (0.7, 0.85, 0.9, 1; 1, 1) (0.8, 0.85, 0.9, 0.95; 0.9, 0.9) |
Very High (VH) | (0.9, 1, 1, 1; 1, 1) (0.95, 1, 1, 1; 0.9, 0.9) |
Linguistic Statements | IT2F Numbers |
---|---|
Absolutely Strong (AS) | (7, 8, 9, 9; 1, 1) (7.2, 8.2, 8.8, 9; 0.8, 0.8) |
Very Strong (VS) | (5, 6, 8, 9; 1, 1) (5.2, 6.2, 7.8, 8.8; 0.8, 0.8) |
Fairly Strong (FS) | (3, 4, 6, 7; 1, 1) (3.2, 4.2, 5.8, 6.8; 0.8, 0.8) |
Slightly Strong (SS) | (1, 2, 4, 5; 1, 1) (1.2, 2.2, 3.8, 4.8; 0.8, 0.8) |
Exactly Equal (EE) | (1, 1, 1, 1; 1, 1) (1, 1, 1, 1; 1, 1) |
Expert | Experience | Qualification | Designation | IP Phase | |
---|---|---|---|---|---|
E1 | 3 | Master graduate | Specialist to Manager | Idea generation | (0.43, 0.58, 0.63, 0.75; 1, 1) (0.51, 0.58, 0.63, 0.68; 0.9, 0.9) |
E2 | 8 | Postgraduate | Manager to GM | Product development | (0.75, 0.88, 0.89, 0.93; 1, 1) (0.81, 0.88, 0.89, 0.9; 0.9, 0.9) |
E3 | 15 | Graduate | Specialist to Manager | Concept Development | (0.65, 0.8, 0.84, 0.93; 1, 1) (0.74, 0.8, 0.84, 0.88; 0.9, 0.9) |
E4 | 9 | Graduate | Executive to Specialist | Launch and market penetration | (0.33, 0.49, 0.54, 0.68; 1, 1) (0.41, 0.49, 0.54, 0.59; 0.9, 0.9) |
Criteria | Criteria Weight | MAW | Selected Criteria | ||
---|---|---|---|---|---|
Technological innovation level | (0.46, 0.66, 0.71, 0.82; 1, 1) (0.57, 0.66, 0.71, 0.75; 0.9, 0.9) | (0.48, 0.61, 0.64, 0.73; 1, 1) (0.55, 0.61, 0.64, 0.67; 0.9, 0.9) | 0.663 | 0.612 | ✓ |
Technological complementarity | (0.38, 0.58, 0.65, 0.82; 1, 1) (0.5, 0.58, 0.65, 0.72; 0.9, 0.9) | (0.43, 0.56, 0.59, 0.67; 1, 1) (0.5, 0.56, 0.59, 0.62; 0.9, 0.9) | 0.608 | 0.561 | ✓ |
Product experience | (0.19, 0.39, 0.45, 0.64; 1, 1) (0.29, 0.39, 0.45, 0.52; 0.9, 0.9) | (0.33, 0.43, 0.45, 0.51; 1, 1) (0.38, 0.43, 0.45, 0.47; 0.9, 0.9) | 0.416 | 0.430 | |
Number of patents held | (0.34, 0.54, 0.61, 0.78; 1, 1) (0.45, 0.54, 0.61, 0.67; 0.9, 0.9) | (0.39, 0.5, 0.53, 0.6; 1, 1) (0.45, 0.5, 0.53, 0.56; 0.9, 0.9) | 0.566 | 0.507 | ✓ |
Expected capabilities of abstraction | (0.09, 0.26, 0.31, 0.48; 1, 1) (0.17, 0.26, 0.31, 0.37; 0.9, 0.9) | (0.26, 0.32, 0.34, 0.39; 1, 1) (0.29, 0.32, 0.34, 0.36; 0.9, 0.9) | 0.280 | 0.327 | |
Technology transfer capability | (0.45, 0.65, 0.7, 0.82; 1, 1) (0.56, 0.65, 0.7, 0.75; 0.9, 0.9) | (0.47, 0.59, 0.63, 0.71; 1, 1) (0.54, 0.59, 0.63, 0.66; 0.9, 0.9) | 0.655 | 0.601 | ✓ |
Overlapping knowledge base | (0.11, 0.27, 0.33, 0.49; 1, 1) (0.19, 0.27, 0.33, 0.38; 0.9, 0.9) | (0.26, 0.33, 0.35, 0.39; 1, 1) (0.3, 0.33, 0.35, 0.37; 0.9, 0.9) | 0.295 | 0.333 | |
Product-specific knowledge | (0.4, 0.6, 0.67, 0.82; 1, 1) (0.51, 0.6, 0.67, 0.73; 0.9, 0.9) | (0.44, 0.56, 0.59, 0.67; 1, 1) (0.5, 0.56, 0.59, 0.62; 0.9, 0.9) | 0.623 | 0.565 | ✓ |
Market knowledge complementarity | (0.27, 0.47, 0.53, 0.71; 1, 1) (0.38, 0.47, 0.53, 0.6; 0.9, 0.9) | (0.38, 0.49, 0.51, 0.58; 1, 1) (0.44, 0.49, 0.51, 0.54; 0.9, 0.9) | 0.495 | 0.491 | ✓ |
Expected knowledge maturity | (0.22, 0.41, 0.47, 0.66; 1, 1) (0.31, 0.41, 0.47, 0.53; 0.9, 0.9) | (0.34, 0.43, 0.45, 0.52; 1, 1) (0.39, 0.43, 0.45, 0.48; 0.9, 0.9) | 0.436 | 0.442 | |
Past experiences | (0.2, 0.39, 0.44, 0.62; 1, 1) (0.29, 0.39, 0.44, 0.5; 0.9, 0.9) | (0.35, 0.45, 0.47, 0.53; 1, 1) (0.4, 0.45, 0.47, 0.50; 0.9, 0.9) | 0.408 | 0.451 | |
Financial assets | (0.06, 0.19, 0.24, 0.40; 1, 1) (0.12, 0.19, 0.24, 0.29; 0.9, 0.9) | (0.2, 0.26, 0.27, 0.31; 1, 1) (0.23, 0.26, 0.27, 0.29; 0.9, 0.9) | 0.216 | 0.260 | |
Expected debt ratio and refund ability | (0.32, 0.52, 0.57, 0.72; 1, 1) (0.42, 0.52, 0.57, 0.62; 0.9, 0.9) | (0.39, 0.5, 0.53, 0.60; 1, 1) (0.45, 0.5, 0.53, 0.55; 0.9, 0.9) | 0.528 | 0.503 | ✓ |
Financial resources demand of the project | (0.09, 0.26, 0.31, 0.48; 1, 1) (0.17, 0.26, 0.31, 0.37; 0.9, 0.9) | (0.23, 0.29, 0.31, 0.35; 1, 1) (0.27, 0.29, 0.31, 0.33; 0.9, 0.9) | 0.280 | 0.297 | |
Return of investment | (0.16, 0.34, 0.4, 0.58; 1, 1) (0.25, 0.34, 0.4, 0.46; 0.9, 0.9) | (0.28, 0.36, 0.38, 0.44; 1, 1) (0.33, 0.36, 0.38, 0.41; 0.9, 0.9) | 0.365 | 0.368 | |
Collaborative behavior | (0.43,0.62,0.67,0.80;1,1) (0.53,0.62,0.67,0.72;0.9,0.9) | (0.46,0.58,0.61,0.7;1,1) (0.53,0.58,0.61,0.65;0.9,0.9) | 0.629 | 0.588 | ✓ |
Mutual trust and commitment | (0.21, 0.41, 0.47, 0.65; 1, 1) (0.31, 0.41, 0.47, 0.53; 0.9, 0.9) | (0.35, 0.44, 0.46, 0.52; 1, 1) (0.4, 0.44, 0.46, 0.49; 0.9, 0.9) | 0.429 | 0.444 | |
Management and organizational culture | (0.24, 0.44, 0.5, 0.69; 1, 1) (0.33, 0.44, 0.5, 0.56; 0.9, 0.9) | (0.35, 0.45, 0.47, 0.54; 1, 1) (0.41, 0.45, 0.47, 0.5; 0.9, 0.9) | 0.461 | 0.454 | ✓ |
Previous relationship | (0.03, 0.14, 0.18, 0.33; 1, 1) (0.08, 0.14, 0.18, 0.23; 0.9, 0.9) | (0.18, 0.23, 0.24, 0.27; 1, 1) (0.21, 0.23, 0.24, 0.25; 0.9, 0.9) | 0.163 | 0.230 | |
Propensity to change | (0.04, 0.15, 0.19, 0.34; 1, 1) (0.09, 0.15, 0.19, 0.24; 0.9, 0.9) | (0.2, 0.24, 0.26, 0.29; 1, 1) (0.22, 0.24, 0.26, 0.27; 0.9, 0.9) | 0.175 | 0.246 | |
Geographical proximity | (0.02, 0.11, 0.15, 0.29; 1, 1) (0.06, 0.11, 0.15, 0.20; 0.9, 0.9) | (0.17, 0.21, 0.23, 0.26; 1, 1) (0.19, 0.21, 0.23, 0.24; 0.9, 0.9) | 0.136 | 0.218 | |
Symmetry of scale and scope | (0, 0.06, 0.09, 0.21; 1, 1) (0.03, 0.06, 0.09, 0.13; 0.9, 0.9) | (0.16, 0.2, 0.21, 0.23; 1, 1) (0.18, 0.2, 0.21, 0.22; 0.9, 0.9) | 0.084 | 0.197 | |
Compatibility of corporation strategies | (0.18, 0.37, 0.43, 0.61; 1, 1) (0.27, 0.37, 0.43, 0.49; 0.9, 0.9) | (0.31, 0.4, 0.42, 0.48; 1, 1) (0.36, 0.4, 0.42, 0.44; 0.9, 0.9) | 0.390 | 0.401 | |
Convergence of expectations between partners | (0.36, 0.56, 0.63, 0.80; 1, 1) (0.47, 0.56, 0.63, 0.70; 0.9, 0.9) | (0.41, 0.53, 0.56, 0.64; 1, 1) (0.48, 0.53, 0.56, 0.59; 0.9, 0.9) | 0.585 | 0.539 | |
Motivation and goal correspondence | (0.22, 0.41, 0.47, 0.66; 1, 1) (0.31, 0.41, 0.47, 0.53; 0.9, 0.9) | (0.33, 0.42, 0.44, 0.5; 1, 1) (0.38, 0.42, 0.44, 0.47; 0.9, 0.9) | 0.436 | 0.446 | |
Strategic objectives of intellectual property management | (0.16,0.35,0.41,0.60;1,1) (0.25,0.35,0.41,0.47;0.9,0.9) | (0.31, 0.39, 0.41, 0.47; 1, 1) (0.35, 0.39, 0.41, 0.44; 0.9, 0.9) | 0.375 | 0.395 | |
Market complementarity | (0.09, 0.25, 0.29, 0.46; 1, 1) (0.16, 0.25, 0.29, 0.35; 0.9, 0.9) | (0.23, 0.29, 0.31, 0.35; 1, 1) (0.26, 0.29, 0.31, 0.33; 0.9, 0.9) | 0.265 | 0.295 |
Criteria | IT2F Criteria Geometric Means | IT2F Criteria Weight | Non-Fuzzy Normalized Weights | |
---|---|---|---|---|
C1 | Technological innovation level | (1.081, 2.353, 6.755, 11.827; 1, 1) (1.308, 2.657, 6.114, 10.414; 0.8, 0.8) | (0.238, 0.245, 0.253, 0.256; 1, 1) (0.24, 0.246, 0.253, 0.256; 0.8, 0.8) | 0.249 |
C2 | Technological complementarity | (0.471, 0.856, 2.12, 3.755; 1, 1) (0.544, 0.943, 1.929, 3.275; 1, 1) | (0.104, 0.089, 0.079, 0.081; 1, 1) (0.1, 0.087, 0.08, 0.08; 1, 1) | 0.088 |
C3 | Number of patents held | (0.258, 0.475, 1.399, 2.974; 1, 1) (0.297, 0.529, 1.241, 2.476; 0.8, 0.8) | (0.057, 0.049, 0.052, 0.064; 1, 1) (0.054, 0.049, 0.051, 0.061; 0.8, 0.8) | 0.056 |
C4 | Technology transfer capability | (0.995, 2.439, 6.777, 10.263; 1, 1) (1.251, 2.778, 6.226, 9.43; 0.8, 0.8) | (0.219, 0.254, 0.254, 0.222; 1, 1) (0.23, 0.257, 0.257, 0.231; 0.8, 0.8) | 0.237 |
C5 | Product-specific knowledge base | (0.409, 0.85, 2.476, 4.616; 1, 1) (0.488, 0.956, 2.225, 3.995; 0.8, 0.8) | (0.09, 0.088, 0.093, 0.1; 1, 1) (0.089, 0.088, 0.092, 0.098; 0.8, 0.8) | 0.093 |
C6 | Market knowledge complementarity | (0.135, 0.224, 0.626, 1.397; 1, 1) (0.151, 0.246, 0.554, 1.144; 0.8, 0.8) | (0.03, 0.023, 0.023, 0.03; 1, 1) (0.028, 0.023, 0.023, 0.028; 0.8, 0.8) | 0.027 |
C7 | Expected debt ratio and refund ability | (0.098, 0.161, 0.459, 1.061; 1, 1) (0.109, 0.177, 0.404, 0.860; 0.8, 0.8) | (0.022, 0.017, 0.017, 0.023; 1, 1) (0.02, 0.016, 0.017, 0.021; 0.8, 0.8) | 0.020 |
C8 | Collaborative behavior | (0.69, 1.404, 3.591, 5.825; 1, 1) (0.822, 1.567, 3.286, 5.227; 0.8, 0.8) | (0.152, 0.146, 0.135, 0.126; 1, 1) (0.151, 0.145, 0.136, 0.128; 0.8, 0.8) | 0.140 |
C9 | Management and organizational culture | (0.048, 0.074, 0.222, 0.571; 1, 1) (0.052, 0.081, 0.193, 0.450; 0.8, 0.8) | (0.011, 0.008, 0.008, 0.012; 1, 1) (0.01, 0.007, 0.008, 0.011; 0.8, 0.8) | 0.010 |
C10 | Convergence of expectations between partners | (0.353, 0.774, 2.254, 3.94; 1, 1) (0.428, 0.874, 2.036, 3.48; 0.8, 0.8) | (0.078, 0.08, 0.084, 0.085; 1, 1) (0.078, 0.081, 0.084, 0.085; 0.8, 0.8) | 0.082 |
Criteria | Alternative | Preference Function Type | Parameters | ||||||
---|---|---|---|---|---|---|---|---|---|
Z1 | Z2 | Z3 | Z4 | Z5 | p | q | |||
C1 | Technological innovation level | L | MH | M | ML | MH | U-shape | - | - |
C2 | Technological complementarity | M | VH | H | H | MH | V-shape | L | |
C3 | Number of patents held | L | MH | ML | VL | M | V-shape | L | |
C4 | Technology transfer capability | L | H | MH | M | ML | U-shape | L | |
C5 | Product-specific knowledge base | H | MH | M | M | ML | V-shape | L | |
C6 | Market knowledge complementarity | VH | VL | ML | H | L | V-shape with indifference | L | M |
C7 | Expected debt ratio and refund ability | MH | ML | VL | ML | L | V-shape | ML | |
C8 | Collaborative behavior | VH | L | M | H | MH | V-shape | ML | |
C9 | Management and organizational culture | H | ML | MH | H | M | V-shape with indifference | VL | ML |
C10 | Convergence of expectations between partners | ML | M | MH | MH | VL | V-shape with indifference | VL | ML |
Non-Fuzzy Net Flow | Rank | ||||
---|---|---|---|---|---|
Z1 | (0.141, 0.128, 0.119, 0.096; 1, 1) (0.140, 0.127, 0.118, 0.114; 0.8, 0.8) | (0.294, 0.302, 0.303, 0.260; 1, 1) (0.297, 0.303, 0.304, 0.270; 0.8, 0.8) | (−0.153, −0.174, −0.184, −0.164; 1, 1) (−0.158, −0.176, −0.185, −0.156; 0.8, 0.8) | −0.1700 | 5 |
Z2 | (0.3, 0.304, 0.271, 0.215; 1, 1) (0.302, 0.305, 0.271, 0.262; 0.8, 0.8) | (0.11, 0.099, 0.094, 0.077; 1, 1) (0.108, 0.099, 0.094, 0.088; 0.8, 0.8) | (0.191, 0.205, 0.177, 0.138; 1, 1) (0.194, 0.206, 0.177, 0.174; 0.8, 0.8) | 0.1824 | 1 |
Z3 | (0.219, 0.224, 0.223, 0.172; 1, 1) (0.221, 0.224, 0.224, 0.189; 0.8, 0.8) | (0.188, 0.179, 0.146, 0.122; 1, 1) (0.188, 0.178, 0.145, 0.143) | (0.031, 0.045, 0.078, 0.05; 1, 1) (0.034, 0.046, 0.079, 0.046; 0.8, 0.8) | 0.0523 | 2 |
Z4 | (0.203, 0.191, 0.187, 0.132; 1, 1) (0.203, 0.191, 0.187, 0.152; 0.8, 0.8) | (0.2, 0.2, 0.197, 0.133; 1, 1) (0.202, 0.2, 0.197, 0.165; 0.8, 0.8) | (0.003, −0.008, −0.01, −0.001; 1, 1) (0.001, −0.009, −0.009, −0.013; 0.8, 0.8) | −0.0050 | 3 |
Z5 | (0.169, 0.165, 0.164, 0.132; 1, 1) (0.170, 0.165, 0.164, 0.137; 0.8, 0.8) | (0.241, 0.232, 0.225, 0.156; 1, 1) (0.24, 0.232, 0.225, 0.188; 0.8, 0.8) | (−0.072, −0.067, −0.061, −0.024; 1, 1) (−0.07, −0.067, −0.062, −0.051; 0.8, 0.8) | −0.0581 | 4 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Puzović, S.; Vesić Vasović, J.; Milanović, D.D.; Paunović, V. A Hybrid Fuzzy MCDM Approach to Open Innovation Partner Evaluation. Mathematics 2023, 11, 3168. https://doi.org/10.3390/math11143168
Puzović S, Vesić Vasović J, Milanović DD, Paunović V. A Hybrid Fuzzy MCDM Approach to Open Innovation Partner Evaluation. Mathematics. 2023; 11(14):3168. https://doi.org/10.3390/math11143168
Chicago/Turabian StylePuzović, Sanja, Jasmina Vesić Vasović, Dragan D. Milanović, and Vladan Paunović. 2023. "A Hybrid Fuzzy MCDM Approach to Open Innovation Partner Evaluation" Mathematics 11, no. 14: 3168. https://doi.org/10.3390/math11143168
APA StylePuzović, S., Vesić Vasović, J., Milanović, D. D., & Paunović, V. (2023). A Hybrid Fuzzy MCDM Approach to Open Innovation Partner Evaluation. Mathematics, 11(14), 3168. https://doi.org/10.3390/math11143168