Life-Cycle Oriented Risk Assessment Using a Monte Carlo Simulation

Featured Application

Product lifetime-oriented risk minimization in the procurement of standard part for a manufacturing and assembly line for roller conveyors.

Abstract

State of the art mechatronic systems are complex assemblies of various parts and sub-systems. In such an interconnected system, even relatively cheap parts can have a major impact on the overall performance due to unexpected failure. Hence, lifecycle management has major implications on the successful modification of existing products. Potential savings due to changes in production and procurement must be compared to the implied risk of products failing in the field due to these changes. This work documents a generic approach for risk assessment based on the distribution of the expected savings and incident costs over the whole lifecycle. To do so, a stochastic model is introduced to quantify the expected savings and costs given a non-risk-free product modification. Using a Monte Carlo simulation, the effects of uncertainty are incorporated into the risk management. The model and simulation are deployed within an industrial use case. The application demonstrates both the appropriateness of the tool and its useability.

1. Introduction

This study’s motivation is based on understanding risk and decision making in manufacturing firms based on situations where there is minimal data available to base decisions upon and where decision making is separated from financial responsibility. Operation environments are best performed with a stable bill-of-materials (BOM) and with standardized assembly instructions and a mature change management process. Procurement’s role is to reduce the input cost by purchasing the parts for assembly at the lowest cost. The role of development is to develop and, with production, industrialize the production process. Risks are created for the business whenever there are changes to the BOM, and these risks can be difficult to quantify without having data beforehand that firms do not always have available [1]. These risks should be managed through a change management process, as they can materialize as a range of quality issues, such as additional assembly work to latent defects hidden in the product that emerge during use and require a product recall.

Risks based on changes to reduce costs may be assessed if data are available, processes described, and the lifecycle is understood. Monte Carlo simulations may be used in place of assumptions that manufacturing firms often use to assess risks [2], albeit with Bayes inference embedded within [3]. A mathematical approach also supports the assessment on a systematic basis rather than an individual event basis. Today, data are available from ERP, PLM and CRM systems, process mapping allows the process to be described, and data can be used in place of assumptions [1,4,5]. In combination with simulation, this allows a decision support tool to be created to support the firm’s decision making, allowing the different departments to understand and quantify their decisions and their potential consequences better. This supports improved risk management by automatically quantifying the risks, capturing the contextual issues, and allowing a more rational decision to be made with a risk mitigation process. It can also support the root cause analysis and allow allocation of risk [6,7].

For complex and nonlinear systems, the use of Monte Carlo simulations in the context of risk management is recommended in literature [8,9,10,11,12,13]. The concept can further be used to estimate the expected lifecycle costs distribution [14]. The purpose of this paper is to describe a case where the existing concept of Monte Carlo simulations for lifecycle cost estimations is used to support risk management associated with the change management process over the lifecycle of a product, including the procurement, manufacturing, assembly, and use phases. The focus is on mass production where a production run can result in more than tens of thousands of products and where variations in the supply chain, manufacturing and assembly may occur. It excludes the initial design and industrialization/qualification phase of the product lifecycle. The simulation aimed to improve the quality of the product that the manufacturing firm was assembling. The research question for this paper is: “how can Monte Carlo simulation approaches be used to successfully translate technical risks into a management risk and so, improve decision-making processes?”

2. State of the Art

The authors have drawn upon relevant papers and articles to develop a detailed understanding of both the digital and quality aspects of risk assessment. They exclude basic documents for quality management, digital lean manufacturing, product lifecycle management, business process management, an application of Monte Carlo simulations in risk management.

2.1. Quality Management

Quality is a concept that compares the characteristics of a product, process, or system against the initial requirements set out for it. This broad definition widens the range of objects to which quality can be assigned [15]. Quality is defined by ISO 9000 as: “[…] the degree to which a set of inherent characteristics fulfils requirements [16].” In production and industrial engineering, this concept turned into an applicable quantity of defects that should be recognizable, measurable, and controllable. The focus of a zero-defects strategy is therefore on prevention and avoidance of failures. As Feigenbaum [17] emphasized in his ‘total quality control’, the responsibility for quality is with everyone involved in the creation and production processes, from top-level management to shop floor. This understanding plays a major role in a corporate environment and is highly affected by the company’s culture.

Garvin [18] assessed the existing definitions and identified five basic approaches: the transcendental, the product-based, the customer-based, the production-based and the value-based approach. Based on the findings of the analysis he derived eight dimensions of quality, which are applicable for every product: reliability, features, aesthetics, service, performance, conformance, durability, and perception. Often the sustainability is added as an additional term. Schmitt and Falk [15] define quality assurance as “[…] all methods preparing and controlling the compliance of processes and products with predetermined quality standards”. Furthermore, “quality assurance” represents the monitoring of process outputs to avoid the procrastination of failures and to identify and reject defective parts. Historically, quality assurance can also be seen first as an antecedent and nowadays as the operational part of quality management” [15]. This can be achieved with multiple methods of measurement and statistics in production. Nonetheless, comprehensive quality assurance can only be achieved by ensuring and gaining internal and external customer requirements and to assess products and processes against these requirements. Therefore, the ongoing change management of these requirements is paramount.

Filz, et al. [19], provide a comprehensive overview of virtual quality gates in manufacturing systems. This method supports the reduction of the total cost of quality while ensuring high transparency about process-product relationships in production. To decrease the total cost of maintaining quality and reduce rework or scrap, it is necessary to consider the complete process chain instead of single and isolated processes. This can only be achieved by understanding the dynamics and interdependencies within the process chain. Furthermore, the “context of product quality, transparency and understanding of process–product relationships, which describe interactions and the impact of the manufacturing processes on the resulting intermediate product features and vice versa, are of great importance” [19]. The concept of quality gates is therefore to systematically divide the manufacturing chain into different quality-relevant decision points.

2.2. Digital Lean Manufacturing

The topic of digital lean manufacturing covers basic definitions of lean manufacturing and enhances its capabilities by applying digital technologies to increase transparency. It is often described as smart manufacturing or Industry 4.0. The cyber-physical-system (CPS) is a core element of digital lean manufacturing. CPS in manufacturing [20] assumes the role of controlling a whole interaction of item planning, acquirement, creation, and coordination, as well as overseeing exhaustive progression of data. CPS is a complex framework that commands adaptable combinations of different advances in technology. It is important to create techniques for interoperability and reconciliation for the advancement of a CPS. The research for the development of the actionable maturity model supports the establishment of digital lean manufacturing by assessing the current challenges. These fundamental challenges must be addressed first to further develop the manufacturing system towards a digital future by increasing the interconnection of assets and the advancement of a digital thread through the entire process chain.

As Liebrecht, et al. [21] elaborate in their paper, the objective of such an implementation process is the case-specific analysis and evaluation of available methods to select the most suitable ones for the individual firm. Therefore, Liebrecht, et al. [21] developed a toolbox with a classification of different methods. Based on these, a classification of methods was made to highlight the differences of the production typologies in this project.

2.3. Product-Lifecycle Management

The purpose of product-lifecycle management (PLM) is to “structure, design, plan, and develop company and business activities based on the guiding principle of sustainable development and a lifecycle perspective” [22]. PLM therefore covers the reduction of costs and optimization of revenues and processes by taking risks into account at all stages of a product lifecycle, from R&D to refurbishment or recycling. Saur [23] regards “lifecycle management as a frame which bases on existing structures, systems, tools, and information. Existing concepts, tools, and programs are not supposed to be replaced, but their application is expected to be improved by a lifecycle management framework”. Such frameworks can provide guidance for the development of a holistic and integrated PLM. A possible starting point can be the internal assessment of processes, and from there must then also include other stakeholders involved over the entire value chain. This is important to avoid data silos or workaround fixes which might be created if the PLM is not sufficiently controlled and creates friction in the value chain. A more generic approach is recommended by Wuest, et al. [24] to assess the three main phases of a product or system: beginning of life (BOL), middle of life (MOL) and end of life (EOL). The forward-directed and especially backward-directed information flows must be established, as Figure 1 shows.

This framework is supported by 15 information quality dimensions to establish the needed information flows. By applying the model, an overview of these flows can be developed and discussed with the stakeholders involved. With an initial focus on internal departments of the firm, extending to encompass external stakeholders such as suppliers, distributors, and customers.

2.4. Business Process Management

In production firms, the term management is used synonymously for business management or corporate management [25]. Boos and Schuh [25] recommend the application of a regulatory framework for production and management. They highlight that “the regulatory framework contains the internal aspects business structure, business development and business processes as well as the stakeholders as an interaction partner with the outside world and the environment as an external factor that affects the enterprise”.

Business process management (BPM) is an integral part of management and can be further categorized as BPM Systems, process modelling, process design, coordination and interoperability, model management, process mining and new technologies. Such frameworks, as introduced in the previous section on PLM, can provide an overview of the complex business structures of today’s firms. This is also applicable to the situation of industrial partners. A firm serving its products to an internal consumer and not having contact with the final customer hinders the flow of information. The identification of the relevant stakeholders and suppliers is therefore essential. This must then be coupled with the business process aspects to enable a holistic picture of the current supply chain structures to be created. Therefore, the development of a reference model is proposed, including the different stakeholders (internal and external) according to the three different timeframes (BOL, MOL, EOL).

Further distinctions can then be made by classifying processes into the main categories of technology management, innovation management, factory planning, production management, logistics management, quality management, purchasing management, service management, and technical sales [25].

Even if the topic of BPM is abstract, it can be developed organically by visualizing processes and connecting them to lager systems. This can be performed collaboratively by integrating the relevant departments and developing a common ground of understanding. This project highlighted the challenges of BPM with its multiple involved internal and external stakeholders and quality measures, which cannot always be clearly quantified and described.

2.5. Monte Carlo Simulation in Risk Management

A systems output probability density function (PDF) depends on the PDF of the systems parameters and/or variables. Especially if the system shows nonlinear behavior, the prediction of the output PDF is not trivial Using Monte Carlo simulation, different outcomes due to different parameter and/or input values can be systematically studied using a model of the underlying processes. A Monte Carlo simulation can either explore all possible combinations of values or a sufficient large sub-set. Using statistical analysis, conclusions about the expected outcome and a probability density distribution of the outcome can be drawn. In the context of risk assessment and risk management, Monte Carlo simulations have various application, such as risk assessment in finances and portfolio management [11,13]. Other management applications deal with the risk of failing project deadlines [10]. Especially in building projects with uncontrollable influences—such as weather—Monte Carlo simulations are used for risk assessments [9,12]. In production environments related risk assessments, Monte Carlo simulations are used for example to evaluate the robustness of supply chains [8]. Wang et al. use a Monte Carlo simulation in [14] to predict the PDF of a buildings lifecycle cost.

3. Materials and Methods

This work builds upon the existing theory and use cases of Monte Carlo simulation in the context of risk management. Scope of this woks is to extend the available use-cases by developing product development and production specific parametrizable lifecycle-oriented risk models suited for the integration into BPM. The goal of this tool is to cope with the uncertainty regarding the effects of changes to the product on its quality and performance. Using a practical use-case from an industrial application, it will demonstrate how the gap between technical and managerial risk assessment can be closed using the computational power of common workstations. This tool shall provide a prediction of the estimated MOL and EOL performance quality wise, using properties determined during the BOL.

The procedure for implementing this lifecycle-oriented risk assessment tool is shown in Figure 2. Firstly, the considered industrial use-case is described and analyzed using the procedures by Voss et al. [26]. This work specifically focuses on the departments involved in the decision making, as well as on the information management and information flows regarding quality management. Using the case topology obtained, a generic system breakdown structure is synthesized. This generic structure enables the development of a generalized approach, by identifying the relevant internal dynamics and couplings of the quality-related dynamics. Based on this generic structure a risk model is synthesized and implemented into an assessment tool. The aim of this tool is to predict the expected performance and its certainty, using the probability density functions of the influencing parameters, i.e., predicting the effects of uncertainties in the BOL on the performance in the MOL and EOL as suggested in [14]. Hence, this tool includes a Monte Carlo simulation to cope with the statistical effects and uncertainties of the available information and data [27]. Thereby, the well-known approach of Monte Carlo simulations in risk management [8,9,10,11,12,13] is extended by the development of new case specific models. These models describe the lifetime-oriented risk for the design, management and production of passive goods with a high life-time expectations.

The tool will be applied on the use-case as a proof of concept. This proof of concept includes the technical and managerial feasibility. The technical proof of concept deals with the question of whether the required parameters and configurations inputs can be provided during a general development or change process of a product in line with the procedures suggested in [19]. The managerial feasibility deals with the process integration of the tool, as well as with the useability of the results it provided. The question to be answered is ‘are the results suitable to optimize the managerial decisions regarding quality management?’.

4. Results

The results start with the case description and then the development of a generic breakdown structure. In the second part the implementation and application of Monte Carlo simulation is introduced.

4.1. Case Description

The firm manufactures engineered-to-order products based on customer requirements. The parts supplied are sold internally within the firm’s group to another business unit where the products are integrated into the systems that are sold to the ‘end-customer’. The firm first developed the assembly and then tested it on a test bench to confirm performance. The development of the assembly led to the bill of materials (BOM) in agreement with the production. The assembly itself is a mix of bought-in materials and self-manufactured parts put together in the assembly workshop before shipping. The sales department initiated the development process and the volume ordering process. In a week, around 20,000 units are manufactured, assembled, and shipped. The departments involved in the development and production processes were sales, development, quality assurance (QA), production, and logistics. Each department is empowered to make its own operational decisions and refer to other departments when necessary.

Investigation of the firm identified two key processes: The first being the new product development. During this process the customer’s use-case and requirements are converted to a BOM following a design and validation sub-process. The second being the production process that used the BOM and issued it to procurement, manufacturing and assembly based on the volume ordered from sales. An overview of the process is described in Figure 3 superimposed on the processes are the two first phases of the lifecycle (BOL and MOL). The warranty period has also been added to the figure, as during the warranty phase the firm has responsibility for the product performance, the warranty period extends only partially into the use phase of the product.

The purpose of the new product development process is to hand over to production a clear BOM that details the specification based on the use case and requirements. Hence, the process identifies, documents, and communicates the product properties critical to quality. This includes the materials, the manufacturing and assembly process steps. These definitions are “frozen” once approved and implemented within the PLM system. Variations to the BOM handed over to production must go through an approval process since a limited number of vendors are approved for supply, that manufacturing is validated on a subset of machines, and that only a limited number of the assembly technicians are qualified for the assembly stage.

The development of a new system follows an engineered-to-order process, where sales have the lead, and the new product development process is used to create a product that fulfills the sales department’s request. The development process creates the BOM as its main output. Sales then take the orders for production volumes of the products. The production of components was batchwise with long (28 days or more) production runs. Boxes would be packed and shipped weekly allowing the parts to be installed, commissioned, and then used by the end-user. The warranty period provided was typically two to five years from delivery.

The effects of changing the supplier of specific bought in materials was considered as part of a cost reduction process. Lower cost materials were procured with the same nominal form and function, however, these failed in use during the warranty phase of the MOL. In this case, it was not considered necessary to re-test the new assembly as procurement considered that there was no impact on the performance. The consequential costs, due to the warranty claim and the subsequent recall and replacement, was many times greater than the cost savings. Based on the experience, the firm’s aim was to have a better risk management approach. This approach must enable them to understand their potential cost of risk exposure and allow them to decide whether to

• Switch to the new materials and accept the risks;
• Test and switch in parallel and accept the additional costs for testing and the risk implied by the ‘blind flight’ until the tests are completed, or;
• Test and then switch, reducing the cost of risk but reducing the cost savings for the firm.

It is worth noting that the variation to the BOM was based on cost-saving, although discussions with product development confirmed that similar changes could be caused though obsolescence, a switch of supplier or a change in one of the manufacturing machines.

The approach taken with the decision-making tool should also be sufficiently generic that it could be used by quality management to identify and help to mitigate risks over their full value chain. The tool should also support joint decision making between multiple departments.

4.2. Generic System Breakdown

To understand how potential warranty issues can occur—i.e., differences between the customer’s expectations and the performance provided—a generic breakdown of the relevant system dynamics is created. Figure 4 shows the suggested system breakdown structure for the subsequent risk assessment. The structure is generic for a multi-system product, i.e., a system consisting of multiple subsystems and parts. A generic product has specifications that originate from the desired outcome and performance of the system. Ideally those are formulated by or together with the system’s user. Based on this general goal, the product’s specification is obtained by formulating use-cases and requirements.

The products need manufacturing and procurement processes to be created during the BOL. These processes will necessarily affect the products’ performance. As for the product, its sub-assemblies have specifications, show performances, need manufacturing and procurement processes, and might include (sub-)sub-assemblies or parts, the details of which are covered in the overall specification. Importantly, the overall performance is affected by the performances of the sub-assemblies.

During its MOL, the product will show a specific performance. The difference between this performance and the specification is denoted as $\Delta \mathrm{y}$. The goal of the risk management introduced later is to minimize the negative entries of $\Delta \mathrm{y}$. In an ideal world, $\Delta \mathrm{y}$ can be quantified precisely. However, in practice, two issues must be addressed:

• Uncertainties in the product specification: These can be caused by a mismatch between the true customer expectations and the assumed expectations. Either the customer has not been sufficiently involved in the specification synthesis or the customer’s desired performance is not correctly achieved. An alternative cause can be differing expectations between different users: what might be good for one customer might be unsuitable for another customer.
• Uncertainties in the product’s performance: The overall performance of a product depends on various factors. A firm must distinguish between a general uncertainty over all manufactured products (general performance offset) and the product’s specific offset.

Given a customer specification $s_j$ and a product with general performance $p_i$ and specific performance offset $\delta p_i$, the resulting specification offset is given as follows:

$${\Delta \mathrm{y}}_{j,i}=s_j-\left(p_i+\delta p_i\right)$$

(1)

As stated above, $s_j$ and $\left(p_i+\delta p_i\right)$ are subject to uncertainties. These uncertainties are here known as the probability density functions (PDF) $s_j\sim f_s$ and $p_i+\delta p_i\sim f_p$. Thus, ${\Delta \mathrm{y}}_{j,i}$ can be estimated using its PDF:

$$f_{\Delta }=f_s\ast f_p$$

(2)

$f_{\Delta }$ represents the probability of having a difference $\Delta \mathrm{y}$ between specified and realized performance. Given the costs of such a difference as $c_{\Delta }$ for a specific situation, the expected risks in terms of costs can be evaluated, i.e., by providing accurate estimations of $c_{\Delta }$ and $f_{\Delta }$ the expected costs and standard deviation of the costs are

$$\begin{array}{ccc}{E}_c& =& {{\displaystyle \int }}_{-\infty }^{\infty }{c}_{\Delta }\left(\mathrm{u}\right)·f_{\Delta }\left(\mathrm{u}\right) \mathrm{du} \mathrm{and}\\ {\sigma }_c& =& \sqrt{{{\displaystyle \int }}_{-\infty }^{\infty }{\left(c_{\Delta }\left(\mathrm{u}\right)-E_c\right)}^2 \mathrm{du}}.\end{array}$$

(3)

4.3. Risk Model and Assessment Tool

Using a combination of a generic cost model and a Monte Carlo simulation, $c_{\Delta }$ and $f_{\Delta }$ are estimated within a risk model for the different use cases. The generic cost model considers a product in line with the structure of Figure 4, where a single part is changed. The aim of the generic cost model is to predict the expected savings from changing this part. The difference in costs consists of three elements: the costs for testing, incident handling, and savings due to modified procurement costs. The change in costs is dependent on the number of parts sold per time unit $N$:

$$c_i=\underset{testing}{\underbrace{\Delta t_t·N·f·\delta c_t}}+\underset{recalls}{\underbrace{\Delta t_r·N·\delta c_i}}-\underset{savings}{\underbrace{\Delta t_s·N·\delta c_s}}$$

(4)

The total savings depend on the savings per product sold $\delta c_s$ and the time span $\Delta t_s$ during which the saving is active. The testing costs depend on the costs per test $\delta c_t$, the duration $\Delta t_t$ during which the tests are performed and the fraction $f$ of products to be tested. Recall costs arise if the product fails during the required lifespan and a recall is required. $\Delta t_i$ is the timespan between the first modified product sold and the first incident detected and $\delta r$ are the costs per recalled product.

The timespans $\Delta t_t$, $\Delta t_r$ and $\Delta t_s$ depend on the selected switching strategies.

Table 1. Dependencies of the timespans and switching scenarios.

	Immediate Switch	Test and Switch	Test, then Switch
$\Delta t_t$	$=0$	$=t_0/g$	$=t_0/g$
$\Delta t_r$	$=\{\begin{array}{ll}0& \mathrm{if} t_i\ge t_0\\ t_i& \mathrm{else}\end{array}$	$=\{\begin{array}{ll}0& \mathrm{if} t_i\ge t_0\\ t_i/g& \mathrm{else}\end{array}$	$=0$
$\Delta t_s$	$=\min \left(t_i, t_0\right)$	$=\{\begin{array}{ll}{t}_0\hfill & t_i\ge t_0\hfill \\ t_i/g\hfill & \mathrm{else}\hfill \end{array}$	$=\{\begin{array}{ll}{t}_0-\Delta t_t\hfill & t_i\ge t_0\hfill \\ 0\hfill & \mathrm{else}\hfill \end{array}$

lists the distinction of cases for the different switching strategies. Relevant for the time spans is the new lifetime $t_i$ after the switch is made. This new lifetime is compared to the required lifetime $t_0$. It is further assumed that the test can speed up the load cycles by a factor $g$.

The true lifetime $t_i$ after the switch is subject to uncertainties. Hence, the result of (4) is also subject to uncertainties. To evaluate (4) for the different switching strategies (3) a Monte Carlo simulation is used. A thousand randomized values for $t_i$ are drawn according to the expected lifetime effect, and certainty for the new part. To facilitate the application of the model and integrate it into the decision-making process and risk assessment, a simple tool is implemented, Figure 5. The tool requires the input of product specific parameters such as expected lifetime, cost structure and others. Based on these inputs, the cockpit enables the risk assessment to consider different degrees of certainty regarding the lifetime of the product after switching parts. The results are shown as a histogram of the expected saving. Additionally, the expected cost-development over time is characterized.

4.4. Application on the Use-Case

The Monte Carlo simulation-based risk assessment tool is used within the presented use-case. The general results from the risk assessment are shown in

Table 2. Results of the risk-assessment tool application on the use-case.

k€	Immediate Switch	Test and Switch	Test, then Switch
MEAN	−4232	179	299
MEDIAN	450	442	442
STV	6859	377	209
Q1	−11,585	−308	−5
Q3	450	442	442
MIN	−17,512	−457	−8
MAX	450	442	442

. In the use-cases’ configuration, an immediate switch will generate savings in the order of EUR 450,000 during the investigated timespan in more than half of the simulated scenarios. This is more than the other two switching scenarios can possibly generate. However, the mean of all the simulated change in costs is about EUR −4.2 M; i.e., a substantial loss. The reasons for this unevenness in the distribution are a few costly incidents: if a switch is performed immediately and the new lifetime falls just a little short, the first incident will require a recall of a lot of products. This observation is in line with the experience made during the use case.

Looking at the expected savings over time (Figure 6), these effects can be studied in more detail. If an immediate switch is performed, the saving becomes effective immediately. During the first days after the change, the expected savings are positive and grow continuously. However, these savings are just “borrowed”. This can be a very dangerous situation in real applications: seeing the savings being effective, might be a premature sign of good performance of this switch. The opposite situation could occur in the case of the third switching strategy (Figure 6c), where even if the strategy shows the best average performance, the savings do not become effective immediately. Moreover, during the testing phase in the beginning of this change, only losses are seen.

By performing a sensitivity analysis of the expected savings on the testing time and the certainty of the specification–performance–match, additional improvements can be identified. The results of this analysis are shown in Figure 7. The sensitivity analysis shows that matching the specification and the performance—i.e., reducing $\Delta \mathrm{y}$ in (1)–provides the biggest savings. If the specifications match the requirements and the product fully fulfills the specification an immediate switch with the risk of costly recalls is enabled. In any other case, testing is recommended. For short testing times, the test should be performed before the switch. For relatively long testing times, one might consider performing the test in parallel to the switch. This decision is affected by the assessed risk of an immediate switch before the test results are in. Hence, the sensitivity analysis is an important tool to identify the approach, and thus the department, offering the biggest leverage towards increasing saving and reducing risk.

5. Discussion

The discussion opens with the implications for decision making and risk management processes within the case. This then moves to academic implications and closes with the managerial lessons from the case.

5.1. Case Implications

The presented use-case shows the importance of incorporating the fundamental system dynamics presented in Figure 4 into the decision-making process. Within the use-case, two basic strategies can be identified:

• Immediate switch: an immediate switch is only reasonable if the specifications are well defined and effects of the switched parts on the overall performance are well known—i.e., if ${{\displaystyle \int }}_{\infty }^0{f}_{\Delta }\left(\mathrm{u}\right)\mathrm{d}\mathrm{u}$ is sufficiently low. This strategy always offers the biggest savings if the specifications and the effects of the change are well known.
• Testing: should there be any significant uncertainty about the specification or the performance of the new product, testing is mandatory. In the presented case, testing before switching was always the preferred strategy compared to parallel testing and switching. Additional soft criteria, such as the demand for immediate effective savings, might favor the parallel approach. This is, however, only recommended if the testing time is short enough and the certainty level of the new performance is high enough.

Generally, the risk-assessment tool used here assists the assessment of the three switching strategies. Additionally, it enables the visualization of the expected development over time. This is a very important indicator to understand the dynamics of the different strategies. The tool further helps to identify additional improvement measures, by quantifying the sensitivity analysis of different parameters. Only by having resilient product specifications and good predictions of the product’s new performance, can the risk be minimized, and the potential saving maximized. This shows that the inter-department work is mandatory: the production department is generally limited to testing the new parts given the—possibly uncertain—specifications.

Relating to the research question, the Monte Carlo simulation was able to map the technical uncertainty to a managerial uncertainty measured by the expected savings. The tool was able to reproduce real scenarios and was proved useful by the involved industry partners. The approach was especially able to close the gap between technical and managerial viewpoints on the problem of how to change the supplier of a part.

The application of the implemented tool in the firms has further triggered an internal discussion during the parametrization: Estimations and quantification of certain parameters required an intensive analysis of the product requirements and performances over several departments. Hence, two case-specific contributions of the developed tool can be identified:

• Quantification of the lifetime-oriented risk given certain product properties and parameter uncertainties and thus assisting the decision-making progress on a managerial level. This advantage is in line with the finding of existing applications described in Section 2.5.
• Internal awareness about missing or misfitting product requirements, specification, and performance characteristics. This insight occurred during the parametrization of the model prior to the Monte Carlo simulation.

5.2. Academic Implications

The application of a Monte Carlo simulation to support decision making is not new; however, described in this paper is an approach that promotes the identification of risk for change management along the lifecycle. The base process here is based on understanding and integrating the new product development process and the production process. It confirms the necessity to have a mature PLM process to manage product and process definitions, as these underpin the risk assessment process described here. The integration of the lifecycle to include the MOL and the warranty phase of the ‘use’ phase is crucial. It means that the consequential risks and their associated costs are captured. The use here of the lifecycle with data and information flows [24] is necessary, as it provides a common basis yet allows for the separation of the business processes.

Risk management is often used as a semi-quantitative approach, where the likelihood and impact are multiplied to provide a risk-adjusted rating. However, all too often, a simple 1–10 score that is uncalibrated is used with scant regard to the commercial aspects. Using the lifecycle approach described in this paper with the process understanding coupled with both technical and commercial insights, it is proposed that the approach could be further adapted for EOT products and could be further calibrated with historical data.

Decision making processes for process changes are poorly described in an Industry 4.0 context. This gap should be further investigated, and Industry 4.0 provides the opportunity to move from silo to systems approaches. Much of the Industry 4.0 approaches focus on process control and creating a history of transactions that, in general, support root cause analysis. Here, the risk is clearly described and quantified and then used to support decision making (i.e., to set up a test in a cell or not) via a form of prediction. Using ‘advanced’ statistics (or other approaches) in decision making requires further investigation as the form of the outputs can be confusing to those who are not used to statistical analysis.

5.3. Managerial Implications

The application described here is a common EOT problem. There are three main themes for the managerial implications, centering around: (i) change management; (ii) risk management, and (iii) process integration.

Process change management can take place in production for many reasons, leading to changes to the BOM that are not discovered until a warranty claim is later made by a customer. PLM systems today offer the promise of improved control of the BOM yet lack the agility to adapt to new situations (i.e., changes in material supplies or manufacturing processes). QC or development is often left with the challenge of allowing a change without understanding if the changes are material or not. The advent of Industry 4.0 techniques (e.g., statistical modelling) and data and information flows along the lifecycle provides the opportunity to understand the risks of proposed changes.

In many firms, risk is managed in isolation, whereas risk is in effect systematic, with the cause being individual decisions made without understanding impacts to the whole system. The implication is that firms should move from a semi-quantitative approach to risk to one where risks are quantified along the lifecycle, using data and statistical simulation to overcome the limited data problem. In the longer-term further quantification and validation of simulations could be made based on the data and information flows.

The first two implications confirm that a need exists for process integration along the lifecycle. Industry 4.0 allows the integration of product development, production, and sales processes as well as the integration of subprocesses. In doing so risk management moves from a “gut feel” input to a quantitative approach based on simulations, and this should support further innovation within the manufacturing process. In particular it should be used to support ‘cost-out’ (or modified re-buy due to change in supplier or obsolescence) where minor modifications to the BOM are made, rather than create a risk-averse approach to change.

6. Conclusions and Recommendations

From the case described in this paper, it has been confirmed that using the developed risk model for the design, management, and production of passive goods with a high life-time expectations in combination with Monte Carlo simulation approaches can translate technical risks into a form that can be understood and interpreted by management and so support the decision-making process in the firm. Similar to the approaches in the finance industry [11,13], this model enables a fact-based BPM and quality management over the whole product lifecycle. The tool developed initially in Excel confirms that the approach could be applied in many firms today; a move to Matlab would provide a more stable environment, although both provide the opportunity for connections with PLM and ERP systems within a firm. The move to more quantitative risk management supports better decision making and linking with the lifecycle clearly describes the risks that are being taken and, therefore, allows the allocation of risks to departments who are ultimately responsible for a decision made. This is in line with the requirements stated by Feigenbaum [17] and assists the implementation of virtual quality gates as in [19]. Integration of such approaches with their high degree of transparency will require a change in management approach to support the adoption of such tools.

The limitation of the study described here is the single case. Therefore, it is recommended that additional studies are performed to confirm the broader applicability of the approach. It is also suggested that such studies compare the application of traditional “office” tools with more advanced techniques (e.g., cloud based or Matlab). Such studies should consider integrating or developing new decision-making processes within firms.