Validating the Whistleblowing Maturity Model Using the Delphi Method

Abstract

Empirical research identifies whistleblowing as one of the most effective internal antifraud controls. Very recently, Directive 1937/2019 became effective in the EU, aiming to deal with the defragmentation of whistleblowing legislation among the member states and provide common minimum accepted standards. The present article aims to provide a verified, weighted comparative maturity model. The suggested model has been constructed based on the methodology for constructing comparative maturity models and validated based on the Delphi method. The weights on each validated component have been calculated based on the summing of votes method. The study resulted in eight main components «scope», «corporate governance», «reporting mechanisms», «protection», «tone at the top», «organizational and human resource practices», «investigations» and «monitor and review» divided further into 18 elements. The suggested maturity model may provide a pathway for organizations to develop and maintain a robust whistleblowing maturity framework that will benefit both the organizations and the public welfare.

1. Introduction

The most recent (ACFE 2022) study identifies whistleblowing as the most effective fraud identification method with a great difference from the second one, while the recent adoption of the (Directive (EU) 2019/1937) in the EU draws the attention of organizations and anti-fraud professionals (fraud examiners and internal auditors).

The article’s objective is to provide a validated maturity framework concerning whistleblowing that organizations can use to enhance their performance regarding whistleblowing and the internal control environment in total, and to allow internal and external benchmarking. Concerning the internal audit, the proposed model aims to provide to internal auditors the adequate, objective criteria to evaluate governance, risk management, and controls (IIA 2017) for whistleblowing.

The first maturity models were developed in the 1990s in management and information technology to assess the level of competency, capability, or sophistication of business processes (De Bruin et al. 2005). Although it is not a requirement for a maturity model to be validated to be helpful, validation by experts may provide valuable inputs, insights, and a degree of assurance in respect to its credibility and relevance. In practice, many maturity models have been validated through the Delphi method (for example CR3M developed by (Głuszek 2021) and the BPAMM developed by (Martinek-Jaguszewska and Rogowski 2022), while others are not (KPMG 2013; ACFE and Grant Thornton 2020).

The Delphi method was developed in the 1950s by the Rand Corporation (Turoff and Linstone 2002), and since then it has been used in various sectors, including public health, transport, and education (Kittell-Limerick 2005). The Delphi Method aims to «obtain the most reliable consensus of a group of experts by a series of intensive questionnaires interspersed with controlled feedback» (Turoff and Linstone 2002). The use of this method is appropriate where the subject matter is complex (Ono and Wedemeyer 1994), where empirical evidence is lacking (Murphy et al. 1998), or when the knowledge is incomplete (Turoff and Hiltz 1996). That is applied to whistleblowing because it is a complex phenomenon that depends on legal, organizational, situational, and personal factors. In addition, although there is extensive research on the factors facilitating or discouraging whistleblowing, the study of whistleblowing as a control mechanism is limited.

The structure of the article is as follows. The next section (Section 2) covers the literature review on whistleblowing; Section 3 covers the theory and the application of the Delphi method. Section 4 presents the collection of data, Section 5 presents the results and the discussion while the last section covers the conclusions.

2. Literature Review

2.1. Whistleblowing

Before developing the maturity model, it is necessary to understand the nature of whistleblowing, its limitations, its place in the internal control framework, and the need for a whistleblowing maturity model.

One of the major limitations in the academic research of whistleblowing is that there is no commonly accepted definition. Near and Miceli (1985) defined whistleblowing as «the disclosure by organizational members (former or current) of illegal, immoral, or illegitimate practices under the control of their employers, to persons or organizations that may be able to effect action», while (Ravishankar 2003) defined whistle-blowers as the «employees who bring wrongdoing at their organizations to the attention of superiors». These definitions indicate the need for specific results. That differentiates whistleblowing from rumors, gossip, or grievances. However, other researchers suggested that considering only employees as whistleblowers may no longer be appropriate and may not adequately portray the whistleblower (Ayers and Kaplan 2005). Empirical evidence (ACFE 2022) shows that reports can also derive from external parties, in rare cases, even from competitors verifying this perspective. In addition, reports from external parties provide «greater evidence of wrongdoing, and they tend to be more effective in changing organizational practices» (Dworkin and Baucus 1998). Other definitions concentrated on whistle-blowers and their virtues rather than on the act of whistleblowing itself (for example (Berry 2004) and (Alford 2002)), hypothesizing that whistleblowers are highly ethical individuals with the courage to face the fear of reprisal. However, other studies have shown that in many cases, whistleblowers act opportunistically (Henik 2015). A more appropriate definition has been provided by (TI-NL 2019) that defined whistleblowing as «the disclosure of information related to corrupt, illegal, fraudulent or hazardous activities being committed in or by public or private sector organizations—which are of concern to or threaten the public interest—to individuals or entities believed to be able to effect action».

Internal controls can be distinguished based on two criteria. The first is whether the internal control is designed to prevent the occurrence of loss events (preventive control) or to identify internal control failures (detective control) after their occurrence. Whistleblowing has been categorized as a «potentially active or passive detection method» by (ACFE 2022) and as «an essential last line of defense in companies’ systems of internal control» (ICAEW 2019), meaning that it is more a detective rather than a preventive one. The second criterion is whether the internal control is specific to certain transactions or business processes (specific control) affects the control environment as a whole (entity-level control). Whistleblowing is an entity-level control with a pervasive effect in the control environment.

Even though whistleblowing is effective internal control, it is also associated with many limitations. Inertia may be one of the most significant ones. Organizations often received early notice but failed to act upon them. The example of Harry Markopolos may be one of the most indicative examples. Harry Markopolos, a financial analyst and fraud examiner, provided red flags of fraud to SEC for the biggest Ponzi scheme in history for over a decade before it imploded. However, his concerns were ignored repeatedly and when the fraud was exposed, the losses for the investors had already reached 65bn US dollars (The Guardian 2010). However, inertia may also derive from those who observe wrongdoing or business risks. Following (Miceli et al. 1987), whistleblowing is a «complex phenomenon that is based upon organizational, situational, and personal factors» which is not entirely within the control of the organizations, and «it is virtually impossible to change individuals’ core values which have been learned and consolidated over a lifetime» (ICAEW 2019). Academic literature and empirical evidence reveal that frequently too little information comes to the attention of the management or the regulators, and if it does come, it may come too late. For example, the Parliamentary Commission on Banking Standards was shocked by the evidence that so many people ignored misbehavior (CIIA 2014). As a result, the main weakness of whistleblowing as an internal control is that it depends on human behavior.

2.2. Maturity Models

Maturity models are often used on a self-assessment basis to help organizations understand their current level of capability in a particularly functional, strategic, or organizational area (OECD 2022). Maturity models provide a holistic approach (Martinek-Jaguszewska and Rogowski 2022) to depict the current situation based on objective criteria; envision the future, and find a way to achieve the desired state by following a disciplined method that is easy to use and implement (IIA 2013). In addition, the maturity models provide an early warning for an organization’s challenges (OECD 2022). Effectively, maturity models contribute to achieving a business process’s full potential. For these reasons, maturity models have been used in many areas, including information systems development (OECD 2022), internal auditing (KPMG 2013; OECD 2021; IIA 2019), fraud prevention and deterrence (ACFE and Grant Thornton 2020), tax law enforcement (OECD 2019a, 2019b, 2020). The fact that the Institute of Internal Auditors issued a practice guidance specifically in selecting, using, and developing maturity models before a decade proves their relevance to the internal audit. In addition, the (ACFE 2022) study reveals that the primary internal control failures that contributed to fraud are the lack of internal controls (29%); override of controls (20%); poor tone at the top (10%); lack of competent personnel in oversight roles (8%). A robust whistleblowing framework may reduce all these internal control failures. For example, management may be reluctant to override controls when the possibility of getting caught is high; additional internal controls may be implemented or the existing ones need to be improved.

3. Whistleblowing Maturity Framework

3.1. Development of Maturity Models

Usually, the maturity models are ranked using five steps. Each step illustrates the current or desired capabilities (De Bruin et al. 2005) and follows a reasonable escalation or path from the lowest to the highest. The maturity models can be distinguished to:

• Descriptives that are useful to depict the current situation;
• Prescriptive maturity models that also provide a roadmap to achieve the desired outcomes and;
• Comparative maturity models that also allow internal or external benchmarking (Becker et al. 2009).

The suggested model falls in the third category. In developing maturity models, the (IIA 2013) suggests the following steps:

• To determine the model and its components;
• To determine its scale and;
• To determine the expectations for each component.

The first step involves ascertaining what is to be assessed and based on that, identifying the components leading to this objective. Key considerations include whether the inclusion or exclusion of a component will increase or decrease the likelihood of achieving outcomes, respectively. In the second stage, 5 scales are usually used (Martinek-Jaguszewska and Rogowski 2022). The lower levels, 0 or 1, illustrate the absence of a capability, competency, or level of sophistication and level 5 illustrates the highest level. The (IIA 2013) also draws attention to the appropriateness of each level’s description. It is noted that scope of this paper is to validate the two first parts of the WBMM.

3.2. Stages

The first stage initially considered is compliance with the (Directive (EU) 2019/1937). The underlying logic is that non-compliance is not an option. Many organizations will consider this stage sufficient. However, it is likely that many organizations will use whistleblowing to comply with other laws and regulations not included in the scope of the Directive or their code of conduct. The «wheel of whistleblowing» suggested by (Culiberg and Mihelič 2017) includes many wrongdoings and threats not included in the scope of the (Directive (EU) 2019/1937). In addition, other organizations may also use whistleblowing to enhance their risk management or to achieve their ESG objectives. In accordance with (ICAEW 2019) whistleblowing provides a weapon to root out complacency and inertia which can be viewed as rigorousness for enhancing the internal control environment. In respect of the contribution of whistleblowing to achieve ESG objectives the (Directive (EU) 2019/1937) includes many wrongdoings that affect society while the «wheel of whistleblowing» suggests even more (Culiberg and Mihelič 2017). Therefore, the maturity levels (see as Figure 1) of the suggested WBMM are:

• Initial;
• Compliance with the Directive;
• Compliance with other laws and regulations;
• Enhancement of internal control environment and;
• Contributing to the achievement of ESG objectives.

3.3. Components

The components identified through a thorough study of the literature are divided into eight categories:

• The scope of the whistleblowing policy;
• Corporate governance;
• Reporting mechanisms;
• Protection;
• Tone at the top;
• Organizational culture and human resource practices;
• Objective investigations and;
• Monitor and review.

Accordingly, these result in 22 elements. The contribution of each main component and element is discussed below.

3.3.1. Scope

The scope of whistleblowing is two-dimensional and comprises what wrongdoings or dangers will be reported and followed up on and who can report them. The first dimension has already been analyzed. Regarding the second, the (Directive (EU) 2019/1937) also determines the persons who have the right to report. However, in some cases, organizations may be appropriate to expand the possible reporting persons.

3.3.2. Corporate Governance

Corporate governance refers to the mechanisms that provide the basis for objective investigations and corporate reporting. This component is divided into three elements:

• Overall responsibility;
• Assurance and;
• Corporate reporting.

Consensus has been established that the overall responsibility needs to be assigned to independent, non-executive directors or committees consisting of them. (PCBS 2013; PCW 2013) suggest a non-executive director, preferably the chairman; (CIIA 2014) proposes the audit committee, while (Greene and Latting 2004) suggest the ethics committee. In accordance with (CIIA 2014), organizations should obtain assurance in respect of whistleblowing either from the internal audit function or elsewhere. In addition, through the reports and the outcomes of the investigations, the internal audit function can confirm or alter its understanding of risks, procedures, and controls, allowing it to fulfill its obligations concerning ERM. The (Directive (EU) 2019/1937) does not set an obligation for disclosures relevant to whistleblowing. However, both (TI-NL 2019) and (GRI 2018) suggest certain disclosures in the annual reports or elsewhere.

3.3.3. Reporting Mechanisms

Reporting mechanisms include all necessary steps to enable a possible reporting person to make informed decisions on whether and how to report and provide secure reporting channels. The elements considered are:

• Anonymity;
• Reporting channels;
• Advice on reporting and;
• Visibility, clarity, and completeness of the information.

The (Directive (EU) 2019/1937) leaves the member states to decide whether anonymous reports will be received and investigated. However, (TI-NL 2019) ranked higher organizations that accept anonymous reports because it provides an additional safety line to the reporting person. Organizations also have to establish secure reporting channels that «protect the identity of the whistle-blower and any other individual included in the report» (Directive (EU) 2019/1937). A step further (TI-NL 2019) suggests that at least one to be available at any time and at least allow oral reporting. In addition, the (ACFE 2022) shows the preference of whistle-blowers for electronic reporting methods (email and web-based forms) compared to the traditional ones. The (Directive (EU) 2019/1937) also predicts free of-charge confidential advice to be provided by the facilitator, a «natural person who assists a reporting person in the reporting process in a work-related context, and whose assistance should be confidential». However, in many cases, whistle-blowers face ethical dilemmas and internal conflicts (Hersh 2002), and assistance in the bureaucratic process of filling reports may not be sufficient in the same cases.

3.3.4. Protection

Academic research, empirical evidence, and the (Directive (EU) 2019/1937) identified the threat of retaliation as a primary factor negatively affecting the decision to report. The consequences to the whistle-blower may vary significantly in terms of severity, from negative perceptions against whistle-blowers (Worth 2013), negative consequences in the work environment such as bullying (Bjørkelo 2013), to blackballing isolation, humiliation (Berry 2004), to psychological effects such as depression (Bechtoldt and Schmitt 2010), anxiety (Bjørkelo 2013), post-traumatic stress (Kreiner et al. 2008) or even to become life-threatening. Protection covers the confidentiality of the reporting person’s identity and any person referred to the reports as required by the (Directive (EU) 2019/1937) and the provision of adequate anti-retaliation measures. A practical implication may arise in cases where the organization has operations in jurisdictions will less robust legal protection. In such a case, it shall determine whether the same protection will be granted voluntarily.

3.3.5. Tone at the Top

The (ACFE 2022) identifies the lack of proper tone at the top as an internal control weakness in many frauds. The relationship between whistleblowing and tone at the top is bilateral, employees will only report if they know the whistleblowing policy and they believe that the top management supports it (Tsahuridu and Vandekerckhove 2008). On the other hand, if top the management is negligent in receiving and investigating reports, the wrongdoing could be seen as a routine practice (Kaptein 2011).

3.3.6. Organizational Culture and Human Resource Practices

For the purposes of this study, organizational culture initially distinguished to:

• Risk culture;
• Ethical culture;
• Learning culture and;
• Cultural differences.

Risk culture is the «part of the organizational culture that helps or hinders the effective risk management» (IIAA 2021) or a «mindful watchfulness for threats» (Berry 2004) that will allow alertness for malpractices and the ethical culture will allow malpractices to be reported once identified. Research shows that possible reporting persons who do not perceive reporting as their duty are unlikely to report it (Miceli et al. 1991), and they become oblivious to those wrongdoings not addressed in the code of ethics (Painter-Morland 2010). In addition, a lack of learning culture contributes to risk management frameworks failing (Schmidt 2020). Learning culture refers to organizations learning from their mistakes and build resilience to risks by creating, transferring, and retaining knowledge. Lastly, many researchers (Tavakoli et al. 2003; Zhuang et al. 2005) found that cultural differences can affect the decision of an individual to report and how to report. The main practical implication of this perspective is the consideration of cultural differences when an organization operates in multiple jurisdictions.

3.3.7. Investigations

The investigation of reports is the ultimate purpose of the whole whistleblowing mechanism. The elements initially considered that will allow unbiased judgments are:

• Organizational independence of the investigation team;
• Professional training;
• Appropriate risk prioritization;
• Investigation protocols and;
• Contribution to risk management.

The organizational independence of the investigation team may be safeguarded from the corporate governance mechanisms and the ethical decision-making of the team members. A reasonable assumption is that professionals, subject to a code of conduct and experience in ethical decision-making, may be less likely to accept undue influence. The professional training of those who manage reports is a mandatory requirement of the (Directive (EU) 2019/1937). In addition, investigation protocols determine key aspects such as the composition of the investigation team and methods of gathering factual evidence. Their contribution is that they enhance the effectiveness of the investigations. The risk prioritization is to ensure that the most severe reports are investigated first with the possible effect of the minimization and/or the recovery of the losses that the organization suffered. Lastly, the outcomes of the investigations may confirm or alter the understanding of the organization on risks and controls, allowing the enhancement of the internal control environment in total.

3.3.8. Monitoring and Review

Consensus on the need for monitoring the effectiveness of whistleblowing has been achieved (TI-NL 2019; IOS 2021, ISO 37002:2021). An interesting aspect of whistleblowing is that it is an entity-level control, not limited to isolated business processes and transactions. Likely, considering whistleblowing in isolation may not provide sufficient grounds to conclude. Therefore, the elements initially considered were (a) the assessment of whistleblowing in isolation of other controls with KPIs and key statistics and (b) considering the effectiveness of whistleblowing in other assessments. The proposed models before the validation by the experts can be illustrated as follows(

Table 1. Initially proposed maturity model.

	Initial	Compliance with the Directive	Compliance with Other Laws and Regulations	Enhancement of Internal Control	Contributing to Achievement of ESG Objectives
Scope
Scope
Corporate governance
Overall Responsibility
Assurance
Corporate reporting
Reporting mechanisms
Anonymity
Reporting channels
Advice on reporting
Visibility, clarity, and completeness of information
Protection
Anti-retaliation
Confidentiality
Tone at the top
Tone at the top
Organizational culture
Risk culture
Ethical culture
Learning culture
Cultural differences
Investigations
Organizational Independence
Investigation Protocols
Professional training
Risk prioritization
Contribution to risk management
Monitoring and review
Monitoring on separate activities
Assess whistleblowing during other assessments

).

4. Theory of the Delphi Method

4.1. Methodology of the Delphi Method

The characteristics of the Delphi method are:

• The anonymity of the members of the panel of experts that will allow them to modify their opinions;
• Statistical analysis and;
• Controlled feedback will provide them with the basis to change their views (Kittell-Limerick 2005).

The steps of this method can be summarized in the following

Table 2. Summary of the Delphi Panel Method.

Researchers define a problem and development of related questions. Researchers select a panel of diverse experts (whose anonymity is generally protected). Researchers distribute the questionnaire to the panel. Researchers analyze and summarize the data and develop follow-up questions. Repeat step 4 as required. As consensus begins forming and issues are clarified, repeat step 4 as required. Researchers invite panelists to revise or review consensus and specify reasons for dissenting opinions. Repeat steps 4–7 as required. Researchers summarize consensus and provide feedback to the panel. Researchers publish the final consensus statement.

Source: Hohmann et al. (2018).

.

4.2. Panel of Experts

In relation to the size of the panel of experts, the minimum is seven experts (Linstone and Turoff 1975). Useful results can be drowned by a panel of 10–15 experts provided that the group is homogenous (Cyphert and Gant 1971). The initial sample consisted of 10 experts who are certified fraud examiners and/or internal auditors and/or PhD holders with relevant research interests. To ensure the completeness of specializations in the panel of experts, they have been asked to identify other professionals who may contribute to the study. Some experts considered external auditors, compliance officers and lawyers. However, the panel already included experts who are also external auditors or compliance officers. As a result, 3 lawyers with sufficient knowledge of labor law and GDPR legislation have been selected.

From the anti-fraud experts ($N=$10) who accepted to participate in the study; ($N=2$) were theorists, ($N=5$) were professionals, and ($N=3$) were both professionals and theorists (

Table 3. Final composition of the panel of experts.

Category	First Group of Participants	Second Group of Participants	Total
Theorists (only)	2	0	2
Professionals (only)	5	3	8
Both theorists and professionals	3	0	3
Total	10	3	13

). In addition, ($N=5$) experts were holders of Ph.D.; ($N=2$) were Ph.D. candidates with relevant research interests, and ($N=4$) had a postgraduate diploma relevant to accounting and auditing (

Table 4. Minimum education level of the panel of experts.

Category	Theorists	Antifraud Professionals	Lawyers	Total
Ph.D.	2	2	0	4
Ph.D. candidates	0	2	0	2
Postgraduate degree	0	4	2	6
University degree	0	0	1	1

).

All antifraud professionals («professionals» and «both professionals and theorists») ($N=8$) had at least one professional qualification in internal audit or fraud investigation ($MIN=1$), on average ($M=3.750$) (

Table 5. Summary of professional qualifications of anti-fraud professionals.

Professional Qualifications	N	Min	Max	Total	Average
Qualifications in Internal Audit	8	0	7	14	1.750
Qualifications in Fraud Examination	8	0	3	9	1.125
Other qualifications	8	0	2	7	0.875
Total qualifications	8	1	9	30	3.750

and

Table 6. Analysis of professional qualifications of anti-fraud professionals.

Qualifications Relevant to Internal Audit	N
Certification in Control Self-Assessment (CCSA)	1
Certified Internal Auditor (CIA)	2
Certified Government Audit Professional (CGAP)	1
Certification in Risk Management Assurance (CRMA)	3
Certified Internal Control Auditor (CICA)	3
COSO ERM	1
COSO IC	1
Registry of Internal Auditors (Ministry of Finance in Greece)	2
Total	14
Qualifications relevant to Fraud detection and prevention	N
Association of Certified Fraud Examiners (ACFE)	6
Certified Global Sanctions Specialist Certification (CGSS)	1
Certified as an Anti-Money Laundering Specialist (CAMS)	1
Cert (ABC)	7
Total
Other relevant qualifications	N
Association of Chartered Certified Accountants (ACCA)	1
Certified Management accountant (CMA)	1
CPA’s training Institute of Greece	3
Institute of Chartered Accountants in England and Wales (ICAEW)	2
Total	7

). They all comply with the continuing professional requirements (

Table 7. Compliance with Continuing Professional Requirements.

Compliance with CPD Requirements	Antifraud Professionals	Lawyers	Total
Yes	8	3	11
No	0	0	0

) they have adequate post-qualification experience (

Table 8. Years of post-qualification experience.

Post Qualification Experience in Years	Theorists	Antifraud Professionals	Lawyers	Total
0–5	1	2	0	3
5–10	1	1	2	4
10–15	0	2	0	2
20–25	0	2	0	2
25+	0	1	1	2

). Most anti-fraud experts have more than one relevant occupation which gives them a broader view of the subject matter (

Table 9. Ranking in the organization of primary occupation (antifraud professionals).

	Secondary Occupation
Primary Occupation	None	Internal Auditor	Fraud Examiner	External Auditor	Compliance Officer	Professor
Internal auditor	1	0	1	0	1	1
Fraud examiner	1	1	0	0	0	0
External auditor	0	1	2	0	1	0

), and they work in senior positions (

Table 10. Ranking in the organization of primary occupation (antifraud professionals).

Position in the Organization of Primary Occupation	N
Partner	2
Head of department	6

).

Antifraud professionals and theorists are self-assessed with adequate knowledge of internal auditing, fraud prevention, and whistleblowing (

Table 11. Self-assessment of theorists and anti-fraud professionals.

Self-Assessment of Theorists and Antifraud Professionals	Very High	High	None
Internal audit	8	1	1
Fraud detection and prevention	10	0	0
Whistleblowing	10	0	0

). Especially in whistleblowing, the theorists ($N=2$) had experience through their research activities, while the antifraud professionals ($N=8$), ($N=5$) replied that their competency derives from practice and ($N=3$) replied that their competency derives from both practice and research.

The lawyers ($N=3$) rank high in their organizations, and ($N=2$) hold a relevant master’s degree; ($N=1$) do not hold a master’s degree, but their experience exceeds 25 years. In addition, they also self-assessed that they have adequate knowledge of labor law, General Data Protection Regulation, the (Directive (EU) 2019/1937) and Law 4990/2022, which incorporates the (Directive (EU) 2019/1937) into the national legislation. As a result, it was ensured (a) completeness of the relative competencies and (b) that the panel meets the qualitative criteria to place reliance on their views.

4.3. Achievement of Consensus

Consensus does not mean absolute agreement. Usually, the consensus is assessed using 5- or 10-point Likert scales. However, even though the Delphi method aims to achieve the most reliable consensus, there is not yet a commonly accepted method or benchmark. The decision criteria usually involve the mean, median, interquartile range, coefficient, and variation; for this reason, considering more than one criterion has been suggested (Głuszek 2021). Lawshe’s Content Validity Ratio (CVR) was also considered in this study. According to this method, experts are asked to determine whether a component is «essential», «useful but not essential», or «not useful». The following formula gives the ratio.

$$CVR=\frac{n_e-\left(\raisebox{1ex}{$N$}\!\left/ \!\raisebox{-1ex}{$2$}\right.\right)}{\raisebox{1ex}{$N$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}$$

where $n_e$ it is the number of experts who indicate that the component is essential, and N is the total number of experts. CVR can rank between −1 and 1. When most experts agree that an item is essential, the CVR is positive, and if there is total agreement, CVR will be equal to 1 (Lawshe 1975). The advantage of this criterion is that provides a clear cut-off decision rule. However, this ratio ignores the useful components. That may be problematic for some organizations that may consider it necessary to include useful, but not necessary elements in their whistleblowing framework. To provide a maturity model of sufficient flexibility, consensus will be achieved where more than half of the respondents consider the suggested elements as «useful but not essential» and «essential», and their contribution will be depicted with lower relative importance.

5. Results and Discussion

5.1. First Round Questions

The first round of questions was to record the competencies of the panel of experts, to ensure completeness in the competencies, and verify the appropriateness of the stages of the WBMM and their sequence. Specifically, in this round, experts were asked to assess the five maturity levels suggested; their sequence or identify other steps not considered initially and to highlight irrelevant steps. All experts agreed that the maturity levels are relevant and in the appropriate sequence, and no additional maturity level has been suggested.

5.2. Second-Round Questions

In the second round of questions, experts were asked (a) to rank each of the elements as «essential», «useful but not essential», or «not useful», and (b) to propose additional elements not identified by the researchers.

Table 12. Results of the CVR test.

	${\mathit{n}}_{\mathit{e}}$	CVR	Decision $\mathit{C}\mathit{V}\mathit{R}\ge 0.54$
Scope
Scope	13	0.85	Retain
Corporate governance
Overall Responsibility	13	0.69	Retain
Assurance	13	0.69	Retain
Corporate reporting	11	−1.00	Reject
Reporting mechanisms
Anonymity	13	1.00	Retain
Reporting channels	13	1.00	Retain
Advice on reporting	13	1.00	Retain
Visibility, clarity, and completeness of the information	13	1.00	Retain
Protection
Anti-retaliation	13	1.00	Retain
Confidentiality	13	1.00	Retain
Tone at the top
Tone at the top	13	1.00	Retain
Organizational culture
Risk culture	13	0.85	Retain
Ethical culture	13	0.85	Retain
Learning culture	13	0.38	Reject
Cultural differences	6	−1.00	Reject
Investigations
Organizational Independence	13	0.85	Retain
Investigation Protocols	13	0.69	Retain
Professional training	13	1.00	Retain
Risk prioritization	13	0.69	Retain
Contribution to risk management	13	0.69	Retain
Monitoring and review
Monitoring on separate activities	13	0.69	Retain
Assess whistleblowing during other assessments	13	−0.85	Reject

summarizes the CVR results of the responses to the second-round questions. The results show that all components that are obligatory by the (Directive (EU) 2019/1937) are considered «essential». Those components are the «scope», «reporting channels», «advice on reporting», «visibility, clarity, and completeness of information», «anti-retaliation», «confidentiality», and «professional training». This finding is also consistent to the maturity levels and their sequence, verified in the first stage.

Moreover, in consistency with prior research, all experts ($CVR=1$) agreed that the commitment from the top management is essential for the success of whistleblowing. Another significant finding is that the panel of experts agreed on the importance of the anonymous reports ($CVR=1$) irrespectively whether it is required by the national legislation. Something that is consistent with the suggestions of (TI-NL 2019). In respect of the corporate governance, experts agreed ($CVR=0.69$) that the ultimate responsibility for whistleblowing should be assigned to the top management, preferably to a non-executive director or a committee consisted of non-executive directors and that assurance ($CVR=0.69$) should be obtained.

Regarding the cultural aspects of whistleblowing, consensus has been achieved in respect of the risk culture ($CVR=0.85$) that will allow malpractices to be identified and the ethical culture that will allow them to be reported ($CVR=0.85$). Learning culture was rejected based on the CVR test ($CVR=0.38$). However, all experts agreed it is either essential ($N=9$) or useful ($N=4$). None of the experts considered it irrelevant. For this reason, the model has included it as an optional element. In contrast, cultural differences failed in both tests since the CVR was negative ($CVR=-1.00$), and fewer than half of the experts ($N=6$) consider it «useful but not essential». As a result, cultural differences are excluded from the WBMM.

In respect of the investigation phase, experts agreed ($CVR=0.85$) that organizational structure is essential to safeguard the objectivity and the impartiality. They also agreed that investigation protocols and risk prioritization enhance the outcomes of the investigations ($CVR=0.69$) and ($CVR=0.69$) respectively. Therefore, both elements included in WBMM as essential elements. The experts also agreed that whistleblowing can contribute to the risk management ($CVR=0.69$) and that its effectiveness shall be monitored through specifically designed assurance engagements ($CVR=0.69$). However, considering the effectiveness of whistleblowing during other monitoring activities failed to pass both tests and is excluded from the WBMM, since the ($CVR=-0.85$) and most experts considered it irrelevant ($N=8$). Lastly, the «corporate reporting» failed to pass the CVR test ($CVR=-0.38$). None of the experts considered it essential. However, most of the experts considered it useful but not essential ($N=11$) and only 2 ($N=2$) as not useful, as a result it will be included in the WBMM as an optional element.

After eliminating the elements of «cultural differences» and «monitoring whistleblowing during other activities». The remaining elements are distinguished into «essential» and «optional» based on the summing of votes technique (Dening et al. 2013; Sanderson et al. 2012), which is usually applied along with the Nominal Group Technique. The nominal Group Technique is also a consensus-seeking technique, with many similarities to the Delphi Method (Figure 2). Under this method, the scores from each participant for every component are summed. The relative importance is given from the formula:

$$Relativeimportance=\frac{scorefortheitem}{maximumpointspergroup}$$

Table 13. Table of relative importance if only the essential elements included.

	Responses from Experts
	Essential	Useful but Not Essential	Irrelevant
	3	2	1	Sum Scores	Relative Importance
Scope	12	1	0	38	5.54%
Overall Responsibility	11	2	0	37	5.39%
Assurance	11	2	0	37	5.39%
Anonymity	13	0	0	39	5.69%
Reporting channels	13	0	0	39	5.69%
Advice on reporting	13	0	0	39	5.69%
Visibility, clarity, and completeness of information	13	0	0	39	5.69%
Anti-retaliation	13	0	0	39	5.69%
Confidentiality	13	0	0	39	5.69%
Tone at the top	13	0	0	39	5.69%
Risk culture	12	1	0	38	5.54%
Ethical culture	12	1	0	38	5.54%
Organizational Independence	12	1	0	38	5.54%
Investigation Protocols	11	2	0	37	5.39%
Professional training	13	0	0	39	5.69%
Risk prioritization	11	2	0	37	5.39%
Contribution to risk management	11	2	0	37	5.39%
Monitoring on separate engagements	11	2	0	37	5.39%
Total	12	1	0	686	100.00%

presents the relative importance of each element of the WBMM if only the essential elements are included;

Table 14. Table of relative importance—essential elements and learning culture.

	Responses from Experts
	Essential	Useful but Not Essential	Irrelevant
	3	2	1	Sum Scores	Relative Importance
Scope	12	1	0	38	5.27%
Overall Responsibility	11	2	0	37	5.13%
Assurance	11	2	0	37	5.13%
Anonymity	13	0	0	39	5.41%
Reporting channels	13	0	0	39	5.41%
Advice on reporting	13	0	0	39	5.41%
Visibility, clarity, and completeness of information	13	0	0	39	5.41%
Anti-retaliation	13	0	0	39	5.41%
Confidentiality	13	0	0	39	5.41%
Tone at the top	13	0	0	39	5.41%
Risk culture	12	1	0	38	5.27%
Ethical culture	12	1	0	38	5.27%
Organizational Independence	12	1	0	38	5.27%
Investigation Protocols	11	2	0	37	5.13%
Professional training	13	0	0	39	5.41%
Risk prioritization	11	2	0	37	5.13%
Contribution to risk management	11	2	0	37	5.13%
Monitoring on separate engagements	11	2	0	37	5.13%
Learning culture	9	4	0	35	4.85%
Total	12	1	0	721	100.00%

presents the relative importance if the learning culture also included

Table 15. Table of relative importance—essential elements and corporate reporting.

	Responses from Experts
	Essential	Useful but Not Essential	Irrelevant
	3	2	1	Sum Scores	Relative Importance
Scope	12	1	0	38	5.35%
Overall Responsibility	11	2	0	37	5.21%
Assurance	11	2	0	37	5.21%
Anonymity	13	0	0	39	5.49%
Reporting channels	13	0	0	39	5.49%
Advice on reporting	13	0	0	39	5.49%
Visibility, clarity, and completeness of information	13	0	0	39	5.49%
Anti-retaliation	13	0	0	39	5.49%
Confidentiality	13	0	0	39	5.49%
Tone at the top	13	0	0	39	5.49%
Risk culture	12	1	0	38	5.35%
Ethical culture	12	1	0	38	5.35%
Organizational Independence	12	1	0	38	5.35%
Investigation Protocols	11	2	0	37	5.21%
Professional training	13	0	0	39	5.49%
Risk prioritization	11	2	0	37	5.21%
Contribution to risk management	11	2	0	37	5.21%
Monitoring on separate engagements	11	2	0	37	5.21%
Corporate reporting	0	11	2	24	3.38%
Total	12	1	0	710	100.00%

if the corporate reposting is included and finally,

Table 16. Table of relative importance—essential elements and corporate reporting and learning culture.

	Responses from Experts
	Essential	Useful but Not Essential	Irrelevant
	3	2	1	Sum Scores	Relative Importance
Scope	12	1	0	38	5.10%
Overall Responsibility	11	2	0	37	4.97%
Assurance	11	2	0	37	4.97%
Anonymity	13	0	0	39	5.23%
Reporting channels	13	0	0	39	5.23%
Advice on reporting	13	0	0	39	5.23%
Visibility, clarity, and completeness of information	13	0	0	39	5.23%
Anti-retaliation	13	0	0	39	5.23%
Confidentiality	13	0	0	39	5.23%
Tone at the top	13	0	0	39	5.23%
Risk culture	12	1	0	38	5.10%
Ethical culture	12	1	0	38	5.10%
Organizational Independence	12	1	0	38	5.10%
Investigation Protocols	11	2	0	37	4.97%
Professional training	13	0	0	39	5.23%
Risk prioritization	11	2	0	37	4.97%
Contribution to risk management	11	2	0	37	4.97%
Monitoring on separate engagements	11	2	0	37	4.97%
Learning culture	9	4	0	35	4.70%
Corporate reporting	0	11	2	24	3.22%
Total	12	1	0	745	100.00%

present the relative importance of the elements of the WBMM if all essential and valuable elements included.

In summary, experts considered as essential not only the elements that derive directly from the (Directive (EU) 2019/1937), but also some elements that derive from best practices, and are likely to make whistleblowing work as intended, for example, commitment from the top management; ethical culture and risk culture.

From the above-weighted factors and the performance in each component, organizations can have a score in each component and an overall score when the individual scores are amalgamated. For example, an organization that assess both the obligatory and the optional elements in its assessment and ranks in the second level of maturity in respect of the scope, its score will be $5.10\%\times \frac{2}{5}=0.0204$ from the 0.051, which is the maximum for this element. Possible assessment criteria for external benchmarking may include mean, median, quartiles, and interquartile range.

However, there are some limitations on this study. One limitation is that the first two steps of developing maturity models have been validated while the third is not. That is partially compensated because having verified the previous steps it is not hard for organizations to follow a reasonable escalation from the lower to the highest level of maturity. Another limitation is that most experts derive from a single member state of the EU, and, likely, research in a larger scale and experts from other member states may be needed. Moreover, the suggested model does not assess the differences in the legal environment and the effectiveness of the judicial system. Although one of the aims of the Directive is to provide common minimum standards across the member states, there are substantial differences in the effectiveness of the judicial systems across the member states (World Justice Project 2022). In addition, the member states can extend the provision of the Directive into their national legislation. In addition, the proposed model does not address differentiations among industries since some of them are highly regulated or differentiations deriving from the different sizes of organizations. Finally, the suggested model provides objective criteria to antifraud professionals to provide assurance or consulting engagements. Assessing the implementation of the policy and if those criteria are met is the next stage of engagement, and it is not addressed in the model.

6. Conclusions

Despite the extensive research on the factors influencing reporting behavior and the effects of whistleblowing, there is limited research on the contribution of whistleblowing as an internal control, despite its importance as a preventive, entity-level internal control and one of the most effective anti-fraud measures. This study identified and assessed twenty-two elements related to whistleblowing based on expert opinions, out of which eighteen were considered essential for effective whistleblowing, while two were rejected and two were considered optional. Weightings were calculated for various combinations of essential and optional elements to create a suggested model that can help businesses understand their current performance and design responses to fully realize whistleblowing’s potential. This model can also assist anti-fraud professionals in conforming to the code of ethics, definition, and standards of internal audit in their engagements, as well as facilitate internal and external benchmarking.

However, there are limitations to this research. Firstly, the expert panel consisted of members from a single member state of the European Union, which may limit the generalizability of the findings to other jurisdictions. Further research on a larger scale, encompassing multiple countries and regions, may be needed to validate the results and provide a more comprehensive understanding of whistleblowing as an internal control. Additionally, the suggested model should be tested and validated in real-world settings to assess its practical applicability and effectiveness. Furthermore, as the regulatory landscape and business environment continue to evolve, future research should also consider the dynamic nature of whistleblowing and its implications for internal controls.

In future research, exploring the impact of cultural and contextual factors on whistleblowing as an internal control would be valuable. Different organizational cultures, legal frameworks, and societal norms may influence the effectiveness of whistleblowing practices, and studying these factors could provide valuable insights for organizations and policymakers. Additionally, the research can further investigate the relationship between whistleblowing and other internal controls, such as risk management and corporate governance, to better understand their interactions and synergies.

To conclude, while this study contributes to understanding whistleblowing as an internal control, further research is needed to address the limitations and explore the potential of whistleblowing in different contexts. This will provide valuable insights for organizations, policymakers, and anti-fraud professionals in designing and implementing effective whistleblowing practices as part of their internal control frameworks.