Environmental Management Systems and Balanced Scorecard: An Integrated Analysis of the Marine Transport

Abstract

Critical aspects of the environment can reduce the efficiency of Environmental Management Systems (EMS) when applied to Marine Transport. Accordingly, this paper focuses on the improvement of the traditional EMS approach through the usage of Balanced Scorecard (BSC). The BSC represents a managing tool able to measure and increase organizational performance, taking into consideration environmental aspects. The proposed method, based on the ISO 14001 standard, allows management of environmental metrics through conventional BSC systems and it is applied to the biggest organization for marine transport in Montenegro as a case study methodology. In this qualitative investigation, particular attention was paid to creating EMS criteria able to orient the complete business operation of the organization but also to test their potential linkage to the conventional BSC approach. Four models of the BSC were created, each one including to a different extent the issue of environmental protection. Finally, an expert’s evaluation of model efficiency, based on the ISO 9126, was carried out. As a result, the best ranked model is recommended for the selection of an approach toward environmental protection based on the use of the EMS metric in a conventional BSC system. This method—in short ECO-BSC—was developed for the specific benefit of those organizations operating on the marine transport market.

1. Introduction

The ISO 14000 group of standards provides recommendations and practical tools for companies and institutions of all types seeking to manage their environmental responsibilities. In particular, the ISO 14001, focused on the Environmental Management System (EMS), gives basic directions for the systematic improvement of environmental protection. Furthermore, it could be applied to all organizations regardless of their size, sector of activity and so on. EMS helps the firms to reduce their adverse environmental impacts while improving their economic efficiency [1]. The proof that an organization respects the requirements of ISO 14001 standards is a special certificate. The certificate compels an organization to pursue a way to improve their environmental management system; however, it is not a guarantee of real ecological enhancement. In other words, the ISO 14001 certification itself cannot ensure improvement of environmental performance.

In addition, it is noteworthy that the initial version of the ISO 14001, dated 2004, did not provide any direct obligation referring to the improvement of ecological goals. The new issue, dated 2015, includes conditions focused on how to improve ecological performance in management but achievements in that direction are still not mandatory prerequisites for obtaining the ISO 14001 certificate. As a consequence, even if this new issue is quite clear in showing the relevance of ecology in business, there is not a clear explanation for how an organization can measure and then improve environmental performance. This limit in the standard is maybe related to the fact that, up to now, practical results of specific managing situations, more than general methodologies, seem the most appropriate way to prove a positive effect of the implementation of an EMS.

Research articles focused on improvements of environmental performance through the implementation of the ISO 14001 standard, in fact, propose unresolved conclusions. On one side, Franchetti [2] is determined to show the positive effect of the application of environmental standards. In particular, the author examined the effect of the ISO 14001 certification on solid waste generation in US industrial organizations and found that solid waste generation rates are significantly reduced with EMS certification. On the other side, several groups of researchers propose discordant indications. In Reference [3,4,5] it was argued, for instance, that the ISO 14001 standard does not improve the environmental performance of organizations, while in References [6,7,8] quite the opposite evidence was provided, suggesting the possibility of performance declination.

Therefore, the ISO 14001 implementation should be amended in its application to include elements of performance management as the only way to ensure the continual harmonization with the requirements set out in the standard. According to this, a possible solution could be setting the ISO 14001 objectives throughout an entire organization and for all employees, through some instrument for performance measurement

In this sense, the Balanced Scorecard (BSC) as a strategic management system and performance management system could turn out to be quite a good choice for organizations that consider the environmental management system as strategically relevant.

In part 4 (four), different models of the BSC have been implemented and discussed, each integrating the same EMS orientations but in a different way. These models have been assessed by Experts with reference to a specific case study on marine transport. The evaluation has been performed in accordance with the requirements for product quality of the consolidated standard ISO 9126: Software engineering- Product quality (later replaced by the ISO 25010 with marginal changes [9]), widely adopted for the assessment of software products and similar applications.

The fundamental scope of the ISO 9126 is to correctly define priorities and assess them in reference to measurable values in accordance with four main directions: quality model, external metrics, internal metrics and quality in the use metrics. This quality approach takes into consideration general criteria:

♦

Functionality

♦

Reliability

♦

Efficiency

♦

Maintenance

♦

Portability

♦

Usability

These quality criteria are detailed by sub-characteristics (as Suitability; Accuracy; Interoperability; Security, …) and further divided into attributes. The most relevant peculiarities of those attributes are related to the fact that they have to be selected in a way that they represent a measurable and verifiable entity of the product. At the same time, those attributes are not rigidly defined a priori through the standard. Thus, the ISO 9126 can be reoriented for its use in the presence of different applications (in respect to the evaluation of software products), as in the present case.

The final assessment is performed in accordance with the procedure of evaluation defined in the standard ISO 14598 (later replaced by the ISO 25040 with minor changes in the procedure of evaluation).

2. Materials and Methods

2.1. Strategic Management System-Balanced Scorecard (BSC)

Among different concepts and tools for achieving high performance and, therewith, an efficient management system—such as SWOT (strengths, weaknesses, opportunities, threats) analysis [10,11], Value Chain [12], Lean Six Sigma [13], Total Quality Management [14,15] etc.—the Balanced Scorecard (BSC) represents a managing tool able to transform the strategy of an organization into objectives and measures (metric) at all organizational levels. Through BSC each individual in an organization has an opportunity to provide a personal contribution to strategy accomplishment and simultaneously has an insight into the value of the work.

Furthermore, the BSC identifies strengths in the company, highlighting in the analysed company the return over assets, return over equity, product quality, operative cycle time and the satisfaction of employees [16]. The strengths and weaknesses identified have a direct relationship with the achievement of goals, which are set according to the historical information of the company and the average values of the industry of stony aggregates.

The graphical presentation of the BSC in Figure 1, based on Reference [17], shows the translation path passing by the Mission, Vision and Strategy concepts into a system of activities grouped through 4 key Perspectives [18]. All perspectives, representing the direction of strategic activities inside an organization, are of equal importance and have to be monitored. In particular, these four perspectives must be continuously measured, analysed and improved so as to conduct comprehensive measurements [19]. This endless process of improvement was represented coupling a Top-Down direct path with a Bottom-Up feedback [20,21,22].

The importance of the BSC is particularly evident in the Bottom-Up path where the activities that are directed to the accomplishment of the target values are defined with the purpose of monitoring the degree of goal fulfilment. Such an approach continually provides management with a real picture of the organization’s status in relation to defined strategic methods and indicates critical points where performance improvements are both possible and desirable. In order to attain the strategy, the objectives and measures of all defined perspectives have to be compatible and connected in a cause-and-effect chain so as to contribute to fulfilment of the strategy. In this way, the activity of an organization aimed at a continual improvement through the PDCA cycle becomes permanently initiated; the environment today is ever changing and therefore it is impossible for one defined strategy to be applied without improvements and reconsiderations.

The Balanced Scorecard recommends 4 standard perspectives (as listed in References [18,23,24]): learning and growth, internal processes, customers and finance; but these perspectives are not strictly defined parts that should be followed completely. Actually, the number and structure of perspectives depend on strategic organizational orientation, so some organizations can include in their strategy only 3 perspectives while other organizations can have 5 or more perspectives. Even though there is no strict form of the BSC system that could be applied to all organizations, it is possible to define two different concepts of the BSC which present the first decision in its implementation [7,20]:

♦

Concept of profit organization

♦

Concept of non-profit organization

Profit organizations are mainly focused on financial profit, which indicates that all objectives of the different perspectives are linked in a cause-and-effect chain, which is driven towards financial gain of the organizations. Non-profit organizations have a budget that services objective realization of all other perspectives, where the perspective of the customer’s satisfaction is the most important and represents the outcome of the cause-and-effect chain of all other perspectives.

2.2. Environmental Management System and Balanced Scorecard

How can the metric of the environmental management system of a certain organization be included in the Balanced Scorecard system and which approach is more effective?

According to References [23,25,26,27,28] environmental and social aspects can be integrated into the BSC, in order to create a SBSC, in 3 different ways:

• Modify the existing perspectives of the BSC model
• Create new perspectives that include these elements
• Create a special ecological/social scorecard

Between them, the most common approach is the first one (through 4 perspectives) but the second approach is also quite common, with the new perspective of focusing on environmental objectives. However, there are many supporters of the following concept: the addition of a new perspective depends on the strategic importance of these objectives [18,25,29,30,31,32,33,34].

However, EMS managers have a justified fear that there is an insufficient number of ecological objectives within the BSC model, meaning that these issues will be more and more neglected. It is also important to mention that this kind of involvement of the environmental objectives into a profit oriented BSC has to be realized in a way that also achieves the objectives of the financial perspective.

On the other side, in the BSC systems of non-profit organizations, the relations between perspectives are defined in order to achieve satisfaction of customers or stakeholders by using the available budget. According to this point of view, the non-profit approach of BSC could be much more efficient for the issue of an environmental management system than the profit oriented BSC model.

The third approach which includes the environmental metrics in the BSC represents the creation of the so-called SBSC (Sustainable Balanced Scorecard) which encompass all metrics that are in relation to the Environmental Management System, the social and financial perspective and which is connected with the BSC of the whole organization. The most important thing is that the SBSC is always profit oriented in profit oriented organizations. The possible negative aspect of its usage in practice is the establishment of a parallel system focused on EMS in relation to the conventional BSC. In accordance with the EMS established by ISO 14001, any other management system which is not entirely included in the BSC can also be considered a parallel system.

In line with this, there is a proposition where:

The Balanced Scorecard system with the ECO BSC system based on the principal of non-profit organizations and oriented toward the environmental protection management system presents an efficient management system which mainly improves ecological performance.

This hypothesis suggests the fact that the key to accomplishing an effective and efficient management system providing and improving environmental performance lies in a good connection between ECO BSC and the conventional BSC system where is ECO BSC is based on a non-profit model of connected perspectives.

2.3. The Case-Study: A Profit Organization Performing Marine Transport

In the paper “AD Barska plovidba”, a profit organization located in Montenegro—an active marine transport organization—was studied as a case of the implementation of the EMS approach based on the conventional BSC system. This action allowed environmental metrics to be included in the company management for the first time with results used to evaluate and select the best model for implementation.

Probably better known as “Montenegro Lines”, the Company was founded with the aim of providing marine transport services, marina activities, international freight forwarding and a marine agency. The Company carries out its basic activity (marine transport) through the transport of passengers and cargos (Figure 2) on the lines between Montenegro and Italy (mainly Bar-Bari-Bar). It is located in the Port of Bar which is traditionally seen as the strategic node in the pan-European corridor connecting East Europe and the Balkans (as Budapest, Belgrade) with Central-Southern Italy (Bari, Taranto) and other Mediterranean ports, relevant in terms of passenger and merchandise flows.

The Company conducts its core business with 2 passenger ships:

♦

“Sveti Stefan”, class “Bureau Veritas”, length 109 m, speed 21 knots, with the capacity of 550 passengers, 240 beds in the cabins and freight capacity of 205 vehicles,

♦

“Sveti Stefan II”, class “Det Norske Veritas”, length 118 m, 22 knots, with the capacity of 920 passengers, 495 beds in the cabins and freight capacity of 36 trucks (up to 12 m) or 24 (up to 18 m) or 225 cars.

Inside this organization, the conventional BSC system allowed planning for the entire business but prior to this survey, the issue of the environmental management was covered by a rather insignificant set of measures. At the same time, an established Safety Management System (SMS), based on the International Safety Management (ISM) Code and mainly oriented towards environmental protection, is prescribed by the International Maritime Organization (IMO) as a mandatory regulation for all marine companies operating in international maritime traffic.

This combination made “Montenegro Lines” a very interesting case study for the current analysis, also taking into account the new company strategy, which focuses on environmental protection, in due accordance with the international standards and the legislative requirements applicable to marine transport organizations.

3. Development

3.1. Approaches to Integrating the EMS into the Balanced Scorecard

Popular books about Balanced Scorecard [20,23,25,32] have not elaborated in detail the issues of implementation of the ecological aspects in BSC models; besides, they have presented strategic maps of specific for-profit organisations with a considerable number of ecological goals within existing BSC perspectives.

The environmental protection goals within these profitably oriented strategic maps (in the way that they lead to the fulfilment of financial goals as the organisation’s ultimate goal) are mainly focused on the offer of a “green” product to sensible “green” buyers and on their financial gain.

Nevertheless, there are research papers on the sustainable concept of the BSC, namely SBSC as Sustainability BSC.

This concept primarily seeks to determine the strategically relevant environmental and social goals of the organization that lead to the creation of economic value by the causal and consequential relationships of the perspective so that it is oriented towards three key elements: finance, ecology and society.

Literary sources show that adding a new perspective or creating a special scorecard actually depends on the strategic relevance of environmental protection for the organisation [23,24,25,29,30,31,32,33,34].

3.2. Integration of Environmental and Social Aspects

Environmental and social aspects can be considered by the existing 4 perspectives through strategic elements, objectives and measures. In this way, environmental and social aspects become an integral part of the conventional BSC and are automatically integrated into a causal chain that is hierarchically oriented towards the goals of the financial perspective in profit organizations. These aspects need to be integrated into the market system and to tend to the ‘green’ products’ buyers.

However, the number of environment-centred objectives and measures that involve such an approach in the BSC is fairly limited and insufficient to cover the overall issue.

However, a large number of organisations that are insufficiently focused on the issue of environmental protection still accept these approaches explaining that this issue is still covered by certain BSC objectives.

3.3. Creating a New Ecological/Social Perspective

With this approach, there is a large number of environmental and social aspects that cannot be fully integrated into the market changes for the simple reason that they are not being explicitly marketed. Creating a new perspective provides a clearer picture of the integration of environmental and social aspects into the conventional BSC model, at the same time creating an opportunity for expanding the metric that encompasses this issue. Elements of an ecological/social perspective must be linked to all other perspectives, not just to the financial perspective. It is also possible to create two perspectives that include independently social and ecological requirements.

However, in practice, the most common approach is the inclusion of sustainable development elements in the conventional BSC model, while adding the perspectives that these goals are unifying is less common because, for most organizations, sustainable development is not strategically relevant. This is the reason why there are numerous advocators of the concept that the addition of a “new perspective depends on the strategic relevance” of these elements [18,23,25,30,31,32,33,35,36].

3.4. Creating A Derived Ecological/Social Scorecard

The previous two concepts always create a justifiable concern with the managers responsible for environmental protection, on the grounds that this problem will be even more neglected due to a scarce number of objectives focused on the financial perspective within the BSC model.

The third approach to integrating environmental and social aspects into the BSC involves creation of a specific ecological/social scorecard in parallel with the conventional BSC. The derived ecological/social scorecard is not detached from the conventional BSC and must be associated with it in order to strengthen the ecological commitments of the organization’s development.

Actually, scientists developed an extended architecture of BSC under the name of SBSC and they did it for two reasons [37]:

• to allow management to define specific metrics in all the three dimensions of the company sustainability: economic, environmental and social;
• to let SBSC merge these three dimensions in a single integrated management system instead of a parallel system.

The point is that this particular scorecard that would manage to cover the overall issue of sustainable development would be profit-oriented. The reason for the rare usage of this model in practice is the creation of a parallel system in relation to the conventional BSC [38,39].

However, in the literature, several examples of the application of SBSC exist but always dealing with EMS in profit-oriented situations [35,36,40,41].

Furthermore, in Reference [42], the authors claim not to have great expectations for SBSC, since this method seems not appropriate to permit radical changes in business operations when oriented at providing a sustainable development. In these terms, it is considered, in practice, a simple managing tool for the implementation of a strategy (as in References [42,43]). This is why the management has to set strategic priorities for sustainable development in the first place and then use SBSC for implementing strategic priorities [42]. The survey [26] is in compliance with the aforesaid as it says that the sustainable development must address a strategic direction of an organization, so as to provide the success of SBSC by continuous monitoring of metrics related to these issues and the continuous work of all employees.

Otherwise, sustainable development becomes an activity which is not considered a factor of competitive advantage for an organization and is therefore not dealt with in the organisation [26].

In Reference [44], an investigation implemented inside the port industry, it is shown how tools like BSC can permit employees to effectively take part in the EMS application; this condition represents an aspect fundamental for gaining environmental performance in real situations.

In Reference [45] it is pointed out that in all organisations where the clear compliance with the regulations governing environmental protection is mandatory, as in the case of port industry organisations, it is possible to combine different management tools with BSC with the aim of increasing the environmental protection effectiveness and efficiency. In this way, the application of management innovations would enable both employees and management to actively and efficiently find ways to work in the field of environmental protection [45].

In Reference [46], an assessment of the effect of different environment management strategies based on the Port of Alexandria, all environmental aspects of the port were presented in the first place for relevance. Then, by modelling the system, it appeared that the application of SBSC system, which provides “more environmentally, socially and economically friendly approach to port operations over the long run”, could effectively lead to the reduction of gas emission.

In Reference [46], in particular, it is also claimed that the SBSC approach is especially suitable for those organisations that have not implemented a BSC yet but want to integrate sustainable development goals into employees’ everyday activities without establishing BSC for complete business practice. It can also be appropriate for those organisations with established BSC but wishing to get strategically oriented to and put special emphasis on the sustainable development [46].

In Reference [47], a survey is reported, done with regards to the relationship between the SBSC and a traditional eco-efficiency analysis. The study highlights that the “eco-efficiency is an instrument for estimating and controlling the appropriate key performance indicators for two major aspects of sustainability; namely, environmental and economic issues”. In this terms, the eco-efficiency analysis may be considered as a connection between SBSC and the EMS, just because both systems rely on material and energy flow analysis and use life-cycle assessment approaches.

Hence, experiences in the application of SBSC are quite different. However, it is typical that these approaches, as the ones previously presented, move toward the inclusion of sustainable development goals and measures into the BSC for profit organisations. Therefore, the environmental protection involves goals and measures connected through BSC perspectives in such a way to be completely in compliance with models for profit organisations. It means that they aim at fulfilment of financial perspective goals, as clearly shown in Reference [18,20,25,29,31,47,48,49,50,51].

In particular, [36] points out that specific conceptual modifications inside the BSC (respect to the original model) have enabled its evolution with the effect to increase its application in practise. The paper creates a new ECO BSC model respecting the operating principle of non-profit organisations but also considering the budget as a relevant issue inside all ecologically relevant measures. It means the attention to the budget is not limited only on those aspects that increase organisation’s profit. At the same time, such model locates financial perspective on the top, in the way that financial effects of such financing of environmental protection can be measured. With this approach, tested in practice, new version of BSC, conceptually oriented to the environmental protection, can be created.

4. Results

4.1. ECO BSC System Created on the Concept of a Non-Profit Organization

With the aim of analysing all the approaches adopted of the EMS by the application of the BSC and with the aim of finding the most efficient way, the definition of ‘ECO BSC’ systems was initiated.

Objectives and measures of the ECO BSC are integrated within the framework of 5 perspectives (budget, learning and growth, internal processes, stakeholders, finance) and mutually connected in a cause-and-effect chain through strategic map created on the approach of BSC for non-profit organizations, as shown in Figure 3. Therefore, the essence of this model is to link the objectives and measures related to EMS so as to establish their mutual correlation and finally their impact on financial results. The point of such a model where objectives and measures are defined at all levels of the organization is that all employees at all levels of the organization are engaged in environmental issues and that such activities can be measured.

Having in mind that the BSC system consists of a well-designed metrics that cascade translate organization’s strategic objectives to all employees, for the purpose of creation of the ECO BSC system, the communication with the employees and the EMS manager was accomplished in this work. Thus, a list of goals and measures was identified, acting as an environmental metric at a corporate level, that depicts the strategic activities of this organization in the area of the EMS. After defining the entire metric, the objectives need to be translated to the lower levels in the organization in cascade-like manner, to reach all the employees and other resources.

Figure 3, shown in line with Reference [41], shows evidence of a portion of the features characterizing the ECO BSC system for the entire organization. This figure shows the general review of the organization in terms of a functional composition and strategic map, as expressed by the software suite QPR. Specifically, the layout splits the system information into three different frames portraying, from left to right: the overall structure with its substructures, list of (several) system features under consideration, their links and mutual influences.

In particular, left frame in Figure 3, it can be noted that the ECO BSC can be regarded as being comprised of 3 separated and ecologically oriented sub-systems, namely:

♦

ECO Catering

♦

ECO ships, specifically, Eco ship Sveti Stefan I and Eco ship Sveti Stefan II

♦

ECO travel agency

It is also evident—see the central frame in Figure 3—that the system, together with its subsystems, have been analysed taking into account traditional features such as: budget, finance, internal processes and such. But also additional aspects, less common in this kind of investigations, as potentially hazardous, stakeholders (complaints and satisfactions) have been included in the study.

This ECO BSC system is created on the approach of BSC for non-profit organizations as a first step; it means that the perspective budget and other similar perspectives are directed to the satisfaction of the stakeholders, instead of generating profit. It is not possible to act differently, since, in strictly financial terms, environmental protection would always be a cost. Any business management tool developed in exclusive profit logic would lead to the exclusion of the environmental protection measures, even at the cost of leading the company to act outside the limits set by the regulations. At the same time, the deflection of a business-oriented concept from the classic concept of non-profit organizations means creation of the fifth—financial perspective in order to measure financial effects of this kind of ECO BSC.

The solution is the one suggested below. The financial perspective in the management system under development does not represent the orientation of the BSC system but only depicts the financial consequences of the EMS in this way.

The ECO BSC can exist independently and cover the overall segment of the Environmental Management System and as such with a purpose of monitoring the entire EMS with measurements and enhancement of environmental performance.

To avoid creating a parallel management system with divergent goals, it is necessary to include a key metric (goals and measures) of the ECO BSC in the conventional BSC system.

This link can be realized in one of the following ways:

• By adding one or more objectives from the ECO-BSC to the perspective of the conventional BSC;
• By creating the fifth perspective with key objectives from the ECO-BSC;
• By combining the two approaches above.

In this way, the ECO BSC would include entire environmental management system coordinated by an EMS manager while the BSC system would be coordinated by top management which is focused on the whole business with the key metric of the environmental management system.

4.2. Approaches of Making Connection between ECO BSC and Conventional BSC

As explained, the ECO BSC needs to be well connected with the conventional BSC model in order to avoid establishing parallel management systems

Aimed at establishing good connection between the ECO BSC and the conventional BSC model, the ECO BSC key metrics referring to the EMS need to be introduced into the conventional BSC. This is possible since each objective or measure in the scorecard framework (map of the cause-effect metric of one organizational level) can present functional dependency of other objectives or measures of the same or other scorecard.

Figure 4 shows connection between the ECO and the conventional BSC. Actually, all the perspectives of conventional BSC involved some EMS objectives and measures which established connections between the conventional BSC and the ECO BSC (for example: EMS process into internal processes of scorecard Ships, as represented in Figure 4). Following this, Model 1 was obtained and implemented. In particular, the left frame in Figure 4 shows how the system under investigation is changed in a way that considers more articulated situation. Instead of three subsystems (displayed in Figure 3, left frame), where only one of them—namely EcoShips—showed a further level of structure, here the overall model is initially divided in two branches: one attributable to the company as a whole and the other to the sustainability part of the business. Also in this case, features were presented, as shown in the central frame but they assumed different relevance and mutual connection in reference to the previous case.

Another way the ECO and the conventional BSC system connection can be integrated, as listed, is when the fifth perspective, the so-called ECO perspective, is added to the conventional BSC system. This perspective includes key metrics of the ECO BSC, as shown in Figure 5 and allows creation of Model 2.

In accordance with literature sources [23,25,26,27,28] apart from the two previously designed models, additional two were created combining the previous approaches:

• Model 3, which corresponds to the inclusion of the EMS metric in the existing BSC: this model is similar to conventional BSC of Model 1 but without the ECO BSC;
• Model 4, which corresponds to the inclusion of the EMS metric in the newly created ECO perspective: this model is similar to conventional BSC of Model 2 but without the ECO BSC.

As a result, 4 different models have been developed that are able to include, in different ways, the aspects of environmental protection inside the conventional BSC system.

Since, for a matter of fact, it is complicated to analyse the efficiency of those alternative models in real working conditions, especially considering the quite long period (up to a decade) that would be necessary for the implementation of each model and for the evaluation of their quality, the choice of the most suitable one to be used in Montenegro Lines was realized based on expert evaluation. In this way, the risk of adopting an ineffective model has also been reduced.

5. Discussion

5.1. Model Assessment

With the aim of carefully implementing this process of model assessment, available standards were followed and, further to this, experts from the BSC and EMS area were asked to provide special support.

The standard ISO/IEC 9126-Software engineering-product quality is used for the assessment of the BSC models. The standard consists of criteria for assessment of internal and external quality of software products. According the standard, the assessment of the four BSC models was conducted on internal and external quality evaluating 6 criteria (Functionality, Reliability, Efficiency, Sustainability, Portability and Usefulness) and their 27 sub-criteria. Evaluation of the software (models) quality in use was postponed after a certain time of application of the best ranking model.

The model evaluation process was conducted by two teams of experts. Experts have been selected with the aim of providing the highest level of expertise in each of the specific areas covered by these models but also to ensure the maximum objectivity during the evaluation process.

Specifically, Team 1, consisting in academic members, was created with the scope to provide competences respect to the wide scientific and research knowledge on the topics covered by the models. In a complementary action, Team 2, consisting in employees and top managers, was created with the scope to provide a wide and deep practical knowledge about the key processes characterizing the organisation; furthermore, Team 2 is also in charge of an evaluation on the application of new methods and technologies for the purpose of innovation and for the improvement of work processes in organisation: exactly the aspects that the present investigation has a chance to witness.

Table 1. Role and fields of expertise for External Experts (Academic Group) and Internal Experts (Company Group) involved in the assessment.

External Experts (Team 1)	Internal Managers (Team 2)
Quality and ISO standards	Environmental Protection
Environmental Management Systems	Financial & Accountability
Performance measurement	Technical & Technological

summarizes the fields of expertise for the external experts and roles of the internal managers involved in the evaluation.

Entering in further details:

(1)

Team 1 experts are university professors and researchers, recognised for their references in those areas that are crucial for performing an evaluation on the considered issues, such as:

♦

Environmental Management System.

♦

Quality and ISO Standards.

♦

Performance measurement.

Their task during the analysis was focused on observing the importance of the models under investigation from the standpoint of scientific soundness and validity. The team conducted the evaluation in order to find the best solution for establishing an efficient and effective management system, respect to the whole organization, which can enable continuous measurement and improvement of environmental performance.

(2)

Team 2 experts are key-member inside the ‘Montenegro Lines’ organisation chart. As a result of their technical competences and long-time employment, these employees have relevant roles (e.g., responsible or managers) in fundamental processes. Their position and experience provide them a deep knowledge about company and work processes, also permitting an access to all available information. Hence, Team 2 is comprised of the following experts:

♦

Environmental Manager in charge to compliance with legal regulations and relevant environmental standards in the organization;

♦

Financial Director in charge to manage the organisation from the aspect of finances and accountability and fully aware of all financial aspects;

♦

Technical Director to charge to directly manage all processes performed inside the organisation, especially in terms of technical and technological aspects, with full access to the related information.

Their task during the analysis was focused on observing the importance of the models under investigation from the standpoint of practical applicability, effectiveness and efficiency respect to the processes performed in “Montenegro Lines.” The team also assessed which model could fulfil the present and future company needs, especially in terms of strategic organizational management and environmental management.

Experts from both sides provided an evaluation based on their knowledge but also based on direct and indirect analysis on data from metrics, as well as by semi-structured interviews.

The evaluation was done in respect to the criteria reported to in

Table 2. Internal and external measures in relation to the 6 evaluating criteria.

Criterion	Sub-Criteria	Internal Measure	External Measure
Functionality	Adjustability	Has the correct functionality been activated correctly?	What is the stability of the operations at the stage of the operation?
	Accuracy	How accurate are the data?	Do you get imprecise data at work?
	Security	How do I access the system and disable data manipulation?	How often is data manipulation occurring?
	Interoperability	Are there good connections between data?	How good is the exchange of data during work?
	Compliance	Have compliance with applicable regulations & standards achieved?	Have compliance with applicable regulations and standards achieved?
Reliability	Availability	How many mistakes have been detected and corrected product?	How many problems can there be that can cause mistakes in future work?
	Errors Tolerance	How much software does incorrectly define connections?	How often systems fail in work or are inability to answer the questions?
	Reparability	How effective is the system after some unforeseen operations?	How much system is able to answer in working on all the required questions?
	Compliance	What is the compatibility of the reliability of the system with the applicable regulations and standards?	What is the compatibility of the reliability of the system with the applicable regulations and standards?
Usability	Understandable	How easy is the connection for users to use?	How many users correctly use defined functions?
	Easy learning	How is the function clearly explained to users?	How many users need time to learn all the functions of the software?
	Operability	How many functions can be changed during operation?	Can the user easily enter the data?
	Attractiveness	How attractive is the interface?	How attractive is the interface?
	Compliance	Have compliance with applicable regulations and standards been achieved?	Have compliance with applicable regulations and standards been achieved?
Efficiency	Behaviour in time	How long does it take to perform a specific task?	How long does it take to perform a specific task?
	Resources Uses	How much resources is needed to perform operations (memory,...)?	Is the software capable of performing operations with expected capacity and resources?
	Compliance	What is the compliance of system efficiency with applicable regulations and standards?	What is the compliance of system efficiency with applicable regulations and standards?
Suitability for maintenance	Possibility of analysis	How good are the functions that provide diagnosis of errors?	Can a user easily identify and correct errors?
	Interchange-ability	Is the process of software changes simple?	Can user’s problems be adequately resolved in an acceptable time?
	Stability	Do certain modifications affect the stability of the software?	Can the software be used without errors after the maintenance process?
	Testing	What is the possibility of software testing?	Can a user easily perform operational software testing?
	Compliance	What is the compliance of system maintenance with applicable regulations and standards?	What is the compliance of system maintenance with applicable regulations and standards?
Transfer ability	Adaptivity	What is the ability to adapt products to organizational and other changes?	Can the software be easily adapted to changes in the environment from the user’s point of view?
	Installation	What is flexibility of software to be installed in a particular environment?	Can users easily install software in a particular environment?
	Coexistence	What is the possibility of connecting this software with other products without a bad interaction?	How often do you make mistakes when connecting with other products?
	Replacement	How many functions must remain unchanged in software product?	Does the addition of new elements provide the same functionality from the user’s point of view?
	Compliance	What is the compliance of the system’s portability with applicable regulations and standards?	What is the compliance of the system’s portability with applicable regulations and standards?

, where each aspect used in classifying the internal and external evaluation items was reported. It represents the list questions the experts were asked to reply, able to represent the mentioned (6) criteria and (27) sub criteria for an accurate evaluation. The (6) experts performed their assessment independently from each other.

Experts performed their task using individual survey sheets, provided by the researchers. In this way, the experts had not idea of others’ opinion or final results. The researchers used a commercial software, Expert Choice, for managing and analysing data. In particular, in the case of inconsistency in values, the researches supported the experts in the way to re-examine those aspects which caused the data inconsistency. This procedure was in line with standards.

5.2. Approach to the Evaluation of Models (Software Solutions) Based on ISO Standards and AHP Method

Among the standards for evaluation, it is possible to list ISO/IEC 9126 and ISO/IEC 14598, used in the case of developed or ready-made software products. The ISO/IEC 14598: Software product evaluation represents a support for applying ISO/IEC 9126. Therefore, standard ISO/IEC 9126 defines general purpose of quality of software solutions, quality criteria and provides examples of measures, while ISOIEC 14598 provides directives in the process of evaluation of software products.

The recommendation of ISO/IEC 14598 standard (later replaced by the less known ISO 25040, with minor changes in evaluation procedure) is that the software evaluation procedure shall include all phases as presented in Figure 6.

Even if this procedure is quite standardized, the related recommendations are not so simple to be implemented: between other aspects, a clear choice of the specific methods to be used is missing.

With the aim of compliance with the evaluation procedure recommended by the ISO/IEC 14598, the AHP method probably represents the best way: [52,53,54,55,56,57,58] report, in fact, that AHP can be considered as the most common procedure in the software products evaluation, whereas evaluation criteria are based on the recommendations under standard ISO/IEC 9126.

In Reference [59], in particular, the application of AHP in choosing the ERP software in production companies has been shown. In Reference [60], the authors have created their own evaluation model in the Component Based Development (CBD) process, respecting the standards ISO 9126 and ISO 14598. Such evaluation model has been tested by applying AHP method in order to confirm the model validity showing satisfactory results.

The general recognition regarding the AHP validity is the reason why this method has be used in the study as a tool to provide as objective model evaluation and as much as possible in compliance with the criteria provided by ISO/IEC 9126 and with the procedure which is fully compatible with the evaluation approach defined under standard ISO/IEC 14598.

5.3. Analytic Hierarchy Process

In brief, it is a quite common idea that any problem that requires structuring, measurement and synthesis of the results represents a good opportunity for applying AHP.

The AHP method has many advantages as following [54,61,62,63,64,65]:

♦

the method is based on a well-defined mathematical model;

♦

quantification of elements structured in a hierarchical model with the possibility of documenting;

♦

applicable in situations involving a number of criteria;

♦

allows subjective assessment;

♦

uses qualitative and quantitative data;

♦

allows to measure consistency;

♦

it is widely analysed and applied in scientific literature;

♦

it has very good software support;

♦

suitable for group decision making.

Furthermore, during its use, the method offered additional advantages [66]:

♦

AHP successfully simulates the decision-making process in all its phases: from defining the goals, criteria and alternatives to evaluating criteria and alternatives and obtaining results.

♦

AHP gives to the decision maker information about the weight coefficients of the criteria relative to the target providing useful index for monitoring the efficacy of the procedure.

♦

When used in group decision-making, AHP significantly improves communication between group members [67,68], which results in better understanding and finding an easier path to consensus and as a final result provides group members with more confidence in the chosen alternative.

♦

AHP successfully identifies problems and points to the inconsistency of decision makers.

5.4. Models Ranking

Results achieved with the Team 1 evaluation (experts not employed in the company) using the AHP method in the commercial software program Expert Choice are shown in Figure 7. They rank Model 2 as the best one. Model 2 consists of ECO perspective in the conventional BSC and ECO BSC focused on the Environmental management system. Results achieved by the Team 1 evaluation (experts not employed in the company) using the AHP method in the commercial software program Expert Choice are shown in Figure 7.

In the same way, the application of the AHP method in the commercial software program Expert Choice helped during the assessment performed by Team 2 (managers from the company), with final results reported in Figure 8. Also in this case, it can be seen that Model 2 shows an important advantage over all other models. At the same time, Team 2 favoured models which imply creation of a separate ECO BSC system (Models 1 and 2) while Model 3 and 4 are estimated with drastically lower marks.

In order to get the final hierarchy for the recommendation of models, an additional AHP approach is used which merged the results obtained through the assessment performed by both teams (Figure 9).

5.5. Further Details on Models

Through this assessment, supported in method by the application of the AHP approach in the evaluation procedure we can notice a certain rule of ranking the models related to the volume of their orientation to the environmental management system. Following considerations were made:

♦

The best estimates were given to the models that most broadly encompass the EMS (Model 1 and 2);

♦

Final evaluation result has shown that Model 2, which involves ECO BSC and Eco perspective in conventional BSC, is the best ranked one. This model was also the best evaluated one by each team individually, what was not the case with Model 1.

♦

The third ranked model is Model 4 which includes eco perspective in conventional model but does not have ECO BSC. This model has stronger orientation towards the environmental protection than Model 3, since Eco perspective in conventional BSC offers more space for environmental metrics.

♦

The lowest ranked model in final evaluation result is Model 3, although it is the most frequently applied one. The reason for this shall be sought in the fact that organisations that have implemented BSC are mainly profit oriented organisations which are not strategically oriented to environmental protection.

As main methodological outcome, these results confirm the risky hypothesis, which argues that the BSC system with the specially created ECO BSC, based on the concept of a non-profit organization and focused solely to the environmental management system, produces an efficient management system that improves ecological performance.

Finally, Figure 10 summarizes the main results dealing with the models’ assessment. In particular, it shows by lines the capability of each model to reach the specific goal and in histograms the alternative options they take into account.

5.6. Comparative Analysis

Table 3. Details in the comparative analysis of the 4 recommended BSC models.

	Model	Model 1		Model 2		Model 3	Model 4
Characteristics
Existence of the ECO BSC		Y		Y		N	N
Existence of a Budget perspective		Y		Y		N	N
Additional ECO Perspective		N		Y		N	Y
Profit Organization		N	Y	N	Y	Y	Y
Base of the strategic map		ECO	BSC	ECO	BSC	BSC	BSC
Inclusion of the significance of goals in the perspectives obtained with the application of AHP		IP		IP	LD	No application of significance coefficients in the goals	No application of significance coefficients in the goals
		ST	US	ST
		LD		LD	ECO
Inclusion of coefficients of significance of measures in AHP targets		IP (AG)	IP (CG)	IP (AG)	LD	No application of coefficients of significance of measures in the objectives	No application of coefficients of significance of measures in the objectives
		ST	US
		LD (AG)	LD (CG)	ST (AG)	ECO (AG)
				LD (AG)

Legend: Yes (Y); No (N); Internal Processes (IP); Learning and Development (LD); Stakeholders (ST); Users (US); All Goals (AG); Company goals (CG).

and

Table 4. Numbers of targets, measures and objectives in the 4 recommended BSC models.

	Model	Model 1								Model 2
		ECO				BSC				ECO				BSC
Number of		A	B	C	D	A	B	C	D	A	B	C	D	A	B	C	D
Eco targets in the scorecard		10	10	10	10	3–13	3–9	3–10	3–10	10	10	10	10	5–12	5–11	5–12	5–12
Eco measures in the scorecard		33	34	21	29	7–27	8–21	5–15	8–18	33	34	21	29	4–24	5–18	2–12	5–15
Related objectives		9	6	6	6	6	6	6	6	8	8	8	8	8	8	8	8
Associated measures		9	9	8	8	6	6	8	8	6	6	5	5	3	3	5	5
	Model	Model 3								Model 4
		BSC								BSC
Number of		A		B		C		D		A		B		C		D
Eco targets in the scorecard		3–13		3–9		3–10		3–10		5–12		5–12		5–12		5–12
Eco measures in the scorecard		10–32		11–24		6–16		11–21		19–38		20–33		10–20		17–26
Related objectives		5		5		5		5		7		7		7		7
Associated measures		12		11		7		11		21		20		11		16

Legend: Scorecard at the corporate level (A); Scorecard at Ship level (B); Scorecard at the Travel Agency level (C); Scorecard at the Catering level (D).

show comparison between the 4 BSC models, both in terms of structure and consistency.

In particular, Table 3 summarizes the inclusion/exclusion of determined perspectives (e.g., budget, eco-sustainability) in the analysis in respect to each model. It can be noted, for instance, that a strategic map based on ECO targets could be considered only for Models 1 and 2.

A clear indication of the profit/non-profit orientation is also reported, together with the applicability of specific coefficients in AHP targeting. Amongst other potential approaches, the investigation was focused on evaluating the effect of changes in the goals between the following options: Internal Processes (IP); Learning and Development (LD); Stakeholders (ST) or Users (US).

Bearing in mind these options, the effects of changing the goals when choosing between a more general attention of All Goals (AG) and the most relevant goals (CG) for the company, such as the profit were also considered.

The following text contains a more detailed analysis of the results.

Table 4 details the values of targets, measures, objectives adopted during the analysis. In particular, it reports the number of:

♦

Eco targets in the scorecard (in relation to the total number of goals)

♦

Eco measures in the scorecard (in relation to the total number of measures)

♦

Related objectives (of the conventional BSC with eco goals at the same level)

♦

Associated measures (of conventional BSC with eco measures)

In particular, four different approaches were considered for defining the scorecards: Scorecard at the corporate level (A); Scorecard at Ship level (B); Scorecard at the Travel Agency level (C); Scorecard at the Catering level (D). Also in this case, it can be seen that a deeper analysis (in terms of combinations) has been performed for Model 1 and 2.

When determining the number of measures and targets (in Table 4), measures that belong to the second hierarchy level were not included because their values were included on a higher level of measures by simple summation. It should be also noted that Models 1 and 3 created a specific objective, which incorporates 3 sub-objectives (controlled aspects, potential hazards and regulatory compliance) for which there are no explicitly defined measures and these sub-targets were evaluated as measures in the table. Analysing the Tables, it can be noted that there are considerable differences between the models. Namely, Models 1 and 2 contain ECO BSC and therefore have a lower number of eco targets and measures explicitly included in the conventional BSC model, while Models 3 and 4 directly include selected eco targets and measures.

In this way, the total number of objectives and measures of Models 3 and 4 increases significantly, with their connection entirely based on models of profit organizations that would correspond to the concept of sustainable development that is not fully justified and applicable to the operations of the company.

Also, in the process of building the ECO BSC model, the significance of metrics was evaluated using the group AHP method to determine the key points that can be operated in order to improve the business results. The results of such measurements were included only in Models 1 and 2 at the corporate level of the ECO BSC and then transferred to the level of the conventional BSC model to the extent to which it ensured feasible connecting.

5.7. Applicability

The synthesis of the evaluation results has shown that those models which include a separately created ECO BSC have a better ranking (in terms of all previously considered criteria) with respect to those models only comprising of a certain number of goals and measures within the conventional BSC model. This general result is exactly where the major contribution of this paper lies. The same model ranking for strategically oriented to environmental protection has been obtained in consideration of a specific real situation and involve a well-definite for-profit organisation. At the same time, it implies that the same recommendations are almost valid in respect to those organisations having similar dilemma with regard to the choice of management’s approach to the performance based on a BSC approach. In details, the recommendations are:

♦

Organisations strategically oriented to environmental protection shall choose Model 1 or Model 2 since these models contain separate ECO BSC which completely covers the entire environmental protection field of an organisation. These models provide a good connection between the ECO BSC and a conventional BSC model, therefore avoiding the possibility of creating parallel systems on one side and on the other side, providing for the conventional BSC model not to be over-loaded with environmental protection goals and measures, what would make environmental protection a priority over other issues.

♦

Organisations not strategically oriented to environmental protection or those that have less significant impact on environment, may choose Model 3 or Model 4, depending on their commitment to this issue. Model 4 is more oriented to the environmental protection due to a special eco perspective, while Model 3 contains only some of the environmental protection targets and measures within existing perspectives.

♦

Furthermore, organisations aiming to implement the ISO 14001 standard but not in a position to implement BSC system on the level of the entire organisation, shall establish the network of goals and measures following the model ECO BSC, so as to enable more efficient monitoring, measuring and improvement of ecological performance, what has been assessed as the greatest shortcoming in application of ISO 14001 standard. ECO BSC model has been created on the operating principle of non-profit organisations, with the aim to grow into a profitable model in case that the amounts of budget and profit from environmental protection management come even in justified conditions. Such approach is also a logical one with regard to environmental protection management, much more logical that the one based on the operating principle of for-profit organisations which includes provision of financial benefits form environmental protection.

An additional contribution of the paper may also be found in the manner of creating the ECO BSC model. Explicitly, ECO BSC has been made on the operating principle of non-profit organisations with the budget as a basic perspective. Thus, the goals set in the budget perspective shall lead to the fulfilment of other perspectives, what is the approach which is suitable for an objective approach to the environmental protection management in most organisations.

Although created on the operating principle of non-profit organisations, having budget as a starting perspective and finance perspective on the top of a strategic map, ECO BSC still tends to grow into a profitable model if the amounts of budget and profit from environmental protection management come even in justified conditions.

6. Conclusions

This manuscript focuses on environmental sustainability issue, specifically on environmental management systems (EMS) and its reading and application by combining the EMS and BSC approach.

This methodological study also explores the case of one marine organization in Montenegro, used as a case study for testing this integrated application of the EMS and the BSC.

The field of investigation receives still little attention by researchers and several suggestions are provided in terms of integrated and more effective managerial instruments and practices that can be adopted to face the barriers that still exist in applying and following the managerial approach based on the respect for the environmental.

In particular, in this paper, four models of the BSC have been presented which, in different ways, include the metric of the EMS applied to the relevant case of the company operating in the marine transport field. Models 1 and 2 include the so-called ECO BSC component, created in relation to the concept of non-profit organizations. Models 3 and 4 do not contain this ECO BSC component, oriented to environmental protection. In that case, this aspect is covered by the framework of already existing perspectives and/or additionally created perspective in the conventional BSC.

Expert assessment, based on the AHP method and with respect to the criteria stipulated under the ISO 9126 standard, largely ranked Model 2 as the best one. This method is based on the special criteria of the ECO BSC system that covers the area of the EMS and has ECO perspective in the conventional BSC system that is able to connect these two systems.

This kind of model considers organization strategically oriented to the environmental protection in line with the robust requirements stipulated by international standards and laws on marine transport organizations in the field of environmental protection.

This general result should be a recommendation for all organizations committed to the EMS which have dilemmas when opting for the BSC approach with EMS elements.

As specific results, the advantages on applying the models 1 and 2 can be listed, in brief, as a:

• continuous monitoring and measurement of all ecological performance, which proved to be the biggest drawback of the implementation of the ISO 14001 standards;
• comprehensive knowledge about the relationship between ecological performance and strategic goals;
• comprehensive knowledge about the relationship between ecological performance and user perspective/user satisfaction;
• comprehensive knowledge about the relationship between EMS and Finance in order to measure the ecological efficiency (if this can be said);
• database containing information to be considered for more efficient decision making in the field of Environmental Management strategies.

Finally, beyond any specific results of the present case, this qualitative investigation also provides information and compares different alternative methods to connect the ECO and the conventional BSC system avoiding the creation of parallel management systems, in a single ECO BSC system able to measures ecological performance in a new environmental management system.

Regarding the future, this analysis could be conveniently oriented to different directions. From one side, the current study can be validated in (at least) a couple of years. In fact, after a reasonable period of time, a consistent database will be available that will allow discussion on assumptions and predictions.

On the other hand, it may be interesting to repeat the same analysis taking in consideration a different maritime transport company. It would be ideal to define a new case study that would present different conditions, while preserving correspondence in a way that simplifies the analysis. For example, it would be extremely interesting to involve a company active in the passenger transport between islands of Croatia or Greece. In this way, general aspects such as variations in the tourist demand for services but also local ones, such as fuel and personnel costs, could be compared.