Next Article in Journal
Relationship between Psychological Capital and Quality of Life: The Role of Courage
Previous Article in Journal
Using Landsat Data to Detect Change in Live to Recently (<6 Months) Dead Coral Cover in the Western Xisha Islands, South China Sea
Previous Article in Special Issue
Application of an Integrated Assessment Scheme for Sustainable Waste Management of Electrical and Electronic Equipment: The Case of Ghana
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Circularity in Waste Electrical and Electronic Equipment (WEEE) Directive. Comparison of a Manufacturer’s Danish and Norwegian Operations

Faculty of Logistics, Molde University College—Specialized University in Logistics, P.O. Box 2110, NO-6402 Molde, Norway
*
Author to whom correspondence should be addressed.
Sustainability 2020, 12(13), 5236; https://doi.org/10.3390/su12135236
Submission received: 29 May 2020 / Revised: 19 June 2020 / Accepted: 22 June 2020 / Published: 28 June 2020

Abstract

:
Waste electrical and electronic equipment (WEEE) as a reverse supply chain (RSC) has a low degree of circularity, mainly focusing on recovering or recycling. Targets to increase the circularity have recently been introduced in the EU WEEE directive. In this case study, we have investigated how WEEE is handled within an electric and electronic (EE) equipment manufacturer. The case study includes findings from two different Nordic countries, Norway and Denmark, with interviews of six stakeholders. The case study shows that there are significant differences in how the case company fulfills its extended producer responsibility (EPR), especially related to reporting. The study also found that there is a mismatch between the ambitions in the WEEE directive and a company’s approach related to circularity in the end-of-life phase of an EE product. Based on the results of this case study and from the literature we propose recommendations on alignment with other directives and on a common information regime within the WEEE RSC.

1. Introduction

The increasing focus on sustainability has led to a move from a linear economy model characterized by a make–use–dispose approach to a circular economy model, in which materials and energy remain in a restorative system [1,2]. Of particular interest is electronic waste (e-waste), one of the fastest growing waste streams in the world [3]. E-waste contains precious and hazardous metals including copper, gold, silver, palladium, cobalt, phosphorous, and platinum, and the amount is increasing, since we use more electronic equipment with short lifetimes [4]. Both practitioners and researchers have an increased interest in how to recover these kinds of raw materials from disposal areas through landfill mining [5,6]. Such recovery is costly, and the circularity approach aims to design-out waste where possible, extending product lifetimes by maintenance and reuse, leaving recycling as a last resort. This will create a circular system, in which restorative services create value by managing electric and electronic (EE) products in the circular system, with the ideal goal of no waste.
The waste electrical and electronic equipment (WEEE) directive regulates the end-of-life (EoL) treatment of e-waste in Europe [7]. This directive is defined as an extended producer responsibility (EPR) principle, giving the producer responsibility for the treatment or disposal of EoL products [8]. However, since the EPR principle allows manufacturers to delegate this responsibility to a third party, a producer responsibility organization (PRO) [9,10], a manufacturer has an indirect relation to the EPR principle. The producer adds a WEEE fee to the product price, and the collected fee is used to pay the PRO for EoL-product management. In Norway and Denmark, both consumers and companies may return their e-waste free of charge at collection points. Retailers of EE equipment are required to receive e-waste from consumers free of charge. The consumers have already paid for the EoL treatment via the WEEE fee since all importers or producers have collected a WEEE fee. In a 2014 study from Ylä-Mella et al. [11] about WEEE in the Nordic countries, Norway and Sweden were seen as global frontrunners of WEEE management. They focus on consumer awareness as a key success factor for sustainable WEEE recovery systems. Statistics from Eurostat show that Sweden and Switzerland have the highest WEEE recycling rates, with annual levels over 60% in the period 2011 to 2016 [12]. Here also Norway and Denmark are performing well, with recycle rates between 38% and 50%.
The WEEE legislation is an EU legislation. Each EU/European Economic Area (EEA) country has to implement the legislation into their national laws. This has resulted in differences in how WEEE is handled between EU/EEA countries related to reporting, WEEE fee levels, and other characteristics.
The recent initiatives to increase the focus on circularity in the WEEE directive have resulted in a new metric: the number of products prepared for reuse or recycling [13]. This product design feature links the EoL phase in a direct manner to the manufacturer. Reporting on the number of products prepared for reuse or recycling will be an incentive for the producer to pay more attention to the EoL phase.
The number of publications about WEEE is increasing. A Web of Science search on “waste electrical and electronic equipment” or “WEEE” gave more than 1400 hits where most of the publications are within the fields of environmental science and engineering environmental. Narrowing down the search to include the topics “reverse supply chain” and “circular economy” reduced the results to 33 and 43 hits respectively. Based on the large number of publications about WEEE, several literature reviews have been conducted within the area [14,15,16,17,18]. According to Islam and Huda [14], WEEE is an active area of research as is evident from the noticeable increase of papers on the theme in recent years [14,15,16,17,18]. The importance of individual manufacturers has increased in going from “cradle-to-gate” to “cradle-to-grave” responsibilities [19].
So far, very little information has been produced on how the new metric is received by manufacturers. Zlamparet et al. argue that even if there are many studies related to WEEE, most of them are related to recycling within the WEEE regime (85%), and significantly fewer studies on reuse and remanufacturing within WEEE [20]. Cesaro et al. remark the historical research focuses on metal recycling and recovery [21]. To improve WEEE management they argue for interdisciplinary research and a trans-national approach. Diedler et al. compared WEEE treatment in Serbia and Germany. They addressed data quality and stated that quality data remains a concern for transparency and reporting adherence [22]. Their recommendation is to develop an open access national online platform where all WEEE-related data is transparently visible to the public. Thus, there is a gap between the increased responsibility of individual manufacturers for circularity, and the knowledge of how manufacturers approach it. This study contributed to fill this gap by addressing the research problem:
How does an EE-manufacturer react to the need for higher circularity in the WEEE directive?
The study was guided by the following research questions:
  • How does a manufacturer view its e-waste handling by the WEEE regime?
  • How does a manufacturer report WEEE-related statistics?
  • How does a manufacturer deliver WEEE-related information with the products?
  • How does a manufacturer focus on for reuse or recycling of its products according to the new metric in the WEEE regime?
The organization of the rest of the paper is as follows. In Section 2, we present background information on e-waste handling according to the WEEE regime, the circularity of EE-products, and the extended producer responsibility. In Section 3, the research method is presented, followed by the results in Section 4. Section 5 analyzes and discusses the results before presenting recommendations in Section 6 and conclusions, limitations, and directions for future research in Section 7.

2. Background

2.1. E-Waste Handling According to the WEEE Directive and EPR

In Europe, the WEEE directive regulates the EoL treatment of e-waste. The first regulation was introduced in 2002 [23], with a new version released in 2012 [13]. Until 2015, the directive used recovered and recycled targets. From 2015 onward, the directive introduced the target “prepared for re-use or recycled”, in addition to the recovery target. The targets are specified as a percentage of what is placed into the market. In recent years, we have witnessed an increase in the target recovery rate and the rate of products prepared for reuse or recycle. The increase in targets is seen as ambitions by the authorities to increase the level of circularity by using the WEEE legislation, as this was one of the central goals of the original WEEE design [24].
When reporting according to the WEEE targets, the basic recovery rate calculation is recovery rate = the weight of EoL EE-products collected/the weight of new EE-products placed on the market. This metric is a rough estimate, since the total amount of products in the market contributes to the amount collected, while the amount of new products placed on the market is per year.
Producers and importers collect environmental fees when selling EEE equipment. This fee finances the collection and treatment of the waste, which normally is done by PROs. Organization of PROs can be done either as several competing collective systems or as one single non-competing collective system [25]. In Norway, there are five approved PROs for WEEE: Renas, ERP Norge, Norsirk, Recipio, and Serva. The five Danish approved PROs for WEEE are Elretur, ERP Danmark, LWF (light sources), Recipo, and Rene.

2.2. The Circularity of EE-Products

In this paper, reverse supply chain management is defined as the activities required to retrieve a product from a customer and either dispose of it, or engage in some kind of reuse/recovery [26,27]. According to Salema et al. [28], there is a loop in the supply chain if there is a relationship between the market for the used products and, in this context, the recovered, recycled, or re-used products. The loop is a closed loop if the two markets coincide and an open loop if not. The difference can also be seen as who the actors within this loop are. In an open-loop supply chain, the actors placing products on the market are not the same as the ones retrieving the products after EoL for reuse/recovery, while in a closed-loop the actors are the same [29]. In the context of a circular economy, the goal should be to achieve as high a level of circularity as possible [30]. Islam et al. identify a gap focusing more on studying WEEE as a closed-loop supply chain (CLSC) and using information technology as RFID and IoT technology to close the loop [14]. An example of a material flow in a forward and reversed supply chain is illustrated in Figure 1. Here we have included the areas where the PROs may have responsibilities for the manufacturer’s EoL treatment.
In a circular economy approach, waste is the last and less preferred option [31]. This is also stated in the EU’s Waste Framework Directive [32] and its ambitions towards a circular economy [33]. While Potting et al. have 10 different levels or strategies, the EU waste hierarchy has five. Here, prevention—or in Potting et al.’s model refuse, rethink, and reduce—is the preferred strategy as this does not produce waste at all. Disposal is the less preferred option; in the ideal circular economy (CE) model it is not even an option. In Figure 2, we have combined Potting et al.’s CE strategies with the EU’s hierarchy. While the WEEE legislation is focusing mainly on the waste dimensions, there are other directives focusing on the non-waste part in the waste hierarchy. The European Eco-design Directive [34] sets requirements for some energy-using or energy-related products put into the common European market. In a recent study, Polverini and Miretti [35] linked different eco-design requirements to different circular economy requirements. Even if they did not have ambitions of making a complete taxonomy, they identified several key connections between eco-design and the circular economy such as extended product lifetime, extended product reparability, extended product refurbishment, extended spare part availability, improved recycling possibilities, and reuse of components and raw materials.
Knowledge about disassembly and possibilities of recycling of components is required. Already in 2010, Kuo recommended a collaboration platform for manufacturers, suppliers, and recyclers with product data, bill of material (BOM), part types, material information, disassembly methods, and recycling analyses [36]. The platform included integrated information from manufacturers’ enterprise resource planning (ERP) systems, product data management (PDM) systems, and computer-aided design (CAD) systems, but also recycling information from the recyclers. Wang and Wang go one step further, introducing digital twins to support the EEE/WEEE manufacturing and remanufacturing process, including the products use phase [37]. They also include a product information model supporting information flow for a product within all its lifetime from design to recycling or remanufacturing.

2.3. Zero Waste vs. Circular Economy

In an ideal situation, the prevention part (non-waste) should dominate, leaving the waste part as minimal as possible, also termed zero waste. The two principles are often used for similar purposes. What is the difference between zero waste and CE? Veleva et al. distinguish the two concepts in the way that zero waste is focusing mainly on the EoL part of a manufacturing process, while CE is more holistic focusing also on the design phase and use phase of a product [38]. The main CE principles are (a) to sort out waste issues already at the design phase; (b) divide the circularity into biological and technical circles, where the biological should be returned back to the biosphere and the technical to be reused; and (c) focusing on renewable energy [31].

3. Research Methodology

We conducted a case study [39] consisting of semi-structured interviews at a manufacturer of EE equipment. The different interviewed stakeholders within the case company came from accounting (A), quality management (QM), product approval (PA), product development (PD), and sales (S). The case study was semi-structured in the way that some of the questions were open, and the informants were able to interpret the questions and provide their answers their own way. The stakeholders represented the management within the different streams, with both national and international stream responsibilities within the case company. All the stakeholders had full decision-making rights within their part of the WEEE legislation, but due to different organization of the WEEE treatment, the stakeholders came from different streams. Even if the WEEE directive applies for all countries under the regulation, the handling within each country might vary. To investigate this, we selected Norwegian and Danish branches of the global EEE manufacturer. By using two branches in similar Scandinavian countries, it should be possible to identify any discrepancies in WEEE handling due to national adoptions since any differences are expected to be even more pronounced in countries that are more diverse. The first part covered the Norwegian part of the case company, while the second part covered the Danish part of the case company. The selection of the panel of managers started with a physical interview with the manager defined as responsible for WEEE reporting within the case company’s head office. She included one of her staff members and the Health, Environmental and Safety-/Quality Manager in the interview, and they could answer some of the questions in the first physical interview. Their knowledge about WEEE in other European countries was limited. Some answers needed to be further investigated by the case company. This part of the interviews was done by email communication between the interviewer and the person responsible for WEEE reporting in the case company. Questions related to product development, product approval, and sales had to be raised to other stakeholders. The stakeholders were identified in the first physical interview, and the interview with them was conducted as phone interviews later in the process. For the second country, new stakeholders had to be identified. Here, a stakeholder representing the sales function had a leading role in answering the questions. Together with a stakeholder from accounting, they could cover the country-specific answers. Some of the answers related to product development and product approval were valid for both countries, but information about PRO and reporting issues were country-specific. All interviews were recorded and transcribed. The transcribed results were sent to the case company for approval. The interviews were done by one of the authors, and were conducted during February, March, and May 2020. The interview was arranged as 10 different questions covering different topics. A short version of the topics and questions, including the responders in the case company is listed in Table 1. A more detailed overview of the questions is listed in the interview guide in Appendix A. Content analysis was used to derive meaning from the interviews. Content analysis is an approach for making inferences by identifying certain characteristics of the data [40]. This wide interview approach was necessary to gain different perspectives on the four research questions. The results of the interviews were analyzed together with data from business documents, allowing for triangulation [40] in the analysis. Access to the resources needed for the successful completion of this research was directly granted, because one of the authors had, until recently, worked at the company. The case study provided in-depth knowledge of the handling of the WEEE directive of the company. In this study, we wanted to go in depth to investigate how one manufacturing EEE company treated their WEEE obligations. According to Yin [39], a single case study is suitable for this kind of study.

The Case Company and Its Subsidiaries

The case company is a multinational EE manufacturer that develops, produces, and distributes EE equipment in a professional market (business to business). Both production and distribution are international, with factories, distribution, and sales units all over the world. The head office is located in Norway and there are presently 2450 employees in the case company. The case company has a significant market share for their product portfolio both in the Nordics, in Europe, and for some product ranges worldwide. The European part of the operation has been covered by the WEEE legislation since the first WEEE directive came into force in 2002 [23]. Presently, the WEEE legislation of 2012, with amendments [13], regulates the company’s EPR related to WEEE. For each EU/EEA country in which the case company is present, the company needs to be a member of an authorized producer responsibility organization (PRO). These PROs may differ from country to country depending on how they are organized and financed and on other characteristics. The PROs may be non-profit member organizations, but they may also be commercial companies. The financing of the operations of the PROs is done by collecting an environmental fee when the EE equipment is sold. The environmental fee may differ from one EU/EEA country to another, depending on both the size of the environmental fee and on how this fee is calculated. The environmental fee may be calculated based on the value of the product, the weight of the product, the volume/size of the product, what kind of material the product is made of, or a combination of these. In some countries, it is expected to separate the environmental fee on invoices and other business documents, while in other countries, this is not necessary. Due to these different WEEE regimes, the case company must keep track of several WEEE dimensions when they place EE equipment into the European market. The case company’s environmental commitment is stated in its environmental policy, and in the case company’s values. Several of the production units within the case company are ISO 14001 environmental management systems certified [41].
The case study was organized in the form of interviews with stakeholders within the case company. This approach was necessary to get adequate answers to the issues covered by the interview.
Each member state in the EU/EEA is responsible to implement the WEEE directive in their country. The implementation may differ from country to country related to several dimensions as collective or competitive systems, financing models, etc. To be able to identify both similarities and differences in how the case company organized their EPR related to WEEE, both the Norwegian part and the Danish part of their operation was covered by the interviews. The company’s head office is located in Norway, with many common functions located there. Some of the answers were valid in both countries, some were country-specific, where stakeholders in one country only focusing on WEEE issues related to their country. As an overall policy within the case company, each country unit is responsible to fulfill the national WEEE regime.
Several production facilities, sales units, and a distribution center are located in Norway. In addition, some common functions such as product development, marketing, ICT, and product approval/laboratory are located here. For the Norwegian operation, the case company decided to join a non-profit member organization as its PRO. The selected PRO claims to be the number one in Norway in terms of the number of member companies, with more than 3000 national and international member companies. It also claims to offer producer compliance for waste batteries and packaging in addition to WEEE compliance.
The Danish part of the EE manufacturer is a pure sales organization. Physical distribution is done from other European countries direct to customers in Denmark. Product development is done outside Denmark, but the Danish operation can affect the development process to include country-specific demands. The Danish operation also had joined a non-profit member organization as their PRO. The purpose of the selected PRO is to “act under Danish legislation on producer responsibility by managing member companies’ interests and obligations, not least by assuming collection and reprocessing of end-of-life products”. The Danish PRO is also a member of a joint venture European take-back system for WEEE and batteries. As with the Norwegian PRO, the Danish PRO also claimed to be the market leader.
The EE manufacturer has over the last 15 years developed an ICT tool to support their WEEE obligations. The tool was built on other ICT systems as enterprise resource planning (ERP), product lifecycle management (PLM), and customer relationship management (CRM) to identify products and customers related to WEEE obligations. The tool is in use both to calculate the WEEE fee for customers, but also to do reporting on WEEE. The ICT tool is in use in about half of the EU/EEA countries covered by the WEEE legislation.

4. Results

In this section, we present the results of the four research questions. Regarding the first question of how a manufacturer views its e-waste handling by the WEEE regime, both the Norwegian and the Danish respondents maintained that, overall, the WEEE is perceived as a well-functioning scheme, collecting an environmental fee from customers to secure proper treatment of the waste after the products’ EoL. However, while the Norwegian response ended there, the Danish respondents also gave some additional responses, including that it is “difficult”, “the regulations are too different from country to country”, “they change too often, which gives a big cost to the businesses”, and “it would have been better to have a common strategy, so a developed joint tool and system could be used by the company”.
For research question two, how a manufacturer reports WEEE-related statistics, the Norwegian respondents mentioned that on a yearly basis, the total value of EE products introduced into the market is reported to the PRO. The same value is used to calculate the environmental fee to be paid to the PRO. Since the case company’s product portfolio only consists of two different WEEE product categories, they only divide the volumes into these different WEEE product categories (in fact, they report three product categories, since one WEEE category is divided into several product categories). On the communication with customers, the environmental fee is specified on outbound invoices and other business documents. In the same way, the environmental fees from all purchases from domestic suppliers are specified on inbound invoices. The calculation of the environmental fee is then based on the total turnover, minus the export share of the turnover, minus the environmental fee paid to domestic suppliers. The Danish responder responded that they could choose to report monthly, quarterly, every half-year, or yearly. The basis that they had to report was turnover, where the weight of the products sold was important. However, here the categorization of the products was more divided. To report WEEE, the case company had to extract this information from some sales statistics combined with information from the specific ICT tool for WEEE, and report to the PRO via a WEB scheme. It was pinpointed that it was important to document all steps in the process due to the fact that there was a WEEE revision on a yearly basis.
For research question three, how a manufacturer delivers WEEE-related information with the products, we also focused on WEEE reports the different units within the case company received from other actors within the supply chain. The responses were that, except for the specification of an environmental fee from domestic suppliers, the different units do not receive any WEEE reports from other business partners. One of the questions was related to product labeling. All products covered by the WEEE legislation must be labeled with a WEEE symbol. Except for that, there is no product labeling related to WEEE product information. The different units within the case company provide declarations, certificates, and product data sheets for all its products, and the units had access to the same set of documents from their suppliers. Both in the installation and maintenance manual and the user manual delivered with the products, there is a general recommendation that the product should be treated according to the WEEE directive after EoL. The recommendation is presented in eight different languages including Norwegian and Danish. Within this area, there was no deviation between the answers from the two countries, but from the Danish respondents’ side, they remarked that specific WEEE product labeling would be challenging due to the fact that then the labeling had to be different from country to country because of the different WEEE categorization between countries. The consequence of that would be that they would need specific WEEE labeling for Denmark, he stated.
For research question four, how does a manufacturer focus on reuse or recycling of its products according to the new metric in the WEEE regime, the Norwegian responses were that the case company had not observed any changes to the WEEE directive described above. They might have been informed about the changes, but as long as the PRO did not require any changes, the case company’s expectation is that the PRO addresses these kinds of changes. The Danish respondent responded that they were aware of the changes in product classification. This was because they had to reclassify their products in their WEEE ICT tool to report WEEE turnover based on the new product classification. The increased recover and recycle targets were unknown in Denmark. In Norway, the case company did see an increased focus on circularity for their products, but this is related to other considerations than the WEEE regulative. Here, two specific demands were mentioned: (1) market demands to reuse physical installations to decrease reinstallation time and (2) demands initiated by other directives or legislation, for example, the EU eco-design directive for the specific EE equipment manufactured by the case company. Related to circularity strategies/levels, key stakeholders from the case company did not see it actually expand the recovery or recycle levels. In Denmark, no extra focus on circularity was observed, neither in general nor more specific related to WEEE. A summary of the results from the interviews is presented in Table 2 below.

5. Discussion

In the case study, various stakeholders within an EE manufacturing company were questioned about how the WEEE directive is treated within the company. Questions were related to (1) WEEE in general, (2) WEEE reporting, (3) WEEE product documentation, and (4) WEEE and circularity.
For WEEE in general (1), the WEEE directive was clearly seen as a good approach to secure proper handling of EE equipment after EoL. The different units within the case company had selected two similar PROs that they observed to be serious and well-functioning organizations related to the WEEE compliant scheme. In addition to the PROs’ collecting and handling of the waste, the Norwegian PRO was also seen as a good partner, handling all aspects of the WEEE legislation, including changes in the directive. Even if this was not specifically stated in the initial Danish respondents’ answers, a follow up question related to how they ranked their PRO, they had a more neutral perception of their PRO’s performance. It was positive that the environmental fee covering the waste collection and handling in Norway had been significantly reduced over the years. The conclusion by the company was that the membership and financing of the actual PRO fully covered the EPR obligations for the case company. The same situation was reported from Denmark. Here, they described the situation that the PRO had collected excessive fees over the years so there was “enough money in the box to handle the waste”. Other researchers also observed a positive perception of the use of PROs. Spicer and Johnson discuss the benefits and challenges of the use of third-party de-manufacturing and found some of the same characteristics as those reported by the case company [42]. They pinpoint that this approach to fulfill EPR is “[i]deal for categories with a wide variety of products and manufacturers” and that, related to efficiency, this form or organization makes competition option for EoL treatment. Both Norway and Denmark have organized the WEEE return with several PROs covering each company’s EPR. There is evidence that this way of organizing the EPR is cost-efficient, at least compared to a single PRO model without competition [9]. However, the organization of EoL treatment using PROs may distance the manufacturers’ obligations, leaving all EoL treatment to the PROs. Competing or non-competing EoL systems both need coordination to perform optimally [25], including coordination with the manufacturer. The studied case indicates that the coordination between manufacturers and PROs are inadequate.
Related to reporting (2) on the amount of EE equipment placed into the market, there was a clear difference between Norway and Denmark. In Norway, the only numbers reported were economic values of the equipment divided into three different product categories. No reporting of weights or volumes were done. This was unexpected, because the WEEE and similar legislation normally use weight or volume as measuring units to calculate how much of the equipment is collected after EoL. This is also how Eurostat reports the development of WEEE in their statistics [43]. In Denmark, the reporting was more complex. As expected, the weight dimension was essential in reporting. The extended reporting regime is also documented by others, where producers of EE equipment here must report data on all EE equipment they place on the market [44]. Other indicators may be valuable to measure the performance of WEEE handling process. Nelen et al. propose the following four WEEE recycling indicators for sustainable materials management [45]: (a) weight recovery of target materials, (b) recovery of scarce materials, (c) closure of material cycles, and (d) avoided environmental burdens. Here, at least two of the indicators can be seen as unrelated to weight/volume dimensions. Anyway, today’s reporting regime on EE products in the market is far from able to fulfill the proposed key performance indicators. Especially in Norway, the current reporting regime needs to be expanded significantly to open this possibility.
In the communication part (3) with partners upstream and downstream in the supply chain, WEEE information such as environmental fees were communicated on business documents, mainly inbound and outbound invoices. In Norway, the information only focused on economic values. In Denmark, more information was available, such as the weight of the different equipment, and in some respect what kind of content the EE equipment was built on. This information was possible to produce due to the specific ICT tool handling WEEE developed in the case company. In Denmark, it was possible to present this information not only on outbound invoices, but also on other outgoing documents as quotations. Other product information is available via data sheets, web pages, and printed catalog material, but very little of this is specifically made for EoL treatment, only standard statements about WEEE and EoL used for all products in the case company’s product portfolio. It was stated that the request for documentation related to EoL, disassembly, or recovery treatment was very limited. Wang et al. argued for a data exchange and management framework based on a cloud-based approach to increase the circularity level in WEEE [46]. In their approach, actors within the lifetime of a product may use this system to enter and update information during both beginning-of life (BoL), middle-of-life (MoL), and EoL in a product’s lifecycle. A QR code should be used to secure the integrity of the product during its lifetime. Sun et al. investigated digital empowerment as new business ecosystem in WEEE collection [47]. This organization is focusing on all three flows between actors within the supply chain: information flow, material flow, and money flow. Zuidwijk and Krikke focus on design for recovery (DfX) and product data management (PDM) with new recovery processes in WEEE treatment [48].
For WEEE and circularity (4), the changes in the WEEE directive described in Section 2, focus is increasingly moving from recovery to recycling and re-use or preparation for re-use. Since the classification in WEEE has been changed, it is not always easy to see this increase, but for some equipment, the increase is significant, for example, lighting equipment [14]. However, the case company had not done any changes to the product development to address this increased ambition within the directive, in neither Norway nor Denmark. Related to improvement in circularity levels, other directives or regulations were seen as more relevant for the products placed on the market in recent years. Products are also increasingly being improved by being made modular, making it easier to replace broken modules, to increase the products’ lifetimes. However, the EU eco-design directive (Directive 2009/125/EC) motivates this approach. One stakeholder within the case company clearly stated that he did not see reuse at the component level on the EE equipment manufactured or imported by the case company as a realistic circularity level. The minimal focus on circularity related to WEEE is also mentioned in another study. Lu et al. indicated that the WEEE legislation is limited in terms of how to evaluate the potential for preparation for reuse [49]. Each EU/EEA country must transfer the WEEE directive into its national laws. Here, the UK is pinpointed as the leading country related to reuse in WEEE. It has also incorporated the reuse in WEEE into its Publicly Available Specification (PAS 141) for reusable electrical/electronic products [50]. There are other barriers to improve circularity in WEEE. Parajuly and Wencel [51] identified obstacles when they collected and investigated reuse and recycle potential of nearly 5 tons of WEEE. They identified several barriers for reuse, much of it related to that current WEEE treatment focus on WEEE as waste, not a potential value for reuse and recovery. One specific recommendation was to label products with their manufacturing year. This to identify the product’s lifetime that can help in making the reuse decision. The current EE case company have labeled their products with this information for the last 20 years.

6. Recommendations

Related to waste management, recycling and recovery of WEEE in Europe generally perform well. Especially the Nordic countries together with Switzerland are seen as frontrunners [11]. One reason for that is that these countries were early in implementing solutions for this kind of waste [12]. After over twenty years of handling WEEE, the recycling and recovery rates are high and increasing. Despite the new targets to increase the circularity introduced in a WEEE directive, the focus in practice is on handling EoL products as waste, and not as a potential value in a circular system. As for supply chain management, also, reverse supply chain management is based on the tree flows, material flow, money/value flow, and information flow [52,53]. Both from the literature and from the result of this study, lack of available information is seen as an obstacle both on today’s WEEE performance, and in a potential, more circular approach in the future. In addition, differences in the WEEE treatment implementation between countries are challenging. Inter-company ICT solutions including unique international product identification have been proposed to sort out these challenges [37,46]. Nevertheless, there is a barrier to these kinds of solutions. Even if all actors within a supply chain identify information sharing as important, most actors are restrictive when sharing information, also known as the information exchange paradox [54]. We support the idea of common ICT solutions to improve the information flow in the WEEE reverse supply chain. However, this ICT solution needs to have the following three characteristics. (1) The system must be network-centric in the way that each actor in the WEEE reverse supply chain is only providing their part of the information. For example, a supplier of an engine in a washing machine should provide their part of the information. The manufacture of the washing machine should assemble information from all suppliers together with the manufacturing part to finalize information about one of its products. (2) The system must reward firms (manufacturers, suppliers, subcontractors, etc.) who provide information. (3) The system must secure the intellectual properties of the firms. This is a prerequisite for firms to share information. Blockchain technology may have the property to solve all these characteristics [55].
A recommendation is to harmonize the EU WEEE directive [13] with the European Eco-design Directive [34] to align both directives with the sustainability goals focusing on unifying the three dimensions—economic, environmental, and social—as one (triple bottom line). It is also argued both in the literature [22] and in our research, lifting the WEEE issue from a national level up to an international level is important.
In light of the drive towards more sustainable products and supply chains, we question whether the use of PROs might slow down the progress towards circularity. Our results indicate that the use of PROs reduces the manufacturer’s attention to details in this development. By this, it is clear that letting the PROs handle the extended producer responsibility (EPR), which is the foundation of WEEE treatment, needs attention. Even if the latest WEEE directive included reuse or recycling, the actual effect of the added aspects can be said to be hampered by the current role of the PROs. An increased emphasis on the producer’s responsibility is called for.
Summing up our recommendations based on our research in four main areas, we call for:
  • network-centric ICT solution in which each manufacturer makes its WEEE related information available through life to secure information flow in the reverse WEEE supply chain,
  • harmonization of the WEEE directive and eco-design directives,
  • an international focus on WEEE due to the global ubiquitous EE products, and
  • stronger governance of WEEE RSC to strengthen the producer’s responsibility.

7. Conclusions, Limitations, and Directions for Future Research

Due to thriving environmental regulations and concern for sustainable practices, circularity in WEEE is an emerging area of research consideration. An investigation was conducted consisting of interviews based on research questions with different stakeholders (finance, quality management, product department, etc.) at a manufacturer of EE equipment. Data from the case company show that the WEEE regime is perceived as a success; the company can fulfill its EPR, EoL treatment is covered in a decent way, and the cost of the EPR is acceptable and decreasing. The organization of their EPR as a membership in a PRO is seen to be a good solution. This secures an economy of scale for the EPR, building up environmental knowledge of e-waste in general and the specific WEEE regime in particular. The PROs seem to generate value from the recovery process because the environmental fee financing the WEEE regime has been decreasing in recent years both in Norway and Denmark. The reporting of WEEE statistics is very limited in Norway, and only the turnover of WEEE categories is reported. In Denmark, the reporting is far more complex. The case company delivers and receives a set of product documentation downstream and upstream of the supply chain, but none of this documentation has its origin from the WEEE directive. The only specific WEEE information delivered is the calculated environmental fee, presented on business documents. The WEEE directive is not seen as an important incentive to improve circularity in the products, and other regulations are seen as far more important. The use of PROs may lead to a gap between the manufacturer and the EoL treatment, where the EE manufacturer is able to outsource their EoL responsibility to the PRO, distancing itself from the EoL treatment, causing less initiative for the manufacturer to increase circularity.

Limitation and Future Research Directions

This study is limited to two units in different countries, each being part of the same multinational company. More organizations in different countries might produce better insight in this area. The recommendations and conclusions should be validated in further studies. In addition, a comparative study focusing on national variations of the single WEEE directive should give interesting insights on how effective the WEEE directive is towards handling e-waste. Studies in this regard could include various approaches in calculations of environmental fees, variations in implementation of the WEEE directive, barriers for implementing WEEE treatment, and a comparison of administrative and waste management costs between countries.

Author Contributions

Conceptualization, T.A. and B.J.; methodology, T.A.; validation, T.A., B.J., and A.M.; formal analysis, T.A., B.J., and A.M.; investigation, T.A., B.J., and A.M.; resources, T.A., B.J., and A.M.; data curation, T.A.; writing—original draft preparation, T.A.; writing—review and editing, T.A., B.J., and A.M.; visualization, T.A., B.J., and A.M.; supervision, B.J. and A.M.; project administration, T.A. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A. Interview Guide

Practical
Interview language? Norwegian or English?
Recording the interview? Yes/No?
Information about anonymity/registration to Norwegian Centre for Research Data (NSD):
Companies/names will not be mentioned in the study.
Link to companies/names may be possible in the raw data.
After the interview is conducted, a copy will be sent to you for approval. What language is preferred? Norwegian or English?
Topic 1—WEEE in your organization.
Q1: Give a short description of how you perceive WEEE?
Q2: What EE return company are you dealing with/member of?
Q3: What type of WEEE reporting and to whom are done (including frequency)? Are copies available?
Q4: What type of WEEE reporting are you receiving and from whom (including frequency)? Are copies available?
Q5: In addition to the WEEE icon, are you providing or demanding extra WEEE product information following the goods (labelling)? Please give examples.
Q6: Are you providing additional WEEE information about your products to customers or other downstream actors? If yes, please give examples.
Q7: Do you have access to WEEE product information from other actors upstream in the supply chain (suppliers). If yes, please give examples.
Topic 2—WEEE directive changes.
Inform about the changes.
Q8: Are you familiar with these changes?
Q9: Have these changes resulted in any changes in (give examples):
WEEE reporting described in Q3 (your WEEE reporting).
WEEE reporting described in Q4 (received WEEE reporting).
Product labelling described in Q5.
Additional downstream WEEE information described in Q6 (customers).
Additional upstream WEEE information described in Q7 (suppliers)
Topic 3—Increased circularity.
Q10: Have the changes in the WEEE directive resulted in a higher focus on circularity within product development? If yes, has this also resulted in higher focus on product information flow upstream and downstream in the supply chain? If Yes, please exemplify.

References

  1. Masi, D.; Day, S.; Godsell, J. Supply chain configurations in the circular economy: A systematic literature review. Sustainability 2017, 9, 1602. [Google Scholar] [CrossRef] [Green Version]
  2. Geissdoerfer, M.; Savaget, P.; Bocken, N.M.P.; Hultink, E.J. The circular economy—A new sustainability paradigm? J. Clean. Prod. 2017, 143, 757–768. [Google Scholar] [CrossRef] [Green Version]
  3. Kumar, A.; Holuszko, M.; Espinosa, D.C.R. E-waste: An overview on generation, collection, legislation and recycling practices. Resour. Conserv. Recycl. 2017, 122, 32–42. [Google Scholar] [CrossRef]
  4. Polat, O.; Capraz, O.; Gungor, A. Modelling of WEEE recycling operation planning under uncertainty. J. Clean. Prod. 2018, 180, 769–779. [Google Scholar] [CrossRef]
  5. Krook, J.; Svensson, N.; Eklund, M. Landfill mining: A critical review of two decades of research. Waste Manag. 2012, 32, 513–520. [Google Scholar] [CrossRef] [PubMed]
  6. Burlakovs, J.; Kriipsalu, M.; Klavins, M.; Bhatnagar, A.; Vincevica-Gaile, Z.; Stenis, J.; Jani, Y.; Mykhaylenko, V.; Denafas, G.; Turkadze, T.; et al. Paradigms on landfill mining: From dump site scavenging to ecosystem services revitalization. Resour. Conserv. Recycl. 2017, 123, 73–84. [Google Scholar] [CrossRef] [Green Version]
  7. Hagelüken, C.; Corti, C.W. Recycling of gold from electronics: Cost-effective use through ‘Design for Recycling’. Gold Bull. 2010, 43, 209–220. [Google Scholar] [CrossRef] [Green Version]
  8. OECD. Extended Producer Responsibility. Available online: http://www.oecd.org/env/tools-evaluation/extendedproducerresponsibility.htm (accessed on 3 October 2019).
  9. Fleckinger, P.; Glachant, M. The organization of extended producer responsibility in waste policy with product differentiation. J. Environ. Econ. Manag. 2010, 59, 57–66. [Google Scholar] [CrossRef] [Green Version]
  10. Wang, H.; Gu, Y.; Li, L.; Liu, T.; Wu, Y.; Zuo, T. Operating models and development trends in the extended producer responsibility system for waste electrical and electronic equipment. Resour. Conserv. Recycl. 2017, 127, 159–167. [Google Scholar] [CrossRef]
  11. Ylä-Mella, J.; Poikela, K.; Lehtinen, U.; Tanskanen, P.; Román, E.; Keiski, R.L.; Pongrácz, E. Overview of the WEEE directive and its implementation in the nordic countries: National realisations and best practices. J. Waste Manag. 2014, 2014, 457372. [Google Scholar] [CrossRef]
  12. Ylä-Mella, J.; Román, E. Chapter 18—Waste electrical and electronic equipment management in Europe: Learning from best practices in Switzerland, Norway, Sweden and Denmark. In Waste Electrical and Electronic Equipment (WEEE) Handbook, 2nd ed.; Goodship, V., Stevels, A., Huisman, J., Eds.; Woodhead Publishing: Sawston, UK, 2019; pp. 483–519. [Google Scholar] [CrossRef]
  13. European Union. Directive 2012/19/EU of the European Parliament and of the Council of 4 July 2012 on Waste Electrical and Electronic Equipment (WEEE); European Union: Brussels, Belgium, 2012. [Google Scholar]
  14. Islam, M.T.; Huda, N. Reverse logistics and closed-loop supply chain of Waste Electrical and Electronic Equipment (WEEE)/E-waste: A comprehensive literature review. Resour. Conserv. Recycl. 2018, 137, 48–75. [Google Scholar] [CrossRef]
  15. Pérez-Belis, V.; Bovea, M.D.; Ibáñez-Forés, V. An in-depth literature review of the waste electrical and electronic equipment context: Trends and evolution. Waste Manag. Res. 2014, 33, 3–29. [Google Scholar] [CrossRef] [PubMed]
  16. Raja, T.Y.; Rahul Gandhi, B. Optimisation of electronic waste recycling network and reverse logistics—A literature review. J. Supply Chain Manag. Syst. 2014, 3, 18–24. [Google Scholar]
  17. Long, E.; Kokke, S.; Lundie, D.; Shaw, N.; Ijomah, W.; Kao, C.-C. Technical solutions to improve global sustainable management of waste electrical and electronic equipment (WEEE) in the EU and China. J. Remanuf. 2016, 6, 1. [Google Scholar] [CrossRef] [Green Version]
  18. Andrade, D.F.; Romanelli, J.P.; Pereira-Filho, E.R. Past and emerging topics related to electronic waste management: Top countries, trends, and perspectives. Environ. Sci. Pollut. Res. 2019, 26, 17135–17151. [Google Scholar] [CrossRef]
  19. Sheffi, Y. Balancing Green: When to Embrace Sustainability in a Business (and When Not to); MIT Press: Cambridge, MA, USA, 2018. [Google Scholar]
  20. Zlamparet, G.I.; Ijomah, W.; Miao, Y.; Awasthi, A.K.; Zeng, X.; Li, J. Remanufacturing strategies: A solution for WEEE problem. J. Clean. Prod. 2017, 149, 126–136. [Google Scholar] [CrossRef] [Green Version]
  21. Cesaro, A.; Marra, A.; Kuchta, K.; Belgıorno, V.; Van Hullebusch, E.D. WEEE management in a circular economy perspective: An overview. Glob. NEST J. 2018, 20, 743–750. [Google Scholar]
  22. Diedler, S.; Hobohm, J.; Batinic, B.; Kalverkamp, M.; Kuchta, K. WEEE data management in Germany and Serbia. Glob. NEST J. 2018, 20, 751–757. [Google Scholar]
  23. Union TEPatCotE. Waste Electrical and Electronic Equipment Directive [WEEE] 2002/96/EC; European Commission: Brussels, Belgium, 2002. [Google Scholar]
  24. OECD. EPR Policies and Product Design: Economic Theory and Selected Case Studies; OECD: Paris, France, 2006. [Google Scholar]
  25. Corsini, F.; Rizzi, F.; Frey, M. Extended producer responsibility: The impact of organizational dimensions on WEEE collection from households. Waste Manag. 2017, 59, 23–29. [Google Scholar] [CrossRef]
  26. Guide, J.V.; Wassenhove, L.V. The reverse supply chain. Harv. Bus. Rev. 2002, 80, 25–26. [Google Scholar]
  27. Prahinski, C.; Kocabasoglu, C. Empirical research opportunities in reverse supply chains. Omega 2006, 34, 519–532. [Google Scholar] [CrossRef]
  28. Salema, M.I.G.; Barbosa-Povoa, A.P.; Novais, A.Q. An optimization model for the design of a capacitated multi-product reverse logistics network with uncertainty. Eur. J. Oper. Res. 2007, 179, 1063–1077. [Google Scholar] [CrossRef]
  29. Gou, Q.; Liang, L.; Huang, Z.; Xu, C. A joint inventory model for an open-loop reverse supply chain. Int. J. Prod. Econ. 2008, 116, 28–42. [Google Scholar] [CrossRef]
  30. Potting, J.; Hekkert, M.; Worrell, E.; Hanemaaijer, A. Circular Economy: Measuring Innovation in the Product Chaing–Policy Report; PBL Publishers: Hague, The Netherlands, 2017. [Google Scholar]
  31. Ellen-MacArthur-Foundation. Towards the Circular Economy. Economic and Business Rationale for an Accelerated Transition; Ellen MacArthur Foundation: Cowes, UK, 2013. [Google Scholar]
  32. European Union. Revised Waste Framework Directive, 2008/98/EC; European Union: Brussels, Belgium, 2008. [Google Scholar]
  33. European Commission. Circular Economy Closing the Loop; European Commission: Brussels, Belgium, 2015. [Google Scholar]
  34. European Union. Directive 2009/125/EC. Ecodesign Directive; European Union: Brussels, Belgium, 2008. [Google Scholar]
  35. Polverini, D.; Miretti, U. An approach for the techno-economic assessment of circular economy requirements under the Ecodesign Directive. Resour. Conserv. Recycl. 2019, 150, 104425. [Google Scholar] [CrossRef]
  36. Kuo, T.C. The construction of a collaborative-design platform to support waste electrical and electronic equipment recycling. Robot. Comput. Integr. Manuf. 2010, 26, 100–108. [Google Scholar] [CrossRef]
  37. Wang, X.V.; Wang, L. Digital twin-based WEEE recycling, recovery and remanufacturing in the background of Industry 4.0. Int. J. Prod. Res. 2019, 57, 3892–3902. [Google Scholar] [CrossRef]
  38. Veleva, V.; Bodkin, G.; Todorova, S. The need for better measurement and employee engagement to advance a circular economy: Lessons from Biogen’s “zero waste” journey. J. Clean. Prod. 2017, 154, 517–529. [Google Scholar] [CrossRef]
  39. Yin, R. Case Study Research and Applications: Design and Methods; Sage Publications: Thousand Oaks, CA, USA, 2018. [Google Scholar]
  40. Bryman, A. Social Research Methods; OUP: Oxford, UK, 2012. [Google Scholar]
  41. ISO. ISO 14001. In Environmental Management Systems—Requirements with Guidance for Use; ISO: Geneva, Switzerland, 2015. [Google Scholar]
  42. Spicer, A.J.; Johnson, M.R. Third-party demanufacturing as a solution for extended producer responsibility. J. Clean. Prod. 2004, 12, 37–45. [Google Scholar] [CrossRef]
  43. Eurostat. Waste Statistics—Electrical and Electronic Equipment. Available online: https://ec.europa.eu/eurostat/statistics-explained/index.php/Waste_statistics_-_electrical_and_electronic_equipment (accessed on 3 October 2019).
  44. Parajuly, K.; Habib, K.; Liu, G. Waste electrical and electronic equipment (WEEE) in Denmark: Flows, quantities and management. Resour. Conserv. Recycl. 2017, 123, 85–92. [Google Scholar] [CrossRef]
  45. Nelen, D.; Manshoven, S.; Peeters, J.R.; Vanegas, P.; D’Haese, N.; Vrancken, K. A multidimensional indicator set to assess the benefits of WEEE material recycling. J. Clean. Prod. 2014, 83, 305–316. [Google Scholar] [CrossRef] [Green Version]
  46. Wang, L.; Wang, X.V.; Gao, L.; Váncza, J. A cloud-based approach for WEEE remanufacturing. Cirp Ann. 2014, 63, 409–412. [Google Scholar] [CrossRef] [Green Version]
  47. Sun, Q.; Wang, C.; Zuo, L.-S.; Lu, F.-H. Digital empowerment in a WEEE collection business ecosystem: A comparative study of two typical cases in China. J. Clean. Prod. 2018, 184, 414–422. [Google Scholar] [CrossRef] [Green Version]
  48. Zuidwijk, R.; Krikke, H. Strategic response to EEE returns: Product eco-design or new recovery processes? Eur. J. Oper. Res. 2008, 191, 1206–1222. [Google Scholar] [CrossRef]
  49. Lu, B.; Yang, J.; Ijomah, W.; Wu, W.; Zlamparet, G. Perspectives on reuse of WEEE in China: Lessons from the EU. Resour. Conserv. Recycl. 2018, 135, 83–92. [Google Scholar] [CrossRef]
  50. BSI (Ed.) PAS 141—Reuse of Used and Waste Electrical and Electronic Equipment (UEEE and WEEE); BSI: London, UK, 2011. [Google Scholar]
  51. Parajuly, K.; Wenzel, H. Potential for circular economy in household WEEE management. J. Clean. Prod. 2017, 151, 272–285. [Google Scholar] [CrossRef]
  52. Pagh, J.D.; Lambert, D.M.; Cooper, M.C. Supply chain management: More than a new name for logistics. Int. J. Logist. Manag. 1997, 8, 1–14. [Google Scholar] [CrossRef]
  53. Kumar, N.; Chatterjee, A. Reverse Supply Chain: Completing the Supply Chain Loop; Cognizant: Teaneck, NJ, USA, 2011. [Google Scholar]
  54. Bogers, M. The open innovation paradox: Knowledge sharing and protection in R&D collaborations. Eur. J. Innov. Manag. 2011, 14, 93–117. [Google Scholar]
  55. Pilkington, M. Chapter 11: Blockchain technology: Principles and applications. In Research Handbook on Digital Transformations; Edward Elgar Publishing: Cheltenham, UK, 2016. [Google Scholar]
Figure 1. Material flow in a forward and reverse supply chain. Adapted from reference [20].
Figure 1. Material flow in a forward and reverse supply chain. Adapted from reference [20].
Sustainability 12 05236 g001
Figure 2. EU’s waste hieraracy and circular strategies. Adapted from references [30,32].
Figure 2. EU’s waste hieraracy and circular strategies. Adapted from references [30,32].
Sustainability 12 05236 g002
Table 1. Topic and questions covered by the interview.
Table 1. Topic and questions covered by the interview.
NoTopicQuestionAnswered by
1WEEE in general.How does the case company perceive the WEEE regulative? All
2 How the PRO was selected, how does the PRO deliver? A, S
3WEEE reporting WEEE reports produced by the case company?A, S
4 WEEE reports received from other actors? A, S
5WEEE product documentation How are products labeled?QM, PA, S
6 What WEEE product information is produced by the case company?QM, PA, S
7 What WEEE product information is provided to the case company?QM, PA
8WEEE and circularity.Is the case company aware of the changes?All
9 Company changes due to WEEE changes?All
10 WEEE changes to support circularity?PA, PD, S
WEEE—waste electrical and electronic equipment, PRO—producer responsibility organization.
Table 2. Case study, summarized findings.
Table 2. Case study, summarized findings.
RQNorwayDenmark
WEEE in general.Very positive. Very positive to the selected PRO.Positive, hard to implement, updated too often, should be more harmonized.
WEEE reporting.Easy. Only focus on economic values. Few categories.More challenging. Weight-oriented. Many categories to report.
Calls for international standards.
WEEE product documentation.Little WEEE-specific documentation.Little WEEE-specific documentation
WEEE and circularity.Low.Unknown.

Share and Cite

MDPI and ACS Style

Andersen, T.; Jæger, B.; Mishra, A. Circularity in Waste Electrical and Electronic Equipment (WEEE) Directive. Comparison of a Manufacturer’s Danish and Norwegian Operations. Sustainability 2020, 12, 5236. https://doi.org/10.3390/su12135236

AMA Style

Andersen T, Jæger B, Mishra A. Circularity in Waste Electrical and Electronic Equipment (WEEE) Directive. Comparison of a Manufacturer’s Danish and Norwegian Operations. Sustainability. 2020; 12(13):5236. https://doi.org/10.3390/su12135236

Chicago/Turabian Style

Andersen, Terje, Bjørn Jæger, and Alok Mishra. 2020. "Circularity in Waste Electrical and Electronic Equipment (WEEE) Directive. Comparison of a Manufacturer’s Danish and Norwegian Operations" Sustainability 12, no. 13: 5236. https://doi.org/10.3390/su12135236

APA Style

Andersen, T., Jæger, B., & Mishra, A. (2020). Circularity in Waste Electrical and Electronic Equipment (WEEE) Directive. Comparison of a Manufacturer’s Danish and Norwegian Operations. Sustainability, 12(13), 5236. https://doi.org/10.3390/su12135236

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop