Radical Prosumer Innovations in the Electricity Sector and the Impact on Prosumer Regulation

Abstract

The electricity sector is in a transition towards a Smart Energy System where the roles of private and institutional actors are evolving. This work deals with the influence of some technological innovations, enabling social innovations such as peer to peer trading and the participation in local energy collectives, on the regulation of the rights and obligations of consumers and prosumers in the electricity sector. It identifies the main radical innovations in the electricity market and analyses the legal and related non-legal obstacles that may impede the empowerment of energy consumers and prosumers. Some recommendations are provided to ensure that consumers and prosumers are empowered and can benefit from these new technological and social innovations in the electricity market. The recommendations relate to an accurate definition of prosumers and active consumers, the integration of demand response, the evolving role of distribution network operators and the birth of peer-to-peer trading.

1. Introduction

The electricity sector is in transition towards a Smart Energy System. A Smart Energy System (SES) is characterised by some radical changes of the market design of the energy sector, affecting the basic assumptions of the current regulatory framework in the EU and its Member States. Among these new developments, ref. [1] (p. 23) first, a move from centrally generated electricity from fossil fuels to electricity generated (locally) from renewable sources which are more volatile and intermittent should be mentioned. Secondly, energy consumers are also becoming more active as they are able to produce electricity themselves, for instance by installing solar panels, and supplying it to the energy network becoming prosumers [2] (p. 1). Thirdly, the traditional demand-driven system will be substituted by a supply-driven system supported by the introduction of demand response (DR) programmes [3,4] and the storage of electricity by means of storage facilities connected to the distribution grid. Demand response (DR) can be implicit or explicit. Implicit DR means that consumers change their consumption patterns voluntarily in response to market signals calculated with dynamic pricing that reflects the real-time price for the electricity. The idea is that consumers could change their consumption patterns avoiding times of high prices. This would reduce peaks of demand which generates benefits for the whole system. The reward for consumers is direct since they pay less for their electricity bills. On the other hand, explicit DR means that consumers organized and helped by aggregators receive a specific reward for their flexibility and willingness to change their demand for electricity at a given point in time. Demand response is already operating in Europe, however its implementation in the household sector is still limited.

Thus, the electricity system is developing towards a Smart Energy System where the roles of the private and institutional actors operating on the distribution and retail markets are evolving and modifying. The objective of the regulation must be to not distort innovations and to facilitate the integration of new environmental-friendly and smart technologies, such as distributed generation, smart meters, blockchain technologies, storage facilities and electric vehicles into the electricity system. The integration of these new technologies in the electricity system facilitates the development of a more decentralised electricity system with new players where traditional utilities may lose relevance and peer-to-peer (P2P) electricity transactions become a reality. As will be discussed below, in this scenario, private law and competition law may acquire more importance under yet necessary safeguards of the remaining regulation to ensure, inter alia, the security of supply, the viability of the public network, protection of vulnerable consumers and non-discrimination regarding access to the energy network [5].

In this new context, Juncker’s Commission has enshrined the Energy Union as one of the priorities for his mandate. The objectives of the Energy Union are the diversification of sources of energy to grant security of supply, an internal energy market, energy efficiency, the decarbonisation of the economy and the support of research and innovation [6]. Consequently, DG ENERGY is particularly active in preparing the modification of the current European regulatory framework of the energy sector. The last proposed amendments constitute the Winter package “Clean energy for all Europeans” [7]. Its three main goals are “putting energy efficiency first”, “achieving a global leadership in renewable energies” and “providing a fair deal for consumers” [8]. The promotion of innovation is a leading thought behind the Winter package since the proposal seeks to “accelerate clean energy innovation” and “create jobs and growth in new economic sectors and business models” [9] (p. 3).

The European Commission thus observes the need for a recast of the current EU and national legislation. The existing framework is still based on the predominant generation technologies of the last decade, that is, centralised large-scale fossil fuel-based power plants and limited participation of (households and business) consumers in the energy system [10]. Consumers are still viewed as passive agents which undermines their capacity of producing and selling their own energy [11]. Moreover, the role that distribution system operators (DSOs) will play in the development of new activities is not sufficiently clear, [12] digitalisation and internet-based metering and trading innovations are forgotten disruptions which, however, will facilitate DR solutions [10]. These innovations shall also have an impact on new forms of (dynamic) contracts, the way that retailers operate and interact with business and households consumers. Technological innovations can also stimulate social innovation as households will make use of new forms of collaborations amongst themselves and with (new) market players. For instance, household consumers will be empowered and will have access to new options, such as aggregators operating as retailers, local energy communities or P2P digital platforms. However, current regulation does not (yet) sufficiently accommodate or may even impede these new innovations (See Section 3). A regulatory disconnection [13] exists, that is, an actual discrepancy between technological developments and the assumptions of the existing regulatory framework, which require a regulatory response to restore the disconnection.

In this scenario, the central question of this work is how do the above mentioned technological and possible social innovations impact some aspects of the regulation of the rights and obligations of consumers/prosumers in the electricity sector and how can regulation accommodate and facilitate these developments to the benefit of consumers and prosumers. It detects the main legal obstacles for consumers to become empowered to integrate in the electricity markets in the short and long term by benefitting from new services and by participating in the energy markets in new ways. This work, particularly, focusses on the ability of household consumers to become active consumers, including prosumers who are able to both consume and produce electricity [14] (See also Section 3.1 infra). For that reason, the work also deals with the role of DSOs, consumer protection provisions and the regulation of the retail market. The objective is to provide some regulatory recommendations for the introduction and integration of these (active) consumers and the related new technological developments and social innovations in the electricity sector.

Accordingly, this work will provide an overview of the main radical technological and social innovations in Section 2. Section 3 will firstly describe the legal obstacles that (active) consumers who make use of these new innovations face. In addition the contribution will also refer to technical and psychological obstacles that need to be considered by policy-makers, as they may impede new innovations and regulatory solutions and therefore are relevant when reviewing and adjusting regulations to facilitate innovations to the benefit of consumers and prosumers. The empowerment of active consumers may culminate in a future where peer-to-peer trading may become a reality. This section also deals with obstacles that prosumers and active consumers could encounter in this future scenario.

Once the obstacles have been detected, Section 4 will provide solutions and recommendations for the regulation of the new market design. Moreover, this section will build on the prospective future to observe and reflect how the tendency of the regulation could be. Section 5 sums up the main conclusions.

The methodology of this paper consists of a literature review of the main technological and social innovations that will impact the electricity market in the next years. It builds on state of the art literature on European and national electricity regulation and market design. This paper goes beyond this by identifying the main prosumer oriented innovations and by giving a detailed analysis of the legal obstacles and legal possibilities for the integration of those innovations in the energy market for prosumers and consumers, while at the same time considering the technical and psychological factors that need to be tackled by policy-makers when adjusting regulations. By way of a legal analysis, it will also assess the current and proposed regulation to evaluate whether it is ready or not to face the new challenges that the transition to a smart electricity system will place on the energy system for the integration of active consumers, including prosumers. The work will also adopt a normative approach to give some recommendations for future regulation.

2. Technological Developments and Radical Innovations

The EU Commission considers that the energy system has reached a “tipping point” [9]. Competitiveness of renewable sources soared and its share of electricity generation has increased [9,15]. The energy transition will lead to a more decentralised system where the paradigm of dominance of large companies, passive consumers and large-scale generation by coal and gas-fired planned disappears. It is, therefore, necessary to identify the technological innovations and market developments that are pushing the energy transition since this new market design will affect the way in which private and public actors relate to consumers in electricity markets.

The first evidence of the energy transition is reflected in the increasing number of prosumers, which enables consumers to develop new roles and participate and collaborate in the market in new ways [16]. The decreasing costs of solar photovoltaic generation, storage technologies and the increasing number of electric vehicles [17] is leading towards a new system in which consumers produce, use, store and sell their electricity [2]. Distributed generation increases energy efficiency since electricity lost in the grid is reduced because the location of the generation facilities and the location of energy consumption are closer. Nonetheless, the growth of distributed generation is also connected to increasing costs for the network, mostly due to increased network tariffs and taxes (financing the production of renewable energy), which translates into a raising of consumers’ electricity bills [18]. Furthermore, the increasing number of solar PV feeding energy into the grid and the increasing number of electric vehicle charging units demanding electricity during peak times complicate the control and operation schemes of the DSOs, as there may be more local imbalances and local congestions that need to be solved [19]. DSOs will have to upgrade their grids by adding new ICT layers, enabling them to transport bi-directional power flows and to balance demand and the more volatile supply in an optimal way [2]. In short, they have to ensure that their distribution grids are becoming smart grids. The term smart grid refers to “a cluster of innovations in metering and communications which allow for more distributed and small-scale generation, more efficient customer consumption decisions, and more efficient and higher utilisation of network resources” [20].

Secondly, the European Commission predicts that close to 200 million smart meters for electricity will be rolled out in the EU by 2020 [21]. The deployment of smart meters facilitates the integration of small prosumers to the grid and the development of DR. Smart meters provide real-time information on cost and usage which are expected to encourage consumers to reduce their demand and switch their consumption patterns to off-peak hours. As will be explained below, they are also essential for the implementation of other innovations such as storage facilities and P2P networks.

In the third place, new possibilities for home storage solutions, distributed generation, smart-vehicle battery-charging strategies and home automation technologies enable consumers to optimize their electricity use and match it with their consumption needs [2]. These active consumers can deliver flexibility services to the markets, by using less in times of an overload of the system and by using more when there is ample energy and network capacity. This flexibility also results in a more optimal use of distribution networks. The idea is that it takes away the need of DSOs to enlarge the network capacity to such extent that all possible (maximum) demand and load peaks can be dealt with, keeping the network tariffs more affordable as well [12].

Fourthly, digitalization will also play a key role by lowering barriers to entry for allowing consumers, local communities and small start-ups to lead the development of energy and related social innovations, for instance by trading energy on P2P digital platforms and by the participation in DR with the option of remote control of energy use [9,22]. The combination of blockchain technology together with the rapid growth of decentralised energy supply bears a considerable disruptive potential for the retail markets and make P2P trading a tangible possibility. Blockchain technology may detach financial transactions and the execution of contractual commitments from a central control unit which, constitutes a threat for the business of traditional utilities [23]. Thus, a next disruption is expected to involve digital platforms, which could make use of blockchain technologies, in which consumers could trade P2P [24].

In the fifth place, although it is not a technological innovation in the strict sense, the role of some relevant actors of the electricity market must evolve and adapt to the new circumstances. After the legal and administrative unbundling requirements that the Third Package mandated for DSOs, a new development of their tasks and functions is required. As it will be explained below (See Section 3.1.2 infra), they ought to become more active managers in order to integrate flexibility services and prosumers, contributing to the new market design in their roles as neutral market facilitators.

3. Towards a New Market Design: Obstacles for Prosumers’ Integration

Nonetheless, the introduction of these technologies and social innovations also raises a number of issues regarding the regulation of the electricity market. This section will focus on the legal and some non-legal obstacles that prosumers face when integrating in the new smart and decentralised electricity system to see how EU and national policy makers could deal with them.

3.1. Legal Barriers for the Integration of Prosumers and Active Consumers

3.1.1. The Concept of Prosumers

As discussed above, prosumers are consumers which not only withdraw energy from the network, but also produce and supply energy into it [11]. Even if they must, generally, be considered as consumers, prosumers participating in energy efficiency and especially demand response programmes are often behaving more like producers than consumers since they provide services that the market needs in exchange for compensation [25]. The definition of prosumers often describes consumers or other agents that rely on smart meters and solar PV panels to generate electricity and/or combine these with home energy management systems, energy storage, electric vehicles and electric vehicle-to grid (V2G) systems [2].

Nonetheless, a first legal barrier that they encounter is that many national regulatory frameworks in the EU do not define who fall within the concept of prosumer yet. The consequence is that their rights and obligations have not been defined and diverge between Member States. The Winter package provides a definition at the EU level for the first time. Nonetheless, in spite of the reception that this term has in the literature on energy law and economics, the Winter Package prefers to use the term active customer [26]. Article 2(6) defines active customer as: “a customer or a group of jointly acting customers who consume, store or sell electricity generated on their premises, including through aggregators, or participate in demand response or energy efficiency schemes provided that these activities do not constitute their primary commercial or professional activity”. It clarifies that the abovementioned commercial activities cannot constitute the “primary commercial or professional activity” of the consumer. This casts some doubts on the ability of active consumers to place competitive pressure on electricity suppliers, [27] which seems to be one of the objectives of the Winter Package [26].

Prosumers, unlike traditional consumers, are more autonomous and participate in a more active and diverse way in the market [25]. Prosumers take more control of their consumption decisions, either taking active steps to regulate their consumption or engaging in self-supply. Moreover, they act as market participants by selling the excess of energy generated to their local utilities, by bidding energy storage services into ancillary services markets, and by receiving compensation through demand response programs [25]. Regulation is then expecting that prosumers become an active and relevant actor in the market.

Demand response programmes provides a way to integrate prosumers and household consumers in the electricity markets as active participants. However, as will be discussed below, there are still some legal and relevant technical problems to allow a massive participation of small prosumers in demand response programmes through dynamic retail pricing (Implicit DR) and the use of aggregators (Explicit DR) which will be explained below.

3.1.2. Legal Barriers for the Introduction of Demand Response (DR)

With regard to implicit demand response, dynamic retail pricing means the calculation of the electricity costs based on real time pricing. It reflects the marginal network costs and/or generation costs of electricity in the wholesale market (See on this [28,29]). Currently, there is significant variation in the penetration of dynamic pricing among households in Europe. So far residential consumers can only take advantage of dynamic pricing in the Spanish, Nordic and Estonian markets [3].

There are several reasons that still impede the penetration of dynamic pricing [3,4]. First, the influence of non-energy components could provide wrong signals that can blur the benefits of dynamic pricing, unless they are also dynamically designed [3]. There is not a prohibition in the regulatory framework that prevents dynamic network tariffs, however they are still not being applied [1] (p. 100). The reason may be that they are complex, as well as dynamic pricing is not a panacea since it can also have socially adverse redistribution effects, for instance for vulnerable consumers with less flexible consumption patterns or less access to flexibility sources [18,30]. In addition, the creation of more individualized data by smart meters raises questions about their treatment [31]. This could create potential problems with the right to data privacy, if unmanaged. CERRE 2017 warns that “thanks to smart meters and demand-response programs, large quantities of precise information will be collected […] Consumers will progressively lose the initial informational advantage on their electricity needs […] and service suppliers will be able to extract more rents from consumers […]. Another question(s) are […] Will it be legal to sell data on consumption profiles? Will data on profiles be viewed as an essential facility for new entrants?” [32] (p. 39).

Regarding explicit demand response, the number of prosumers has increased together with the number of aggregators. Article 2(14) of the proposal for the revised Electricity Directive defines an aggregator as a market participant that combines multiple customer loads or generated electricity for sale, for purchase or auction in any organised energy market [26]. These companies offer their services as middlemen between prosumers and electricity markets, [25] for instance they aggregate many small demand response commitments and bid them as a package into retail or wholesale electricity markets. Aggregators capitalise on economies of scale and scope and the savings that they get are passed on their clients retaining a fee [33]. The aggregator can act as an incumbent electricity supplier offering aggregation services, a service provider specialized in aggregation services collaborating with a supplier, a joint venture between a traditional supplier and a service provider or as an independent market actor (independent aggregator) [34]. In addition, regulation can permit opt-in models where the consumers choose to join the services provided by an aggregator or opt-out models where the consumer belongs to the aggregation scheme unless he expresses he does not want [35,36]. It is a regulatory decision to make the abovementioned business models available for consumers [37,38,39]. The value of aggregators in the retail market is that they offer a service to consumers that do not have the means and knowledge to trade in the market. However, explicit DR still encounters some regulatory obstacles in Europe [37]. First, in some markets demand-side resources cannot access the market since they are closed to explicit demand response. This is for instance the case in the balancing markets in Spain or Italy, as well as in the re-dispatching market in Germany. Furthermore, in the Netherlands, aggregators can only participate in the market through balancing responsible parties (BRPs) [37]. Moreover, some countries (Polonia and Slovenia) have not opened their markets to aggregated load, only allowing industrial customers to access the market. In addition, some countries have still neither defined what an independent aggregator is, nor clarified its role under a principle of fair competition that let them access the market without an agreement with the consumers’ retailer.

3.1.3. Evolving Roles of DSOs

In this new market design, where consumers become more active, DSOs are one of the key contract partners of the prosumers and active consumers. The DSO’s key task [40] (Article 25) is the responsibility to ensure the long-term ability of the distribution system, as well as to manage a secure, reliable and efficient electricity distribution system [12]. Furthermore, DSOs usually are also responsible for measuring the energy used and produced by household consumers and usually make measurement devices available for small connections [12]. Prosumers and distributed generation together with the roll-out of smart meters ought to change the role of DSOs vis-à-vis the consumers to a more active system manager [41]. Nonetheless, this should not change core responsibilities of DSOs (e.g., maintaining grid security and ensuring security of supply).

There is also wide consensus among policy-makers that DSOs are well suited to act as neutral market facilitators in the transition towards a Smart Energy System [42]. This notion expresses the idea that the DSOs can enable and facilitate the connection of flexibility services to their grids to develop local flexibility markets in a neutral way, for instance by sharing data on the availability of flexibility sources in an independent way. Flexibility can be seen as the modification of generation, injection and/or consumption patterns in reaction to an external signal in order to provide a service within the power system [12]. The evolving DSOs’ tasks related to the role of neutral market facilitator are not addressed by the current European and national legislative frameworks [42].

Moreover, as it is expected that more imbalances and congestions may occur locally, DSOs may also assume part of the transmission system operators (TSOs)’ main task which is the balance of demand and supply. Accordingly, DSOs could locally balance demand and supply and apply local congestion management in the distribution systems [42]. This development requires more coordination and cooperation between the DSOs and TSOs.

3.1.4. Legal Barriers for the Birth of Peer-to-Peer (P2P) Networks

In a decentralised electricity market, where active consumers and prosumers are progressively being empowered, the technological process seems to head towards a paradigm where these consumers could buy and sell electricity among themselves. In April 2016, the project Transactive Grid did the first P2P exchange of energy between five homes in Brooklyn, NY, USA. The excess of energy produced on one part of the street was sold to the other part without the need of a utility Transactive Grid helped to manage the microgrid that supported the transaction and used a blockchain software run by Ethereum project. After this first pilot, the US start-up LO3, responsible of the Transactive Grid project, is collaborating with Siemens to develop blockchain microgrids [43]. Blockchain is a decentralized internet protocol that facilitates transactions, such as the sale of electricity, directly between peers without an intermediate institution. In the energy sector, decentralised storage of transaction data could increase security and may ensure greater independence from a central authority [24]. It can also help to digitize contracts, to manage digital content and to execute trades. Nonetheless, unlike the financial sector, where blockchain is firstly being developed, electricity requires a network infrastructure to be transported. Thus, even if it is possible to conceive a future with a majority of autonomous off-grid and self-sufficient agents, it is more likely that prosumers are still connected to a (micro)grid where they sell their excess of electricity [44].The emergence of blockchain technology [24,45] and the raising of prosumers with storing mechanisms facilitate the booming of P2P markets [2] where prosumers, and consumers, could buy and sell energy between each other without an intermediary.

Parag et al. envisages three possible models of prosumer-integrated markets [2]. P2P models strictu sensu interconnects prosumers directly with each other wherever they are. These models are the less structured ones. A P2P platform would allow prosumers to bid and sell electricity directly between each other. Secondly, prosumer-to-grid models connect prosumers to a local microgrid where they buy and sell their services. There would be brokerage systems where prosumers would sell the electricity to a central (local) market where other prosumers and consumers would buy it. Thirdly, in organized prosumer group models, groups of prosumers aggregate their resources in pools and buy and sell to other prosumer groups. This model is more structured and centralized than the other two. Accordingly, the first model requires a larger number of contractual relations and the development of its network is more costly. More decentralisation also entails a more uncertain attribution of liability and accountability for the energy services. Those problems are less relevant in the other two models where the use and demand of energy is more aggregated. Nevertheless in the last two models, prosumers mainly provide a service to the grid, rather than becoming a competitor for generation [2].

Even if P2P electricity trading is still in an embryonic state, it allows to conceive a future where the electricity could be sold for a trading period (30 min or 1 h) between consumers/prosumers without the intervention of a traditional utility. This raises some legal questions on how the energy market could be and how the new roles, responsibilities and rights of market participants evolve.

The first question that P2P networks raises is whether the prosumer that sells the excess of energy to another peer is a business or still a consumer. With the possibilities to store energy and to speculate with the price of electricity through storing capacities, the prosumer that sells the excess of energy that it does not use might be more similar to a business. This question is also being discussed in the context of the shared economy and collaborative consumption [46]. In the EU, the ECJ will have to solve a preliminary reference made by a Spanish Court (C-434/15—Asociación Profesional Elite Taxi) which could solve some of the questions about the sharing economy. If prosumers were considered businesses, they would have to comply, inter alia, with stricter consumer provisions when they sell the electricity or with a different tax regime. This could also result in prosumers becoming balance group managers and submitting their own demand forecasts to the network operator [24]. This would place an unjustifiable burden on the prosumers that would likely prevent the rise of P2P trading.

A second issue is that it is possible to conceive some self-sufficient microgrids, albeit it is more likely that they are still connected to a macrogrid to ensure the security of supply. A “manager” should demand energy to the macrogrid to grant the supply in the microgrid when the latter is exhausted due to the intermittent flows of renewable generation. Regulation ought to designate who should be this microgrid manager. Taken the role of the DSO in ensuring a secure and reliable operation of the distribution grids, it would not be strange to give that function to the DSO.

A third question concerns the elimination of intermediaries. Even if blockchain technology could help to eliminate the retailer in some transactions, others may appear, such as the digital platform that keep peers in contact. Moreover, every private transaction between peers would have to be levied for the use of the DSO’s infrastructure. For instance, [47] proposes a model where DSOs charges a fee for the use of its infrastructure. In this model, the supply company or aggregator earns a small fee for operating the blockchain-based platform that connects peers and consumers still pay a retail tariff, including taxes and distribution charges. Blockchain technology could help as the ledger for all those transactions.

The introduction of these new digital platforms in the market raises some concerns as well. They may be a retail market platform or central backbone which will make profit of interconnecting sellers and buyers of energy (in exchange for a fee per transaction or per kWh) such as Uber or Airbnb do in other sectors. Nonetheless, the transaction that peers negotiated through the platform would be different and add to the transaction that the peers make with the DSO to transport the electricity fed into the grid to the demand point. Thus, the role of the peer-to-peer platform at least in a first stage could be more to complement than substitute the incumbents, which will still participate in the market and ensure the non-P2P transactions [48]. The introduction of new business models would inevitably reduce the gains of some incumbents, such as traditional suppliers, unless they are the ones that operate the new platforms. This could lead to a “death spiral” of current utilities when new businesses thrive [49,50]. It could also create unfair competition problems for P2P platforms in the energy sector, as Uber is experiencing in the transport sector, due to the considerable market power that some traditional suppliers still have in some countries.

An important question to answer is thus the legal nature of these digital platforms. Would the platforms and their members be subject to the same obligations as energy suppliers? Would they be considered an “information society service” that keeps consumers in contact? Advocate General Szpunar considers that Uber falls within the field of transport, rather than information society services, and therefore it can be subject to licenses and authorizations (See C-434/15 Asociación Profesional Elite Taxi v Uber Systems Spain, SL (2017) ECLI:EU:C:2017:364). The consequence of being considered as an “information society service” would mean a different application of the freedom of services. In that case, licences and authorizations by Member States would be more difficult to justify.

As a last remark, a second stage of booming of P2P transactions also suggests that the incredibly high number of transactions that would be generated between peers would have to be automatized. As in financial markets, the parties may have to define a trading strategy (for instance, more flexible consumers (more risk) may be able to make more profit selling energy), which may be set in computer code. The transactions would then be executed impartially by the computer matching buy and sell orders. Potentially, consumers could delegate their trading functions to professional managers that could manage their “electricity portfolio”, as investment funds in financial markets. This may require a market regulation similar to a stock exchange providing a framework contract for some microgrids which would take into account possible contingencies and would establish obligations of the parties involved. It is likely that those framework contracts have to be approved by a public authority to provide the rules that would guide the platforms and designate the manager that secures the balance of supply and demand.

3.2. Non-Legal Barriers for the Integration of Prosumers

In addition to legal obstacles, there are also other technical and psychological obstacles for the integration of prosumers in the electricity markets that need to be considered by policy-makers when adjusting electricity market regulation. One important barrier is technical, and relates to the absence of metering equipment that allows to have access to real time user data to calculate the price of energy use. Smart meters equipped with some functionalities are an essential development to make, inter alia, dynamic pricing available to consumers. The current EU deployment will make dynamic pricing available for more consumers, although some countries, such as Portugal or Czech Republic, have decided not to participate in the current EU roll-out for economic reasons [21]. These countries are required to assess the cost-effectiveness of a large-scale smart metering deployment on a regular basis. Although the confidence of European Commission in smart meters is high, academics have also raised some problems such as excess of information for consumers, the possibility of suppliers to discriminate between price-sensitive consumers or that consumers do not sufficiently react to information provided by smart meters [27].

Moreover, consumers also face psychological and behavioural barriers. While it is certain that the abovementioned developments substantially impact the market design of the electricity sector, it still remains uncertain to what extent and within which time limits these developments will be realized. The integration of new technologies and possible social innovations in the electricity system will to a large extent depend on the consumers’ acceptance of these technologies [51,52]. For instance, consumers’ lack of information and awareness of the possible benefits of dynamic pricing is an important barrier to the introduction of demand response. Current market situation suggests that they might prefer stability in pricing over a financial reward for adjusting their consumption. This may be suggested by the increasing presence and popularity of fixed contracts among household consumers in several European countries [4,14]. In addition, the benefits of demand response, such as dynamic retail pricing, depend on the characteristics of the market, such as the volatility of wholesale price. For example the shifting potential of Scandinavian countries, where volatility of wholesale prices is higher, is not observed in other EU countries (See on volatility of prices in Europe, [4,53]). Thus, while energy consumers are expected to actively participate in the energy market, by informing themselves about alternative energy (service) contracts and by consciously making choices about their energy contracts and energy use, empirical studies into consumer behaviour in the energy markets show otherwise [14,54,55,56,57]. Various academic research, as well as studies, conducted in the energy sector indicate that, when entering into energy supply contracts, consumers remain very often inactive or fail to make optimal choices in terms of price and quality of the energy supply contracts [14]. In addition to mal-functioning retail markets, this is partly the result of the psychological factors that influence consumers’ behaviour, such as the status quo bias, loss aversion and bounded rationality (See on these factors: [58,59,60].

3.3. Observations

In light of some of the problems discussed above, a crucial question, therefore, is whether the legislator can reasonably expect the energy consumer, who already experiences severe difficulties to properly assess the complex information concerning contracts in the market for traditional energy supply, to become more active and make well informed and optimal choices in a Smart Energy System. Even if some legal obstacles for the introduction of dynamic pricing and the use of aggregators are removed, prosumers still encounter psychological bias to become more active. In addition, in the new decentralised system, DSOs are increasing their relevance for the integration of active consumers in the market. A new configuration of their functions and responsibilities seems necessary to facilitate the introduction of new innovations and eventually let P2P networks appear. It is precisely the path towards P2P trading which could disrupt the electricity markets and the way energy prosumers and consumers collaborate and trade in the next years. Nonetheless, P2P trading is still in an embryonic state and has to respond several questions. How many consumers will become a prosumer engaged in P2P trading? Or how many consumers will actually go off-grid? In the electricity market, where incumbents’ business models depend largely on the regulatory framework and large investments are necessary, the introduction of new innovations and new paradigms are usually progressive and require pilots and serious tests. Thus, in the short-term future, a significant part of the consumers will still depend on big power stations and big grids, especially in the winter. It should be recognized, that the world where distributed generation and small microgrids can do it all is still far, although they are already able to disrupt the markets and regulations that currently govern the power system [19]. ‘Business as usual’ could lead to malfunctions, with grid congestion, expensive short notice network upgrades and inadequate generation at peak times [19]. The objective of the next section is to provide some recommendations to the regulation of electricity markets that takes into account the legal and relevant non-legal obstacles for the integration of active consumers and prosumers detected in this section.

4. How to Deal with the Obstacles for the New Market Design?

4.1. Dealing with the Rights and Obligations of Prosumers

The Winter package tries to provide a legal framework for the protection of prosumers. As mentioned above, Article 2(6) of Proposal for the amendment of the electricity Directive defines active customer as a “customer or a group of jointly acting customers who consume, store or sell electricity generated on their premises, including through aggregators, or participate in demand response or energy efficiency schemes provided that these activities do not constitute their primary commercial or professional activity” [26] (Art 2(6)). If the active customer consumes, stores or sells the energy which is generated within his or its premises, including a multi-apartment block, a commercial or shared services site or a closed distribution system, then the active customer is according to the proposal for a revised Renewable Energy Directive also a renewable self-consumer [61] (Article 2(aa). Article 21(1)).

The Winter package assumes the necessity to protect prosumers since they contribute to a clean energy system and they could encounter some retaliation by traditional utilities, which could face a financial ‘death spiral’ [49,50]. Thus, the active consumers, including renewable self-consumers, are granted some specific rights. First, according to the proposal for a revised Electricity Directive they cannot be subject to disproportionately burdensome procedures [26] (Article 15(1)). Moreover, they shall be subject to cost reflective, transparent and non-discriminatory network charges, accounting separately for the electricity fed into the grid and the electricity consumed from the grid ([26] Article 15(1)). In addition, the proposal stipulates that their distributed generation facilities may be managed by a third party for installation and operation, including metering and maintenance [26] (Article 15(2)). According to the European proposals active consumers can also participate in the markets through local energy communities, [26] (Article 2(7)) which are also protected due to their specific characteristics in terms of size, ownership structure and the number of projects that can hamper their competition on equal footing with large-scale players ([61] Recital 55.).

In addition, prosumers which are also self-consumers are entitled to be part of power purchase agreements ([61] Article 2(cc)) to maintain their rights as consumers, and not to be considered as energy suppliers if they feed into the grid less than 10 MWh for households and 500 MWh for legal persons on an annual basis ([61] Article 21). Perhaps, it is not the best idea to regulate the amount of energy at the EU level. Energy generated by SV and wind varies depending on the weather conditions which are different in every Member State. A better approach may be to regulate a maximum and allow MS to establish lower values at the national level. Prosumers are also entitled to receive remuneration according to the market value of the electricity fed in [61].

4.2. Dealing with the Evolving Role of DSOs

The Winter package also acknowledges the evolving role of DSOs. The proposal for the amendment of the Electricity Directive conceives that DSOs play a key role in facilitating the flexibility services delivered by prosumers, aggregators and demand response to reach the market in a transparent and non-discriminatory way ([26] Article 32). The assumption of the European proposal is that the DSO procures the services for an efficient operation of their networks, but they do not compete in delivering these services themselves [62]. An important, and sometimes forgotten, aspect for the introduction of competition in retail markets is data management [63]. The Winter Package addresses this issue leaving Member States leeway to designate the eligible parties that have access to the data (including metering and consumption data) ([26] Article 23). Nonetheless, the Commission reserves the competence to regulate a common European data format and non-discriminatory and transparent procedures for accessing the data ([26] Article 24). The European proposals stipulate, that Member States also decide which data management model will apply provided that they ensure efficient data access and exchange in a transparent and non-discriminatory manner [26]. Thus, it opens the possibility for DSOs to have access to consumption data and act as data manager by providing to any eligible party, with the explicit consent of the final customer, access to data of the final customer [26]. It could be argued, that in their role as neutral market facilitator, DSOs are in a good position to manage commercial data that could help integrate flexibility services under the supervision of regulatory authorities [12] (p. 29).

The Winter package also recognizes that DSOs and TSOs need to enhance their cooperation in system operation, investment planning, data exchange and in ensuring the security and safety of the system ([10,42] (Article 51(2) and Article 32). The Winter Package also clarifies the role of DSOs beyond their core functions [1,12]. The DSOs’ core function is to transport the electricity from the distribution network to the consumers. They have to operate and develop their distribution networks to grant long-term viability of their networks. Some examples of non-core activities are the provision of flexibility services (storage services, demand response programs) or infrastructure provision for electric vehicles. In theory, most of these activities are, or can be, competitive. Even if there are good arguments to defend that system operators must confine themselves to their core functions since DSOs have network monopolies that could potentially distort competition ([42] pp. 6, 9–11), the evolving role of DSOs and the urgency of the energy transition could provide arguments to leave some leeway to system operators to innovate by developing other tasks under some strict conditions [1,64]. It could be argued, that DSOs should be allowed to fulfil competitive non-core functions temporary to stimulate the energy transition and promote competition, provided that they are supervised by national authorities and are sufficiently unbundled from commercial players in the market. From this perspective, the Winter package stipulates that DSOs shall not own, develop, manage or operate energy storage facilities or recharging point for Electric Vehicles, unless others have not shown their interest -following an open and transparent tendering procedure- and the National regulatory authority has assessed or approved that a derogation is necessary [26] (Articles 33 and 36).

If the DSO takes on new roles, sufficient control mechanisms and structural measures will be required to ensure that it does not use access to commercial data to gain undue competitive advantages or create market distortions [12] (p. 27). There is consensus around the need to impose a minimum of unbundling requirements on DSOs active in competitive downstream markets which are supervised by the national regulatory authorities, but not around their scope and depth across the Member States. The legal and administrative unbundling requirements have not been strengthened so far by the European proposals [12] (p. 27). In any event, national authorities should balance carefully the short and long term interests of the consumers in delineating the DSOs’ core and non-core tasks, considering the relevant economic, social, legal and technological circumstances of each region and DSO [26,65] (Articles 33 and 36).

4.3. Dealing with the Introduction of Implicit Demand Response

As explained above, demand response can be implicit or explicit. Both permit consumers to participate more actively in the market. Demand response is already operating in Europe, however its implementation in the household sector is still limited. Once the obstacles have been identified, some recommendations are provided to an adequate introduction of this development in the energy markets.

First, the Winter package gives active consumers an entitlement to a dynamic electricity contract, meaning an electricity supply contract between a supplier and a final customer that reflects the price at the spot market or at the day ahead market at intervals at least equal to the market settlement frequency [26] (Article 11). However, it should be reminded, that with regard to implicit demand response, as explained above, the characteristics of the market influence the benefits of dynamic pricing. This advises against a “one size fits all” approach and justifies that market mechanisms introduce dynamic pricing step-by-step when consumers preferences really demand it. For instance, benefits of dynamic prices without automatization are less relevant. The project EU Ecogrid showed that a customer with manual control gave a 60 kW total peak load reduction, while automated or semi-automated customers gave an average peak reduction of 583 kW [66]. Retailers should be free to design tariffs that include dynamic pricing but consumers should be free to choose between tariffs that do not fluctuate during the day since they may not be able to change their personal circumstances and daily routines [67].

Secondly, it is important to inform consumers about the possible costs and benefits of dynamic retail pricing. BEUC proposes an information campaign targeting different consumer segments and coordinated by consumer organizations and national regulatory authorities [67]. The complexity that dynamic pricing introduces in the calculation of electricity prices may increase the importance of the role of consumer protection provisions, including transparency requirements. Although specific consumer protection provisions already exist, [40,68] the Winter package introduces new obligations regarding the billing requirements, [26] (Article 18) it prohibits all switching related charges, except for early termination fees on fixed term contracts, [26] (article 12) and recognizes a right to free access to at least one certified energy comparison tool [26] (Article 14). Nonetheless, the success of these provisions may depend more on the development of consumer law in the Member States and the monitoring of the market by national regulatory authorities and consumer associations rather than their recognition at the EU level. European Commission justified the introduction of these new provisions after a study [69]. However, as that study also shows, the problem sometimes is not the lack of regulation but that electricity companies do not comply with the information requirements which are already in the regulation and a lack of private and public enforcement. The relationship between the consumers and the retailers in a desired liberalized market, although special, in principle should be governed by private law and general consumer protection provisions rather than introducing detailed ex ante regulatory requirements of, inter alia, the billing conditions. Consumer law, as it is in the European Union, has enough provisions and mechanisms to deal with, for instance, the transparency or unfairness of the clauses to switch from one provider to another. Even if the role of national consumer law is still very important, ECJ has been lately very active in the protection of consumers with regard to financial-related contracts which are complex contracts like the electricity supply contracts [70]. Though, more detailed, special consumer protection provisions seems to be justified in the initial stage of the transition towards a Smart Energy System; as competition flourishes and consumers may be more empowered, the role of these special provisions may diminish and the role of general consumer law (and competition law) may grow in protecting energy consumers.

In the third place, consumers can only benefit from dynamic retail pricing, if they have the access to a smart meter and a dynamic price contract. The Winter package requires that consumers who do not have the metering equipment which is necessary to engage in dynamic pricing should be entitled to a smart meter with that functionality, although they may need to bear the cost of the installation [26] (Articles 21 and 11). Thus, the access to dynamic pricing is also projected in countries where a wide roll-out of smart meters has not been planned. In addition, the roll-out of smart meters should be combined with a scrutiny of the market by data privacy authorities too.

In regard to the contract offers, retailers should have contractual freedom to design the offers that include dynamic pricing, which is in harmony with the principle of free competition in the retail markets [3]. They are better placed to target consumers which are prone to this kind of contracts. Nonetheless, under certain conditions, a market intervention by the regulatory authority can be justified in the general economic interest; it should comply with the principle of proportionality; and the public service obligation should be clearly defined, transparent, non-discriminatory and verifiable; as well equal access for EU electricity companies to consumers should be guaranteed (See Case C-265/08 Federutility and Others v Autorità per l'energia elettrica e il gas. (2010) ECLI:EU:C:2010:205).

In the Spanish case, those consumers that still do not have a smart meter which is able to calculate dynamic pricing are priced according to the consumption patterns of an “average consumer” which is calculated by the TSO. Member States could be entitled to allow transitory regulated tariffs that could help to calculate the tariffs for those consumers that do not have a smart meter yet and protect vulnerable consumers. According to the Spanish regulator, the Spanish experience, where the consumer can select to stay in a dynamic regulated tariff, which is calculated by the government and paid to the five reference suppliers, or switch to the liberalized market where they negotiate the price of kWh and the conditions directly with the supplier, has shown that the regulated tariff was the cheapest option in times where the energy prices were low [71]. However, this is not the Commission’s desired option [66].

In addition, dynamic pricing will probably only work when the energy component has a sufficiently high share in the electricity bill to provide incentives to the energy consumers. Otherwise non-energy components, such as network tariffs, will probably blur the benefits of dynamic pricing [3]. Nonetheless, it is possible to design dynamic network tariffs that could avoid distortions on the consumer’s perception of the energy component (Portugal and France) [72] and at the same time it allows DSOs to recoup their costs in a context where their role also increase. The onus is on the national regulatory authorities that are responsible for setting tariff methodologies for access to energy networks because current European legislation includes only the main principles for network regulation, namely the principles of transparency, non-discrimination, objectivity, flexibility, non-distortion, cost recovery, cost-reflectiveness and predictability [73]. With regard to tariff design, the Winter package only emphasizes the relevance of the flexibility principle, since Article 16(1) of the proposed amendment of the Electricity Directive adds “and flexibility” as a principle to regulate network charges, and the mandate for national authorities to incentivize consumer participation in Demand Response Programmes [10]. It also clarifies how some of these principles should be applied [11,26] (Article 16). Thus, the recognition of these principles at the EU level leave the national regulatory authorities sufficient leeway to design the tariff methodologies that fit best both geographical conditions including the extent of introduction of distributed energy in a certain area [11]. A fairness element should also be taken into account since there is a risk that consumers with less access to flexibility are charged higher tariffs, while prosumers that can access to storage solutions and temporarily go off-the grid contribute less to the maintenance of the network [11].

4.4. Dealing with the Introduction of Explicit Demand Response

Regarding explicit DR, regulatory burdens that restrict aggregators to operate independently from the consumer’s retailer, which is potentially its competitor, must be removed [37]. In this sense, it is welcomed that the mandate of the Winter package ensures that the national regulatory framework encourages the participation of aggregators in the electricity retail market. Article 17(3) of the proposal for a revised Electricity Directive recognises a right for each aggregator to enter the market without the consent from other market participants and the prohibition of requiring them to pay a compensation to suppliers. It also emphasizes the need of transparent rules in regard to data exchange and the creation of resolution mechanisms between market participants [26]. Nonetheless, Member States should not wait for the Directive coming into force to remove these obstacles. In addition, the Winter package also acknowledges the right of consumers and small businesses to participate in the market through aggregators subject to compliance with competition rules [26] (Article 3(d)). DSOs should not act as an aggregator, as according to unbundling requirements they should not participate in the electricity supply market [39,40] (Article 27). Moreover, about the possibility of introducing local opt-out schemes in Europe, they should not be forbidden and some pilots would be welcomed to analyse the possible effects. Nonetheless, under an assumption that consumers should not be tied to goods and services that do not reflect their personal circumstances, opt-in schemes are more advisable. BEUC recommends opt-in schemes [67].

4.5. Dealing with the Birth of P2P Trade and the Tendency of the Future Regulation

Although many questions remain unanswered, P2P, implying technological and social innovations, might become a reality in the next decade. After Section 3 has envisaged how a P2P trading scenario could look like and some regulatory obstacles that its birth and rise confront, this section tries to provide some recommendations to integrate adequately this development in the energy system as a way to fully empower consumers.

In a first stage, regulation should eliminate some regulatory constrains to promote P2P systems, such as licensing requirements to supply electricity ([13] detects some of them in Dutch Energy law, for instance, the impossibility of supplying energy to small and medium consumers in the absence of a license). Those licences generally try to grant a minimum of organizational, financial and technical characteristics for a good performance of the tasks as energy suppliers [13]. It is thus to be welcomed that the Winter package prohibits self-consumers which feed into the grid less than 10 MWh for households and 500 MWh for legal persons on an annual basis from being considered as energy suppliers. It implies that prosumers that would engage in P2P trading would not lose their rights as consumers and thus they would not have to comply with the stricter requirements of business-to-consumer (B2C) regulation and more severe tax regimes.

The prohibition of being considered as energy suppliers also suggests that prosumers that would like to engage in P2P trading would not have to meet Universal Service Obligations (USO) [40] (Article 3(3) and 3(7)). Nonetheless, in a context where P2P digital platforms thrive and more consumers have access to facilities to sell energy to consumers directly, future research should evaluate whether these exemptions could create an uneven playing field. This could be the case where P2P platforms capture the most profitable consumers, while traditional supply companies, which face the USO, are left with the most expensive consumers, for instance, island consumers who do not enjoy economies of density and may not have access to a liquid P2P market [74]. Nonetheless, this problem would be less relevant if the traditional suppliers and those who runs the P2P platforms are the same entity. In this case they would have just to adjust their business models. However, it is expected that some business models only participate in the P2P market. The idea for introducing a USO is protecting unprofitable, and thus weak, consumers from exclusion in a liberalised market [75]. It should be examined whether prosumers engaged in P2P platforms could cherry pick the consumers that can have access to their service while traditional suppliers would have to supply all the consumers that demand them. A relevant question therefore seems to be how the USO ought to be designed in a context of P2P trading where prosumers would not be capable of facing that requirement and there would be consumers that would not have access to liquid P2P markets ([75] provides different designs of the USO and the way to finance it).

Nonetheless, it is easier to visualize that the role of regulation in the retail market seems to evolve towards more relevance of private law, namely, contract law, consumer law and competition law, still within the framework of necessary safeguards of ex ante regulation to ensure security of supply, protection of vulnerable consumers and the viability and safety of the network. The reason is that prosumers that sell their energy to other peers would have to deal more with terms and conditions in the contracts which they would sign with other parties or digital market platforms, rather than with the security of supply in general, since non-specialised prosumers/consumers are not the ideal market players to deal with this. The context of more relevance of private law suggests that energy regulators may have to become more interdisciplinary and take into account not only regulation but also competition concerns, data privacy and general consumer law. This could generate arguments to change the institutional model of energy market regulation towards a model of a multiple interdisciplinary agency, such as the ones already in place in UK, the Netherlands, Spain or Finland (It must be raised that the models are quite different among the cited countries [76]. This agency would be in the position to assess which is the most appropriate legal regime to deal with legal hurdles that consumers may face, and apply the rules that are most efficient and effective considering the development of competition and consumer empowerment in the energy markets.

5. The Future of Electricity Regulation: Conclusions

This paper has sought to answer the central research question what are the main influences of the prosumer oriented technological and social innovations on some aspects of the regulation of the electricity sector and how can regulation accommodate and facilitate these developments to the benefits of consumers and prosumers. It has been made clear that in the context of a transition to a more sustainable energy system, consumers may be part of social innovations in which they will play more important and active roles in a new decentralised energy system. Some technological innovations, such as blockchain technology and digital platforms, ought to help them to achieve these new roles where they become more active by, individually or collectively, storing, buying, producing and selling electricity. Thus, the role of regulation should be to integrate these new technologies and roles of new market participants in the retail market in order to achieve energy efficiency objectives. However, it has been made clear that consumers/prosumers may still face several barriers when making use of new technologies and social innovations.

This work has identified the main legal barriers in the current legal framework that prevent a wider implementation of these technologies and possible social innovations; such as an unclear definition of rights of prosumers/active consumers, an unclear definition of the roles of DSOs and aggregators vis-à-vis prosumers/active consumers and regulatory restrictions, including unnecessary licensing requirements for prosumers and aggregators to supply and trade energy and to access peer to peer markets. Furthermore, not all consumers/prosumers have access to a smart meter and a feedback tool which helps them to understand the information generated by smart meters, which is necessary for enabling consumers to assess their real time energy use and real time prices to adjust their behaviour in an efficient way. The work has tried to provide some regulatory recommendations for an appropriate integration of prosumer oriented innovations in electricity market regulation to overcome the legal obstacles identified. Appendix A provides for a table identifying the main legal hurdles and the possible solutions. The recommendations foresee a future where consumers will make use of all of these developments and will be empowered to trade energy among themselves.

First of all, it is important to recognize the definition of prosumers in the regulation. In this sense, it is to be welcomed that the Winter Package brings legal certainty in this regard. The so-called active customers are entitled to specific rights, including the prohibition from being considered suppliers if they feed into the grid less than a maximum of megawatts, which may vary during the legislative process. They are also entitled to be subjected to cost reflective, transparent and non-discriminatory network charges and they cannot encounter disproportionately burdensome procedures when assessing flexibility markets.

Secondly, it has been argued that the new evolving role of the DSOs as neutral market facilitators should be recognized in regulations. DSOs are well suited to facilitate that flexibility services connected to their grids can reach the market and to facilitate the developments of local flexibility markets. It is also recommended that Member States investigate the possibility that the Winter package opens for designating DSOs as an eligible party that could become a data manager and has access to commercial data. This could also help to integrate other market players, such as aggregators, into the energy market in a neutral way. Lastly, DSOs should only be allowed to fulfil competitive non-core functions temporarily. All of these functions should be performed under the supervision of national authorities and be subject to a sufficient degree of unbundling of the system operation and data management tasks, from commercial players and competitive activities in the energy market.

Thirdly, in regard to the introduction of DR, a “one-size-fits-all” approach has not been recommended. The different characteristics of the markets across the EU, such as geographical conditions, volatility of prices and automatization influences the benefits of dynamic pricing for consumers. Thus, it is better that retailers target through market mechanisms the consumers that really demand it. Retailers should be free to design dynamic tariffs but consumers must be free to choose between tariffs that do not fluctuate during the day. Moreover, Member States should also clarify the role of aggregators in the retail market and let them operate independently of retailers to introduce more choices for consumers. In addition, consumers should be able to switch between different options easily which make opt-in schemes more advisable.

Fourthly, another objective of this work has been to assess the evolvement of regulation in order to empower consumers to trade energy between themselves by making use of digital platforms and blockchain technology. It has been argued that regulation should remove some regulatory burdens which make the transactions of electricity among peers difficult, such as licensing requirements or USOs. Even if the rise of some technologies, such as blockchain, could eliminate some intermediaries, such as the retailers, it also make others appear such as the digital platforms to keep in contact buyers and sellers. This new scenario of shared energy raises new legal questions about the legal nature and the obligations of these new platforms that may be solved by a CJEU’s future decision in the case Asociación Profesional Élite Taxi v Uber Spain. Nonetheless, at least in a first stage, P2P networks are more likely to complement, rather than substitute, the traditional retail model which will face the USO of supplying every customer that demand energy and to guarantee non-P2P transactions.

In conclusion, though, more detailed consumer protection provisions seems to be justified in the most immediate future and regulators may have to intervene to limit the market power of some service suppliers promoting the rise of P2P networks, the tendency seems to be that the role of ex ante provisions diminish and general consumer law and competition law become more relevant legal frameworks to protect energy consumers. This suggests that the role of future regulation may be more focused on granting consumer rights in civil courts, the empowerment of consumer organizations and (ex post) enforcement of competition and consumer law by market authorities. Regarding the institutional design, this expected importance of competition law, consumer law, as well as of data privacy issues may justify the creation of regulators with more interdisciplinary expertise. This could stimulate Member States to be in favour of a model of inter-disciplinary market authorities which are in charge of different legal areas, as it is the case in UK, Spain, the Netherlands or Finland, which could combine the legal concerns coming from several fields.