Multi-Agent Real-Time Advanced Metering Infrastructure Based on Fog Computing

Abstract

This effort to make the power grid more intelligent is tightly coupled with the deployment of advanced metering infrastructure (AMI) as an integral part of the future vision of smart grid. The goal of AMI is to provide necessary information for the consumers and utilities to accurately monitor and manage energy consumption and pricing in real time. Immediate benefits are enhanced transparency and efficiency of energy usage and the improvement of customer services. Although the road map toward successful AMI deployment is clearly defined, many challenges and issues are to be solved regarding the design of AMI. In this paper, a multi-agent AMI based on the fog-computing approach is presented. Architecture follows structural decomposition of AMI functionalities encapsulated in a form of local and area-specific service components that reside at the different tiers of hierarchically organized AMI deployment. Fog computing concepts provide the framework to effectively solve the problems of creating refined and scalable solutions capable of meeting the requirements of the AMI as a part of future smart grid. On the other hand, agent-based design enables concurrent execution of AMI operations across the distributed system architecture, in the same time improving performance of its execution and preserving the scalability of the AMI solution. The real-time performance of the proposed AMI solution, related to the periodic and on-demand acquisition of metering data from the connected electricity meters, was successfully verified during one year of pilot project operation. The detailed analysis of the performance of AMI operation regarding data collection, communication and data availability across the deployed pilot AMI, covering several transformer station areas with diverse grid topologies, is also presented.

1. Introduction

As one of the key components of the smart grid concept, the set of AMI technologies is responsible for energy monitoring, load profiling, and energy management, at the same time providing the link between the energy consumers and distribution system operators. The road-map of the smart grid [1] and representative surveys [2,3,4,5] point out the essential role of AMI in the joint effort of smart grid subsystems to enable sophisticated real-time services for monitoring, management, and optimization of the power distribution from generation sources to the end-users.

However, comprehensive reviews of the smart grid application in different technology areas [6,7,8], as well as the analyses of the smart grid infrastructure, architecture, and communication requirements [9,10,11,12,13,14,15], identify the most recent open issues of AMI implementation, regarding large-scale deployment, communication, big-data analysis, real-time operation, information security and scalability. There are many scientific papers and research studies dealing with different aspects of AMI implementation alluding to the integration of smart electricity meters. On the other hand, there are only a few papers analyzing the possibility of upgrading the existing metering infrastructure for their integration in smart grid, although the majority of existing electricity meters are legacy models with constrained communication capabilities.

The identification of communication problems as the main concerns in design of AMI introduces a gap between the AMI solutions proposed so far and the possibility of their application both in the existing energy grid and in the envisaged smart grid. Focusing on smart meters that play a key role in the transition from traditional to smart grid elevates the necessity of a communication network that provides both comprehensive coverage and sufficient network capacity. In the actual implementation, neither the communication infrastructure meets all the technical requirements and long-term availability, nor are the electricity meters equipped with communication technologies for last mile and/or wide area network connectivity. Even in the European Union, the penetration rate of smart electricity meters is expected to grow to 71% by the end of 2023 [6]. Therefore, the scope of our work is to offer a flexible solution for AMI applicable both to the existing metering infrastructure and the future smart one. In order to be able to offer such a solution, we treat AMI design as an architectural problem, where the limited communication capabilities of utilized electric meters and communication networks do not represent a limiting factor for achieving real-time system performance.

Concerning the functional requirements of AMI, the list of requirements is adopted from the recommendations defined in smart grid roadmap [1] where AMI presumes two-way flow of information, providing customers and utilities with data on electricity price and consumption. The set of AMI functionality include real-time reporting of energy consumption data, energy diagnostics regarding detailed load profiles, ability to identify outages, ability to report consumers the pricing information at the time-of-use, detection of losses and theft, etc. Regardless of the extensive list of the requirements the major concern for their implementation is related to the scale of the AMI deployment. The large-scale aspects introduce scalability issues limited with the data communication and real-time performance of AMI operation [16].

This paper is focused on employing a distributed model of AMI operation, which alleviates the impact of the utilized communication technology on AMI performance. Proposed hierarchically organized implementation of AMI enables its simple integration with other applications [17] and technology areas of smart grid, including consumer-side systems, renewable and distributed energy sources, distribution grid management, electric vehicle charging infrastructure, etc. The proposed AMI architecture based on fog computing is capable of handling communication requirements of large-scale AMI deployment, at the same time sustaining its real-time performance [18]. Regarding the AMI deployment, the real-time operation presumes the metering information from all connected electrical meters is available for further processing within and outside of the metering infrastructure in the 15-min interval. The time interval is selected as a widely adopted averaging period for power consumption measurement and billing. Periodic readings of metering data are not excluding the possibility for on-demand reading of individual electricity meter (EM) data in the minute intervals. Availability of real-time metering data about end-consumer consumption, available either in the separate or aggregated form, may provide many benefits regarding energy generation, distribution and transmission as well as the end-consumer electricity management [19,20]. The scope of the information collected through the AMI deployment will not only encourage customers to manage their electricity consumption, but also enable utility companies to better understand and meet end-user needs.

The proposed AMI architecture extends the basic layered model of AMI from [21,22], utilizing fog computing as a background for building metering infrastructure in smart grid. Adopted fog computing approach resembles tiered architecture of AMI, where the implementation of AMI functionalities is given in the form of cross-fog application utilizing services that reside at different tiers of AMI deployment. As the AMI can be seen as an infrastructure of interconnected computing resources distributed over a wide area resulting in the distribution of capabilities, control and information, the decomposition of the interactions between the software elements is needed in order to fulfill the functional requirements of AMI in a ubiquitous way. We adopted a multi-agent approach as a flexible background for the implementation of AMI requirements over the underlying fog computing architecture [23]. The role of agents is to enable concurrent execution of tasks, fulfilling the functional requirements of AMI, across the distributed system architecture, in the same way preserving the scalability of the solution.

Thus, we found our solution as a universal approach for the integration of all electrical meters with communication capabilities, regardless if they are smart or legacy one. Therefore, our approach is directed to the architectural concept of real-time AMI, applicable in the existing grid infrastructure, where the metering data from all connected meters is available at the highest architectural level in every 15-min interval. Introduction of the edge tier and bringing the intelligence closer to the consumer premises brings many opportunities for flexible smart grid integration even outside the considered scope of AMI. Therefore, the proposed solution could be seen as a milestone in establishing a viable path toward infrastructure capable of responding in real-time to the challenges of future smart grid.

Immediate benefits and the contributions of the presented fog-based architectural solution of advanced metering infrastructure are:

• Fog computing based architectural framework for AMI implementation provides scalable distributed system solution capable to seamlessly integrate a large number of electricity meters with communication capabilities, at the same time preserving operational performance of AMI operation;
• Introduction of local metering controller tier enable concurrent execution of the periodic data readouts from all connected Electricity Meters (EMs) with the limited communication capabilities;
• AMI operation regarding real-time readout of metering information from all connected electricity meters was successfully verified under the conducted case study during one year of pilot project AMI operation;
• The detailed verification of AMI operations, including the inter-tier node-to-node communication is performed based on the QoS implemented at the different tiers of AMI architecture;
• Introduction of service agents as a unique active system component, enables efficient and flexible model for the implementation of both periodic and on-demand metering operations;
• Agent-based design approach supports the loosely coupled model for the decentralized and concurrent execution of system level functionalities across the tiered AMI deployment;
• The agent-based design of AMI based on fog computing elevates the impact of the metering interface and the area-specific communication technology on the system scalability and performance.

This paper is outlined as follows. In Section 2, a review of the most recent scientific studies and research efforts regarding the design, operation, and investigation of performance of advanced metering infrastructure. The details of the proposed solution for AMI, including the AMI architecture, service deployment and collaboration is presented in Section 3. Analysis of data flow across the AMI deployment related to the execution of periodic and on-demand meter readings is given as well. The analysis of the performance of AMI operation, concerning command execution, data communication and data availability is presented in Section 4. Concluding remarks and the directions for future work are given in the final Section 5.

2. Related Work

Focus of the research studies concerning AMI is analyzing its functional requirements, role, and position inside the smart grid concept, as well as various aspects of its implementation and deployment addressing communication and performance issues. Regardless of its implementation, AMI is considered as an integrated system of metering equipment, communications, and information management systems that enable two-way communication, providing customers and utilities with data on electricity price and consumption [1,7,13]. The main components of the AMI system are electricity meters, data concentrators, communication infrastructure and a data management system. Since the number of installed electrical meters is enormous, it is not hard to perceive the volume and the velocity of the produced data that need to be handled through the metering infrastructure. Therefore, designing communication infrastructure capable of delivering huge amounts of data in an accurate and reliable way between the electricity meters and the data management system under the time constraints is found to be one of the major challenges in successful AMI deployment. The solution for the obvious bottleneck problems found in the communication infrastructure can be found in the domain of communication capabilities and technologies, data aggregation and data representations, as well as at the architectural level.

Large numbers of electricity meters, connected to data centers through data-gateways or data concentrators, represent a problem that can be formulated as a scalability issue from the architectural viewpoint, where large-scale aspects of AMI deployment need to be solved. The requirement of real-time AMI operation elevates the observed problems in the communication infrastructure since the rate of data communication introduces additional challenges.

The rest of the section presents the details of the most recent studies and approaches for addressing identified issues in the design and development of AMI, as well as challenges related to the AMI deployment and operation.

The group of papers [24,25] analyze the decentralized approach in the designing AMI system. In [24], the authors investigate the scalability of different AMI architectures and propose hybrid approach as an optimal solution compared with centralized and decentralized AMI architectures. The selection criterion is formulated as a deployment cost optimization problem taking into account introduced cost-related metrics parameters. The scalability analysis presented in [25] is based on the total cost of the communication system with regard to the traffic load on the smart meters for centralized and distributed communication architectures. They also introduce a novel performance metric parameter, as an accumulated product of bandwidth and distance, representing total communication resource usages. The simulation results show the advantages in utilizing distributed communication architectures compared to the traditional centralized model.

The requirement to fulfill the real-time requirements of the AMI operation leads to extensive research efforts, where some of them are conducted at the architectural level [21,22,26,27]. The layered model of AMI architecture, with the distributed implementation of traditional data-concentrator functionality is presented in [21,22]. The introduction of the local concentrator layer in [21] enables command request-response communication with locally connected electrical meters with communication capabilities. The proposed architecture supports development of real-time applications and services inside the metering infrastructure. The analysis of the performance of AMI operation regarding the periodic and aperiodic meter reading at the transformer station areas utilizing different grid topologies and communication technologies successfully confirmed real-time performance [22].

Other research efforts to meet the real-time performance of metering infrastructure were directed toward solving communication related problems or to the introduction of novel communication technologies [16,17,27,28,29,30,31,32].

The work presented in [28] analyzes the operation and performance of large scale centralized or distributed smart grid deployed over a large geographical area. They found that varying communication delay significantly influences the network performance and its application. They developed the queuing model to describe the communication traffic across the grid infrastructure with different topologies in order to analyze the effects of communication inconsistency.

Research presented in [29,30] investigating the properties and the solution for communication infrastructure in smart grid in order to support development of reliable, high-speed and secure grid applications. The paper [29,31] gives a comprehensive survey on the communication architectures in the power systems, including the technologies and the composition of communication infrastructure as well as major research challenges. The study presented in [30] analyzes network requirements, in terms of data payloads, communication rates, latency regarding the different smart grid applications deployed at Home Area Network (HAN), Neighborhood Area Network (NAN) and Wide Area Network (WAN).

In [32], the authors pointed out the benefits of using PLC (Power Line Communication) technology for the communicating in AMI since it is not required to deploy an additional communication infrastructure. However, they identified several issues regarding such communication since the electrical network was not initially designed for communications. Identified network communication problems in AMI are found to be dependent on the connected loads, resulting in the network impedance variation, frequency selectivity or noise. Therefore, they developed a Web-Based Toolkit to facilitate the deployments based on PLC networks, supporting performance analysis and identification of problems in PLC networks.

The analyses presented in the study [33] emphasizes the role of AMI as the one of the basic elements of Smart Grid capable for transmission of transparent information about events occurring inside the low voltage grid enabling localization of disturbances and for determining reliability indices regarding the specific end-consumer, network tracks, transformer stations, etc. This way AMI increases reliability of the grid and contributes better consumer satisfaction by supporting identification and the notification about anomalous, faults or breakdowns found in the grid operation.

The possibility to construct an efficient on-site communication network for an advanced metering infrastructure was investigated in [34]. Based on the quantitative analysis of the communication performances of high-speed power line communication (HS PLC), wireless smart utility network (Wi-SUN), and ZigBee modems used in AMI, through both experimental testbeds and practical environment sites, communication models for classified area types are derived. The study further suggests that through the constructed models, it is possible to efficiently choose an appropriate communication method and plan a methodology for building an AMI network depending on the particular area type. Moreover, electricity providers can apply presented results to select a proper communication method for the particular end-consumer offering quality of services suitable for more reliable meter readings, dynamic tariff, and power remote control.

This article [35] analyzes the smart grid state of play within China, the US, and the EU, assessing the completion state of each smart grid technology and integrated asset. Although the share of smart meter deployments is significantly higher, the analysis shows that the smart grid overall state of play in China, the US, and the EU are equal to 18%, 15%, and 13%, respectively, unveiling the need related to further efforts and investments in these countries for the full smart grid development. Beside the benefits of knowing the information of load variation at each grid point, AMI is seen as one of the key technologies contributing to the overall reliability of smart grid by reducing outages and power restoration costs and enhancing the power quality.

Since the current communication technologies were developed for conventional data traffic with different velocity and volume requirements, a key challenge in designing AMI is to select cost-effective solutions that are capable of delivering the required level of service.

Paper [36] investigates broadband power line communications (BPLC) as a backhaul solution in AMI. Results show that although BPLC has certain limitations; however, with the modifications in the network topology, it can successfully fulfill most AMI traffic requirements even in time-bounded applications. Simulation confirmed that BPLC can support flat and clustered AMI structures with cluster size up to 150 smart meters.

The papers [37,38] deal with the design of an Information and Control Technology network for an advanced metering infrastructure. Paper [37] investigates the potential in using PLC communication systems in delivering minimum performance requirements for AMI. Simulation results comply with the results obtained from the case study of 330 smart meters deployed in the low voltage (LV) grid and 33 data concentrators in the medium voltage (MV) grid. PLC communication has proven to be a cost-effective solution for this AMI application with scope for further scalability without changing the ICT network. The simulation-based performance assessment of BPLC communication infrastructure, based on realistic PLC channel model implementation in Network Simulator 3, presented in [38] is supposed to be used to understand, evaluate, and test the grid configuration before deployment. Simulation results of the basic network topologies showed that the noise sources have the largest impact on the capacity, while the attenuation of the power line is proportional to their length. The simulation also shows the significant influence of the overhead power line and the transition between underground and overhead power lines on the throughput.

The paper [39] states that the successful deployment and operation of AMI is directly impacted by the effectiveness and the efficiency of the communication between electricity meters and Data Concentrator Units (DCU). In order to assess and supervise the communication performance, a paper proposes introduction of Communication Performance Index (CPI) parameters calculated from the acquired success rate and response time.

The communication performance and limitations of AMI deployment accommodated by a real distribution system, from the context of traffic requirements resulting from smart meters message size and transmission rate settings, are investigated in [18,40]. The data traffic analysis, presented in [18] covers the various communication scenarios found in a low-voltage distribution network at urban, semi-urban and rural grid areas. The results show that the traffic can reach extremely high values, depending on the message packet size and on the frequency with which the messages are forwarded to the data concentrator. On the other hand, there are inherent system limitations with respect to the overall amount of data that can be handled due to limited storage resources or resulting from the total transmission time that is required for all smart meters to transmit their data. The paper [40] gives the use case where the methodology for interoperability testing is applied on describing the interaction between a data concentrator and one or more smart meters. The interaction is tested under different conditions by varying two parameters: the rate at which meter data is requested by the data concentrator and the number of smart meters connected to the data concentrator. The obtained results confirm the effectiveness of the methodology for testing interoperability of AMI under regular operation and under stress conditions.

The paper [41] analyzes AMI operation from the perspective of Big-Data analytics in implementing knowledge hierarchy, where wisdom applicable in different areas of smart grid is extracted from raw AMI data. The proposed framework joins the vision of AMI given from three perspectives: an architectural viewpoint for the deployment of AMI in the context of smart grids with the identified business goals, from the perspective that gives the value to data by using different analytic techniques and finally, the evolution from knowledge to wisdom as an ability to involve human judgment and reasoning as a part of the consumer interaction with the smart grid.

The architecture for building smart metering infrastructure, presented in [42], is based on cloud computing, allowing the communication between distribution grid services and smart meters. It also addresses the issues regarding the role of cloud-based solution for obtaining scalability and interoperability, in the same time providing the interfaces for the different services intended for the automation of distribution system.

Extensive literature study given in [43,44,45,46,47] introduces fog computing and its fundamentals along with the discussion which led to the appearance and foundation of fog computing. Fog computing was introduced as an intermediate platform between the devices located at the network edge and the cloud data centers located at the network core. Since the platform moved the processing closer to the edge of the network and adopted its distributed implementation form, it became an interesting solution for the system design, offering location-awareness, real-time performance and low-latency as its inherent architectural property [43,44]. Additionally, fog computing resembles hierarchically organized data transport, introduced through the adopted tiered architectural style, resulting in a scalable solution for its application in different large-scale deployment scenarios [45,46].

Consequently, fog computing was recognized in [47] as a potential solution applicable in various domains as smart grid, community MicroGrid, smart healthcare, smart cities, etc. It was identified that smart grid, which ensures reliable, secure, and cost-effective power supply and offers the solution for coordinated real-time monitoring, control and automation of grid operation, effectively needs fog computing to address numerous problems in its implementation. The study presented in [48] proposes the Serverless AMI implementation at the national level, based on fog-edge computing. The proposed design solution was benchmarked against the traditional cloud computing-based implementation. The presented results show that the proposed design offers an improvement of 20% to 65% performance on network traffic load, latency, and time to respond, with a reduction of 50% to 67% on the total cost of ownership, lower power and cooling consumption compared to the traditional cloud-based design. Therefore, the research confirmed that the fog-edge based approach in AMI design can effectively contribute to increasing the scalability, interoperability, automation, and standardization of future smart grid solutions.

The work presented in [49] comprehends the opportunities and prospects of building fog-based real-time analytic and data aggregation services that enable fulfilling the mission critical computing demands of smart grids. It also outlines the significant adoption challenges encountered in fog based smart grid deployment and architecting. Toward the same line, paper [50] presents architecture and methodologies to support the integration of data analysis process in smart metering systems based on edge-fog-cloud computing approaches in multi-tier context.

A summarized review of recent studies addressing different issues regarding the design and the development of AMI or some of its features is presented in

Table 1. Summarized review of recent studies regarding the design, analysis and AMI implementation.

Research	Approach	Targeting	Applicability	Analysis
Our solution	Experimental on Multi-agent Fog-based solution	AMI design framework, Traffic engineering	Existing grid and smart metering infrastructure	Real-time performance, Multi-tier data availability analysis
[42]	Simulation of cloud-based solution	AMI data management services	Smart metering infrastructure	Interoperability analysis
[50]	Experimental in Fog-based solution	AMI analytics services	Smart metering infrastructure	Scalability analysis
[40]	Experimental evaluation	Communication requirements	Smart metering infrastructure	Interoperability analysis
[18,37]	Simulation and experiments	Design of networking infrastructure	Smart metering infrastructure	Traffic analysis, E2E communication
[38,39]	Simulation and experiments	Performance assessment	Smart metering infrastructure	Data rate analysis, performance analysis
[36]	Simulation based assessment	BPLC performance	Clustered AMI network design	Network reliability, performance analysis
[34]	Experimental on site measurement	PLC and wireless Field network design	AMI Field network design	Performance analysis
[27]	Experimental in two-tier infrastructure	End-to-end communication	Smart grid communication	Routing and delivery, performance analysis
[24,25]	Simulation-based cost optimization	AMI communication architectures	Communication architecture design	Performance evaluation, scalability analysis

. The selection of viewpoints was performed in order to reveal some of the critical aspects regarding the research approach, research directions and intended applications.

3. Materials and Methods

The methodology used to obtain the proposed solution was highly influenced by experimental findings from the previous pilot project of AMI integration [21]. The aspects of the requirements analysis, physical and logical network design, AMI design validation, and verification of real-time performance are as follows.

Requirement analysis of AMI included the review of several international standards. The most relevant ones were found to be IEC 61968-9, IEC 62056-21 and IEC 62056-31. The standard IEC 61968-9 is targeting specification of the interfaces for meter reading and control in the broader context of application integration at electric utilities. IEC 62056-21 and IEC 62056-31 are devoted to the direct local data exchange and use of the local area networks, all as a part of data exchange meter reading, tariff and load control standard in electrical metering. The intention of our analysis was not only to take into consideration all existing technical requirements and technology details, but also to understand the broader context of binding AMI to several other technology areas of the smart grid.

The physical and logical design of the network was conditioned by the need to utilize available broadband powerline (B-PLC) and GPRS/EDGE/3G/HSDPA communication technologies for the field area communication in various deployment topologies intended in pilot project, but at the same time providing direct communication with electricity meters over serial interfaces. During this stage, research efforts were directed toward carefully tuned traffic engineering since it has been found that an adequate management of data traffic is one of the critical factors for the successful AMI deployment.

Validation of AMI design was performed during experimental and in-field testing. It revealed the various performance and operation issues regarding network topology, data access and the impact of methods used in data transport management. The analysis confirmed that the command execution model and the engineering of data traffic strongly influence communication reliability, performance and data availability across the layers of AMI deployment.

The real-time performance, regarding periodic acquisition of load profile data and its availability across the metering infrastructure, required implementation of QoS services at different tiers of AMI deployment. Dedicated services were deployed to evaluate time requirements and the success-rate regarding the inter-node data communication in the execution of operations.

The research path toward the presented AMI solution can be considered as an evolutionary design process bound to the emerging smart grid concept and its roadmap. The evolution of the initial design was motivated by recent technological advances and innovative concepts. The introduction of new features and redesigns were performed through an iterative and incremental process of acquiring in-depth knowledge of system characteristics and operational properties and improving them. The brief timeline of the important events and decisions that have arisen during the system evolution are given in the following paragraph.

Development of information and communication technology infrastructure and systems capable of gathering and processing end-consumer energy consumption data has been a research topic for decades. Moreover, the possibility to monitor such information in real-time, control and optimize energy usage has been seen as a part of a strategy of electricity consumption management at the industrial, service and residential levels. Both monitoring and control were the subjects of our requirement analysis and the initial research efforts were directed toward the integration of existing electricity meters as a part of a unique information system solution. PLC was selected as a cost-effective communication solution for building such a system, complementary to the legacy GSM/GPRS communications. Obvious advantages of using PLC lie in utilizing the utility’s existing power lines for data communication as an added value, as well as simple deployment and maintenance of such communication infrastructure. One should have in mind that the communication capabilities of the metering equipment were highly limited at that time. Since the standard electricity meters offer only serial communication interfaces, their direct integration into the PLC-based communication infrastructure was not possible. The logical solution that enabled us to surpass such a problem was the introduction of intermediary system component in the form of embedded device. At the early stage of physical and logical network design, the device was acting as a traditional gateway which offered just simple network protocol conversion. Later on, new features were added to the gateway device to address data buffering, aggregation, and access control, which finally resulted in the introduction of intelligence at the electrical meter premises. Hypothesis that the real-time performance of AMI operation is achievable regardless of the constrained communication capabilities of both metering equipment and the communication infrastructure was successfully verified on the implemented AMI system [21,22].

Since the AMI deployment is associated with large-scale integration requirements, scalability and interoperability were imposed as obvious issues affecting the applicability of our proprietary solution. Therefore, we directed efforts toward seeking a general framework that will be capable of addressing previously stated issues, but also the issues of security, reliability, availability, serviceability, openness, manageability, etc. In line with the emerging standardization in the field of computational architectures, we adopted fog computing as a general framework that addresses the majority of the AMI concerns. However, the necessity for real-time availability of metering data across the different tiers of AMI architecture was identified as a particular application-specific requirement which was beyond the scope of OpenFog reference architecture since it was attributed to implementation issues regarding the data flow mechanisms for communication across the system architecture. To address this issue, we adopted agent-based design where the management of data acquisition and transport is handled by service agents as dedicated system components that reside at the different tiers of system architecture. The essential contribution in AMI design can be found in the domain of engineering of data traffic. Positioning of service agents at the edge-tier enabled real time acquisition of metering data and its availability across the system hierarchy.

The scope of adopted methods used in AMI design include hierarchically organized distributed processing model as a systematic solution for aggregation and data access, and traffic engineering with carefully tuned data communication patterns and path management supported through agent-based design.

4. AMI Deployment and Operation

The section gives the overview of the AMI system architecture where its features and capabilities are encapsulated in the form of microservices adopted from the OpenFog reference architecture [43]. The AMI operation is given in the form of collaboration of a group of physically distributed services that reside at the different tiers of the AMI architecture. Although implementation of AMI according to the fog computing approach addresses all functional aspects of AMI deployment, including the discovery, provision, configuration, life-cycle management, security, etc., focus of our research is directed toward the design and implementation of the native ability of AMI to collect, store and report customer energy consumption data at the required time intervals.

Section 4.1 introduces distributed components of fog-based AMI architecture, their responsibilities, physical deployment, and the details of communication infrastructure. Section 4.2 gives the description of AMI operation and service deployment, including the details of the adopted mechanism for data communication and command execution patterns.

4.1. Overview of AMI Architecture

Description of AMI, presented in Figure 1, accommodates tiered architectural style adopted from the fog computing approach, where local specific functionalities are supported by Tier 1 fog node named Local Meter Concentrator (LMC) node, area-specific functionalities are supported by Tier 2 fog node named Transformer Station Concentrator (TSC) node, while the Metering Data Collection (MDC) fog node, at Tier 3, integrates the head-end system functionalities and other services [51]. Tiers overlaying tier 3, are integrating the functionalities of the supervising system as a part of Meter Data Management (MDM), which is out of the scope of our paper.

The bottom tier of the system architecture given in the Figure 1, includes the collection of electrical meters, where a single electrical meter or a group of EMs are locally connected to single LMC over node-to-device pathways over standard point to point or multi-drop communication interface [52]. The tier 1 concentrator supports integration of both smart electrical meters and traditional electrical meters with communication capability. Physical LMC installation is possible in the form of an internal electrical meter module (in-meter) or a separate LMC hardware unit within the metering cabinet. In-meter LMC is capable of communicating with the other, external, electrical meters over a separate physical interface. A group of LMC are connected to a single tier 2 TSC, over the vertical node-to-node pathways utilizing broad-band power line [53,54,55], GPRS [17,56] or other communication technology [57,58,59,60]. Responsibilities of tier 1 fog node is to handle data transport between the group of connected electrical meters connected to the single LMC and the particular TSC node as a representative neighborhood area fog node. Data communication over node-to-device and node-to-node pathways is initiated through the execution of periodic and aperiodic operations triggered at different Tiers of AMI. Periodic operations are related to acquiring regular periodic meter readings and they are responsible for the majority of data transport, while the aperiodic operations conform to the on-demand meter readings, meter configuration and settings, configuration of Tier 1 services, event notification, etc.

Responsibilities of the TSC node are to handle configuration and discovery of underlying LMC nodes located at the same transformer station area, to execute automatic operation at the reference area regarding time-synchronization, to perform aggregation and management of retrieved metering data, and to handle data transport over different node-to-node data pathways.

The MDC node is providing the primary interface between the metering infrastructure and back-office application at MDM offering interoperability between different metering systems and MDM, performing data collection and time-synchronization of underlying infrastructure. MDC is responsible for executing commands received from overlying MDM, directed toward the TCS, LMC or metering devices, to handle underlying node provisioning and registration, to aggregate and store all metering data and to participate in the execution of other AMI services.

Communication across the metering device to the cloud continuum is organized through the different pathways, which come with their common properties. Cloud-to-cloud pathways are utilized to support back-end data sharing models and the integration of AMI with other smart grid technology areas and third-party services involved in application regarding energy monitoring and management applications, integration of renewable energy resources, supporting dynamic tariff, prepaid services, etc. Node-to-cloud pathways, according to the adopted OpenFog reference architecture, comply with the ITU-T X.800 security architecture, where communication is usually given as a web service transaction utilizing SOAP over HTTP or RESTful HTTP/COAP messaging [43].

Node-to-node communication is strongly dependent on particular inter-tier communication since a variety of communication technologies are used. As the MDM tier can be considered as a boundary among the utility network and the core network node-to-node pathway between Tier 3 and Tier 2 is given in the form of WAN or Field Area Network (FAN). This communication pathway is usually referred to as an AMI backhaul enabling the communication between the utility head-end system as a part of MDM tier functionality and in-field metering equipment and data concentrators as TSCs nodes are. Data communication over Tier 2 to Tier 3 communication pathways is utilized for collecting aggregated metering data information from connected transformer station areas, execution of automated operations at Tier 3 or operation triggered through MDM interface, configuration and management of in-field devices and network equipment, etc. Since the expected data volume and the rate of the communication toward head-end applications and services, this node-to node pathway is given as robust IP-enabled communication infrastructure based either on public cellular service or fiber-optical or twisted-pair Ethernet network. Node-to-node communication across the transformer station area is organized in the form of Neighborhood Area Network. Corresponding TSC to LMC node communication pathways are utilizing PLC or wireless RF networks. Data transmission over the LMC-to-TSC data path utilizes Automatic Repeat Request (ARQ) as an error-control mechanism which uses acknowledgements and timeouts to achieve reliable data transmission over an unreliable communication link.

Node-to-device pathways correspond to LMC to meter communication or TSC to meter communication depending on the communication capabilities of the metering equipment. Compared with the AMI infrastructure, that is integrating smart meters capable of communicating at the NAN level, LMC nodes are introduced at the consumer level to support data aggregation and data communication with the legacy electricity meters over UART, RS485 or RF network.

4.2. AMI Services and Operation

Although fog-computing deployment of AMI assumes the implementation of a number of services, the main focus of our discussion is directed toward the implementation of AMI’s native ability to collect, store and report information of customer energy consumption data at required time intervals or on-demand. Since this ability is related to the execution of commands, triggered at the different tiers of AMI architecture, in the rest of the section we are discussing the command execution mechanisms and the succeeding flow of information across the metering infrastructure.

Deployment of AMI services with the details of data flow during the command execution are presented in Figure 2. The majority of the services reside at the application service layer described at the OpenFog reference architecture model [43]. The services and the functionalities at the application support layer and node management and software backplane layer of software architecture view are out of the scope in the following discussion.

During the AMI operation, there are two different command execution patterns; the first one is periodic and it is related to the acquisition of data profiles, and the second one, aperiodic, related to manual, on-demand command execution. The command execution mechanism presumes request-response message exchange pattern, where the command requests are generated at the different tiers of AMI deployment. The propagation of command request and command response across the AMI tiers defines the command request data path and response data path, respectively.

As the execution of periodic commands is essential for AMI operation, their execution is managed by the introduced command management agents implemented in the form of software services. In the further discussion command management agents are referred to as service agents.

The AMI functionality of period data acquisition is delegated to the LMC tier and it is distributed between the LMCs for their simultaneous execution. Concurrency of LMCs operation is ensured since each of LMCs operates with locally connected EMs. In our implementation periodic acquisition of the load profile data, is the automatic operation triggered by the local service agents at the LMC tier. On the other hand, transport of information collected by LMCs toward TSC and aggregated data from TSCs toward MDC is triggered by service agents at the LMC and TSC layer, respectively. Since these automatic operations are locally triggered at the appropriate service agent layer, their request-response data path consists only of the response data path. Effectively, from the MDC’s point of view, the mechanism of period data acquisition is identical to the push-data model triggered at the LMC tier.

In the case of aperiodic operation, on-demand commands are mainly triggered at the MDC tier, where their request-response data path consists of both command request data path and response data path. Effectively, from the MDC’s point of view, aperiodic operations are executed in the “pull” manner, where the command execution is triggered by the local service agent at the MDC tier. It needs to be mentioned that information carried by the commands requests and command responses are formatted as the time stamped XML based data structure, although another format can be adopted to support different data representation formats and messaging data-exchange protocols.

As we discussed the AMI operations regarding the command execution and the adopted model for their implementation, the rest of the section give the details of other important system services, their collaboration and physical deployment at different tiers of AMI architecture.

The main role of the MDC tier is to process commands from the supervising system of Meter Data Management (MDM) and to collect data from all subordinated TSCs. Entire data flow, including command requests and data acquisition process is controlled by the Task Manager, as the central MDC module. Automatically acquired metering data are verified and stored into a central database, further accessible through the MDM service interface. All metering and system events, used for supervisor notification or Network Management System (NMS) analyses, are also stored into the central database. MDC also supports other standard AMI functionalities, such as automatic time synchronization, fraud and tampering detection, meter tariff management, etc. Local automatic actions are accompanied by appropriate notifications to supervising MDM. Its interaction with MDM are out of the scope of this paper. Web service transactions are adopted as the model of MDC network communication with other system components, including the internal communication between MDC’s software modules in the case of preferred distributed architecture.

TSC fog nod implementation presumes both fog connector services utilizing node-to-node communication over web-service based communication interface towards upper MDC tier and implementation specific communication interface towards underlying LMC devices. Communication mechanism and protocol stack are implemented in the form of separate communication managers, which handle request-response messaging patterns for data exchange. All node-to-node data exchange is based on time-stamped XML-based data format. Service execution at the TSC tier is performed by OS-dependent Task Manager Thread, implemented according to thread-based POSIX standards. TSC-tier functionality is organized in the form of independent system services, accessible and configurable through appropriate TSC service interface. Introduction of the particular software layer at the TSC tier given as TSC Service Agent Layer enables execution of the automatic operations scheduled at the TSC level. These operations are involved in the time-synchronization, configuration of registered LMCs, discovery of electrical meters, etc. Data management, including data storage and retrieval, is performed through the local database server, accessible through Database Manager Service. Appropriate TSC configuration, including TSC identification and location information, underlying TSC subsystem topology is handled by the TSC Configuration Manager as an independent software component at the TSC service layer. Time Synchronization service is also triggered by the automatic operation at the TSC Service Agent Layer. The execution of such operation presumes the time synchronization of all underlying LMCs and EMs at the transformer station area. The QoS service at TSC service layer is used to evaluate time requirements and the success-rate regarding the execution of the operations triggered either on MDC tier or the operations automatically triggered at the TSC Service Agent Layer. Beside the data transport responsibilities toward the both, MDC and LMC tier nodes, LMC correspondent Communication Manager supports routing of command requests to a single, group or to all connected LMCs. Additionally, aggregation of the information that resides* in command responses is delegated as one of its duties.

Deployment of services at the LMC tier resembles the TSC node architecture. Communication toward TSC is performed via Communication Manager, while communication toward EMs is performed over several physical wired EM interfaces, such as RS-232 and RS-485. Data communication with the particular electrical meter is vendor specific and depends on adopted communication standards, such as IEC 62056-21, DLMS specifications, etc., and is implemented as a separate thread of program execution, managed by OS dependent LMC Task Manager. Beside the communication with EMs over Metering Interface, Task Manager is able to access other services, such as for instance QoS service in order to provide the success rate information and elapsed time information regarding the data communication with EMs or with the upper TSC nodes. Setting and getting the configuration info from LMC, as well as the information about connected EMs, is handled by the LMC Configuration Manager Service. Correspondingly to the TSC node operations, LMC Service Agent Layer enables execution of automatic operations at LMC tier, including the acquisition of 15-min load profile, collecting of the events at the EM side, etc. Functionality related to the execution of commands on a single EM or group of connected EMs is implemented in the form of Metering Service, as the separate thread of program execution managed by the LMC Task Manager. Metering Service operation is triggered over Metering Interface, either directly by Communication Manager, in the case of TSC-triggered command execution, or by any of the LMC’s Service Agents, in the case of automatic command execution. The Metering Service actually implements the functionalities dedicated to access metering info and meter management. The metering info relates to the load profile, instantaneous values, billing and meter status information, including phase outages, tampering and fraud detection. On the other hand, meter management is linked to the getting and setting of general configuration parameters, regarding the time settings, tariff plan, configuration of relay state and power limits for alarm notifications and auto-disconnect features.

5. System Verification

The operation of system services and the real-time potential of the proposed AMI architecture based on the fog computing is investigated through the conducted pilot project case study. The operation of AMI with the deployment presented in Figure 3, is analyzed during a one-year period. The transformer station areas are selected to resemble AMI topologies in the urban, rural and island grid scenarios. Presented AMI architecture was physically deployed on the existing power network grid, without any grid reconstruction or modifications.

5.1. Pilot Project Deployment Details

The topology of AMI deployment, presented in Figure 3, consists of a single MDC node, three TSC fog nodes, 387 LMCs and 848 electrical meters in total. All the EMs are connected to LMCs over RS-232 or RS-485 serial communication interface, where the majority of the EMs are compliant with IEC 62056-21, while other EMs are compliant with DLMS. The first transformer station area, with single TSC, covers the urban grid topology with 441 EMs integrated in AMI through 61 LMCs. In most cases, the LMCs are connected to the group of six to ten EMs. Communication between LMCs and TSC in this transformer station area is performed over Broadband Power Line designated as B-PLC #1. Single TSC in the second transformer station area mainly covers the rural grid topology, where the majority of 403 EMs is connected to a single LMC. As shown in Figure 3, TSC communication with 323 LMCs in this transformer station area is also performed through Broadband Power Line designated as B PLC #2. The AMI deployment in the third transformer station area, resemble island grid topology and is installed in order to investigate TSC-LMC communication over GPRS communication link designated as GPRS #1. In this segment, a single TSC is covering four EMs integrated through the three LMCs. While the MDC subsystem is located in the Data Center, the other AMI nodes are deployed in appropriate transformer station areas.

As the hardware requirement for MDC node implementation depends on the number of installed TSCs, MDC subsystem hardware platform, used in this pilot project, is a high-availability cluster platform based on Windows Server, hosted at the Data Center at the tier 3. MDC and TSCs in B-PLC #1 and B PLC #2 are connected within VPN that is realized over xDSL, while the TSC in GPRS #1 is connected to the Data Center through the dedicated GSM-APN. TSC is implemented on an embedded hardware platform, based on the ARM Cortex family of processors, running Linux OS. The TSC integrates B-PLC Head-end server, based on Marvell’s DSS 9503 chipsets, to maintain the both of B-PLC networks, B-PLC #1 and B-PLC #2, and 3G/GPRS communication module, based on Sierra Wireless SL8082T module, for GPRS communication over GPRS #1. Two different LMC hardware platforms are used, depending on the required communication technology. LMC based on Marvell’s DSS9501 chipset provides B-PLC communication channel, while LMC based on Sierra Wireless SL6087 module provides GPRS communication channel with the TSC. TSC and LMC, as dedicated hardware, are compliant with industrial grade requirements, containing no internal wiring connection and no moving parts.

5.2. Experimental Results

The following section analyzes AMI operation regarding the on-demand and the periodic acquisition of load profile data, including the data availability analysis, command processing and data communication time analysis for the specific transformer station area or for complete AMI deployment.

Timing analysis of the information flow across the EM-to-MDC data path for the scenario of real-time acquisition of load profile data is shown in Figure 4. The scenario assumes the execution of the operations automatically scheduled at the LMC service layer, which are periodically triggered in 15-min intervals. The load profile data contain typical dataset of time-stamped values of active and reactive energy registers, as well as the information of 15-min average active power. Results of more than one-year measurement of load profile data availability are obtained by QoS services based on time-stamp information from XML-based data payloads.

The histogram in Figure 4 represents the time-distribution for the availability of acquired 15-min load profile data at the MDC tier, given for all the 848 EMs within the pilot project.

As one could notice, the real-time performance of AMI operation is achieved since all load profile data are available at the MDC layer after a seven-minute interval, which is well before the next sampling instance given at the rate of 15-min. In order to investigate the origin of the distribution shape from Figure 4, a more detailed analysis, presenting the distribution of the time required gathering and transfer load profile data across the EM-to-MDC data path is given in Figure 5.

Each segment of the individual column chart, given in Figure 5, corresponds to the time-interval between the data available at the two adjacent AMI tiers. There are two characteristic segments of column charts noticeable in Figure 4. In the first segment, where the data are available at the MDC tier before the interval of three-minute is elapsed, the significant part of the data propagation time is related to the TSC-LMC data path, while the LMC-EM data path time requirement is rather constant. The prolonged data propagation time, in the second column chart segment, is dominantly related to the LMC-EM data path.

Since the actual reasons for the prolonged load profile availability up to MDC tier could not be found within the results presented in Figure 4 and Figure 5, the performance of the AMI operation is further analyzed for each of the transformer station areas, as presented in Figure 6.

Obviously, as given in Figure 6, the different area-specific topology of AMI deployment causes a different distribution of time required for the load profile data to be available at the particular tier. Since the number of meters in B-PLC #1 (441 EMs) and B-PLC #2 (403 EMs) areas are rather comparable, the significant difference in the availability of load profile data at the MDC tier is a consequence of particular EM to LMC grouping. Since the number of LMC nodes in B-PLC #1 is 61 compared with 323 nodes in B-PLC #2, the different time distribution along the EM to MDC path is rather expected. The corresponding distributions of inter-node data availability time for different transformer station areas in AMI deployment are given in Figure 7. The area-specific distribution of inter-node data availability along the LMC-to-TSC data path is presented in Figure 7a, while the analogous distribution for EM-to-LMC data path is presented in Figure 7b.

As expected, in the case of B-PLC #1 transformer station, where 61 LMCs are connected, and GPRS #1, with three connected LMCs, the TSC-LMC communication is less time demanding and all load profile data sets are transferred within a minute interval. On the other hand, The TSC-LMC communication in case of B-PLC #2 transformer stations are, as expected, more time consuming, since 323 LMCs are connected in a single B-PLC network, prolonging this time significantly. As obtained from LMC-QoS service utilizing ARQ-based data transmission mechanism with acknowledgments for node-to-node communication, the average time for the single successful LMC-to-TSC data transfer is 4 s for the B-PLC #1 in, 8 s for B PLC #2 in, while in case of GPRS #1 this time is 7 s. The prolonged communication time along the LMC-TSC path in the case of B PLC #2 communication is taken as an operational property of B-PLC network since the more demanding network topology compared with the B-PLC #1, and it is not further analyzed.

Even though the TSC-LMC data transfer in case of the B-PLC #1 is less time demanding, data availability at the MDC layer is significantly prolonged, compared with the B-PLC #2, since the performance of LMC-EM communication, as shown in Figure 7b. In case of B-PLC #1, gathering of load profiles, for all EMs connected to a single LMC, requires significant time, because a single LMC is connected through a serial communication interface to a group of six to ten EMs. The corresponding time, in the case of the B-PLC #2 is much shorter, since AMI topology at the particular transformer station area presumes that in the majority of cases a single EM is connected to a single LMC. Therefore, the resulting data availability at the LMC tier is achieved within the first minute, except for a few LMCs integrating a group of three and four EMs, where the data availability is prolonged up to the 90 s. Concerning node-to-device communication, as similar to node-to-node communication, data availability is obtained from LMC-QoS service, where QoS service provides the statistics of command execution time regardless the execution is triggered externally, by the service agents from MDC or TSC tier, or automatically scheduled by LMC Service Agents. Execution time is defined as the time interval between the moment when the request is sent to the Metering Service over the LMC Service Interface and the moment when the response from the LMC Metering Interface is received.

In the scenario where the execution of on-demand requests is analyzed, the readout of instantaneous values from a single EM or the group of EMs, all from the same vendor type, is triggered at the TSC tier. QoS measurements at the TSC tier, averaged for a group of EMs that belongs to the same vendor type are presented in Figure 8.

The LMC-EM communication protocol with the vendor #1 and #2 EM types is performed according to IEC 62056-21 specification, while the communication with vendor #3 and #4 EM types is performed according to DLMS specification. Averaged on-demand command execution time at the LMC tier given for the group of EMs connected to the single LMC is in correlation with the command execution time for the single EM and it is directly proportional to the number of the EMs involved in the communication. Since the results obtained for vendor #1 and #2 EM types as well as for the vendor #3 and #4 EM types are similar, one can conclude the choice of the physical metering interface communication technology dominantly influences the execution performance of the LMC-to-EM communication, independently on EM vendor-specific implementation. In the case of a larger group of EMs connected to the single LMC, where up to 20 EMs are supported to be connected to the single LMC, the choice of the LMC-EM communication technology dramatically impacts the overall AMI system performance. However, architectural benefits from introducing LMC tier provide the noncumulative effect of LMC-EM time requirements on TSC level command execution time requirements, since the operations on all the LMCs, as mentioned before, are performed concurrently, rather than in sequence, as in the centralized AMI topologies.

5.3. Discussion and Limitations

The approach in this paper pursues the fog-computing based framework for the AMI design and implementation, where the focus is shifted from solving communication related problems in creating smart metering infrastructure, addressed in the existing literature, to the comprehensive architectural design that effectively alleviates capabilities and limitations of the existing metering infrastructure. Our approach is targeting metering infrastructure with the existing installments, where the proposed solution is offering the upgrade of the legacy meters to smart metering platforms, capable to be are fully integrated into the smart grid. From the viewpoint of the recent research studies, our approach can be seen as an operative extension of existing theoretical and practical efforts that will enable occurring smart metering rollouts to be, not only the solution for local energy consumption awareness of end consumers, but also the building element for the last-mile integration of end consumers into the broader scope of the smart grid services and applications.

Presented experimental results successfully verified the ability of the designed and implemented AMI system to deliver metering data in real-time to all tiers of AMI infrastructure and to ensure real-time command execution properties for all commands, either issued to the particular meters or broadcasted to all meters in the managed deployment area. The proposed synergy of the distributed system architecture and the agent-based data traffic engineering is found to be the key factor for achieving observed real-time data availability.

According to the presented results, the further optimization of real-time performance of AMI operation could be achieved by balancing the number of the LMCs implemented in the transformer station, and the number of EMs connected to a single LMC. In case when the higher number of LMCs is integrated into a single B-PLC network, the TSC-LMC data transfer time will be prolonged. Similarly, connecting the higher number of the EMs to a single LMC will correspondingly prolong time requirements for LMC-EM data communication influencing resulting data availability at the MDC tier. If analyzing the AMI deployment from the cost-optimization viewpoint, grouping of EMs and connecting to a single LMC reduces the overall number of LMCs and the costs of the overall AMI deployment.

Once more, one should have in mind that all automatic operations are triggered and executed in parallel on every LMC, so the achieved LMC-EM communication time requirements is defined by a particular, e.g., local LMC-EM connection and not by the total number of LMCs. On the other hand, TSC-LMC communication time requirements are in relation with area specific network configuration, and therefore dependent on overall number of LMCs and their connection to the corresponding TSC.

Even though the GPRS area within this pilot project covers a small number of LMCs and EMs, the results on a larger scale are not expected to be much different. It is expected that this communication technology should also provide satisfactory real-time performance, if it is not used as a primary TSC-LMC communication technology in the system, but only for the isolated grid connected islands in AMI deployment, where GSM communications is the only available choice providing wide-area wireless Internet access.

Anyway, regardless of the LMC-EM and TSC-LMC communication properties, metering infrastructure enables real-time acquisition of load profile data and its availability across the hierarchy from EM to MDC tier. On the other hand, execution of on-demand requests is directly influenced by the properties of LMC-EM communication.

From the architectural viewpoint, the design of AMI in the proposed way brings some challenges and limitations that also need to be pointed out. From the security perspective, distributed systems are exposed to security threats that centralized systems are not subject to. Extensive inter-node communication brings challenges in achieving data confidentiality and disabling interference and attacks by unauthorized and possibly malicious parties. Additionally, the security model in such a distributed environment is based on establishing the trustworthiness, with the specific role of each system component, as the security is a fundamental cross-cutting issue that pervades the design of the entire system. Such complex trust, authentication and confidentiality requirements are involved in the provisioning and management of nodes and their resources. Required security policies and implementation of secure communications inevitably burden the intended real-time performance of the overall system. From the manageability perspective, distributed AMI architecture brings many challenges and limitations regarding the management of fog nodes’ life cycle and their maintenance. These limitations are related to the necessity to handle node commission, discovery, identifications of features, capabilities, trust relationships, but also handling faulty node operation, recovery, firmware/software updates, etc.

Generally speaking, a distributed architecture that brings intelligence to the edge of the metering infrastructure, itself imposes restrictions on various aspects of information sharing and access.

6. Conclusions

The presented AMI architecture, based on fog-computing, defines the flexible and scalable framework for the implementation of all the AMI functionalities envisioned in the Smart Grid concept. The real-time operation is provided by adopting hierarchically organized tiered model for AMI implementation, where LMC tier can be considered as a conceptual upgrade of EM communication module, and the TSC tier as an upgrade of a traditional gateway-based data concentrator. The architecture is applicable to the existing electrical grid and supports the integration of any communication-capable meter irrespective of communication interface type. The LMC tier delivers concurrency in the execution of the periodic data readouts from all EMs as the most demanding system level operation. In addition, introduction of the service agents enables a unified model for the automated execution of the tier-specific services, without any unnecessary involvement of upper-tier AMI services. This resulted in significant improvement of overall AMI system performance, including dramatically decreased timing requirements and communication overhead in the typical system operation. In the case of load profile data acquisition, the metering data is imported into the system at the LMC tier, being accessible for the real-time processing services deployed at any tier of AMI deployment, supporting development of consumer-side location-aware and neighborhood area-specific grid services. Online monitoring and evaluation of AMI operation, including fault detection and diagnostics, is supported by the deployment of QoS services across the AMI tiers. The result obtained from the pilot project operation analysis has confirmed the AMI architecture is applicable to complex AMI grid network topology, covering the urban, rural and island grid scenarios. The AMI system topology, the choice of primary area-specific communication technology as well as the metering interface communication technology were identified to have the dominant impact on the overall system real-time performance. However, the architecture alleviates these impacts by introducing decentralization and concurrency in the AMI operation.

As a prospective field of future research, authors see the integration of real-time services and applications that bind different technology areas defined within the smart grid concept. The domain of special interest is energy management, where the synergy of AMI and the customer side systems is expected to provide benefits to all parties involved in the process.