Hybrid Multi-Agent Control in Microgrids: Framework, Models and Implementations Based on IEC 61850

Abstract

Operation control is a vital and complex issue for microgrids. The objective of this paper is to explore the practical means of applying decentralized control by using a multi agent system in actual microgrids and devices. This paper presents a hierarchical control framework (HCF) consisting of local reaction control (LRC) level, local decision control (LDC) level, horizontal cooperation control (HCC) level and vertical cooperation control (VCC) level to meet different control requirements of a microgrid. Then, a hybrid multi-agent control model (HAM) is proposed to implement HCF, and the properties, functionalities and operating rules of HAM are described. Furthermore, the paper elaborates on the implementation of HAM based on the IEC 61850 Standard, and proposes some new implementation methods, such as extended information models of IEC 61850 with agent communication language and bidirectional interaction mechanism of generic object oriented substation event (GOOSE) communication. A hardware design and software system are proposed and the results of simulation and laboratory tests verify the effectiveness of the proposed strategies, models and implementations.

1. Introduction

A microgrid is commonly defined as an integrated power system consisting of distributed generators (DGs), distributed energy storages (DSs) and interconnected loads, which can operate in a grid-connected mode or in an intentional island mode. Operation control is a vital and complex issue for microgrids due to the intermittent and fluctuating output of distributed energy resources, the low inertia of DGs and the diversity of operation states (i.e., in parallel with the grid, intentional island and transition states during the switch between the two states) [1,2,3,4,5,6].

Presently, there are three basic control strategies for microgrids: centralized control, decentralized control and local control [7,8,9]. Compared with local control, centralized control and decentralized control are facilitated to implement more complex operations through information sharing and cooperation among DGs. Decentralized control attracts more attention, since centralized control is considered to be less robust because any failure of the central controller could compromise the operation of the whole system [10,11,12,13,14,15].

Multi-agent system (MAS) is a perfect implementation scheme for decentralized control [16,17]. With MAS, each control unit (CU) in a microgrid (e.g., DG, DS, or load controller (LC)) is designed as an agent, and the operation of the microgrid is implemented through intelligent decisions and collaborations of these agents [18]. In [19], the authors proposed a framework of an energy management system in microgrids based on MAS; in [20], self-healing control and fault restoration of microgrids by MAS was realized. Furthermore, in [21,22,23], the Java agent development framework (JADE) platform was used to design and verify the control strategy based on MAS.

However, previous research based on MAS has mostly focused on the theory and simulation analysis of control strategy, ignoring implementation schemes for the proposed strategy. The JADE platform mentioned above is based on the JAVA program language and TCP/IP communication protocol, which is hardly applicable to practical projects due to its non-real-time characteristics and the function and performance limitations of DGs.

This paper will focus on decentralized control and propose a hierarchical control framework (HCF) and a MAS control strategy named hybrid multi-agent control model (HAM) in an effort to propose a practical decentralized control method for microgrids. Furthermore, the IEC 61850 Standard [24] is used as the modeling method and communication protocol of HAM based on the analysis of its necessity and feasibility, and some new implementation methods are proposed to apply HCF and HAM to practical projects and devices.

The rest of this paper is arranged as follows. Section 2 first presents a HCF to flexibly implement the decentralized control concept in microgrids. Next, a HAM is proposed to implement HCF, and the properties, functionalities and operating rules of HAM are described in detail. Section 3 discusses the implementation of HAM based on the IEC 61850 Standard [24]. Some new methods, such as extended information models of IEC 61850 with agent communication language and a bidirectional interaction mechanism of generic object oriented substation event (GOOSE) communication are proposed. Section 4 briefly describes the hardware platform and the software system of HAM and presents the results of simulation and laboratory tests on HAM. Section 5 concludes this paper and provides advice on future research.

2. Design of Hybrid Multi-Agent Control Model Based on Hierarchical Control Framework in Microgrids

2.1. Hierarchical Control Framework in Microgrids

Given the diversity and complexity of microgrids as stated in Section 1, there are a variety of control strategies and purposes applicable to them. Hence, this paper proposes a HCF for microgrids, which is made up of four control levels to implement different strategies and methods for different control purposes. The four control levels of HCF are detailed as follows:

(1)

At the local reaction control (LRC) level, CU implements the basic functions according to local electricity information (e.g., the magnitudes of voltage and current at the point of coupling between CU and microgrids). LRC indicates CU can work independently without external control.

(2)

At the local decision control (LDC) level, CU implements some optimizations without external control. LDC indicates CU can autonomously regulate the control strategies, references or parameters to work more efficiently and economically.

(3)

At the horizontal cooperation control (HCC) level, CU communicates and cooperates with others. HCC indicates CU can participate in coordinated control. With the advantages of the cooperation among CUs, some more complex and optimized operations can be implemented, which is superior to LDC and LRC especially when there is a large disturbance.

(4)

At the vertical cooperation control (VCC) level, there is a central controller (e.g., microgrid control center (MGCC) or microgrid energy management system, which distributes operation commands to other CUs with the global optimization. VCC indicates CU can be dispatched, which could make microgrids operate most efficiently and economically under the control of central controller. The response delay of VCC is generally the longest among the four levels, due to the complex calculation for the global optimization.

2.2. Properties of Hybrid Multi-Agent Control Model

Nowadays, MAS is attracting widespread attention in the microgrid field, since MAS is suitable for different applications [20], can reduce a complex control issue to several simple issues [25], and enhance the robustness of control systems [26]. In this paper, a HAM is proposed to implement HCF mentioned above. Compared with common MAS, the proposed HAM includes four basic properties to meet the four control levels in HCF, respectively. Some significant relations between HAM and HCF are described in Figure 1 and the details are listed below:

(1)

Reflex property of HAM is related to LRC in HCF. It means that the local controls of CU will be implemented by the reflex control strategies of HAM.

(2)

Intelligence property of HAM is related to LDC in HCF. It means that the local optimizations of CU will be implemented by the intelligent control strategies of HAM.

(3)

Sociality property of HAM is related to HCC in HCF. It means that the coordinated controls among CUs will be implemented by the social control strategies of HAM.

(4)

Initiative property of HAM is related to VCC in HCF. It means that the centralized controls and global optimizations under MGCC or the master agent implemented by the initiative control strategies of HAM.

Figure 1. Relations between hybrid multi-agent control model (HAM) and hierarchical control framework (HCF). LRC: local reaction control; LDC: local decision control; HCC: horizontal cooperation control; and VCC: vertical cooperation control.

Figure 1. Relations between hybrid multi-agent control model (HAM) and hierarchical control framework (HCF). LRC: local reaction control; LDC: local decision control; HCC: horizontal cooperation control; and VCC: vertical cooperation control.

Hence, HCF for microgrids could be designed as consisting of several HAMs (HAM1–HAMn), and each HAM is a CU. Furthermore, each property of HAM could be treated as a sub-agent of HAM (e.g., HAM1_1, HAM1_2, HAM1_3, HAM1_4) in order to respectively implement the four control levels of HCF.

2.3. Functionalities of Hybrid Multi-Agent Control Model

Although HAM is suitable for implementing HCF, it is necessary to define the functions of different HAMs and their sub-agents. Several typical HAMs are listed in Table 1, for instance, point of common coupling (PCC), DS, micro-turbine (MT), DG and LC. The details of the functionalities of different types of sub-agents include:

(1)

HAM_1, related to LRC, implements local reflex controls and protections under the rules established by HAMs themselves. As listed in Table 1, HAM_1 usually solves events such as protections and breaker operations that require instant actions.

(2)

HAM_2, related to LDC, implements local optimizations depend only on the decisions from HAMs themselves. HAM_2 are responsible for independent events only requiring simple analysis, such as power quality analysis, maximum power point tracking (MPPT) and so on.

(3)

HAM_3, related to HCC, implements decentralized controls through negotiation among HAMs with communication. The main function is for all the HAMs to solve system power shortage together by regulating power output of each after the negotiation based on their current state and capacity. This function will be verified in Scenario IV, Section 4.

(4)

HAM_4, related to VCC, implements centralized controls and global optimizations under the master agent as for HAM_4. The functions of HAM_4 include system operation controls, system protection, economic optimization and so on, with the highest priority and the lowest time-constrain. Taking system operation control function as an example, first, HAM_4 of PCC agent makes an operation plan of the microgrid based on forecasts about generation and load. Second, the PCC agent dynamically regulates the plan according to the real-time state of the microgrid features, such as voltage, current and power measured by HAM_1 levels. Third, HAM_4 distributes operation commands to other HAMs for execution without refusal.

Table 1. Typical HAM in a microgrid. PCC: point of common coupling; DS: distributed energy storage; MT: micro-turbine; VF: voltage-frequency; SOC: state of charge; SOH: state of health; DG: distributed generator; PQ: active power and reactive power; MPPT: maximum power point tracking; and LC: load controller.

HAM	HAM_1 (LRC)	HAM_2 (LDC)	HAM_3 (HCC)	HAM_4 (VCC)
PCC	Measurement; breaker operation; local protection; island detection (switch to island); synchronizing close (connect to grid)	Power quality analysis; power out-of-limit analysis; non-scheduled island	Decentralized sharing power; coordinated control; scheduled island	System operation controls; system protection; forecast; economic optimization; scheduled island; local charging/discharging optimization; maintenance
DS/MT	Measurement; breaker operation; local protection; VF/droop control (stabilization)	System-state analysis; local charging/discharging optimization; SOC/SOH optimization; emergency control; power quality analysis	Decentralized sharing power; coordinated control
DG	Measurement; breaker operation; local protection; PQ control (generation)	MPPT control; island protection; low-voltage ride through; power quality analysis	Decentralized sharing power; coordinated control
LC	Measurement; breaker operation; local protection	Emergency load shedding; power quality analysis	Decentralized sharing power; coordinated control

Table 1. Typical HAM in a microgrid. PCC: point of common coupling; DS: distributed energy storage; MT: micro-turbine; VF: voltage-frequency; SOC: state of charge; SOH: state of health; DG: distributed generator; PQ: active power and reactive power; MPPT: maximum power point tracking; and LC: load controller.

HAM	HAM_1 (LRC)	HAM_2 (LDC)	HAM_3 (HCC)	HAM_4 (VCC)
PCC	Measurement; breaker operation; local protection; island detection (switch to island); synchronizing close (connect to grid)	Power quality analysis; power out-of-limit analysis; non-scheduled island	Decentralized sharing power; coordinated control; scheduled island	System operation controls; system protection; forecast; economic optimization; scheduled island; local charging/discharging optimization; maintenance
DS/MT	Measurement; breaker operation; local protection; VF/droop control (stabilization)	System-state analysis; local charging/discharging optimization; SOC/SOH optimization; emergency control; power quality analysis	Decentralized sharing power; coordinated control
DG	Measurement; breaker operation; local protection; PQ control (generation)	MPPT control; island protection; low-voltage ride through; power quality analysis	Decentralized sharing power; coordinated control
LC	Measurement; breaker operation; local protection	Emergency load shedding; power quality analysis	Decentralized sharing power; coordinated control

2.4. Operating Rules of Hybrid Multi-Agent Control Model

As stated above, each HAM consists of four sub-agents. It is necessary to establish some operating rules to make them work orderly or cooperatively. The details are discussed as below, and also shown in Figure 2:

(1)

First, the operation events of microgrids are divided into two basic types: internal events and external events. Internal events include the state changes of microgrids and CUs (e.g., the changes of system operation mode, frequency, voltage, power, power quality, etc.), faults and other emergencies, which can be measured and detected by HAM itself (LRC level). External events include dispatching command, manual command, interaction information and so on, obtainable through external communication.

(2)

When HAM (e.g., HAM1) detects an internal event happening, the operation process will be:

• First, HAM1_1 works immediately and uploads the operation result directly to HAM1_2 whether the operation is successful or not. HAM1_2 either optimizes or corrects the result or directly uploads the result to HAM1_3 by determining whether HAM1 can independently deal with the event or not. Once HAM1_3 obtains the event from HAM1_2, it works and sends request to other HAMs, among which there are cooperation relations established beforehand.
• Next, a solution based on a certain optimization object for the event is deployed through several times of interactions among the HAMs. Meanwhile HAM1_3 uploads the solution to HAM1_4 whether the interaction is successful or not. Lastly, if HAM1_4 has the authority to control the other HAMs, a new operation plan with global optimization is made by HAM1_4, and sent to HAM1_1–HAMn_1 respectively.

(3)

When HAM (e.g., HAM2) detects an external event happening, the operation process is briefly described as follows: first, HAM1_2 determines whether the command is from HCC level (e.g., HAM1_3) or VCC level (e.g., HAM1_4). If the command is from HAM1_3, HAM2_3 starts to work and participates in the interactions initiated by HAM1. If the command is from HAM1_4, HAM2_1 starts to work and performs the command directly.

Figure 2. Operating rules of HAM.

Figure 2. Operating rules of HAM.

3. Implementations of Hybrid Multi-Agent Control Model Based on the IEC 61850 Standard

3.1. Analysis on Necessity and Feasibility

When a smart grid is discussed, IEC 61850 [27] is usually mentioned since it is the first and the only global standard that considers all the communications needed in substations and even in the entire power system. IEC 61850 was originally developed for implementing interoperability among different devices and systems of different vendors in substation, and it currently has been extended and applied in various fields of power system [28,29,30,31,32]. The necessity of implementing HAM based on IEC 61850 is described as follows:

(1)

IEC 61850 is the global standard for power utility automation. Currently, most intelligent electronic devices (IEDs, such as protection, control and measurement unit) in power systems support IEC 61850. Hence, microgrids should adopt IEC 61850 in order to implement necessary interoperability with other IEDs and systems.

(2)

IEC 61850 combines the advantages of mainstream communication technologies (e.g., Ethernet) with the performance and security, enabling integration of protection, control, measurement and other functions. Hence, microgrids should adopt IEC 61850 due to its advances in communication and automation.

The feasibility of implementing HAM based on IEC 61850 is described as follows:

(1)

IEC 61850 is not merely a communication protocol, but also a modeling standard for automation. To be specific, IEC 61850 provides a series of data models and communication models as basic function entities to standardize various automation applications in power systems. Moreover, some concepts of MAS (e.g., intelligent agent unit, decentralized control, hierarchical control and so on) are consistent with some rules in IEC 61850 (e.g., abstract and free combination of automation applications, both vertical and horizontal function structure, and so on). Hence, it is convenient to develop HAM by using the data models and communication models defined in IEC 61850.

(2)

GOOSE is a communication model defined in IEC 61850, which provides the fast and reliable data transmissions among IEDs. GOOSE adopts some measures to ensure speed, convenience and reliability of data transmission, which are also the essence of HCF:

• GOOSE messages are directly mapped to Ethernet data packets for the sake of improving transmission efficiency and ensuring that the transmission time of each message is less than 4 ms. Hence, GOOSE is suitable for the time critical applications in microgrids and the current hardware and software conditions of practical devices due to its simple mapping rules.
• GOOSE adopts a publisher-subscriber mechanism in broadcast or multicast communication for the sake of conveniently transmitting data among different IEDs to implement different applications. Hence, it is convenient to implement multiple controls in different areas or under different time scales in HCF.
• GOOSE adopts the IEEE 802.1p priority tag for the sake of setting appropriate transmission priority levels for different operations. Hence, it is convenient to set priorities for different microgrid operations.
• GOOSE adopts enhanced retransmission mechanism with varying and increasing re-transmission intervals for the sake of improving transmission reliability as well as restricting network traffic.

In summary, IEC 61850, and particularly GOOSE, is suitable for implementing HAM according to the analysis discussed above. Furthermore, there are two major steps for implementations of HAM based on IEC 61850: (1) building a data model for HAM based on IEC 61850; and (2) building a communication model for HAM based on IEC 61850. The details will be discussed in the rest of this section.

3.2. Building a Data Model for Hybrid Multi-Agent Control Model Based on IEC 61850

In IEC 61850, all automation applications have been split into the smallest entities which can communicate with each other. These entities, called logical nodes, contain all functionally-related data and their communication attributes. Each logical node has a mnemonic name of four letters. Moreover, logical nodes for common applications are grouped into logical devices, possibly one or more logical nodes in each logical device, and one or more logical devices in each IED as well.

Since logical nodes and logical devices are the basis of the IEC 61850 data model, designing appropriate logical nodes and logical devices for HAM is the key to building a data model for HAM based on IEC 61850. Furthermore, a standard named agent communication language has been widely applied to the understanding of agents in MAS. In agent communication language, some specific data segments are defined to standardize the descriptions of agents’ attributes in order to conveniently implement interaction and cooperation among agents. Hence, these data segments should be embedded into the HAM data model. This paper proposes extended information models of IEC 61850 with agent communication language, and the details are described as follows.

3.2.1. Logical Nodes of Hybrid Multi-Agent Control Model

As stated in Section 2.3, each HAM (e.g., HAM1) consists of four sub-agents (e.g., HAM1_1, HAM1_2, HAM1_3 and HAM1_4) to match four control levels of HCF, respectively. Hence, three extended logical nodes (HAM_LNs, namely EXPC, EXPD and EXPR) are designed to contain the typical ingredients of the four sub-agents of HAM, respectively.

EXPC is designed to contain the typical ingredients of HAM1_3 and HAM1_4. EXPC consists of nine data objects as shown in Figure 3, which includes four common data objects describing common properties of HAM according to IEC 61850, and five new extended data objects representing the sociality and initiative properties of HAM according to agent communication language. The details of the five extended data objects are illustrated as below:

• AgentHost is used to denote HAM1 working as HAM1_3 or HAM1_4. If AgentHost is equal to 0, it means HAM1 works as HAM1_3, namely the common agent in the HCC level. If AgentHost is equal to 1, it means HAM1 works as HAM1_4, namely the master agent in the VCC level.
• AgentPerformative is used to define twenty-two types of actions and attitudes that might be adopted by agents during interaction as shown in Figure A1a in the Appendix.
• AgentContent is content of interaction, which includes two data attributes respectively contains some standardized control commands as shown in Figure A1b in the Appendix, and includes some supplementary specifications for content.
• AgentParticipants includes three parameters named sender, receiver and reply-to respectively, which are used to denote the sender, receiver and replier of specific interaction separately. AgentParticipants is useful for agents to understand interaction process.
• AgentConversation includes four parameters named id, reply-with, in-reply-to and reply-by, respectively. “id” is the unique identification of interaction. “reply-with” is used to describe whether message needs to be replied or not. The other two parameters are used to describe properties and time of message separately.

Figure 3. Data model of EXPC.

Figure 3. Data model of EXPC.

EXPD is designed to contain typical ingredients of HAM1_2, which represent the intelligence property of HAM. EXPR is designed to contain typical ingredients of HAM1_1, which represent the reflex property of HAM. The data models of EXPD and EXPR are similar as shown in Figure 4, whose structures are simpler than EXPC due to their fewer requirements for information exchange. The details of the two extended data objects in EXPD and EXPR are illustrated as below:

• ActState is used to denote working state of HAM1_1. If ActState is equal to 1, it means HAM1_1 is activated by internal or external operation event mentioned in Section 2.4. If ActState is equal to 0, it means HAM1_1 is not activated yet.
• ReaEvent is used to denote whether the operation event has been resolved by HAM1 or not. If ReaEvent is equal to 1, it means that HAM1 is capable of dealing with the event independently. If ReaEvent is equal to 0, it means that HAM1 demands help from other HAMs by activating HAM1_3 or HAM1_4.

Figure 4. Data model of EXPD and EXPR.

Figure 4. Data model of EXPD and EXPR.

3.2.2. Logical Devices of Hybrid Multi-Agent Control Model

As stated above, logical nodes should be grouped into logical devices in the IEC 61850 data model. Hence, three logical devices (HAM_LDs) are proposed to include EXPC, EXPD, and EXPR respectively and other functional logical nodes defined in IEC 61850. For instance, a data model of a photovoltaic inverter (e.g., HAM1) is shown in Figure 5, which includes HAM_LDs, HAM_LNs and other logical nodes related to the functions of the photovoltaic inverter.

Figure 5. IEC 61850 data model of photovoltaic inverter.

Figure 5. IEC 61850 data model of photovoltaic inverter.

The compositions and information exchange methods of HAM_LDs are:

• HAM1_LD1 includes EXPR and other functional logical nodes (e.g., measurement, breaker operation and so on).
• Moreover, HAM1_LD2 includes EXPD, and HAM1_LD3 includes EXPC. As to the meaning of these functional logical nodes, please refer to the IEC 61850-7 Standard Document.
• Furthermore, EXPR exchanges information with other logical nodes in HAM1_LD1 through calling internal functions or address mapping. EXPD exchanges information with EXPR and EXPC by calling internal functions. EXPC exchanges information with other HAM through GOOSE communication.

3.2.3. Representation of IEC 61850 Data Model

In the IEC 61850 Standard [24], a representation format named substation configuration description language is proposed to organize the four types of configuration files used to standardize represent abstract data model at different stages of the engineering process. Given the limited space in this paper, readers are referred to the IEC 61850-6 Standard document for the complete definitions of substation configuration description language. One segment of substation configuration description language file used to represent HAM_LNs and HAM_LDs mentioned above is shown in Figure A2 and Figure A3 in the Appendix.

3.3. Building Communication Model for Hybrid Multi-Agent Control Model Based on IEC 61850

As stated above, IEC 61850 GOOSE is an appropriate communication model for HAM. However, the original GOOSE model might not fully meet the requirements of HAM. Hence, a bidirectional interaction mechanism of GOOSE communication and other improvements are proposed below.

3.3.1. Bidirectional Interaction Mechanism of Generic Object Oriented Substation Event Communication

Originally, GOOSE adopts a unidirectional broadcast or multicast communication mechanism that needs to be enhanced to a bidirectional communication mechanism, in order to meet the bidirectional interaction requirement of HAMs. As mentioned above, the data objects of AgentParticipants and AgentPerformative are used to describe the members and actions of specific interaction, which are also contained into GOOSE messages as extended data segments. Hence, the receiver of a GOOSE message can know whether the message relates to itself according to the data segment of AgentParticipants, and whether the message needs to be replied to according to the data segment of AgentPerformative. Some rules of the proposed bidirectional interaction mechanism are briefly described as below:

• If AgentPerformative is equal to twenty-one or seven, it means that the GOOSE message is a subscribed or informed message, and the receiver needs to reply with an informed message whose AgentPerformative is equal to seven, as shown in Figure 6a.
• If AgentPerformative is equal to three, it means the GOOSE message is a proposal message, and the receiver needs to reply with an agree or failure message whose AgentPerformative is equal to one or six, as shown in Figure 6b,c.
• Similarly, agree or refuse messages are used to reply with request, request when or request whenever messages, reject proposal or accept proposal messages are used to reply to propose messages, confirm messages are used to reply with query if or query ref message, and disconfirm messages are used to reply with query if messages, as shown in Figure 6b–e, respectively.
• Moreover, propagate and proxy messages do not need to be replied to in normal conditions. Only if the receiver fails to completely understand the received propagate or proxy message, will it reply with a not understood message. Then, the sender of the propagate or proxy message will additionally send inform if, inform ref or cancel messages to further explain the validity, details or cancellation of the former message, respectively.

Figure 6. Bidirectional interaction mechanism of GOOSE communication: (a) replies of subscribe and inform messages; (b) reply of proposal message; (c) replies of request messages; (d) replies of propose message; and (e) replies of query messages.

Figure 6. Bidirectional interaction mechanism of GOOSE communication: (a) replies of subscribe and inform messages; (b) reply of proposal message; (c) replies of request messages; (d) replies of propose message; and (e) replies of query messages.

3.3.2. Improvements on Transmitting, Capturing and Decoding Generic Object Oriented Substation Event Messages

Some improvements on transmitting, capturing and decoding GOOSE messages are proposed, for the sake of improving communication efficiency and ensuring that HAM is suitable for the time critical controls in microgrids:

• Transmitting: some steps to transmit GOOSE messages are briefly shown in Figure A4a in the Appendix. Firstly, a data model of HAM including HAM_LDs, HAM_LNs and data objects is structured by parsing the substation configuration description language file organized beforehand. Secondly, a communication model of HAM is configured according to the GOOSE control block defined in the substation configuration description language file. Thirdly, the related dataset will be updated once the data value is changed due to an internal or external event. At last, a GOOSE message with the updated dataset is framed and transmitted under the rules of the data coding and communication mechanism defined in the GOOSE control block.
• Capturing: as stated in Section 3.1, a GOOSE message is directly mapped to an Ethernet data packet in order to reduce the time for framing, transmitting and decoding. Hence, it is necessary to bypass the surplus protocol stacks so that HAM could capture GOOSE messages from the data link layer directly. The procedures of capturing GOOSE message are briefly shown in Figure A4b in the Appendix.
• Decoding: to reduce the time on decoding, it is not necessary to decode the whole GOOSE message except for some data segments, such as appID, StNum, SqNum and so on. For instance, StNum and SqNum need to be decoded to know whether the event in the captured message is a new operation or not, as shown in Figure A5a in the Appendix. Moreover, some judgments need to be done in turn as shown in Figure A5b in the Appendix. The details of the process are discussed below:

  ➢

  Firstly, the receiver re-checks if the message is targeted to itself according to the AgentParticipants data segment.

  ➢

  Then, the message sent from either a master agent (working on the VCC level) or a common agent (working on the HCC level) is judged according to the appID and AgentHost data segments.

  ➢

  As a result, if the message is from the VCC level, the receiver will directly execute the order and reply with the results.

  ➢

  Subsequently, some other data segments, such as AgentPerformative, AgentContent and so on, are decoded respectively and responded to with the communication mechanism proposed above.

4. Simulation and Laboratory Tests

4.1. Simulation Study

Figure 7a shows a simulation of a multi-agent control system based on GOOSE by using the OPNET simulation tool, which is an advanced software tool for high-efficient analysis of performances and behaviors of various communication networks, technologies, protocols and applications. The simulation system includes one master agent and thirty-one common agents, and is used to study the communication delay of coordinated control among HAMs when they are connected together via 100 Mbps Ethernet. Figure 7b shows that the transmission time of the GOOSE message with 486 bytes in length is less than 1.5 ms. The simulation result illustrates that GOOSE is suitable for the time critical controls in microgrids.

Figure 7. Simulation study system by using OPNET: (a) simulation topology; and (b) simulation result.

Figure 7. Simulation study system by using OPNET: (a) simulation topology; and (b) simulation result.

4.2. Laboratory Test

4.2.1. Introduction to the Test System

A test system is constructed in the laboratory to verify the proposed strategies, models and implementations in this paper, as shown in Figure 8:

• Construction of the test system: the test system is a three-phases 380 V/50 Hz ac power bus connected with a wind turbine generation simulation system (WTSS), a photovoltaic generation simulation system (PVSS), a micro-turbine generation simulation system (MTSS), a measurement and control unit (MCCU) installed at the PCC and some actual loads (e.g., computers, printers, lamps, air conditioner, and so on) with LC. WTSS and MTSS respectively consist of a programmable dc power source (Chroma 62150H-600S) and a 3 kW general controllable current source inverter (SMA Windyboy3000). PVSS is simulated by a programmable ac power source (Chroma 61511).
• Development of HAM: based on the hardware design shown in Figure A6 in the Appendix, some devices named ECP plus are developed, instead of using common computers to implement the functions of the proposed HAM, specifically the controls in the HCC and VCC levels with GOOSE communication. The ECP plus hardware platform is based on an Atmel AT91RM9200 microcontrol unit, and the software platform is a VxWorks embedded operation system, and standard C programming language is used for software development. In the laboratory tests, each ECP plus (HAM) belongs to a specific CU as shown in Figure 8, and all the HAMs are connected together via 100Mbps fiber Ethernet. Photographs of the ECP plus and its main control board are respectively shown in Figure A7 and Figure A8 in the Appendix.

Figure 8. Laboratory test system for HAMs. MTSS: micro-turbine generation simulation system; PVSS: photovoltaic generation simulation system; WTSS: wind turbine generation simulation system; and MCCU: measurement and control unit.

Figure 8. Laboratory test system for HAMs. MTSS: micro-turbine generation simulation system; PVSS: photovoltaic generation simulation system; WTSS: wind turbine generation simulation system; and MCCU: measurement and control unit.

4.2.2. Test of Controls in Local Reaction Control and Local Decision Control Levels

Table 2 shows three test scenarios for verifying the controls in the LRC and LDC levels of the HAM of WTSS (HAM_WT).

Table 2. Descriptions of Scenarios I–III.

Scenario	Time	Control Function	Description
I	0–21 s	MPPT mode	WTSS increases generation while wind speed increases, and stays rated while the speed exceeds rated but below maximum limitation.
	>21 s	Generation limitation mode
II	0–21 s	MPPT mode	WTSS autonomously cuts off while wind speed exceeds maximum limitation.
	21–32 s	Generation limitation mode
	>32 s	Cut off WTSS
III	0–40 s	MPPT mode	WTSS intelligently adopts MPPT control strategy while wind speed is fluctuating.

Table 2. Descriptions of Scenarios I–III.

Scenario	Time	Control Function	Description
I	0–21 s	MPPT mode	WTSS increases generation while wind speed increases, and stays rated while the speed exceeds rated but below maximum limitation.
	>21 s	Generation limitation mode
II	0–21 s	MPPT mode	WTSS autonomously cuts off while wind speed exceeds maximum limitation.
	21–32 s	Generation limitation mode
	>32 s	Cut off WTSS
III	0–40 s	MPPT mode	WTSS intelligently adopts MPPT control strategy while wind speed is fluctuating.

The details of each scenario are described as follows:

• Scenario I: Scenario I presents a local control of HAM_WT where WTSS increases generation following the growth of wind speed (v) and stays rated while the wind speed exceeds the rated value (v_n = 10m/s) but stays below the maximum limitation (v_q = 25 m/s) as shown in Figure 9. During the process, the LRC level of HAM_WT continuously measures the generation of WTSS and updates it to the LDC level for further optimization, such as MPPT control and generation limitation and so on, once a change of wind speed is detected (at 8 s and 21 s).

Figure 9. Generation of WTSS in Scenario I.

Figure 9. Generation of WTSS in Scenario I.

• Scenario II: Scenario II presents WTSS cuts off generation when the wind speed exceeds the maximum speed limitation as shown in Figure 10. From 0 s to 32 s, the situation is the same as Scenario I. After 32 s, HAM_WT detects that the wind speed has exceeded the maximum speed so that LDC level instructs the LRC level to cut off WTSS, presenting a sudden decrease of generation in Figure 10.

Figure 10. Generation of WTSS in Scenario II.

Figure 10. Generation of WTSS in Scenario II.

• Scenario III: Scenario III presents that WTSS adopts a MPPT control strategy when the wind speed fluctuates in a certain range (v_r < v < v_n, v_r = 3 m/s) as shown in Figure 11.

Figure 11. Generation of WTSS in Scenario III.

Figure 11. Generation of WTSS in Scenario III.

4.2.3. Test of Controls in Horizontal Cooperation Control and Vertical Cooperation Control Levels

Table 3 shows the descriptions of Scenario IV for verifying the controls in HCC and VCC levels of HAMs.

Table 3. Descriptions of Scenario IV.

Time	Control level	Control function
8–15 s	LRC	The input of WTSS&PVSS is regulated to zero.
15 s	LDC	Internal event (excess of CUHA.MMXU.TotW) is detected.
15–17 s	LDC	Continuous detecting of the excess.
17 s	HCC	HCC of HAM_CU starts negotiation.
18–28 s	HCC and LDC	MTSS performs a two-stage decrease of CUHA.MMXU.TotW.
28 s	VCC	VCC of HAM_CU commands load shedding.
28–30 s	LDC	LC performs load shedding.
>30 s	HCC	CUHA.MMXU.TotW is less than 1.65 kW. The coordinated control is finished.

Table 3. Descriptions of Scenario IV.

Time	Control level	Control function
8–15 s	LRC	The input of WTSS&PVSS is regulated to zero.
15 s	LDC	Internal event (excess of CUHA.MMXU.TotW) is detected.
15–17 s	LDC	Continuous detecting of the excess.
17 s	HCC	HCC of HAM_CU starts negotiation.
18–28 s	HCC and LDC	MTSS performs a two-stage decrease of CUHA.MMXU.TotW.
28 s	VCC	VCC of HAM_CU commands load shedding.
28–30 s	LDC	LC performs load shedding.
>30 s	HCC	CUHA.MMXU.TotW is less than 1.65 kW. The coordinated control is finished.

Figure 12 shows the processes of negotiations among HAMs in scenario IV, and the details of the processes are described as follows:

• From 8 s to 15 s, the inputs of WTSS and PVSS are gradually regulated to zero to simulate the intermittent nature of renewable energies. At 15 s, the LRC level of HAM_CU measures the active power absorbed from the connected grid (CUHA.MMXU.TotW) having exceeded 15% of the total rated generation of test system (TotW), which is made up as below:

  $$\text{TotW}=\text{PVHA}.\text{ZINV}.\text{OutWSet }(5\text{ kW})+\text{WTHA}.\text{ZINV}.\text{OutWSet }(3\text{ kW})+\text{MTHA}.\text{ZINV}.\text{OutWSet }(3\text{ kW})$$

  (1)

  PVHA.ZINV.OutWSet, WTHA.ZINV.OutWSet and MTHA.ZINV.OutWSet are the rated output power of PVSS, WTSS and MTSS, respectively. 15% of TotW is 1.65 kW. After 2 s of continuous detection, the LDC level confirms the excess and informs the HCC level of HAM_CU.
• At 17 s, since HAM_CU has no ability to control the power flow independently, it broadcasts an inform message to other HAMs. The inform message organized as a GOOSE message is shown in Figure A9 in the Appendix. The HCC levels of HAM_PV, HAM_WT, HAM_MT and HAM_LC obtain the coordinated control invitation from the received inform message, and deliver it to the LDC levels.

Figure 12. Processes of negotiations among HAMs in Scenario IV.

Figure 12. Processes of negotiations among HAMs in Scenario IV.

Then, HAM_MT and HAM_LC accept the invitation since they have certain ability to regulate power and an inform message is respectively sent from HAM_MT and HAM_LC to HAM_CU as an affirmative reply, whereas HAM_PV and HAM_WT refuse the invitation due to their MPPT operation modes and send a respective failure message to HAM_CU as a negative rely. This means PVSS and WTSS will not take part in the power flow control at the PCC.

Next, a query ref message is transmitted from HAM_LC to HAM_CU to inquire what other HAMs can take part in the coordinated control. Then, HAM_CU sends an inform message to tell HAM_LC that HAM_MT also participates in the coordinated control.

After that, the query if message sent by HAM_LC and the inform message replied by HAM_MT show the process of negotiation between these two HAMs on how the HAM_MT increases its output power to decrease CUHA.MMXU.TotW efficiently, which can be observed in Figure 12:

• From 18 s to 28 s, HAM_LC and HAM_MT implement the negotiation stated above twice, and then HAM_MT increases its output power twice resulting in a two-stage decrease of CUHA.MMXU.TotW that can be observed in Figure 13, and a detailed process is presented in Table 3. During the process, the LRC level of HAM_CU continuously measures CUHA.MMXU.TotW.
• At 28 s, HAM_CU measures that CUHA.MMXU.TotW is still more than 1.65 kW. Hence, a request message is transmitted from HAM_CU to HAM_LC commanding LC to perform load shedding directly. Then, HAM_LC replies with a confirm message and acts according to the setting values in the command message. As a result, CUHA.MMXU.TotW decreases rapidly as shown in Figure 13.
• After 30 s, CUHA.MMXU.TotW is less than 1.65 kW, a propagate message is transmitted from HAM_CU to HAM_LC to inform LC that the power flow control has finished and load shedding need not be implemented. Subsequently, a propagate message is transmitted from HAM_LC to HAM_MT to inform MTSS that the coordinated control is finished and the output should remain constant.

Figure 13. Changes of the active power absorbed from the connected grid in Scenario IV.

Figure 13. Changes of the active power absorbed from the connected grid in Scenario IV.

Scenario IV shows how HCC and VCC levels make DGs and LCs effectively participate in coordinated control through the interactions among DGs and LCs, or under the control of a master agent, and Table 4 shows the summary of controls implemented in Scenario IV.

Table 4. Summary of controls implemented in Scenario IV.

Level	Times	Description
LRC	1	Measurement on the active power absorbed from the connected grid implemented by HAM_CU.
LDC	4	(1) Optimal power flow control implemented by HAM_CU. (2) Judgments on the ability of power regulation implemented by HAM_MT, HAM_LC, HAM_PV and HAM_WT. (3) Increasing the output implemented by HAM_MT. (4) Load shedding implemented by HAM_LC.
HCC	5	(1) Negotiation among HAM_CU, HAM_MT, HAM_LC, HAM_PV and HAM_WT to share the control object and decide which HAMs take part in the coordinated control. (2) HAM_LC inquires what other HAMs take part in power regulation from HAM_CU. (3) The first negotiation between HAM_LC and HAM_MT to make WTSS increase its output. (4) The second negotiation between HAM_LC and HAM_MT to make WTSS increase its output further. (5) Negotiation among HAM_CU, HAM_LC and HAM_MT to share the control result.
VCC	1	HAM_CU requests HAM_LC to implement load shedding.

Table 4. Summary of controls implemented in Scenario IV.

Level	Times	Description
LRC	1	Measurement on the active power absorbed from the connected grid implemented by HAM_CU.
LDC	4	(1) Optimal power flow control implemented by HAM_CU. (2) Judgments on the ability of power regulation implemented by HAM_MT, HAM_LC, HAM_PV and HAM_WT. (3) Increasing the output implemented by HAM_MT. (4) Load shedding implemented by HAM_LC.
HCC	5	(1) Negotiation among HAM_CU, HAM_MT, HAM_LC, HAM_PV and HAM_WT to share the control object and decide which HAMs take part in the coordinated control. (2) HAM_LC inquires what other HAMs take part in power regulation from HAM_CU. (3) The first negotiation between HAM_LC and HAM_MT to make WTSS increase its output. (4) The second negotiation between HAM_LC and HAM_MT to make WTSS increase its output further. (5) Negotiation among HAM_CU, HAM_LC and HAM_MT to share the control result.
VCC	1	HAM_CU requests HAM_LC to implement load shedding.

4.2.4. Analysis on Generic Object Oriented Substation Event Communication Delay

• Scenario IV was tested five times. The coordinated control was successful every time, and the delay including computing time, transmission delay of GOOSE message and control execution time was always less than 1 s, as can be seen in Figure 13.
• In order to analyze the communication delay in Scenario IV more specifically, the negotiation between HAM_LC and HAM_MT based on GOOSE messages mentioned above was tested several times. The value of the communication delay obtained with Wireshark (a kind of package capturing tool) shows that it takes approximately 1 ms from HAM_LC sending a query ref message to HAM_LC receiving an inform message from HAM_MT as can be seen from Figure A10 in the Appendix, which illustrates that GOOSE messages are suitable for the time critical controls in a microgrid.
• The realization of control in HAM_3 and HAM_4 in different scenaerios such as Scenario IV is essentially based on an open-loop control instead of real-time feedback control. It means that the delay can only have an influence on the time of corresponding controls’ achievement but not the stability of the system.

5. Conclusions and Future Work

This paper presents a HCF comprising four control levels to implement flexible control strategies in microgrids, and a hybrid MAS control model of four sub-agents to implement the control framework. Furthermore, this paper proposes the implementation of HAM based on the IEC 61850 Standard as a practical method to apply a variety of control strategies and models in actual microgrids and devices. The extended information models of IEC 61850 with agent communication language, bidirectional interaction mechanism of GOOSE communication and some improvements on transmitting, capturing and decoding GOOSE message are proposed in the implementation scheme.

The simulation tests illustrate that the proposed HAM with GOOSE communication is suitable for time critical controls in microgrids. The laboratory tests show that LRC and LDC levels in HAM can make DG regulate their power generation autonomously based on the local optimization, HCC and VCC levels in HAM can make DGs and LCs effectively participate in coordinated control through the interactions among DGs and LCs, or under the control of a master agent, and the control delay is less than 1 s. The results of laboratory tests verify that the proposed strategies, models and implementations can meet a variety of control purposes in microgrids.

The test system and scenarios presented in this paper are simpler than actual microgrids, and the number of HAMs is limited. Building a test system close to some actual microgrids and doing more complex tests are the future works to be done. Meanwhile, applying some advanced decentralized control algorithms, such as average consensus control, pinning control and so on, based on the proposed framework, models and implementations in this paper, is also future tasks to be done.