SDN-OpenFlow Topology Discovery: An Overview of Performance Issues

Abstract

Software-defined networking (SDN) is an innovative architecture that separates the control plane from the data plane to simplify and speed up the management of large networks. This means the control logic has been moved from the network hardware level to the centralized control management level. Therefore, the use of the OpenFlow Discovery Protocol (OFDP) is one of the most common protocols used to discover the network topology in a data plane and then transmit it to the control plane for management. However, OFDP has various shortcomings in its performance such as exchanging too many messages between both levels (control and data), which in turn increases the load on the SDN-Controller. Additionally, since the application layer depends entirely on the network topologies plotted in the control plane, it is very important to obtain accurate network topology information from data plane. Therefore, after providing background on topology discovery protocols to the reader, we will concentrate on performance issues. The present study identifies and discuss the primary concerns involved in the complex query process, infrastructure, influencing factors, and challenges for the topology discovery process. Furthermore, this paper will present several recent studies that have overcome and enhanced these issues. In addition, open discussion and future work concerning these issues are also discussed.

1. Introduction

Software-defined networking (SDN) architecture is a promising solution to overcomes the limitations of traditional network architecture in terms of control, scalability, and management [1,2]. The basic idea of SDN is to separate the control plane from the data plane [3], where the control logic is extracted from network hardware and is centralized into an independent control plane. However, the process of separating the control plane from the data plane is not new, but it has limited to some applications such as NETCONF, SANE, and Ethane [4,5]. However, recently, this method of managing networks of various kinds has been introduced. The control plane can physically include a stand-alone SDN-Controller or multiple SDN-Controllers that collaborate among themselves and then act as a central SDN-Controller [6]. However, the standard process for separating SDN levels is illustrated in Figure 1.

Based on the open networking foundation (ONF) reference model for SDN architecture [7], the SDN infrastructure layer (data plane) includes the forwarding devices that forward packets based on a set of streaming rules configured by the control layer (SDN-Controller) [8]. The SDN-Controller also acts as a proxy between the infrastructure layer and the application layer. It translates high-level instructions at the application layer into low-level rules and then forwards them to the SDN-Switches at the infrastructure layer [9]. In the application layer, the network applications used such as adaptive routing, network management, traffic monitoring [10,11], intrusion detection system, and intrusion prevention system [12]. Moreover, the same figure shows that these SDN layers communicate among themselves using two interfaces; southbound interface (SI) and northbound interface (NI). NI is needed to support and optimize a variety of networking applications, and it is used at the application layer to query the state of the infrastructure layer through the control layer. Examples such as the network topology, flow statistics, link bandwidth, and other information about the state of the network are queried. Furthermore, the application layer also uses it to control the infrastructure layer to implement high-level policies such as new configurations, respond to topology changes, or traffic requirements [13,14,15,16,17,18]. SDN-Controllers such as OpenDaylight [19], Floodlight [20], NOX [21], Ryu [22], Onix [23], and Beacon [24] are defined by their specific northbound application programming interfaces (APIs).

On the other hand, SI acts as a link between the SDN-Controller and SDN infrastructure layer to establish settings and manage the SDN-Switches. Also, there is no standard southbound interface and there are different protocols that are used for this proposal such as ForCES [25], POF [26], OpFlex, OpenState, and OpenFlow [27]. However, OpenFlow is the most widely used protocol [28,29]. OpenFlow [28] defines a physical channel as a dedicated TCP link between SDN-Controllers (control layer) and SDN-Switches that serves as an interface for exchanging control messages using the OpenFlow protocol and also using an optional encryption mechanism called Transport Layer Security (TLS) [17,18,30,31]. Additionally, the most important role of the SDN-Controller is to maintain a consistent network topology for the infrastructure layer. The network topology maintained by the SDN-Controllers are entirely dependent on OpenFlow Discovery Protocol (OFDP) [32]. Therefore, any falsification or expiration in the data of network topology inside the SDN-Controller will directly affect the SDN-dependent services and applications [33,34,35]. As a result, the topology discovery service in the SDN-Controller is one of the most important.

Breitbart et al., in [36], define the network topology as a description of the physical communication relationships between network devices in networks (how SDN-Switches communicate with each other) and how the hosts communicate with them. Therefore, the topology discovery protocol consists of three main processes: SDN-Switch discovery, link discovery (between SDN-Switches), and host discovery [34]. The SDN-Controller uses link discovery to discover links between forwarding devices in the infrastructure layer. Therefore, the OFDP will be the mediator between SDN-Controller and network devices in the infrastructure layer to discover these links. OFDP uses the Link Layer Discovery Protocol (LLDP) message format for that purpose and the SDN-Controller sends a large number of LLDP advertisements at relatively large, fixed intervals to each active SDN-Switch port in the network to discover links between SDN-Switches. The SDN-Controller uses Packet_Out OpenFlow message to send the LLDP advertisement to each active SDN-Switch port, and the SDN-Switch on the other side (data layer) will send the link information by encapsulating the LLDP packet in a Packet_In OpenFlow message. This process is cyclic every 10 s, which means that every 10 s the SDN-Controller will send several Packet_Out messages equal to the number of active SDN-Switch ports [34,35,37,38,39]. The results of the previous processes are a burden on the resources of the SDN-Controller and are inefficient in detecting network topology changes in a timely manner. This leads to many studies in the literature that OFDP has critical limitations in terms of performance which makes it inefficient especially for large SDN networks. Another issue related to topology discovery between SDN and non-SDN networks [40,41]. Therefore, attention to improving the performance of ODFP is an important priority that must be discussed to improve the performance of SDN technology. Different studies have been working to improve OFDP protocol in [38,42,43,44,45,46], but still, this domain needs more attention and research.

Moreover, in the literature, many surveys discuss SDN in general [13,47,48], and others focus on the features that have been added to SDN [49,50]. Another group discussed the adoption of SDN architecture [51,52,53]. Furthermore, most of the SDN OpenFlow surveys have discussed the security and scalability [3,15,29,31,51,54,55], and the authors in [56] had worked on the OpenFlow in term of performance but did not discuss the details of the topology discovery process in the SDN. Therefore, this survey focuses on the performance issues in SDN-OFDP networks. The main contributions of this paper can be summarized as follows:

• Explain in-depth how OFDP works, the factors that affect its performance, and OFDP limitations.
• A survey of the recent existing techniques found in the literature in response to enhance the performance of OFDP. The pros and cons of each technique also are highlighted.
• OFDP’s open challenges and future research solutions.

The rest of the paper is organized as follows. Section 2 introduces a brief background about SDN architecture and OpenFlow southbound interface. Section 3 presents the topology discovery in SDN networks and its de facto OFDP, performance issues in OFDP, and the current proposals to enhance the performance of OFDP. Section 4 discusses the pros and cons of the proposals for OFDP and Section 5 concludes the paper.

2. Topology Discovery in SDN Networks

As we discussed in the introduction, the SDN technology is based on separating the control layer from the data layer (forwarding devices) to facilitate network scalability and management operations. However, the process of discovering the network topology relies on cooperation between the SDN-Controller and forwarding devices layers, and the responsibility lies primarily with the control layer. The control layer topology discovery task mainly consists of three main operations: SDN-Switch discovery, link discovery (i.e., links between SDN-Switches), and host discovery [33,34] as illustrated in Figure 2.

Since we need OpenFlow to act as a data pipeline between the control layer and the data layer, the loading of the topology discovery process is entirely on the controller layer. Therefore, the OpenState protocol has been proposed as a modification of OpenFlow that attempts to divide the load of the topology discovery process between SDN-Controller and SDN-Switches. As a result, OpenFlow SDN-Switches can be directly programmed, allowing them to implement redirection rules without relying solely on the remote controller. As a future improved release of OpenFlow, OpenState has not yet been deployed [57].

The OpenFlow architecture combines two layers through the use of flow tables in the forwarding devices layer. Moreover, each flow table entry contains three features (rule, action, and statistics) [15] and each flow table contains action fields that are linked with each flow entry. These flow tables data and commands are transmitted between two layers by using a control channel. In detail, the use of flow table and control channel techniques will be discussed in Section 2.3 and Section 2.4.

However, the topology discovery services are discussed in detail as the follows.

2.1. SDN-Switch Discovery

According to [58] version 1.5.1, each SDN-Switch must consist of a set of tables, namely, a flow table, a matching table, and a missing table, as well as a control channel to monitor the changing flow in the different SDN-Switches via SDN-Controller. Therefore, in the SDN-OpenFlow, routing processes are based on standard flow tables rather than destination (i.e., IPs or MAC) as in the traditional networks. The flow table is used in conjunction with the logical data structure where packets are processed based on the list of priority entries in these flow tables. In each flow entries for up to 15 fields can be stored in OpenFlow version 1.10 [57], 5 of which are mandatory and the rest are optional. The matching, action, priority, timeout, and counter are the most common fields. Moreover, the SDN-Controller pins these flow entries to the flow table in two approaches: reactive and proactive, and the SDN-Controller determines which one is based on the occurrence of some events. When network activity starts, the SDN-Controller in a reactive approach does not initialize the flow table with any rules. The SDN-Controller will insert rules into the flow table whenever data arrives at switches while the network is running. For the proactive approach, the SDN-Controller will pin the flow entries into the flow table beforehand, when the network is started. The choice of rules is essential in optimizing network performance, particularly in large-scale networks. When packets reach at a switch while the network is running, the incoming packet flow matches the flow entries in the flow table. If no match is found, the SDN-Switch will call the SDN-Controller to request that entries to permit the packet to reach its destination. This includes frequent connections between the SDN-Controller and the SDN-Switch as well as delays before the packet can be transmitted to the next hop. The proactive technique was implemented to reduce the amount of time SDN-Switches and SDN-Controllers had to communicate.

In addition, each OpenFlow SDN-Switch is configured with the IP address and the TCP port number of the SDN-Controller. Thus, to join the network, the OpenFlow SDN-Switch creates a TCP session using a three-way handshake (SYN, SYN/ACK, ACK) to initiate the communication with the SDN-Controller. Next, the SDN-Controller sends an OFPT_Features_Request message to the SDN-Switch requesting its current configuration as Media Access Control address (MAC address) and network interfaces. Then, the SDN-Switch will reply with an OFPT_Features_Reply message which contains the requested information. The SDN-Controller stores and uses such information for future network management tasks including re-processing of topology discovery [39]. Figure 3 presents the process of establishing SDN-Switch discovery in SDN networks.

After the process of establishing OpenFlow SDN-Switch connection, flow tables will contain the SDN-Switch header information and actions (responsible for giving commands). As illustrated in Figure 2, when the packets are streamed from the data plane to the SDN-Switch, each incoming packet will be checked against the flow tables, when the packet’s matching header meets one in the pipeline flow tables, the rules related with the flow entry will be triggered. For each successful match between incoming packet and flow table entries, the counter field will be increased. When the packet flow reaches the input flow table (ingress) and a match is made against flow entries, if no match occurs, it moves to the next flow table using GoTo-Table instruction and then again performs a match with flow entries. This process continues seamlessly until all flow tables are finished, and therefore, the packet will be treated as miss_flow if it does not get a match in one of the flow tables. According to the instructions in the miss_flow entry, it either drops the packet or resends it to the other flow table [56]. The packet flows via an OpenFlow SDN-Switch in both directions (ingress and egress) are illustrated in Figure 4.

In the case of the network traffic being huge and complex, many unknown stream packets will arrive at the forwarding node, thus, it will produce a large number of Packet-In messages. On the other hand, sending a flow request (Packet-In message) to the SDN-Controller for each unknown packet will confuse the SDN-Controller because the SDN-Controller has to compute the forwarding rules for each new packet and then install it to the flow tables in all the data forwarding nodes (SDN-Switches). This high volume of traffic and computational overhead will cause SDN-Controller overhead and increase the time it takes for flow rules to be placed, affecting network efficiency and scalability [1,2].

Moreover, [59] claimed that the data center with a 4 K server can handle up to 200,000 flows per second. Another study found that the average flow width was roughly 20 packets per flow, with latency between flows less than 30 milliseconds [60]. These costs are very high, but the memory available to hold sending entries is limited. However, the entries for typical OpenFlow SDN-Switch flow tables were stored in Ternary Content Addressable Memory (TCAM), a type of high-speed memory that allows looking up a continuous flow entry in a clock cycle (1). Although the TCAM search is fast, its capacity is limited to a few thousands of entries [61]. On another hand, Increasing the TCAM size raises additional issues such as cost, and will require high power consumption. Therefore, the researchers tried to optimize the flow schedule to take full advantage of it.

2.2. Host Discovery

In the process of communicating and exchanging packets (in and out) in OpenFlow protocol, and when an OpenFlow SDN-Switch receives a packet that does not match any flow rule in its flow table. In this case, one possibility is that a new host is connecting to the network. The new host starts sending the packets to SDN-Switch, then the OpenFlow SDN-Switch will encapsulate that package with a Packet_In message and send it to the SDN-Controller. The SDN-Controller will in turn use this Packet-In message to discover the hosts on the network, then the SDN-Controller will extract the host’s location (i.e., to which SDN-Switch port it is connected to), host’s IP address, and its MAC address from Packet_In messages [32].

2.3. Link Discovery

The goal of the link discovery process is to discover the existing links between connected OpenFlow SDN-Switches and also to efficiently detect changes to the network topology. Additions and deletions are among the most common examples of network topology changes. Link deletion occurs when an existing link is removed (physically) or access fails due to other reasons. The link deletion also occurs when an existing SDN-Switch is removed or failed to access for various other reasons as well. Moreover, the processes of link additions will be similar to the processes of deletions [38]. In all of these processes, the occurrence of link changes is directly related to the SDN-Controller. Therefore, given the importance of this part in our topic, further discussion will be given in Section 2.2.

2.4. Link Discovery Protocol

As we noted in the introduction that there is no standard protocol in SDN networks to discover the links between SDN-Switches, and most of the existing SDN-Controllers use Link Layer Discovery Protocol (LLDP) [33,38,39] such as OpenDaylight [19], Floodlight [20], POX [62], Ryu [22], Beacon [24], Cisco Open SDN-Controller [63], and Open Network Operating System (ONOS) [34,38,43] to discover these links. LLDP is considered a layer-2 (Data link) protocol that is used by network devices to their identity, capacities, and neighbors’ devices on a Local Area Network (LAN) based on IEEE-802. Moreover, each LLDP discovery message is embedded in a layer-2 frame, which is a type of Ethernet and the Data Unit called (LLDPUD) [56]. The data obtained by LLDP is placed into the management information database of the SDN-Switches, which can then be queried while crawling the network’s nodes to retrieve the network topology using a network management protocol.

At a later stage, the NOX SDN-Controller [21] implemented the LLDP development process to improve the discovery of the link between SDN-Switches and created the first version of the OFPD protocol [64]. OFDP does not depend on a centralized control such as LLDP (i.e., switches send and receive LLDP advertisements autonomously). It is a request-response discovery protocol that can send a Packet-In message to the SDN-Controller to receive the discovery information collected. Nevertheless, OFDP uses the LLDP packet format with few modifications and operates in a slightly different way than LLDP protocol for compatibility with the SDN architecture, where the control logic is central to the SDN-Controller. Therefore, OFDP SDN-Switches do not initiate LLDP advertisements but the SDN-Controller has full control over the link discovery process.

Table 1. Similarities and differences between LLDP and OFDP protocols [64].

Features	LLDP	OFDP
Type of Ethernet frame	LLDP’s EtherType = 0 × 88cc	OFDP’s EtherType = 0 × 88cc
Destination address of the frame	bridge-filtered multicast MAC (01:80:C2:00:00:0E)	normal multicast MAC (01:23:00: 00:00:01)
Mode of operation	Advertisement only	Advertisement only
What will the SDN-Switches do with advertisements?	SDN-Switches will not forward LLDP advertisements	SDN-Switches will forward OFDP advertisements
SDN-Switches’ neighbor table	SDN-Switches that support LLDP build a table for directly connected neighbors	OpenFlow SDN-Switch does not keep any information about its directly connected neighbors
How is the topology obtained?	By crawling the neighborhood tables of SDN-Switches	By inferring the information from LLDP Packet-In messages

presents the main differences between LLDP and OFDP protocols.

The SDN-Controller initiates the discovery process in OFDP by sending an LLDP discovery announcement encapsulated in a Packet_Out message to the forwarders (parent SDN-Switches) that are directly associated with OpenFlow using a multicast address. When the forwarder device receives the announcement message, it floods all of its ports with an LLDP discovery announcement, and the only SDN-Switch that supports OpenFlow updates its OFDP table.

To illustrate how OFDP uses LLDP advertisement messages, the LLDP considers a layer 2 frame that consists of a header and a payload [39] as shown in Figure 5. In the header portion of the frame, the Ethertype field is set to 0x88cc, and the destination MAC address field is set to a multicast address as we discussed in Table 1. The Ethertype field is used by OpenFlow SDN-Switches to distinguish LLDP frames from others. The payload portion is called LLDPDU which is shaded grey. The payload consists of a number of fields with a Type-Length-Value (TLV) structure and ends with the “End of LLDPDU TLV” field. Some TLV fields in LLDPDU are mandatory while the others are optional. The mandatory fields contain the information that the SDN-Switch wants to advertise to its neighbor which is: Chassis-ID (which is a unique switch identifier), Port ID (which is its egress port), and time to live.

Therefore, to illustrate the OFDP function, we explain the cooperation between the ODFP sections based on each other, as illustrated in Figure 6. The SDN-Controller sends every specified period (i.e., 10 s) an LLDP packet encapsulated with a Packet-Out message to each active port in each SDN-Switch [38]. From Figure 6, the SDN-Switch (s1) has three active ports, which means three LLDP packets and each of these packets has Port-ID TLVs and a Chassis-ID configured accordingly. In OpenFlow protocol, if the SDN-Controller wants to send a packet to the OpenFlow SDN-Switch it will encapsulate it in a Packet-Out message. The Packet_Out message structure contains a field called the instruction field. This field is responsible for deciding what the SDN-Switch should do in that packet. Therefore, the LLDP packet is obtained from all ports in SDN-Switch except for the one connected to the SDN-Controller. This port communicates with SDN-Controller via an OpenFlow Packet-In message. Moreover, it is used to collect and aggregate all the port information in each SDN-Switch into a single “Packet-In” message and sends it to the SDN-Controller according to a proactive rule installed in the flow tables of all SDN-Switches. In the Figure 6 scenario, the LLDP packet on port-ID 1 is sent from SDN-Switch (s1) and received by SDN-Switch (s2) through port 1 as well.

Moreover, the Packet_Out message has a field called instruction which is configured to forward the encapsulated LLDP out of the corresponding port on the SDN-Switch [39,42]. Next, the SDN-Switch (s1) will, in turn, receive Packet-Out messages, de-decompile the LLDP packet from the Packet-Out message, and forward only the LLDP packet that has exited from each matching port (based on the port’s MAC address in LLDPDU). When SDN-Switch (s2) receives an LLDP packet, it will parse the LLDP packet, write its SDN-Switch Chassis-ID, and add Port-ID (i.e., the port through which the SDN-Switch received the LLDP packet). Then, SDN-Switch (s2) will encapsulate the LLDP packet in a Packet_In message and send it to the SDN-Controller. The SDN-Controller will in turn parse the Packet_In message and discover the new links represented by the mapping between s1 and s2 [65]. However, the same method is repeated to discover the remaining links in the network [39].

2.5. Control Channel

Each OpenFlow Logical SDN-Switch is connected to an OpenFlow SDN-Controller through the OpenFlow channel. The SDN-Controller installs and maintains the SDN-Switch using this link, collects events from the SDN-Switch, and transmits data from the SDN-Switch to the SDN-Controller. The SDN-Switch’s control channel can handle a single OpenFlow channel with a single SDN-Controller or several OpenFlow channels with multiple SDN-Controllers that share SDN-Switch management. In addition, the communication connection between both the data stream and the OpenFlow channel is managed independently, but it must be liable to the OpenFlow SDN-Switching protocol rule. Moreover, the OpenFlow channel is normally performing over TCP and is secured using TLS, [58].

Furthermore, if an SDN-Switch becomes connected to a group of SDN-Controllers, the SDN-Controllers status update should be sent to the SDN-Switch with only one SDN-Controller and the other SDN-Controllers in ‘standby’ mode if the first controller stops. Additionally, the SDN-Switch should provide an improved Controller-Status indication for all SDN-Controllers when the OpenFlow channel is reconnected. Moreover, If the SDN-Switch loses connectivity to all SDN-Controllers for various reasons, including echo request timeouts, TLS session timeouts, or other disconnects, it should switch to “Fail-Safe Mode” or “Fail-Standalone Mode”, depending on the SDN-Switch design and configuration. The only difference in the SDN-Switch’s behavior in Fail-Safe Mode is that packets and messages addressed to the SDN-Controllers are dropped. In Fail-Safe Mode, flow entries should terminate according to their timeouts, while in “Failure-Standalone”, the SDN-Switch uses the OFPP_NORMAL reserved port to process all packets. In other words, the SDN-Switch acts like an old Ethernet SDN-Switch or router. Additionally, the SDN-Switch can use flow tables in any way it wants when in “Fail-Standalone” and can delete, add, or edit any flow entry. By the way, only Hybrid SDN-Switches usually have a Fail-Standalone mode [58].

Moreover, according to [66,67], a slow control channel significantly reduces data layer throughput and response time in addition that it will threaten network availability. Therefore, several recent works have been proposed to help maintain OpenFlow availability.

2.6. Performance of the Link Discovery (OFDP)

One of the most important services of the SDN-Controller is to provide an updated and comprehensive network topology under its control. All network applications depend entirely on the network topology received from the SDN-Controller. Thus, any issues with the performance of this service in the SDN-Controller will negatively affect the performance of the entire SDN [6,46,68,69]. These performance issues are more noticeable in the case of dynamic and large networks.

Regarding the topology discovery service, several experiments were performed on different SDN-Controllers with only one discovery unit running. The results showed that when the number of SDN-Switches (i.e., network size) reaches a certain limit, there is a significant increase in the CPU usage of the SDN-Controller and a significant decrease in the network performance [39,69,70]. In [71], the authors empirically evaluated the performance of the OpenDaylight and the ONOS SDN-Controllers in terms of updating the topology discovery process. The authors used the topology discovery time and throughput as performance metrics, and their results showed that the ONOS performed better in terms of network throughput in the event of topology changes while the OpenDaylight outperformed ONOS in topology discovery time. Therefore, in this subsection, we will analyze how the network size affects the SDN-Controller performance during the network topology discovery process.

2.6.1. OFDP in Huge and Dynamic Environments

However, there is not much research analyzing the OFDP performance on the SDN networks and most of the researchers in OpenFlow performance analysis have concentrated on SDN-Controller types such in [55,72,73,74]. Therefore, ODFP performance analysis would be appropriate to open up this issue to the researchers to search in these domains.

In this work, we will discuss the performance issues of topology discovery in the OpenFlow protocol and its impact on the SDN-Controller, where OFDP will have a major role.

In [69], the authors investigated the OFDP performance for transport networks (i.e., backbone service provider networks) and found that when carrier rank requirements are met, the transport networks should recover from link failure within a maximum of 50 ms. Thus, to achieve a 50 ms recovery time for OpenFlow-based carriers, the topology discovery service at the SDN-Controller should run approximately every 10 ms. Therefore, the SDN-Controller must check hundreds of LLDP Packet_Out messages per active SDN-Switch port every second to detect one direction per link. Moreover, it also must receive and process two hundred LLDP Packet_In messages every second for each link and end-to-end tunnel. In addition, transport networks have hundreds of links and thousands of tunnels. This means that the SDN-Controller has to handle millions of messages per second just to monitor the health of the network. This undoubtedly imposes a large load on the SDN-Controller and also a large overhead on the control network, especially for in-band control channels. In addition, another type of network environment where OFDP shows performance issues is multi-tenant cloud data centers. In such environments, the network topology is dynamic, because the tenant can build and modify their network; They can add and remove SDN-Switches or links at any time. This means that the topology can dynamically and continuously change [38]. Hence, the SDN-Controller must be efficient enough to maintain an updated network topology. However, for OpenFlow networks where OFDP is the link discovery protocol, the SDN-Controller only discovers the topology at periodic, constant, and relatively long intervals (floodlight controller for example every 15 s). Therefore, the SDN-Controller only realizes the new topology changes in each discovery round, and if an error occurs, the error correction needs to wait for the next topology discovery round which is too long [75]. As a result, application-level network applications such as routing will use the wrong network configuration until at least the next topology round.

2.6.2. OFDP Performance Metrics

As we discussed earlier, three main entities participate with each other in the OFPD protocol to get work done: the SDN-Controller, SDN-Switches, and control channels between the SDN-Controller and SDN-Switches. The SDN-Controller will send, receive, and process messages related to the discovery process. Likewise, SDN-Switches will receive, send, process messages, and all of these messages will use the control channel as arguments. Ultimately, these processes are an overhead on all participating entities. The overhead incurred by OFDP consists of the OFDP-connection to the connector (control channel) from one side and processing overheads on the SDN-Controller and SDN-Switches on the other side. Until this moment, there are no performance metrics that are adopted by any of the standard organizations to measure the performance of the topology discovery service in SDN, but there is some research suggesting and using some of the commonly recognized performance metrics. Therefore, we will discuss the performance metrics used in these studies and suggest some others for their importance as follows:

1.

The number of packets sent and received by the SDN-Controller

The authors in [39,42,45,76] used the number of packets sent and received by the SDN-Controller as a performance metric. To discover the links between SDN-Switches, in each discovery round the SDN-Controller sends several LLDP Packet_Out ($P_{OUT}$) messages equals to the number of active SDN-Switch ports in the network and will receive a number of Packet_In ($P_{IN}$) messages equals to twice of the number of links as shown in Equations (1) and (2).

$$P_{OUT}={\displaystyle \sum }_{i=1}^N{P}_i$$

(1)

$$P_{IN}=2L$$

(2)

where $N$ is the number of SDN-Switches and $P_i$ is the number of ports of DS-Switch $i$, $L$ is the number of links between the SDN-Switches in the network.

As illustrated from Figure 6, the SDN-Controller will send three LLDP Packet_Out messages equal to the number of active ports on SDN-Switch (s1) and it will receive one LLDP Packet_In message to discover the one-way link from s1 and s2. The SDN-Controller also needs to send another three LLDP Packet_Out messages to s2 to discover the one-way link from s2 to s1 and it also needs to receive an LLDP Packet_In message. In total, the SDN-Controller will send six LLDP Packet_Out messages (i.e., number of active ports on s1 and s2) and two LLDP Packet_In (i.e., double of the number of links) messages to discover the link between s1 and s2.

Moreover, the authors in [39] performed three experiments with different topologies as shown in

Table 2. Number of Packet_Out messages in different topologies [39].

Topology Number	Topology Type	Topology Parameters	Number of SDN-Switches	Number of SDN-Switches	Number of Packet_Out
Topology 1	Tree	d = 4, f = 4	85	424	424
Topology 2	Tree	d = 7, f = 2	127	380	380
Topology 3	Linear	m = 100	100	298	298

. The letters ‘d’ and ‘f’ donate the depth and fan-out parameters in tree topologies as well as the letter ‘m’ refers to the number of SDN-Switches in linear topologies. The experimental results reveal that the number of LLDP Packet_Out messages is equal to the number of active ports regardless of the topology type as previously mentioned in Equation (1).

There are two types of SDN network topologies; Linear and tree [77]. In a linear structure, each SDN-Switch is associated with a single host, and in a tree, the SDN-Switches are arranged like tree branches as well as terminal branches associated with hosts. It is the only structure without loops, and the tree topology has the highest throughput with OpenDaylight, which is approximately 13 gigabits per second (Gbps) [77]. However, the drawbacks of the linear structure are the occurrence of disturbances in the network if the network element fails or smashes, and the troubleshooting difficulty is high and time-consuming.

2.

Average CPU Utilization of SDN-Controller

In [38,39,75], the authors use this metric to measure the extent to which OFDP is using the SDN-Controller to obtain the topology. The SDN-Controller uses its CPU to create, send LLDP Packet_Out messages, and process LLDP Packet_In messages. The average CPU utilization increases when the number of packets sent and received by the SDN-Controller increases.

3.

Accumulative CPU Utilization of SDN-Switches

The authors in [42] used this metric to measure the extent to which OFDP is using the CPU for SDN-Switches. SDN-Switch is an essential part of discovering the topology. The SDN-Switch receives LLDP Packet_Out messages from the SDN-Controller and sends them to its active ports. As a result, the number of packets sent or received by the SDN-Switches will also increase the CPU utilization ratio.

4.

Bandwidth Consumed by OFDP

As described in the OFDP methodology, there are two types of connections: between the SDN-Switches themselves and between SDN-Switches and the SDN-Controller. This metric is determined by the size of the exchanged OFDP packet to maintain the topology. In [39,42], the authors used this metric to evaluate OFDP. Thus, this bandwidth can be especially important for measuring the performance of large networks and in-band control channels.

5.

Learning Time

Some researchers [38,42,76] have used this metric to evaluate OFDP and the performance of their topology discovery. Learning time is the time the SDN-Controller needs to learn about topology changes. The discovery process will be repeated every discovery interval. The discovery interval is the time interval between two discovery rounds. The problem is that when a topology change occurs, the SDN-Controller will wait for the next discovery round to learn about new topology changes. This means that learning time is at least equal to the discovery interval.

2.7. Challenges of the Link Discovery

As we discussed in Section 2.4, on each fixed time interval (10 s) the SDN-Controller sends an LLDP packet encapsulated with a Packet-Out message to each active SDN-Switch port in the network. This discovery mechanism could present serious performance issues to SDN networks, especially of large networks. Therefore, we will summarize the OFDP link discovery challenges as follows:

1.

Overhead to SDN-Controller and Control Channel

This point of the challenge has been clarified in details in the OFDP performance subsection, and therefore we will avoid re-explaining it

2.

Inefficient Link Failure Detection

In most SDN-Controllers, each round of detections is performed approximately every 10 s and the SDN-Controller will become aware of new topology changes [44]. This is a very long process for dynamic network environments where the changes to the topology occur frequently over a short period [38]. This greatly affects the operation of network applications that depend entirely on the SDN-Controller’s network topology [78]. For example, if the learning time is long and there is a link failure on router links, the routing application will still redirect traffic on that path based on a legacy network topology, which means this will cause many packets to be dropped. Figure 7 shows the relationship between learning time and topology changes. When a topology change occurs after the first discovery round, the SDN-Controller will wait for the next round to re-detect changes to the topological structure, which is too long. On the other hand, there is a suggestion to reduce the interval of discovery changes, but this will cause the number of the Packet_Out messages to increase significantly, which in turn will increase the load on the SDN-Controller and uses more bandwidth in the process of discovering the topology. In addition, an increase in the discovery interval means fewer LLDP messages and less overhead, but also means more time required to learn about new changes.

Regarding large and dynamic network environments as we discussed in Section 2.6.1 these challenges appear clearly, and based on performance metrics, we expect that OFDP performance will perform as shown in

Table 3. OFDP performance in large and dynamic networks.

Performance Metrics	Description	Outcome
Number of packets sent and received by the SDN-Controller	The number of packets sent equals the number of active SDN-Switch ports The number of packets received equals twice the number of links	Considerable Reasonable
Number of packets sent and received by each SDN-Switch	The number of packets sent equals the number of its active ports that connected to SDN-Switches The number of packets received equals the number of its active ports	Reasonable
Average CPU utilization of SDN-Controller	CPU overhead added by OFDP to SDN-Controller	Considerable
Accumulative CPU utilization of SDN-Switches	CPU overhead added by OFDP to SDN-Switches	Reasonable
Bandwidth consumed by OFDP for in-band control channels	How much does OFDP use from control channel bandwidth?	Considerable
Learning time	How long does it take for OFDP to learn about topology change?	Considerable

. The table shows that the value of some performance metrics is considerable and cannot be ignored [38,69].

3.

Security Issues

Adding security to the topology discovery processes in OFDP also poses a new challenge in terms of Quality of Services (QoS). Secure OFDP is a great idea to protect the real-time topology from attacks such as inserting malicious rules at SDN-Switch, denial of services at SDN-Controller, and man-in-the-middle in control channel [52], but it causes other issues related to the performance of the device and time-consuming. Therefore, given the contrast of the two trends between network performance and security in sensitive issues (network topology discovery), the issue of balancing between them is also important. Therefore, this makes the door of research open to researchers to balance security and performance according to the needs of the network.

However, with challenges such as these, the SDN discovery topology protocol needs to provide the SDN-Controller with a real-time view of the network topology to meet the application and dynamic routing Quality of Service (QoS) demands.

3. Recent SDN Topology Discovery Performance Studies

In the literature, the studies that discuss the issue of performance in OFDP or OpenFlow are considered rather scarce, and most of the studies dealing with and maintaining OpenFlow security [33,34,35,79,80,81]. Therefore, in this paper, we will focus on OpenFlow, and in particular on OFDP protocol, along with the factors that have a direct impact on its performance. This section provides summaries of this research in detail as follows:

3.1. Link Discovery Improvement Algorithms

In this subsection, we will discuss the proposed solutions that are related to OFPD link discovery. These proposals can be categorized into two domains based on their procedural release (event or periodic). Moreover, a comparison of these proposals is provided in

Table 4. Comparison between different proposed approaches to enhance OFDP.

Proposal	Methodology	Location of Logic	Operation Mode	Advantage	Disadvantage
OFDPv2	Merge Packet_In messages from each port to each SDN-Switch	Controller	Periodic	(1) Reducing CPU overhead. (2) Reducing the bandwidth overhead on the control channel.	It works periodically, and this may introduce unnecessary discovery traffic
sOFTDP	Triggering topology discovery service by link failure events monitored by Bidirectional Forwarding Detection.	Switches	Event	(1) Reduce learning time. (2) Reduce adaptation time. (3) Reduce CPU overhead.	It restricts the controller’s ability to collect statistical data on discovery traffic
ForCES	Delegating the logic of topology discovery to the SDN-Switches.	Switches	Event	(1) Reduce the learning time of link changes.	It is only applicable for ForCES as a southbound API.
ESLD	Reducing messages between the SDN-Controller and SDN-Switches for topology by restricting the sending discovery packets to only SDN-Switch ports connected to switches and not to hosts.	Controller	Periodic	(1) Reducing CPU overhead. (2) Reducing the bandwidth overhead.	Port classification consumes more messages
SLDP	A new packet format was used for topology discovery messages with a random source MAC address.	Controller	Periodic	(1) Reducing CPU overhead. (2) Reducing the bandwidth overhead.	Increases Flow_Mod messages to enable authorized packet forwarding
TEDP-S	Reducing messages between the SDN-Controller and SDN-Switches by sending only one discovery packet to the root SDN-Switch.	Controller and Switches	Periodic	(1) Reducing CPU overhead. (2) Reducing the bandwidth overhead.	Increasing CPU overhead on the switches.
TEDP-H	Offloading the process of discovering the topology from SDN-Controller to the root SDN-Switch.	Controller and Switches	Periodic	(1) Reducing CPU overhead. (2) Reducing the bandwidth overhead.	Increasing CPU overhead on the SDN-Switches.
SDN-RDP	Sharing network state management between multiple SDN-Controllers.	Controller	Periodic	(1) Reducing the number of messages. (2) Reducing the computation time.	Manual configurations
GTOP	Improve topology discovery process in PCE to be as OpenFlow	PCE and Switches	Periodic	(1) Reduce link failures (2) Reduce updating times	Legacy domain
SONT	Test-signal mechanism to detect network links	Controller and Optical switches	Periodic	(1) Reduce updating times	Fault tolerance is not checked despite its importance in optical networks
HDDP	A lightweight agent and network exploration model based on flooding	Controller and Switches	Periodic	(1) Support different type of networks	CPU Overhead Increasing packet messages
eTDP	Distributed topology discovery process on layer 2 and uses shortest control paths	Switches	Periodic	(1) Reduce discovery time and cost	Back to traditional networks
TDP	Rely on network partitioning and using a timer to send topology discovery packets	Wireless nodes	Periodic	(1) Reducing send packets (2) Reduce topology discovery energy	Suitable for tree network topology only

as well.

3.1.1. Periodic

This domain is named periodic because the discovery process is carried out periodically for every period. The authors in [39,42,43,45] used this type as we will discuss.

In [39], the authors proposed a new approach called OpenFlow Discovery Protocol version 2 (OFDPv2) to enhance the OFDP performance. OFDPv2 reduces the number of LLDP Packet_Out messages to only one LLDP Packet_Out message per SDN-Switch instead of per active port. They proposed two versions of OFDPv2. OFDPv2-A for SDN-Controller and OFDPv2-B for SDN-Switch. In OFDPv2-A, the SDN-Controller will install an additional set of flow rules in each SDN-Switch using the OFPT_FLOW_MOD message to forward the LLDP packets from each port on the SDN-Switch. The OFPT_FLOW_MOD message is used by the SDN-Controller to process the flow tables of the OpenFlow SDN-Switches. It can add, update or delete flow entries from the flow tables of OpenFlow SDN-Switches. However, added rules consume a lot of Ternary Content-Addressable Memory (TCAM) which is already a scarce resource in SDN-Switches. In OFDPv2 B, they did not install flow rules on SDN-Switches but they send an action list (i.e., a set of instructions to forward LLDP packet for each port) with the LLDP Packet_Out message. However, this makes OFDPv2-B withstand a large bandwidth, especially for in-band control channels. These processes are illustrated in Figure 8.

Based on Figure 8, the SDN-Controller will send one LLDP Packet-Out for each SDN-Switch in the network. S1 will receive a Packet_Out message and extract an LLDP packet from it. Then it will forward the LLDP packet to all active ports in the SDN-Switch and replaces the source MAC address of the LLDP packet with the MAC address of the egress port. S2 will receive the LLDP packet and encapsulate the LLDP packet with a Packet_In message and send it to the SDN-Controller. The SDN-Controller will parse the incoming Packet_In packet and learn about the new link. Finally, the results show that OFDPv2 uses 63–80% fewer LLDP Packet_Out messages than the same process in standard OFDP. Furthermore, in measuring the CPU overhead between OFDP and OFDPv2, the results show that OFDPv2 reduced the CPU utilization of the SDN-Controller by up to 45% compared to the standard OFDP.

Additionally, in [42], the authors presented a lightweight, efficient, and secure approach to discover the links between SDN-Switches in SDN called Secure and Lightweight Link Discovery Protocol (SLDP). In general, the proposal uses a new packet format for link discovery by using minimal features of the frame and removing unnecessary features from the standard LLDP frame. Moreover, for each iteration of the link discovery process, the SDN-Controller will generate an SLDP packet and send it with a random source MAC address to SDN-Switches. Then the SDN-Controller installs a flow entry in each SDN-Switch flow table to generate the packet with that random source and agree to these values when a message is returned to the SDN-Controller. In brief, the SDN-Controller will initially send an SLDP packet to each SDN-Switch port in the network, and the subsequent discovery iterations will only receive ports that are eligible for SLDP packets. As result, only legitimate SLDP packets will be sent to the SDN-Controller to build the topology and this will reduce the number of packets used in the discovery process and prevent unqualified ports from receiving SLDP packets. In evaluation metrics, the authors used the Mininet emulator and compared it to the OFDP in various network topologies with different numbers of SDN-Switches, hosts, and links. Moreover, they used the number of packets sent by the SDN-Controller, CPU of the SDN-Controller, and validation time. For all of these metrics, SLDP outperforms the standard OFDP.

Moreover, [43], the authors proposed another approach called Efficient and Secure Link discovery scheme (ESLD) by also limiting the transmission of LLDP packets to SDN-Switch ports connected to SDN-Switches. This approach is illustrated in Figure 9.

The basic idea of the ESLD is to classify SDN-Switch ports into two classes, either ‘Switch’ or ‘Host’. ‘Switch’ ports are those ports connected to SDN-Switches and ‘host’ ports connected to users. Moreover, ESLD uses some well-known OpenFlow messages such as ‘Feature-Replay’, ‘State-Reply’, and ‘Port-Status’ messages to label these ports in both types (‘Host’ or ‘Switch’). Therefore, the efficiency of ESLD is directly dependent on the number of SDN-Switch ports in SDN. For evaluation, the authors used a host scale in different scenarios with different topologies and compared ESLD to the de facto OFDP and OFDPv2. Moreover, a number of LLDP packets handled by the SDN-Controller, CPU utilization for SDN-Controller, and SDN-Switches were used as performance metrics.

In a different study, the authors suggested two new applications of the OFDP to improve the link discovery process: the Enhanced Topology Discovery Service (ETDP-SDN) and the ETDP-Hybrid [45]. In ETDP-SDN, the Discovery Service is centralized inside the SDN-Controller, and the SDN-Controller identifies each iteration of the root SDN-Switch and then sends an ETDP discovery packet to the root SDN-Switch. SDN-Switch will in turn flood this packet to all of its ports except for the port where ETDP is received. Next, the SDN-Controller installs a flow rule to force SDN-Switch to send a Packet_In to the SDN-Controller and another flow rule to flood the ETDP frame. Next, after the SDN-Controller receives the Packet_In message, it will send an additional FLOW_Mod to remove the flood rule, which was previously installed to prevent loops. However, the ODFP performance is improved by sending a single ETDP packet encapsulated with a Packet_Out message to the SDN-Switch root from the SDN-Controller. This will reduce Packet_Out messages per discovery frequency to just one packet. ETDP also provides the minimum latency path between any two SDN-Switches.

In the second suggested application (ETDP-Hybrid), the discovery service is shared between the SDN-Controller and SDN-Switches. SDN-Switches in this application do not depend on the SDN-Controller to start the discovery process, it initiates the discovery process and installs a flow entry in its flow table to send the topology information to the SDN-Controller. In discussing the results, the authors evaluated their application using Mininet in different topologies and different experimental times to compute the mean and standard deviation. The Packet_out count, Packet_In count, and Flow_Mod count messages were used as performance metrics. For ETDP-SDN, the results showed that only one Packet_out message per SDN-Switch. The number of flow_Mod messages was expected to be twice the number of flow_mod messages in standard OFDP, but that was not the case because the locking mechanism (to stop flooding frames and prevent loops) is not fast enough to stop annoying ETDP packets before installing block flow rules. In ETDP-Hybrid, there were no Packet_Out and Flow_Mod messages sent by the SDN-Controller.

Similarly, the authors presented the SDN Resource Discovery Protocol (SDN-RDP) in [82] as a solution to sharing network state management between multiple SDN-Controllers. Each SDN-Controller discovers a portion of the network topology to maintain distributed node management and enhance protocol accuracy. The presented approach is asynchronous, does not require full network information, and does not need a global startup step. According to the simulated results, the proposed method effectively reduces the overload of the SDN-Controller.

Other topology discovery methods have used centralized techniques, such as the Path Compute Element (PCE). The authors in [83] suggested a centralized topology discovery algorithm called Generalized TOPolgy (GTOP) for PCE. GTOP works by enabling PCE to mechanically create a network topology without employing a global routing protocol such as the Open Short Path protocol (OSPF). The GTOP uses an out-band control channel to proactively collect topology costs from the SDN-Switches. Additionally, it uses the same control channel to update the topology changes in an interactive way. However, concerning the testbed system, the total time for the suggested protocol was 10 ms to update the topology changes. The drawback of this work was using an out-band control channel, which may not be feasible to deploy in large-scale states.

In discovering SDN-Optical Network Topology (SONT), the authors in [41] adopted a sequence of signal checking to detect the links one by one. This technique fits into the pre-service category of layer 1 convergence discovery cases and provides a correct links mapping at SDN-Controller despite limiting scalability and time efficiency, especially in large networks. Therefore, the same authors [84] used a parallel mode rather than a sequential mode to overcome their limitations.

Likewise, the authors in [85] proposed Hybrid Domain Discovery Protocol (HDDP) to enhance the mechanism of discovery topology between SDN and traditional networks. HDDP is managed by SDN-Controller and works as OFDPv2 through the use of a lightweight agent that implements HDDP and transmits topology information indirectly to the SDN-Controller. HDDP uses a network exploration model based on a controlled flooding mechanism (Packert_Out) to discover non-SDN devices. Subsequently, the authors in [40] developed the HDDP to cover the topology discovery process in different types of wireless networks.

Furthermore, other proposed work transfers the topology discovery process from SDN-Controller to layer-2 (SDN-Switch) completely as described in [66]. The authors proposed a new method called enhanced Topology Discovery Protocol (eTDP) which has been classified the SDN-Switches into three types (core, leaf, and v-leaf). Leafs called for SDN-Switches that have only one adjacent switch while SDN-Switches that have more adjacent Switches called v-leaf. The remaining SDN-Switches are called a core. In addition, SDN-Switch ports also have different states according to their positions in the control tree formed by the topology discovery module (Standby, Parent, Child, and Pruned). These performed functions of ports and SDN-Switches are rotated between managed components and independent managers. Each performed function has four tasks: monitor, analyze, plan, and execute. Finally, the SDN-Controller will draw the network topology from the shortest control paths transmit to it by the SDN-Switch core.

Moreover, other works related to topology discovery processes in low power and CPU devices have been discussed in [67,86]. The authors of [67] analyzed the effect of OpenFlow performance over wireless networks in terms of QoS metrics. Meanwhile, the authors in [86] proposed a new topology discovery method called efficient Topology Discovery Protocol (TDP) to achieve simultaneous adaptive SDN-Node ID mapping and topology discovery for Underwater Acoustic Networks (UAN). To save power, the proposal eliminated outdated information about the metering network and node identification from the network topology procedure, leaving only the node identifier. Instead, the SDN-Controller separates the network into layers and employs a local timer in each SDN-Node to ensure that topology discovery packets are sent on time. Each node could determine the network topology and create its SDN-Node ID separately by utilizing the received topology discovery packets.

3.1.2. Event

This domain is named by event, because the changes come along each discovery takes place on every event, such as updating, adding, and deleting links. The authors in [38,76,87] used this type as we will discuss.

In different techniques related to improving OpenFlow topology discovery called Secure and Efficient OpenFlow Topology Discovery Protocol (sOFTDP) [38]. The authors in [38] have transferred the control logic for topology discovery and its security from SDN-Controller to SDN-Switch in order to reduce arithmetic operations from SDN-Controller. SDN-Switches are now responsible for monitoring the status of their ports, and they have used Bidirectional Forwarding Detection (BFD) protocol [88] as a mechanism for detecting port vitality. Each SDN-Switch will establish BFD sessions with their neighbors using three-way handshaking. The SDN-Switches then will use the established BFD session to exchange control and echo messages to monitor link vitality. When a link failure occurs, the affected SDN-Switches will detect this failure by BFD and immediately send a notification called BFD-Status message to the SDN-Controller which also immediately removes this link from its links table. For a link addition, asynchronous notifications using the OFPT__Port_Status message will be sent to the SDN-Controller which in turn will discover this link by following the same procedure that is used in OFDP but only for this link. Moreover, port-status messages (OFPT_Port_Status) are sent from the SDN-Switch to the SDN-Controller when a change in port status occurs or when a new port is added, removed, or modified in the SDN-Switch data path.

The experiments were performed using learning time and CPU utilization performance metrics to evaluate sOFTDP. In link addition, the average learning time for 50 experiments was 5.68 ms while in link removal the learning time was 3.25 ms. Also, they evaluated sOFTDP against standard OFDP and OFDPv2 in terms of CPU utilization of the SDN-Controller throughout 200 s with one topological change in second. The results show that sOFTDP induced the least overhead versus standard OFDP and OFDPv2.

In the same procedural, the approach called ForCES based optimal network topology discovery [76] was proposed. In ForCES, the authors improved OFDP performance by offloading the SDN-Controller from sending or receiving LLDP advertisements. The LLDP protocol will work as it is and without any modifications. The SDN-Switches transmit LLDP advertisements and build their topology tables without the intervention of the SDN-Controller. Then the SDN-Controller pulls the topology information from the SDN-Switches continuously. This technique has been used in [87] in a small-scale testbed. As a result, this proposal can be beneficial by reducing the learning time, and the affected SDN-Switches will inform the SDN-Controller of this change immediately when a change to the topology occurs. However, this solution has been suggested for SDN networks that use ForCES protocol as a southbound interface but can be used with the logic for OpenFlow networks. Finally, the average time for the SDN-Controller to learn about topology changes is 10 ms which is less than 90% of the learning time in standard OFDP (i.e., 100 ms).

3.2. Flow Table Management Algorithms

As we mentioned earlier in the section of switches discovery in SDN technology, the adoption of switch discovery depends mainly on the data and instructions of those flow tables. Each OpenFlow in each switch contains at least one flow table and a set of flow entries within that table. These flow entries contain matching fields, counters, and instructions to apply to matched packages. Typically, it will have more than one flow table, so it is important to note that matching starts from the first flow table and may continue with additional pipeline flow tables. The packet will first start in

Table 5. Comparison of flow table management to improve OpenFlow performance.

Proposal	Methodology	Controller Placement Mode	Operation Mode	Goals
Rifai et al. [89]	Flow entry compression	Reactive	Traffic engineering	Maximize the utility of flow tables
Panda et al. [90]	Dynamic hard timeout allocation	Reactive	LRU	Maintain unpredictable flow for a limited period
Isyaku et al. [91]	Dynamic idle and hard timeout based on traffic pattern to reduce overhead	Reactive and Proactive	LRU	Improved the restricted flow table
Xu et al. [92]	merging flow table and cost of the SDN-Controller	Reactive	Traffic engineering	Adjusts the idle timeout value based on the flow
Kotani and Okabe [93]	packet filtering scheme	Proactive	Traffic engineering	Reducing multiple packet-in messages forwarded to the SDN-Controller
Favaro and Ribeiro [94]	Blackhole mechanism	Reactive	Flow-table management	Maintain visibility for each new flow.
Leng et al. [95]	Rule optimization and binary tree aggregation	Reactive	Flow-table management	Reduce the number of flow entries
Li et al. [96]	Used Q-Learning rule for selecting effective timeout values	Proactive	Machine Learning	Adjusts the idle timeout value based on the flow
Yang and Riley [97]	Classify flows into active and inactive to decide the right flow to remove intelligently	Proactive	Machine learning	Increase the flow table capacity

and check these entries based on priority. It will match the highest priority first (e.g., 200, then 100, then 1). If the stream needs to continue to another table, the go-to instruction tells the package to go to the table specified in the instruction. Therefore, improving the technique of dealing with tables will improve the performance of the entire network. The researchers attempted to improve the mechanism of flow tables in the process of topology discovery. Whereas SD-OpenFlow uses TCAM memory to store flow tables, it is very expensive and has a limited size. Thus, the number of flow entries that can be accommodated is limited, therefore, the researchers tried to optimize the flow schedule to take full advantage of it. Moreover, a comparison of these proposals is provided in Table 5 as well.

Timeout mechanism is one of the techniques used by the SDN-Controller to calculate the lifespan of the forwarding entry in the switch flow table. When no packet matches an item during its timeout period, the entry is ejected from the flow table, making room for fresh arriving packets [89]. There are now two methods for installing the timeout mechanism in the OpenFlow SDN-Controller: idle and hard timeout. Typically, the OpenFlow SDN-Controller configures each stream’s flow entry with a preset idle timeout value in seconds. However, when it comes to packet inter-arrival time, the flow table has enough room to accommodate all flows, such a timeout number may provide superior performance. In [90], the authors consider a dynamic allocation of hard timeout for identified and unidentified flows. The goal is to maintain unidentified flow for a limited period while also maintain the identified flows. This is accomplished by studying packet traces to determine the nature of each packet’s arrival and then modifying the hard timeout accordingly. When a flow table’s load is exceeded, expires Least Recently Used (LRU) is utilized to remove an entry with a maximum timeout. Even when the strategy has succeeded in maintaining identified flows, the challenge of a bad entrance removal scheme remains. LRU is not an ideal eviction method for SDN since it is a packet-driven method that can only be executed in theory but may not be suitable with OpenFlow in action. In the same dynamic allocation technique, the authors in [91] used dynamic inertia value for short live flows and set the hard timeout value for long live flows with a short time between packets arrival. This has greatly reduced the Packet_In numbers and as a result, reduced SDN-Controller overhead. By utilizing OpenFlow’s built-in data collecting, the limitation of LRU is resolved. Flows with a low packet count are victims, so deleting them improves the amount of the limited storage. Without modifying the design of SDN, it is effective to conclude that their proposal reduced network cost and improved the restricted flow table usage to some level. In [92], the authors offered an adaptive flow table alteration based on the fraction of active flow entries by merging the flow table and cost of the SDN-Controller. The algorithm constantly examines traffic, and as a result, the procedure adjusts the idle timeout value based on the flow. Surprisingly, the algorithm was able to set different timeout settings for distinct flows. On the other hand, the process of determining the cost of flow table entries in the cost of the switch and SDN-Controller computing adds additional computational expenses to the SDN-Controller.

Another technique that was used to reduce SDN-Controller overhead, the authors in [93] employed a packet filtering scheme. The mechanism started by inspecting the header in Packet-In messages and dropping the duplicate one. Moreover, in [94] the authors devised a blackhole technique to reduce SDN-Controller overhead and preserve the advantage of visibility for each new flow. The packet forwarding architecture between SDN-Switch and SDN-Controller has been changed so that only the first packet is routed to the SDN-Controller and successive table-misses are dealt with locally. In addition, its architectural design leaves it vulnerable to dropping alerted packets, which could result in a substantial number of packet losses. However, the amount of events transmitted to the SDN-Controller is minimized. In addition, another technique based on flow entries aggregation has been proposed in [95] to reduce the load on TCAM. Theoretically, the strategy reduces the number of entries to be saved by compressing fine-grained sending entries into less coarse-grained entries with a somewhat greater matching range. Surprisingly, it is a software application that is easy to install as an additional plug-in on the OpenFlow SDN-Controller and does not require additional hardware. However, since the aggregation methodology fails to preserve the original semantics of the rule in most circumstances, the SDN-Controller has certain issues when changing the direction of entries or querying the traffic stats counter.

A different technique based on machine learning has been proposed to identify the classes of traffic flows [96,97] to predict the duration of the flow entry. In [97], the authors used machine learning techniques to determine which flow should be eliminated. The algorithm, which is based on past data of flow entries, forecasts and with the flow entry with the shortest time being the victim. Depending on the output of the algorithm used, it is determined whether the flow entries are active or inactive. Moreover, in [96], the authors also used the Q-Learning rule for selecting effective timeout values for flows to improve OpenFlow SDN-Switch performance. These techniques, on the other hand, may provide superior performance in a small–medium-sized network, but a large-scale dynamic network may necessitate a more complex training set, which in turn necessitates greater storage to handle more historical data.

3.3. Control Channel Improvement Algorithms

Furthermore, in Section 2, we mentioned the importance and how the OpenFlow protocol control channel works. Where it is the hot line of communication between the controller and the plane layer devices. There are a number of constraints for both out-of-band and in-band controls. On the one hand, out-of-band control is costly and inconvenient. This is due to the fact that each data plane device requires additional physical interfaces and cabling to link to the controller. Furthermore, data plane devices may be physically positioned far from the controller, necessitating a significant additional cost to construct a network protocol for out-of-band control. In this subsection, we will discuss the proposed solutions that are related to control channel failure detection and recovery, where sustaining communication between the SDN-Controller and SDN-Switches becomes the main difficulty during network disruptions. Moreover, a comparison of these proposals is provided in

Table 6. Comparison of control channel schemes to improve OpenFlow performance.

Proposal	Methodology	Controller Placement Mode	Operation Mode	Logic Location
Asadujjaman et al. [100]	Combined between topology type and source-routed forwarding to support local failure recovery	In-band	Recovery	SDN-Switch
Fan and Yang [99]	Centralized trust management system for in-band control channel	In-band	Protection and Recovery	SDN-Controller
Osman et al. [101]	The hybrid controlling mode that dynamically changes between centralized and distributed	In-band and Out-band	Protection	SDN-Controller and SDN-Switches
Alowa and Fevens [98]	Trusted control pathways for in-band control channel	In-band	Protection and Recovery	SDN-Switches
Hwang and Tang [102]	weighted function (Complete Bipartite Graph) technique is used to select the alternative control channel path	In-band	Recovery and Protection	SDN-Switches
Ko et al. [103]	Dijkstra algorithm is used to calculate the shortest control channel pathways	In-band	Protection and Recovery	SD-Swatches
Chan et al. [60]	K-best is used to find control channel paths in between multiple controllers	In-band	Protection	SDN-Controller and SDN-Switches
Ibrar et al. [104]	Logistic regression and support vector machine algorithms to predict the link status	In-band	Protection	SDN-Controller
Yang and Riley [97]	Classify flows into active and inactive to decide the right flow to remove intelligently	Proactive	Machine learning	Increase the flow table capacity

as well.

As the control channel is used to establish in-band and out-band connection between the SDN-Controller layer and data layer in OpenFlow-SDN, the control channel failure threatens the network availability. According to [98,99] a lossy control channel dramatically reduces data layer throughput and response time. Therefore, several recent works have been proposed to help sustain OpenFlow availability. The authors in [100] proposed control channel recovery for in-band and out-band control links. To enable local recovery from failure, the author integrated logical ring topology with source-routed forwarding. The SDN-Switch-to-SDN-Controller communication was made more robust using a ring-based local recovery technique. A source forwarding method was employed to ensure the SDN-Controller-to-SDN-Switch communication was robust, as a robust SDN-Switch-to-SDN-Controller communication channel allows the SDN-Controller to be updated of the complete topology. However, while keeping the functionality of data plane devices, this recovery does not necessitate SDN-Controller involvement. Moreover, [98] protected the in-band control channel from failure by finding a set of ideal pathways. To recover from SDN-Switch failure in an SDN, this research used a bypass and non-intervention backup recovery methodology. When an SDN-Switch fails, the goal is to deal with it as quickly as possible, especially if it is part of an SDN-Switch group. If a group of dependent SDN-Switches is affected by the failure, all control traffic from the impacted SDN-Switches is re-routed to a different recovery path.

Using a full cartelized control channel, the authors in [99] suggested a centralized trust management system for selecting the most trustworthy data transfer pathways (in-band control) in the SDN. The authors viewed the network as a multi-agent system, with SDN-Switches and routers serving as the agents. The authors thought that their centralized unit had complete control over the route choices made by the agents. Fully centralized systems, in most circumstances, necessitate highly complex resources for the central unit, while also introducing a single point of failure. The central unit must be completely operational, tamper-resistant, and physically secure. It is also necessary to keep a close eye on all of the system’s agents. The cost of applying such systems rises dramatically when all of the above-described qualities are combined. In contrast, the authors in [101] proposed a new method called hybrid-SDN to operate the network regardless of the level of control vulnerability by shifting network control from central to distributed. This approach depends on the Control Packet Loss Ratio (CPLR). The OpenFlow-SDN is converted to a hybrid-SDN (distributed control channel to SDN-Switches) approach when the CPLR value reaches an undesirable level.

In terms of restoring the OpenFlow control channel failover mechanism, SDN-Switches can detect a broken link but must wait for the SDN-Controller to generate alternate pathways. Therefore, the authors in [102,103] looked at how to protect and recover from control channel failures. Authors in [102] employed both recovery and protection methods for data channel failures. In the recovery process, the SDN-Controller constructs alternative paths depending on the complete two-segment graphs of the topology of the network, and the SDN-Controller will use the weighted functions to the alternative paths to re-built SDN-Switches control channel path. Moreover, in control channel protection, the SDN-Controller identifies flow entries for the SDN-Switches. However, this proposal has drawbacks related to both processes (recovery and protection) as the increase of failover times as the number of SDN-Switches grows and handling with rapid changes in network topologies. Therefore, these drawbacks were overcome in [103] by using both a flexible network hypervisor-based structure. The suggested solution involves calculating backup pathways (Dijkstra algorithm) in a finer timeframe (e.g., 5 s), and only configuring recovery flow rules if a physical network failure is detected. In addition, another recovery control channel based on the use of multiple SDN-Controllers in in-band networks has been proposed in [60]. The master SDN-Controller is in charge of network control at the stable level, while the other SDN-Controllers are on standby to take over the network control in the event of failure. The authors planned the control paths using the K-best path technique on a modified graph, and in this way, each SDN-Switch would be serviced by several SDN-Controllers with discontinuous pathways.

Furthermore, the authors in [104], attempted to reduce the impact of control channel service failure by predicting link failure before it occurred. The authors used a machine learning technique for their proposed method that re-computes the locations of access control policies and reduces their violation in the event of link failure.

4. Discussion and Open Issues

Each of the proposals presented in the previous section has its characteristics of improving one or more parts of the topology discovery process. All of these approaches that seek to improve the process of discovering different or similar topologies must be aligned with the basic criteria of OpenFlow-SDN protocol, as we will discuss below.

4.1. Location of the Topology Discovery Logic

In these OFDP, OFDPv2, SDN-RDP, ESLD, and SLDP protocols, the control logic for the topology discovery remained in the SDN-Controller. In OFDPv2, the SDN-Controller starts the discovery process and sends LLDP advertisements to the SDN-Switches. The SDN-Switches have a secondary role in the discovery process; It just consists of forwarding the SDN-Controller’s LLDP advertisements to its neighbor, or encapsulating the LLDP advertisement in a Packet_In message and send it back to the SDN-Controller. Whereas in TEDP-H and sOFTDP protocols, almost all of the logic is in SDN-Switches. The SDN-Switch is responsible for monitoring the status of links with its neighbors and for sending notifications when there is a change in the topology. The SDN-Controller plays a secondary role and is only responsible for sending an LLDP advertisement when it receives an OFPT_Port_Status message indicating that the SDN-Switch port state is changing. In eTDP and ForCES, the logic of topology discovery is entirely in the SDN-Switches. SDN-Switch is responsible for creating and forwarding LLDP advertisements independently and the SDN-Controller only captures link information from the SDN-Switches. In HDDP and TEDP-S, the logic is distributed between the SDN-Switches and the SDN-Controller.

4.2. How Much Do Methods Differ from OFDP

This standard discusses the number of modifications of each proposal to standard OFDP. OFDP takes advantage of the OpenFlow channel to exchange messages between the SDN-Controller and SDN-Switches. HDDP, SDN-RDP, and OFDPv2 make minor modifications to OFDP, as they still use OpenFlow messaging. In sOFTDP, the topology discovery depends entirely on the BFD protocol to detect link failures, which is considered a major change in the methodology of OFDP. In addition, eTDP and sOFTDP rely on a new notification message (i.e., BFD-Status) that is not defined in OpenFlow specification. Moreover, the eTDP uses a new technique that depends on SDN-Switch classes. In ForCES, it is just an implementation of the LLDP protocol for ForCES-based SDNs. The methodology of the discovery process is very different from that of OFDP and requires modification of OFDP to a large extent. In ESLD and SLDP, the topology discovery methodology is the same as in OFDP except for some other steps to limit the transmission of discovery packets to SDN-Switch ports that connect to other SDN-Switches. In TEDP, the methodology is very different from OFDP. The SDN-Controller sends a Packet_out message to each active SDN-Switch port in standard OFDP, but in TEDP, the SDN-Controller sends only one Packet_out to the root SDN-Switch, which in turn completes the task.

4.3. Operation Methods

By operation method, we mean that if the discovery of the topology is performed in a periodic mode or is triggered by events. ‘Periodic’ meaning that the SDN-Controller inquires about the network topology within a specified period, while the ‘triggered’ type means that the process of changing the network topology takes place only when changes occur in network topology. However, most of the reviewed works that we reviewed were using the periodic method and two of them only used the triggered. OFDPv2, ESLD, SLDP, and TEDP periodically discover the topology as in standard OFDP. Whereas in sOFTDP, the topology discovery is triggered by topology change events. In ForCES, the LLDP advertisements are periodically sent between SDN-Switches and if there is a change in the topology, the affected SDN-Switches will notify the SDN-Controller about this change.

However, each type of operation method has its pros and cons. This opens the door to the field of application of a hybrid model between the two methods, which can take advantage of the advantages of each type.

4.4. CPU Usage

While OFDPv2 reduces the number of LLDP Packet_Out messages from the number of active SDN-Switch ports in the network to only the number of SDN-Switches. Results in reducing the CPU usage of the SDN-Controller by up to 45%. In eTDP and sOFTDP, the discovery process runs without the need for frequent advertisements. This significantly reduces the number of LLDP Packet_Out and Packet_In messages and then reduces CPU usage. However, sOFTDP requires SDN SDN-Switches to support a completely independent protocol (i.e., BFD protocol) beside OpenFlow protocol. In ForCES, the SDN-Controller is offloaded from sending LLDP advertisements, which reduces CPU usage drastically. In addition, the learning time for topology changes, particularly link failures are still relatively high because the SDN-Controller will capture the information about topology change after notifications from the affected SDN-Switches. Moreover, CPU utilization in SDN-Switches is increased ESLD, HDDP, and eDTP due to the transformation of the process of topology discovery to the SDN-Switches. In SLDP, the results showed significantly lower CPU usage compared to OFDP. In TEDP, Packet_Out messages sent from the SDN-Controller are distributed to SDN-Switches which in turn reduce the CPU usage. In HDDP, also the SDN-Controller will be overhead due to the flooding Packet-Out messages. TDP and eTDP their technique is offloaded from SDN-Controller to SDN-Switches. Moreover, in [105], the functionalities were offloaded from the data plane to the smart network interface.

4.5. Learning Time

In OFDPv2, ESLD and SLDP learning time is still equal to OFDP, which is high. In SDN-RDP, eTDP, and sOFTDP, learning time is less than OFDP and OFDPv2. In ForCES, the learning time for topology changes, particularly link failures is still relatively high because the SDN-Controller will capture the information about topology change after notifications from the affected SDN-Switches. In HDDP and TEDP, the learning time is the largest because the discovery packet will traverse the whole of the network before discovering the topology change.

However, the process of topological discovery in SDN networks is one of the essential principles that must be given priority in terms of performance and security. The three main components (i.e., Link, Control Channel, and Flow-Table) of the topology discovery process must ideally work together or close to ideal to achieve optimal performance. Thus, optimizing the control channel communication in both methods (protection and recovery) was discussed in the above section but still needs to be covered based on the two redundant paths (in-band and out-band). Moreover, the recent references to OpenFlow defined by ONF [58] already define the architecture of an OpenFlow generically, in a client-server manner, rather than in a data-control plane. Therefore, in our opinion, the future directions of topology discovery should focus on offloading most of the data layer functionalities to programmable hardware such as smart network interfaces are a good solution. Moreover, heterogeneous networks such as wire and various types of wireless also should be included in the topology discovery process, to become more realistic in real environments. In addition, the use of machine learning techniques with programmable devices at the data level to reduce the incidence of errors in topology discovery to rates close to zero. Furthermore, most of the studies reviewed in topology discovery focused on periodic operational mode and ignoring the event. Even the event operational mode will provide a good solution to reduce the process’s cost. Moreover, the combination between both operation modes will take into account another good solution to improve the topology discovery process.

5. Conclusions

While the SDN architecture appears to solve problems within the traditional network architecture, it also comes with some major challenges. In this paper, we highlight one of these challenges, which is related to the topology discovery service. In standard SDN, the SDN-Controller is responsible for maintaining an updated network topology through using OFDP protocol to discover the links between SDN-Switches in the data plane. However, it has major limitations in its performance, especially in huge and dynamic networks. Moreover, applications at the application layer depend entirely on that topology. Therefore, several limitations related to the performance of the link discovery protocol used in SDN networks (OFDP) are presented in this paper. Furthermore, the elements (flow tables and control channels) related to the OpenFlow protocol and optimization studies on these elements are explained, which in turn leads to improving the performance of OFDP. However, there are not a large number of studies that attempt to address these limitations in the literature. Therefore, it is a good opportunity for researchers to work on it for their future studies. Moreover, five fundamentals that measure the similarity between the proposed technique and the standard (OFDP) used in the topology discovery are discussed, including topology discovery logic location, operation mode, OFDP compatibility, CPU usage, and learning time. Based on these fundamentals, each of these proposals has some limitations in one or more of these fundamentals. Therefore, we still need a new proposal that takes into account all five fundamentals. One of the research directions in the development of a new topology discovery approach that distributes the logic between SDN-Switches and SDN-Controller. This solution will take as little workload as possible on both and reduce learning time as much as possible. As another research direction for proposals that discover the topology periodically, the trade-off between SDN-Controller overhead and learning time to meet the requirements of the environment must be determined. This trade-off should be based on the criticality of the environment and topology change rate. In some environments, such as enterprise networks, the rate of topology change is low and less significant; thus, it is possible to increase the discovery interval. In contrast, in data-centers and transport networks, the rate of topology change is high and critical, thus, it is recommended to decrease the discovery interval as much as possible.