7.3.1. Evaluation of the Smart Office Scenario
The first analysis we propose is to compare the outcomes of several network metrics when applying the proposed algorithms to the smart office scenario described in
Section 7.1.1. As introduced in
Section 7.1, we assumed a variation of
= {1, 30, 60, 90, 1800, 3600, 14,400} s to analyze the impact of the frequency of message generation on the network performance. Therefore, a total of
simulations have been run varying the components of pairs of type
; where
is the set of different routing algorithms. Moreover, four network metrics were examined in each simulation: (i) average delivery probability; (ii) overhead ratio; (iii) average latency; and (iv) average number of hops to reach the destination. In the following, the definition of each metric is provided.
Let us define the number of messages that are created by node
i and sent towards node
j as
. Moreover, the number of delivered messages to node
j starting from node
i is given by
. The average delivery probability
in the network is given by Equation (
2):
In order to determine the performance of a routing algorithm in terms of number of relayed messages, i.e., the number of times each message is received by an intermediate node and relayed to another node in its path towards the destination, the overhead ratio
defined by Equation (
3) is considered:
where
represents the number of relayed messages during the process of sending a message from node
i to node
j. Finally, the average latency,
, and the average number of hops,
, were also considered in the performance analysis to evaluate the effectiveness of each algorithm in terms of response time.
Once the different network metrics have been defined, we focus on analyzing them in the case of the smart office scenario. As previously introduced, the delivery probability
evaluates the likelihood of a message to reach its destination. Since all the nodes in the scenario are able to send messages toward the rest of nodes,
Figure 8 reports the values obtained for
as a function of
.
By inspecting
Figure 8, several considerations emerge. At first, it can be seen that SACAR-IR presents a
delivery probability regardless of the frequency of message generation. The explanation of this situation is that the logic of this algorithm is different in nature compared to the rest of the evaluated algorithms. In SACAR-IR, a source node will only send a message to a destination node that (i) is located within its action range and (ii) has the same interests (goals/skills matching). Since this solution [
13] highly restricts the set of potential destinations, each created message will definitely reach its destination, with a resulting average delivery probability of
.
Regarding both the rest of SACAR versions and the benchmark solutions, it is necessary to remark that all of them exploit the store-carry-and-forward technique on packet forwarding. In these cases, source and destination nodes are not required to be within the same range, allowing intermediate nodes to receive a packet, temporarily store it, and forward it to a node that will (potentially) be met along the path. Looking again at
Figure 8, we can notice that SACAR-OCVN is the store-carry-and-forward algorithm that presents the best results in terms of delivery probability, reaching a peak value of
for
s. The exploitation of the OCVN concept in the forwarding actions of a node makes this solution particularly appealing for scenarios such as a smart office, with a limited size and a big ratio of individuals sharing common interests. Interestingly, the rest of the algorithms revealed a similar trend: when
was increased, i.e., the time period between rounds of messages was longer,
tended to increase. In particular, for values of
(i.e., when messages were generated very frequently), GSaR presented the best outcomes among this last set of algorithms. This algorithm presented a linear increase of
for values of
. Concretely, for
, it provided better results than SCAR-OCVN. In turn, SWR behaved better for values of
s, slightly overcoming our SACAR-OCVN solution for
s and
s. Finally, while SACAR-HC remained at the average of the benchmark algorithms, SACAR-Hybrid suffered for large values of
.
The second metric we aimed to analyze is the overhead ratio,
, which is reported in
Figure 9 as a function of
. From the figure, we can extract that, apart from the solutions that do not consider the possibility of relaying messages (SACAR-IR and DDR) where no overhead was experienced (
), SACAR-OCVN, SWR, and GSaR presented a similar flat trend with low values of
independently of the value of
. This means that a very small gap between the number of relayed messages with respect to the number of delivered ones was obtained. Therefore, nodes’ resources such as buffer occupation and energy consumption due to forwarding actions were not highly impacted because of the use of these algorithms. Instead, a clear increase in the value of
with respect to
was experienced when using MPR and ER. The spreading logic behind these solutions, where several copies of the message are created under different circumstances, severely impacted the network performance in terms of overhead ratio.
In the following, we evaluate the average latency,
, experienced by a message on its way from the source to the destination node. By inspecting
Figure 10, it can be seen that the average latency tended to increase with
. Clearly, SACAR-IR presented the best results in terms of average latency, since the source-destination pair was constrained to be physically placed within the same range. If we focus on the store-carry-and-forward-based algorithm set, we can see that both SACAR-OCVN and GSaR suffered for small values of
compared to the rest of the algorithms. The explanation of this situation for SACAR-OCVN is that a node will only forward the message to another node with the same interests (goal-skill matching). Similarly, for GSaR, a message was only forwarded if the nodes followed a specific trajectory. Therefore, although there may be several possibilities for forwarding the message when two nodes meet each other, a further constraint in the checking must be also satisfied. Remarkably, although SACAR-OCVN produced interesting results for large values of
, SWR was the algorithm that in general best fit the average latency among the ones that allowed the nodes to have a buffer to temporarily store messages.
Finally, we move our attention to analyze the number of hops a message needs to take from the source node towards the destination node.
Figure 11 reports the average number of hops as a function of
. Once more, the generic trend was that the number of hops increased with
, except for SACAR-IR, DDR (since the logic behind them is to create and send a message only to destinations that are one hop away), and SACAR-HC, for which the average number of hops was stabilized around one for each value of
. For GSaR, the number of hops was also stabilized around one because it is focused on different types of scenarios. Remarkably, and differently from the previous analysis in which the average latency was studied, SACAR-OCVN and SWR presented comparable results in terms of the number of hops.
As a summary, after the evaluation of the proposed algorithms and their comparison with the benchmark ones in a small scenario such as a smart office, we can state the next remarks: (i) if most of the nodes are within the same range, the best solution is to use SACAR-IR; (ii) if (i) is not the case, SACAR-OCVN presents the best outcomes among the set of store-carry-and-forward routing algorithms on delivery probability and overhead ratio, although an increase in the average latency and in the number of hops is experienced; (iii) if a balanced trade-off between delivery success and latency is required, then the best option according to the obtained results is to use SACAR-HC.
Having evaluated a small scenario, in the next section, we analyze the outcomes of the proposed solutions over a bigger one, both in terms of area to cover and in the number of nodes.
7.3.2. Evaluation of the Mall Scenario
A mall scenario is simulated by selecting a subset of nodes as devices with skills (screens), and the rest of them are people that go to the mall with the aim of buying items (i.e., their goal is to purchase). As in
Section 7.3.1, our first objective is to analyze the performance of each algorithm in this scenario. In addition, since the routing protocols proposed in this paper take into account the similarity between the goals/skills of the encountered nodes in order to forward a message (creating the OCVN), we also evaluate the impact of increasing the number of nodes with goals and the number of nodes with skills on the network performance.
The first network metric we aim to analyze is the average delivery probability. The resulting values of
as a function of
are reported in
Figure 12. Remarkably, SACAR-OCVN again outperformed the rest of the algorithms, especially for low values of
. In fact, the best gains were obtained when messages were continuously generated and the network traffic was high (
of gains when
s). Although SACAR-OCVN
slightly decreased with
, it was almost stable and always above
. On the contrary, the rest of the algorithms presented an increasing trend as a function of
, but their outcomes were worse than the ones obtained when applying the concept of OCVN. Among them, GSaR must also be highlighted because it provided better results for
. In fact, for
= 14,400, it presented even better results than SACAR-OCVN.
In a big scenario such as a mall, it is necessary to evaluate the time a message is stored in the buffer of a node. Since there are many nodes in the network and the traffic can be significantly high, an efficient usage of nodes’ storage capabilities to perform store-carry-and-forward actions must be performed. Therefore, a new metric is analyzed in this scenario,
, which is the average time a message is stored in a buffer (since the obtained results for the rest of metrics considered in
Section 7.3.1 presented similar conclusions in the mall scenario, we avoid to including them in this analysis, and we only focus on
and
).
Figure 13 reports the values of
as a function of
. From the results, we can highlight that a division of the algorithms into two groups according to their behavior respecting
can be done. At first, there were four solutions (DDR, SACAR-HC, GSaR, and SWR) for which the average time a message spent inside a relaying node increased with
, reaching prohibitive results when the time period between consecutive rounds of message generation was high. On the contrary, there was no impact on
when
was varied for the rest of algorithms. If we focus on our best solution in terms of delivery probability, i.e., SACAR-OCVN (see
Figure 12), we can state that, although the logic of the algorithm restricts the number of nodes that can be used for relaying a message (there must be a goals/skills matching), and therefore messages must spend longer times in buffers,
results reported in
Figure 13 indicate that these times are comparable with solutions where such a constraint is not taken into account and where messages can be relayed by any node found along the path.
In the last analysis, our aim is to evaluate what is the impact of increasing the number of nodes with goals and the number of nodes with skills on the different considered metrics when our SACAR-OCVN solution is applied. To do that, a multivariate analysis has been performed considering the number of nodes having goals
, as well as the number of nodes with skills
in the scenario as independent variables, while the considered dependent variables were
,
,
,
, and
. Thus, the idea is to know the impact of each independent variable on the selected dependent variables. As a summary, the set of variables used in the statistical analysis are defined and categorized in
Table 3.
Regarding the setting of the simulations to obtain the results to be statistically analyzed, we considered the mall scenario with a value of s, i.e., the time period between two consecutive rounds of messages creation was one hour. This interval was selected due to the fact that it allows the nodes to move along the scenario and exploit their store-carry-and-forward capabilities while no other aspects can interfere with the algorithm’s functioning, e.g., an increase in the amount of traffic flowing through the network. Several tests were performed by varying the number of nodes and the initial position of each of them in the scenario. In fact, a total of 25 independent runs were carried out for each test to retrieve statistically-significant outcomes.
Table 4 shows the outcomes after performing the Ordinary Least Squares (OLS) regression considering the number of nodes with goals as independent variable, achieving an average value of
. First, all the variables considered in the analysis were statistically significant (
) except
, for which no conclusions can be drawn (
). Looking at unstandardized coefficients (B column), it is clear that there existed a directly proportional relationship between the number of nodes with goals,
, and the overhead ratio,
, experienced in the network. In particular, an increase in one unit in the number of nodes with goals in the scenario was associated in a statistically-significant way with an increase in
in the overhead ratio. In the same way, an increase of
s on the average latency,
, was paid as a penalty if a new node became part of the network. The number of hops,
, in turn, was not highly affected by the considered independent variable, with a slight increase per added node. However, it is interesting to highlight that the only variable with negative (but also statistically significant) coefficients was the average buffer time,
, with
. This means that buffer time was reduced with the increase in the number of nodes with goals, following a decreasing progression of
s per new node. This situation can be explained as follows. Since there are more nodes in the network to share common interests with, the probability for a node to store a message temporarily that is intended for a different destination decreases. Therefore, the average buffer time in the network also decreases.
Similar results emerge by inspecting
Table 5, when the number of nodes with skills,
, is taken as the independent reference variable. Again, an increase in the overhead and in the average latency was obtained as the result of adding a node with skills to the network; whilst a reduction of more than four points in the average buffer time was achieved. Similarly to
Table 4, the impact of increasing the number of nodes over the number of hops a message must traverse remained negligible (
). However, the main difference with the previous analysis was that the
variable was statistically significant. Therefore, we can extract from the results that the impact of adding a node with skills to the network is associated with an improvement of
in the delivery probability.
To sum up the OLS regression outcomes, the higher the number of nodes in the network, the higher the overhead ratio and the average latency imposed by the application of SACAR-OCVN. However, a remarkable point is that nodes’ resources, such as their buffer to store messages temporarily, were efficiently used by our proposed solution. Finally, the delivery probability was increased in the case of adding nodes with skills to the network.