Echo State Learning for User Trajectory Prediction to Minimize Online Game Breaks in 6G Terahertz Networks

Abstract

Mobile online gaming is constantly growing in popularity and is expected to be one of the most important applications of upcoming sixth generation networks. Nevertheless, it remains challenging for game providers to support it, mainly due to its intrinsic and ever-stricter need for service continuity in the presence of user mobility. In this regard, this paper proposes a machine learning strategy to forecast user channel conditions, aiming at guaranteeing a seamless service whenever a user is involved in a handover, i.e., moving from the coverage area of one base station towards another. In particular, the proposed channel condition prediction approach involves the exploitation of an echo state network, an efficient class of recurrent neural network, that is empowered with a genetic algorithm to perform parameter optimization. The echo state network is applied to improve user decisions regarding the selection of the serving base station, avoiding game breaks as much as possible to lower game lag time. The validity of the proposed framework is confirmed by simulations in comparison to the long short-term memory approach and another alternative method, aimed at thoroughly testing the accuracy of the learning module in forecasting user trajectories and in reducing game breaks or lag time, with a focus on a sixth generation network application scenario.

1. Introduction

Recently, online games have presented new challenging opportunities, enabling real-time interactions and cooperation in a shared virtual world. In particular, the improved reliability and flexibility offered by new wireless technologies will boost the emerging mobile gaming paradigm. Mobile games have improved hugely—from a snake that hunts little pixels to 4 K-quality behemoths. Online games face strict quality of service (QoS) constraints, typically expressed in terms of service delay and interactivity, and are extremely susceptible to network conditions, implying that slight changes in network performance will drastically impact the user experience, causing gaming breaks.

In such a context, maintaining a coherent game state (i.e., a connection continuum with the game server) that is shared by all players’ applications is not a trivial problem. In particular, accidental gaming breaks may cause a mismatch between user game states. As an example, we can look at online poker. The user may have just acquired the best hand they have ever seen, and the stake is quite interesting. The user is just about to show their cards, but the wireless connection fails, e.g., due to user mobility. Suitable methodologies to prevent this unfortunate event are appropriate to avoid annoying the users. Furthermore, it should be highlighted that online poker is currently evolving, suggesting new challenges that replicate the ambiance of a real poker room. This makes it necessary to resort to new wireless network technology (i.e., sixth generation (6G) wireless network technology) to allow very fast data delivery to enable new scenarios where players will have the perception of being in the same room, even by touching the cards they have been dealt. New online games must allow real-time interactions, as well as remote immersive interaction (RII), hence making the connection continuum a mandatory requirement. In particular, to provide an immersive and engaging gaming experience, it is essential to guarantee a connection connectivity continuum between players and the game server and fast responses to player actions. Likewise, RII is related to VR/AR contexts, where remote players can interact with each other. In this case, many new information types, i.e., haptic, are dispatched by the network to meet the timing and reliability requirements that enable a high-quality immersive experience. Although real-time communications are not strictly mandatory in RII, latency plays a significant role. Excessive latency can impede the sense of presence and immersion, as it hinders a player’s ability to correlate cause and effect, thus negatively affecting the overall experience [1]. The latency threshold for VR/AR online games should not exceed 20 ms [2]. Figure 1 illustrates a VR/AR-based online gaming scenario where mobile players access the system from different locations. Here, the assumption, as in [3], of a round-trip latency of less than 20 ms will leave approximately 7 ms for data transmission. This makes it mandatory to use upcoming sixth generation (6G) technology that allows lightning fast speed, enabling near-instantaneous downloads and updates. What takes 5 h to download under 5G technology will take 18 s in upcoming 6G networks. When compared to actual 5G networks, 6G will allow a huge improvement in terms of performance. The arrival of 6G will make use of terahertz (THz) bands between 100 GHz and 10 GHz to achieve a peak data rate of 1000 gigabits per second with an air latency of fewer than 100 microseconds. New 6G technology will be 50 times faster, 100 times more dependable, and accommodate ten times more devices per square kilometer in comparison to 5G. Moreover, upcoming 6G wireless network technology has overcome the limits of previous network generations (i.e., 3G/4G/5G) in terms of the low bandwidth and latency, in addition to scarce interactivity, conferring a new impetus on the achievement of online gaming experiences characterized by a high interactivity, immersiveness, and communication reliability. In particular, the main breakthrough of 6G network technology is its extremely low latency via the exploitation of high-frequency bands including $\left[30,300\right]$ GHz-band millimeter waves (mmWaves) and free-space optical in the range of $\left[200,385\right]$ THz [4,5]. One of the most ambitious challenges in 6G generation networks for both researchers and practitioners is the proper exploitation of the network facilities and resources in order to meet the quality of service (QoS) requirements needed to support highly interactive online games [6] and guarantee the connection continuum to users [6].

Recently, the trend in the games industry market has been to attract new customers by introducing novel technologies and features such as online functionalities, especially the chance to remotely play through a domestic or an internet network. Accordingly, the users are able to play games far from their own dedicated game device [7], exploiting the potentialities of novel computing paradigms such as Pervasive Edge Computing (PEC). According to PEC, computation is offloaded to processing nodes, i.e., small base stations (SBSs), deployed on the network edges, reducing the typical latency of the centralized cloud computing approach.

It is important to highlight here that, according to the emerging trend, users may also require online gaming when they are on the move, making the prediction of the communication channel conditions in relation to the mobility pattern a crucial point in order to properly support the service provision. Despite the fact that 6G technology will empower the practical realization of interactive online games, some points have still to be deeply investigated. In particular, since the coverage area of a 6G small base station (SBS) is typically limited, user mobility represents a critical issue in relation to the possibility of experiencing temporary game breaks (i.e., game lag) that annoy users and make them perceive low quality of service (QoS) and quality of experience (QoE) levels. In particular, the prediction of the connection conditions with a given SBS at the next user location represents an important issue to efficiently support the growth of location-based and mobile service applications (see [8] and the references therein). Furthermore, connection condition prediction is of special interest in 6G networks due to the high susceptibility of THz frequencies to blockage, and molecular absorption makes the investigation of predictive methods to effectively manage the service crucial to avoid cellular dead zones and consequent game breaks. A machine learning strategy is required to provide a reliable connectivity prediction with respect to the SBSs of a 6G network to allow seamless services.

In such a context, recurrent neural networks (RNNs) [9,10,11,12,13,14,15] have provided an interesting tool to perform scalar time series predictions. In this regard, an echo state network (ESN) consists of an RNN with an intrinsic simplicity in the training and architecture structure [15], particularly able to capture the nonlinearity of the series dynamics.

In summary, the main contributions of this paper are as follows:

• A deep investigation and discussion of 6G networks to properly support online gaming services in the presence of user mobility, taking into account both the user blockage attenuation and the possibility of not having a line-of-sight (LoS) link;
• Application of an ESN module in order to provide a connection continuum towards the game server by selecting the most suitable SBS to support seamless online gaming services along the user path and avoid game breaks.
• Validation of the proposed approach in a high-density 6G network scenario by comparing analytical predictions with simulation results and alternative method performances.

Finally, it is important to note that this paper contextualizes the application of the proposed framework to online gaming, but it can be applied to a rich variety of different service classes with strict delay constraints operating in mobile environments. The rest of the paper is organized as follows. In Section 2, an in-depth review of the related literature is provided. Section 3 presents the system model description, while Section 4 proposes the ESN-based user trajectory forecasting approach. Likewise, Section 5 details the performance analysis, while our conclusions are drawn in Section 6.

2. Related Works

In reference to online games, paper [16] addresses the problem of user communication during prolonged online gaming sessions. In particular, the authors take into account the voice, the textual chat, and the predetermined commands as a method to communicate the team strategy. Consequently, the authors claim that a more effective communication strategy is represented by a combination of voice and predetermined commands. Comparison results were provided after extensive qualitative and quantitative analyses. Then, in paper [17], the relationship between players’ personalities and the corresponding game strategy is investigated for popular online games such as League of Legends. The authors start by identifying the main five personality traits. Then, they analyze the existing correlations between each personality profile and the strategy adopted in the game session. In fact, paper [17] exhibits an evident correlation between the user personality type and the actions performed during role-based online games. The authors of paper [18] propose a large-scale real-time analysis of churn users, also considering the case in which there are millions of active users in the game. In detail, the paper presents a predictive framework able to forecast at what point a user will leave the game and the amount of gaming hours played before a break. The proposed study contributes to analyses of game industry revenue, since user participation is fundamental for the monetization of the game. Moreover, paper [18] exploits a big data analysis to provide an effective churn user prediction framework. In contrast, in paper [19], the game servers are the focus of the work. In fact, game servers play the role of verifying the status and user connections. The VENUS II system is proposed to make game testing automatic, supporting blackbox testing and both scenario-oriented testing and load testing. The framework provided is able to test games without knowledge of the game client code. VENUS II acts on the basis of game grammar and game maps to describe the game logics. The correlation between payment and player churn is extensively studied in paper [20]. A data analysis was conducted on data from two real-world online games, combined with information stemming from player portraits, behavior sequences, and social networks. On the basis of this, a multi-source data, multi-task learning approach has been developed to capture the existing multi-source implicit information, aiming at predicting the churn and payments of each player simultaneously [20]. A data collection strategy about player churn is developed in paper [21], where the playing time from two online games is mined and the information about the regularity with which players are in-game is represented in the universal features form, resulting in a valuable tool for churn prediction. The validity of the solution proposed in [21] is evident from a performance analysis conducted using the developed and standard features, highlighting the superiority of the developed features when exploited to predict the churn. Paper [22] deals with the potential of a massive multiplayer online game framework for collaborative learning by identifying the conditions which trigger and promote collaborative interactions for learning among players. The framework takes into account the following aspects affecting collaborative learning processes: (a) player cooperation in tasks, (b) player features such as behavior or preferences, (c) social interactions between player groups, (d) the group structure, (e) the environment design, and (f) the social and ethical environment defined by the player community. In paper [22], focusing on group collaboration, specific patterns are defined, highlighting the existence of interconnections between the features of the environment and the community processes. Trajectory prediction through machine learning is addressed in paper [23], where the long short-term memory (LSTM) model and the bidirectional LSTM model are applied to predict the next user position. Similarly, in paper [24], the authors propose the joint application of LSTM and sequence-to-sequence learning to design a region-oriented multi-user multi-step trajectory forecasting framework. Trajectory prediction in 6G networks is the objective of paper [25], where the trajectory of a vessel is predicted through LSTM and its application is devoted to automatic identification in maritime IoT systems. Similarly, paper [26] addresses the problem of proactive handover and beam selection in THz networks for drone communication, supported by intelligent reflecting surfaces. The authors apply a Gated Recurrent Unit to jointly predict the serving base station and the serving beam. LiDAR sensors are exploited in [27] to predict dynamic human blockage considering indoor scenarios. An ESN-based predictive approach has been pursued in paper [28], in which mobile communication traffic has been forecasted considering complex network structures such as small world topologies. Then, an ESN has been also adopted by the authors of [29], contextualizing its application in time series predictions and involving a binary particle swarm algorithm to determine the ESN matrix weight connections. Furthermore, in paper [30], the particle swarm strategy has been applied to design both a single- and the multi-objective optimizer structure. Then, in paper [31], a novel learning algorithm based on the regularization method has been proposed, providing a stable solution to the approximation function and guaranteeing a good tradeoff between accuracy and smoothness. The problem of power supply prediction has been addressed in paper [32], in which an ESN has been applied. Likewise, the prediction of network traffic has been investigated in [33], aiming at planning resource allocation and optimal congestion control strategies to guarantee the reference QoS constraints. The considered state-of-the-art research is provided in a concise form in

Table 1. State-of-the-art research.

References	Topic Covered
[9,10,11,12,13,14,15]	RNN description
[16,17]	Efficient communication strategies for online gaming.
[18]	Proposal of a prediction approach of the time when a given user has to stop the game
[19]	Game server approach
[20,21]	Payment solutions
[22,23,24,25,29]	Collaborative learning process
[23,24,25,29]	User trajectory prediction
[26]	Proactive handover approach
[27]	Dynamic human blockage prediction
[30]	Multi-objective optimization
[31]	Regularization method
[32]	ESN-based power supply prediction
[33]	Optimal congestion control strategies

.

Finally, in comparison to Table 1 (the existing literature), the main contributions of the paper can be summarized as follows:

• Contextualization of the mobile online gaming to 6G network environments, considering the advantages and the drawbacks deriving from the exploitation of THz communications;
• The application of an ESN empowered by the genetic algorithm (GA) to predict the mobile gamer trajectory, aiming at guaranteeing a seamless service provision.

3. System Model

This paper focuses on the scenario sketched in Figure 2, consisting of a set of SBSs $\mathcal{B}$, whose cardinality is b, i.e., $\left|\mathcal{B}\right|=b$, which provide a connection continuum with a given game server to support online gaming services for a given user i (i.e., the tagged user) following a specific mobility trajectory (the proposed approach is unaware of the presence of more than one cloud provider and the specific service placement). Let ${\mathcal{T}}_i$ express the trajectory of the user i, represented as a set of 2D user positions $(x_t,y_t)$, where the subscript t denotes the instant of observation. Consequently, considering T at the instant at which the user trajectory observation ends, we have that

$${\mathcal{T}}_i=\left\{(x_t,y_t)\right|t=1,\dots ,T\}.$$

(1)

Futhermore, it is important to highlight here that the tagged user i, tracing the trajectory, moves through the coverage area of different SBSs, which commonly overlap, experiencing different receiving signal power levels. In reference to this, by assuming $\beta$ as the SBS serving user i, we identify ${\mathcal{P}}_t^{\beta }$ as the associated power received by the tagged user side at time t. In order to avoid a definite game break, user i has to switch off their connection to the most convenient SBS in terms of a higher received power level whenever ${\mathcal{P}}_t^{\beta }$ drops below a fixed threshold ${\mathcal{P}}_T$, under which the link with the original SBS is dropped.

Moreover, in the case of a 6G network, due to the high susceptibility of the terahertz frequencies, the presence of the LoS 6G channels is significantly influenced by the surrounding environment, i.e., other mobile users, objects such as foliage [34,35], or the oxygen level. This makes the connection loss of a given SBS strongly susceptible to the variations caused by external conditions. To perform a proper investigation of this issue, the broadband statistical spatial channel model (SSCM) proposed in [36] has been adopted, together with the human blockage model given in [37,38], as detailed later. Then, the close-in path loss considered here is compliant with [36,39,40], and, in particular, it is given as

$$\begin{array}{cc}\hfill {\mathrm{PL}}^{\mathrm{CI}}(f,d)\left[\mathrm{dB}\right]& =\mathrm{FSPL}(f,1\mathrm{m})\left[\mathrm{dB}\right]+10n{log}_{10}\left(\frac{d}{d_0}\right)\hfill \\ \hfill & +\mathrm{AT}\left[\mathrm{dB}\right]+{\chi }_{\sigma }^{\mathrm{CI}},\hfill \end{array}$$

(2)

where $d\ge d_0$, f represents the carrier frequency in GHz, d denotes the 3D T-R separation distance, n is the path loss exponent, $d_0$ is the space reference expressed in meters, and AT denotes the attenuation term due to the atmosphere. In addition, ${\chi }_{\sigma }^{\mathrm{CI}}$ is a non-zero mean Gaussian random variable, whose standard deviation is $\sigma$, and $\mathrm{FSPL}(f,1\mathrm{m})\left(\mathrm{dB}\right)$ is given by [41]

$$\begin{array}{cc}\hfill \mathrm{FSPL}(f,1\mathrm{m})\left[\mathrm{dB}\right]& =20{log}_{10}\left(f\right)\left(\frac{4\pi f\times {10}^9}{c}\right)\hfill \\ \hfill & =32.4\left[\mathrm{dB}\right]+20{log}_{10}\left(f\right),\hfill \end{array}$$

(3)

in which c is the speed of light. Moreover, we have that

$$\mathrm{AT}\left[\mathrm{dB}\right]=\alpha [\mathrm{dB}/\mathrm{m}]\times d\left[\mathrm{m}\right],$$

(4)

in which $\alpha$ is the attenuation factor which takes into account the effects due to the dry air, water vapor, rain, and haze as in [36,37,38,39,40]. To model human blockage events, we have resorted to the four-state Markov model proposed in [38], in which the mean attenuation of a blockage event is approximated by a log-normal random variable with the average mean attenuation ${\mu }_A$ expressed by

$${\mu }_A\left[\mathrm{dB}\right]=10{log}_{10}\left(g+\frac{180}{r}\right),$$

(5)

where g is a constant and r is the receiver antenna azimuth, expressed in degrees from 1 to 180 degrees. Then, in accordance with the channel assumptions assumed in [36], we have that the probability of having the LoS link as a function of the Euclidean distance d between the SBS and the receiver can be expressed as

$$P_{\mathrm{LoS}}\left(d\right)={\left[min\left(\frac{d_1}{d},1\right)\left(1-e^{-\frac{d}{\nu }}\right)+e^{-\frac{d}{\nu }}\right]}^2,$$

(6)

where $d_1$ is the breakpoint distance at which the LOS probability is no longer equal to 1 and $\nu$ (m) is a decay parameter.

It is important to highlight here that the game break, hereafter referred to interchangeably as the service break, occurs when ${\mathcal{P}}_t^{\beta }<{\mathcal{P}}_T$ for a tagged user linked to a given SBS and they are not able to switch to a different SBS due to human blockage attenuation and/or the propagation conditions. Usually, in the practical implementation of 6G networks, due to a dense SBSs deployment, non-LOS (NLOS) conditions have a low probability of taking place. However, whenever they occur, the proposed approach can avoid used being bored due to game breaks by resorting to suitable countermeasures, trivially suggesting users to stop in advance in order to complete the game or to store the session before suffering a connection loss and resuming it as soon as a new one is set up. The influence of NLOS conditions on the performance behavior of the proposed approach by resorting to the use of the game store/resuming solution is investigated in Section 4.

4. Echo State Network

In order to improve user decisions regarding the selection of the serving base station, through trajectory prediction and avoiding game breaks as much as possible to lower the game lag time, we involve the usage of an RNN. RNNs are generally credited as one of the most effective tools to perform time series forecasting in the presence of temporal dependencies among successive samples. Since RNNs have exhibited remarkable abilities in performing time series predictions [42,43], their application may represent a valuable approach to forecast the nonlinear behavior of the trajectory ${\mathcal{T}}_i$, the object of the analysis. The ESN, belonging to the class of RNNs, has an intrinsic simplicity in training and in its architecture [15], resulting in a fast and efficient RNN. A conventional ESN is formed by an input layer, a recurrent layer, i.e., the reservoir layer, and an output layer. Therefore, an ESN is characterized by [42,43,44]:

• An input layer;
• A reservoir layer;
• A large number of sparsely connected neural units;
• An output layer;
• Efficiency in reference to both the time complexity and the energy consumption.

This paper applies an ESN combined with the GA, aiming at finding the suitable ESN parameters, as reported in Figure 3. It is important to highlight here that since the connections of the reservoir layers are randomly initialized, the ESN is a light machine learning solution capable of being run on mobile devices with a limited battery lifetime. In particular, by considering $\mathbf{u}\in {\mathbb{R}}^{q\times 1}$ as the input vector of the trajectory position values, in which the generic element $s_t$ represents the user position at time t, i.e., $s_t={(x_t,y_t)}^T$, we have

$$\mathbf{u}\left(q\right)={[s_1,\dots ,s_t]}^T,$$

(7)

in which T denotes the transpose operator. As highlighted in Figure 3, the input $\mathbf{u}$ is linked to the first reservoir layer through ${\mathbf{W}}_{\mathit{in}}\in {\mathbb{R}}^{m\times q}$, an input weight matrix, in which m is the number of neurons in the reservoir component. The procedure starts with the input processed by the first reservoir layer, then its output serves as the input of the output layer. Consequently, the reservoir weight matrix update is [44]

$$r^{m\times 1}\left(q\right)=tanh({\mathbf{W}}_{\mathit{in}}^{m\times q}{\mathbf{x}}^{q\times 1}\left(q\right)+{\mathbf{W}}_{\mathit{r}}^{m\times 1}(q-1)),$$

(8)

where $r^{m\times 1}$ is a vector of internal units in the reservoir part, while ${\mathbf{W}}_{\mathit{in}}^{m\times q}$ represents the weights matrix associated with the connections existing between the input layer and the reservoir level. Finally, ${\mathbf{W}}_{\mathit{r}}^{m\times 1}(q-1)$ is the recurrent weights matrix. Let $v\left(q\right)$ be the output vector and ${\mathbf{W}}_{\mathit{out}}^{q\times m}$ be the weight matrix associated with the connection between the reservoir and the output layer. Therefore, the relationship between the reservoir and the output level is the following

$$v\left(q\right)={\mathbf{W}}_{\mathit{out}}^{q\times m}{r}^{m\times 1}\left(q\right).$$

(9)

The application of the genetic algorithm (GA) [45] has been realized to select suitable global parameters for the ESN which are the number of the nodes in the reservoir and the spectral radius. Generally speaking, in ESNs, these parameters stem from an exhaustive search or from random experiments, which are typically time and computationally consuming. In accordance with the reference literature, the GA [46] is an iterative algorithm producing a range of solutions utilizing selection and reproduction processes. More specifically, the next generation of solutions derives from the current population, in which the GA identifies individuals with the best values of the fitness function, i.e., the metric directing the selection criterion. Then, the individuals with the best fitness values, typically referred to as elite, are directly admitted to the next generation, together with the children derived from the crossover and the mutation procedures [43,46] with valuable values of the fitness function. By denoting $\gamma =(N,\theta )$ as the solution parameters which are object of our focus, where N is the amount of nodes in the reservoir and $\theta$ expresses the spectral radius, the GA acts as follows [45,46]:

• A random initial population of 100 individuals $\gamma$ is generated; the ESN is set in accordance with $\gamma$. The fitness function, computed as the percentage of MSE of the training data as in [45], is evaluated.
• Among the individuals belonging to the current population, some elements are selected to produce the next one, following the steps:
  • Rank each individual in accordance with their corresponding fitness function values;
  • Identify the elite and incorporate them into the next population;
  • Select parents among the individuals with a high-value fitness function. By performing random changes from a single parent, or by combining the parameters of a pair of parents, i.e., by crossover, children are generated. Then, the next generation is created by replacing the parent with the children.
  • Terminate when the maximum number of mutations $\mathcal{M}$ is reached.

The proposed SBS switch strategy framework, summarized in Figure 4, acts as follows:

• For each instant t, the position ${\widehat{s}}_{t+1}$ is predicted, exploiting the sample points $s_1,\dots ,s_t$;
• Then, ${\mathcal{P}}_{t+1}^{\beta }$ is computed;
• If ${\mathcal{P}}_{t+1}^{\beta }<{\mathcal{P}}_T$, then select the SBS $i^{\ast }\in \mathcal{B}\setminus \left\{\beta \right\}$ such that

  $$i^{\ast }={\mathrm{argmax}}_{i\in \mathcal{B}\setminus \left\{\beta \right\}}{\mathcal{P}}_{t+1}^i,$$

  (10)

  and leave $\beta$ in favor to be served by $i^{\ast }$ at time $t+1$;
• Otherwise, if ${\mathcal{P}}_{t+1}^{\beta }\ge {\mathcal{P}}_T$, the migration is not needed and the user continues to be served by the SBS $\beta$.

It is crucial to highlight here that the application of the predictive ESN module allows an improvement in the system’s ability to support seamless service continuity, forecasting the movement behavior of the user one step ahead and anticipating a possible decline in service quality.

5. System Performance

Since this paper focuses on a 6G network environment, performance results have been obtained by resorting to extensive computer simulations exploiting the open source 5G and 6G channel simulator NYUSIM [41]. The system parameters corresponding to the scenario object of our analysis have been summarized in

Table 2. Simulation parameters (set in accordance with [41]).

Simulation Setting
SBS TX power	30 dBm
SBS height	3 m
Foliage attenuation	0.6 dB/m
Humidity	50%
Bandwidth	28 GHz
Carrier frequency	1 THz
Barometric pressure	1013.25 mbar
Temperature	20 degree
User terminal height	1.65
Number of SBSs	9

, whereas

Table 3. ESN parameters.

ESN Parameters
Population size	100
N	$[100,800]$
Elite count	5
Crossover fraction	0.78
$\theta$	[0.75, 1.4]
Activation function	tanh
Loss function	MSE
$\mathcal{M}$	15
${\mathcal{P}}_T$	−115 dBm

details the ESN parameters. In particular, we have assumed a total of nine SBSs with a height of 3 m and a transmitting power equal to 30 dBm. Furthermore, the user terminal height has been approximately considered at around 1.65 m, and the humidity percentage has been set to 50%, with a temperature of 20 degrees and a barometric pressure equal to 1013.25 mbar. Then, the carrier frequency was set to 1 THz, whereas the bandwidth was set to 28 GHz. Likewise, the parameters of the ESN have been reported in the ESN parameter table. In detail, as a loss function we have considered the Mean Squared Error (MSE) given by

$$\mathit{MSE}=\frac{1}{M}\sum _{i=1}^M{({\widehat{s}}_{i+1}-s_{i+1})}^2,$$

(11)

where M represents the number of the samples in test data and ${\widehat{s}}_{t+1}$ and $s_{t+1}$ are the predicted and the actual values at time $t+1$, respectively. The connections between the units in the ESN have been randomly generated, whereas the number N of units in the reservoir, as well as the spectral radius $\alpha$, has been selected by applying the GA. The optimization process involving N and $\alpha$ was performed considering the range $[100,800]$ and $[0.75,1.4]$ for the two parameters, respectively. Then, as stopping criterion of the GA, the achievement of the maximum number of generation, here set to 15, has been used. Moreover, the elite selection process terminates when the best five individuals advance to the next population generation. In contrast, the crossover fraction per generation has been set to 0.78. As the user mobility pattern, we used the traces for locations reported in [47], collected from 50 students at the Beijing University of Posts and Telecommunications.

Aiming at testing the validity of the ESN-GA module, we compared the proposed solution with the standard version of the serving SBS decision policy (SP), acting in accordance with the following steps:

• For each instant t, ${\mathcal{P}}_t^{\beta }$ is computed;
• If ${\mathcal{P}}_t^{\beta }<{\mathcal{P}}_T$, then select the SBS $i^{\ast }\in \mathcal{B}\setminus \left\{\beta \right\}$ such that

  $$i^{\ast }={\mathrm{argmax}}_{i\in \mathcal{B}\setminus \left\{\beta \right\}}{\mathcal{P}}_t^i,$$

  (12)

  and leave $\beta$ in favor to be served by $i^{\ast }$ at time t;
• Otherwise, if ${\mathcal{P}}_t^{\beta }\ge {\mathcal{P}}_T$, the switch off is not needed and the user continues to be served by the SBS $\beta$.

Furthermore, the proposed approach has also been compared with the long short-term memory (LSTM) neural network [15] and the ARM approach [48]. First of all, the accuracy of the proposed ESN-GA-based prediction strategy has been tested. Note that in our case, by using a single core of an Intel Xeon Gold 5120 CPU (2.20 GHz) equipped with 32 GB of RAM, the training time and setting parameters were around 3.567489 s, with a runtime prediction time of 0.008 s.

In reference to this, Figure 5 exhibits the accuracy of the user trajectory prediction provided as the time horizon increases. In particular, we would like to note that the maximum confidence interval for the proposed approach was $\pm 3.35\times {10}^{-4}$, which validates the performance of our strategy. More in depth, the time horizon represents the number of steps ahead at which forecasting is performed. As it is straightforward to note, the prediction accuracy reduces as the number of steps ahead grows. In fact, long-term predictions are more challenging than those provided for short time horizons. Nevertheless, the ESN-GA strategy adopted here results in an acceptable MSE, guaranteeing an error of 0.84 when the prediction horizon is 16. In contrast, the LSTM approach reaches higher values of the MSE.

In reference to the service continuity performance, from Figure 6, it is clear to note that the ESN-GA strategy, which provides an estimation of the actual power level of the signal received by the SBS in charge of the tagged user i in advance, notably limits the game breaks. In particular, the proposed strategy makes it possible to yield the estimation of the power level of the signal received by all the SBSs to which the tagged user may be linked in the new position. In this way, it is possible to select in advance the most suitable SBS which provides the highest received signal with the highest power level, hence speeding up the switch off procedure and lowering the game lag time for the ongoing online game. Specifically, the switch off procedure has to be started by the tagged user whenever the power level of the received signal for the linked SBS drops below −115 dBm, hereafter referred as the quality target, i.e., ${\mathcal{P}}_T$. Furthermore, as stated before, the proposed ESN-GA strategy efficiently manages NLOS conditions between the user and all possible SBSs, prompting the user to stop and complete the game or store the game state to prevent sudden interruptions of the game session. This behavior is validated in Figure 6, where it is shown that the proposed strategy achieves a better performance in service continuity in comparison to the SP alternative and the LSTM approach, in which the GA optimization routine is not applied. Nevertheless, in the presence of heavy attenuation conditions due to human blockage, the ESN-GA approach is able to reach a higher level of service continuity, minimizing the game breaks. Similarly, by considering Figure 7, the advantages introduced by the ESN-GA strategy are more appreciable once we refer to the time spent without connection (i.e., service outage). As it is evident from this figure, the overall outage time in which the user experiences a loss of connection is significantly lower in the ESN-GA case in comparison to the SP alternative and the LSTM scheme. The time represented in Figure 7 takes into account both the time required to perform channel access and the set-up time needed to switch the connection from one SBS to another. Clearly, the time spent accessing a channel in a 6G network is significantly lower (ns) than the set-up time (ms). For this reason, the trend of the curves in Figure 7 is highly impacted by the number of game breaks suffered by the user. In addition, Figure 8 highlights the number of service breaks experienced as the number of SBSs grows. Furthermore, in this case, the superiority of the ESN-GA approach is evident in comparison to both the SP and the LSTM strategies. As depicted in Figure 8, when considering eight SBSs, the number of service breaks drops to zero. Furthermore, Figure 9 expresses the probability of having a service break, i.e., a service interruption due to harsh propagation conditions or due to the lag time when performing the SBS switch off. As evident from the SP behavior, and confirmed by Figure 9, the SP strategy does not anticipate the SBS switch, starting the SBS switch off once the received signal power drops below ${\mathcal{P}}_T$. Consequently, the corresponding service interruption probability is always equal to 1 for the SP approach. In contrast, this probability can be significantly lowered by using the proposed ESN-GA approach, since the ESN-GA strategy anticipates the selection of the most suitable SBSs, hence strongly decreasing the switch off time. Furthermore, also in this case, the improved behavior of the ESN-GA approach is justified by the fact that the LSTM strategy is not optimized with the GA routine, resulting in a lower performance than the ESN-GA strategy. Finally, we would like to stress that due to the high levels of human blockage attenuation considered in Figure 9 (16 dB), the ESN-GA strategy exhibits a better behavior as the number of SBSs increases.

6. Conclusions

This paper focused on a 6G environment operating at terahertz frequencies to provide an online games service. In particular, the paper has addressed the problem of providing seamless online gaming in the presence of user mobility. In more detail, the user movement trajectory has been forecasted by using an ESN empowered by GA involvement. User trajectory forecasting has been pursued to predict the network performance to opportunistically select the serving SBS when a user has to switch off from the linked SBS to a different one to prevent an online game drop and to lower the game lag time. Finally, the validity of the proposed approach has been confirmed considering both the accuracy of the trajectory prediction provided by the ESN and the gaming experience breaks suffered by the user when adopting the proposed strategy in comparison to both the scheme in which prediction is not performed and that based on an LSTM neural network. Future works may include the integration of novel ML paradigms, such as federated or democratized learning, to enable cooperative training to predict the whole user trajectory and orientation.