Hypervector Approximation of Complex Manifolds for Artificial Intelligence Digital Twins in Smart Cities

Highlights

What are the main findings?

• The proposed AI approach is effective at hypervector approximation of complex manifolds in smart city settings.
• The Hyperseed algorithm can generate fine-grained local variations that can be tracked for anomalies and temporal changes, as well as incremental changes in dynamic data streams.

What is the implication of the main finding?

• This approach can be integrated into AI digital twins that have to process complex manifolds of high-dimensional datasets and data streams generated by smart cities.
• The interplay between digital twins and novel AI approaches is crucial in unpacking the complexities of urban systems and shaping sustainable and resilient smart cities.

Abstract

The United Nations Sustainable Development Goal 11 aims to make cities and human settlements inclusive, safe, resilient and sustainable. Smart cities have been studied extensively as an overarching framework to address the needs of increasing urbanisation and the targets of SDG 11. Digital twins and artificial intelligence are foundational technologies that enable the rapid prototyping, development and deployment of systems and solutions within this overarching framework of smart cities. In this paper, we present a novel AI approach for hypervector approximation of complex manifolds in high-dimensional datasets and data streams such as those encountered in smart city settings. This approach is based on hypervectors, few-shot learning and a learning rule based on single-vector operation that collectively maintain low computational complexity. Starting with high-level clusters generated by the K-means algorithm, the approach interrogates these clusters with the Hyperseed algorithm that approximates the complex manifold into fine-grained local variations that can be tracked for anomalies and temporal changes. The approach is empirically evaluated in the smart city setting of a multi-campus tertiary education institution where diverse sensors, buildings and people movement data streams are collected, analysed and processed for insights and decisions.

1. Introduction

More than half of the global population currently resides in urban areas, and this rate is predicted to reach 70% by 2050 [1]. Smart cities have been proposed as an overarching and enabling framework that can support and sustain this rapid urbanisation [2,3]. However, smart cities are impacted by a unique set of challenges, including the development and interoperability of new technologies with legacy systems, safety, privacy and confidentiality of people, systems, operations, regulatory frameworks that promote citizen, community and organisational engagement in decision-making and preparation for potential disasters and emergencies [4]. Smart cities have been defined and described by various research studies, organisations and government entities, and an aggregate of these definitions can be presented as, “urban environments that deploy and leverage innovative technology solutions and platforms to enhance the quality of life through improved urban services, reduced costs, and increased engagement across all stakeholders” [5,6].

Artificial intelligence (AI) has been recognised as a potential foundational technology, if not the strategic layer of technological infrastructure, that promotes innovation, efficiencies, productivity and resilience within smart city environments [7,8]. With the recent developments in generative AI (Gen AI), the value and contribution of AI-generated content in improving workflows, increasing engagement, fostering innovation and reducing complexities within smart city environments has been amply demonstrated and studied [9,10]. Collectively, AI and Gen AI has been defined by the Organisation for Economic Cooperation and Development (OECD) as “a machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments. Different AI systems vary in their levels of autonomy and adaptiveness after deployment” [11]. Despite the collective definition, the distinction between narrow AI (or task-oriented AI) and Gen AI is crucial in recognising the potential application areas and domains of relevance for AI in smart cities [12]. Narrow AI can be further deconstructed into the core capabilities of prediction, classification, association and optimisation. Diverse applications have been researched and reported in smart city settings, such as energy systems [13,14], intelligent transport [15,16], and digital health [17,18]. Alongside its transformative potential, AI is also being scrutinised for the risks posed from bias, inaccuracies, lack of transparency, accountability and potential risks to physical and mental health [19]. Responsible AI adoption and AI regulation are making headway into addressing some of these risks [20,21].

Digital twins are a complementing technology capability to AI that have also found diverse and effective applications in smart city settings. Digital twins are defined as “virtual representations (or replicas) of a physical asset—an object, person, process or system that can be used to simulate behaviours to improve our understanding and management of this entity” [22,23]. The Internet of Things and edge computing have accelerated the development and adoption of digital twins to detect the physics of structures and evolving conditions measured by sensors and IoT driven by edge computing [24,25]. Digital twins run simulations that explore limitations, including challenges with options for improvements through service updates to the physical asset [26,27]. Digital twins for smart cities have been deliberated in various studies, including inclusion of socio-economic components [23], virtual feedback loops where citizens can interact and report feedback on planned changes in the city [28], disaster management [29], transportation and smart grids [30] and climate adaptive resilience [31].

A computational challenge in smart city digital twins is in the complexity and large volumes of data that need to be processed, analysed, learned and leveraged for AI capabilities such as predictions and classifications [32,33]. Smart cities by definition are large urban areas that will continue to grow and evolve as populations within those settings increase, and this means the volumes and complexities of data will continue to grow. These data typically exist in high-dimensional spaces and can exhibit intricate, non-linear relationships. The diversity, magnitude and variability of such datasets and data streams with intersecting properties can be recognised as “complex manifolds” [34]. A manifold is a mathematical structure that locally resembles Euclidean space but can have more intricate global properties, making it a useful concept for understanding non-linear data landscapes [35]. In the context of smart cities, manifolds can model multiple interacting components, accommodating non-linearities and relationships between various urban systems such as transportation networks, energy grids and technology platforms. Manifolds, including examples like Riemannian and Finsler manifolds, are defined over the field of real numbers and provide flexibility for modelling a wide range of complex systems. While these manifolds capture the complexity of urban environments, novel AI algorithms need to be developed to navigate and analyse these spaces, generating meaningful insights for predictions and classifications [35,36]. Recent work includes AI digital twins that mitigate undesirable emergent behaviours [37] and digital twin-driven prognostics for complex medical equipment [38]. Hypervectors are the encoded data vectors of hyperdimensional computing, an unconventional computing paradigm that aims to emulate brain-like processing, where the encoding process maps the input data from the scalar domain to a hyperspace. Hypervectors have very low computational needs compared to standard counterparts due to straightforward processing through simple logic elements. Hypervectors are also robust to errors and missing data, which makes them further ideal candidates for representing noisy and irregular data streams found in smart city settings.

In this paper, we present a novel AI approach for hypervector approximation of complex manifolds in high-dimensional datasets and data streams such as those encountered in smart city settings. This approach begins with the simple yet effective clustering capability of the K-means algorithm, followed by the Hyperseed algorithm [39] that unpacks each cluster to preserve internal structures and similarities while overcoming limitations in handling complex, non-linear manifolds. Additionally, we extend the Hyperseed algorithm to support incremental learning, enabling the model to adapt dynamically to continuous data streaming at varied frequencies and magnitudes. This approach is empirically evaluated within the smart city setting of a multi-campus tertiary education institution where diverse sensors, buildings and people movement data streams are collected, analysed and processed for insights and decisions. These experiments conducted on temperature sensor data streams collected at 15-min intervals from all buildings across campuses demonstrate its effectiveness in capturing local structures and incrementally adapting to new data streams.

The rest of this paper is organised as follows. Section 2 provides the necessary background on vector symbolic architectures (VSAs) and related concepts essential for understanding our proposed approach. Section 3 details the methodology, introducing the proposed approach and its incremental extension for approximating complex manifolds within high-dimensional datasets. Section 4 presents the experimental setup, including the dataset description, implementation details and the results obtained from applying our approach to real-world temperature sensor data from the La Trobe Energy Analytics Platform. Section 5 concludes this paper by summarising the key contributions and discussing the implications of our findings.

2. Background

A comprehensive review of hyperdimensional computing and hypervectors can be found in several recent studies. In this section, we outline only the aspects directly relevant to the proposed approach: hypervector binding, fractional power encoding and the resonator network. The bundling and ordering operations relevant to encoding input data involve combining multiple hypervectors to represent complex data structures and arranging them in a specific sequence to preserve temporal or spatial relationships. Bundling refers to the process of summing or combining hypervectors to aggregate information, while ordering ensures that the sequence of data points is maintained within the hyperdimensional space. These operations are integral to accurately encoding input data into hypervectors that capture the essential characteristics and relationships inherent in the original data [39].

2.1. Hypervector Binding

The binding operation is used to associate two hypervectors together, where the result of the binding is another hypervector. For example, for two hypervectors ${\mathbf{v}}_1$ and ${\mathbf{v}}_2$, the result of binding of their hypervectors (denoted as $\mathbf{b}$) is calculated as follows: $\mathbf{b}={\mathbf{v}}_1\circ {\mathbf{v}}_2$, where the notation ∘ is used to denote the binding operation. In Holographic Reduced Representations [40], the binding operation was implemented as a circular convolution of ${\mathbf{v}}_1$ and ${\mathbf{v}}_2$ that can in turn be implemented as the component-wise multiplication in the frequency domain. This observation inspired the Fourier Holographic Reduced Representation (FHRR) model, where the atomic hypervectors are already in the frequency domain in a form of phasors so that the component-wise multiplication, which is equivalent to the addition of phasors modulo $2\pi$, plays the role of the binding operation. There are two important properties of such binding operation. First, the resultant hypervector $\mathbf{b}$ is dissimilar to the hypervectors being bound, i.e., the similarity between $\mathbf{b}$ and ${\mathbf{v}}_1$ or ${\mathbf{v}}_2$ is approximately 0. Second, the binding operation preserves similarity. That is, the distribution of the similarity measure between hypervectors from some set $\mathcal{S}$ is preserved after binding each hypervector in $\mathcal{S}$ with the same hypervector $\mathbf{v}$. The binding operation is reversible and has been demonstrated in generalisations that require only integer operations for weight updates [41]. The unbinding, denoted as ⊘, is implemented via component-wise multiplication with a complex conjugate of the corresponding hypervector: ${\mathbf{v}}_2\oslash \mathbf{b}={\mathbf{v}}_1$. Being the inverse of the binding operation, the unbinding obviously has the same similarity preservation property when performed on all hypervectors in $\mathcal{S}$ with same hypervector $\mathbf{v}$.

2.2. Fractional Power Encoding

Fractional power encoding (FPE) was first proposed in [42,43] as a method for representing subsymbolic data in a vector space. The method was further generalised to represent functions in a vector space in [44]. The data sample is encoded by exponentiating a fixed random base hypervector. Consider an example of transforming a two-dimensional data domain. Let x and y denote each dimension. To represent a value in each dimension, we first generate two random base hypervectors ${\mathbf{x}}_0,{\mathbf{y}}_0\in {\mathbb{C}}^d$ as ${\mathbf{x}}_0\sim e^{j·2\pi ·U(0,1)}$ and ${\mathbf{y}}_0\sim e^{j·2\pi ·U(0,1)}$. In this context, d is set to a high value to ensure that each hypervector has sufficient dimensionality to capture the complexity of the data being encoded. The imaginary unit j is crucial for generating hypervectors with properties suitable for representing complex data structures in hyperdimensional space. Let us denote the bandwidth parameter $\beta$ regulating the similarity between the adjacent values $(x_i,x_{i+1})$ and $(y_i,y_{i+1})$ in the vector space. Value i along the x and y dimensions will be transformed via FPE as: ${\mathbf{x}}_i={\mathbf{x}}_0^{\beta ·i}$,${\mathbf{y}}_i={\mathbf{y}}_0^{\beta ·i}$. FPE together with VSA operations allows representing multi-dimensional continuous-valued spaces by binding together FPE hypervectors representing the values of each dimension. Hypervector ${\mathbf{p}}_{(i,j)}$ representing a pair of values $(x_i,y_j)$ is represented as ${\mathbf{p}}_{(i,j)}={\mathbf{x}}_i\circ {\mathbf{y}}_j$. With this transformation, the inner product between the hypervectors of any two data vectors represents a similarity kernel applied to the original data domain. Specifically, the kernel of FPE, when the base hypervectors are generated in the interval $(0,2\pi )$, is the $\mathrm{sinc}$ function: $K\left(x\right)={\displaystyle \frac{1}{\sqrt{2\pi }}}\mathrm{sinc}\left(\pi d\right)$. Figure 1 shows the similarity between FPE of the values $x=15$ and $y=15$ and all other FPEs of integer values on the $50\times 50$ 2D grid.

2.3. Resonator Network

The resonator network [45] is a method for factoring compound hypervectors constructed by binding several hypervectors (e.g., $\mathbf{a}\circ \mathbf{b}\circ \mathbf{c}$). Finding hypervectors by brute force checking of all possible combinations of the arguments is impractical since the number of such combinations grows exponentially with the number of arguments being involved. A resonator network depicted in Figure 2 solves this problem by a parallel search in the space of all possible combinations.

The resonator network assumes that none of the arguments are given but that they are contained in separate dictionaries (item memories), which are known to the resonator network. In a nutshell, the resonator network is a recurrent neural network that uses VSA principles to solve a combinatorial optimisation problem. Due to the space limitations, we omit the detailed description of the concept and refer the reader to [45] for examples of factoring hypervectors of data structures with it and to [46] for its comparison with other standard optimisation-based methods.

3. The Proposed Approach

The proposed approach aims to approximate a complex manifold by first applying the K-means algorithm to capture local neighbourhoods and then using the Hyperseed algorithm on these local clusters. Each cluster can subsequently be represented in a 2D map using FPE. This approach leverages FHRR representations and FPE to preserve the internal structure and similarities within each cluster. By applying K-means, initially, the complexity of the manifold is reduced, allowing Hyperseed to operate more effectively within these local clusters, thereby mitigating its limitations when dealing with complex, non-linear manifolds. The result is an efficient factorisation and representation of the data. This combined method facilitates the management and analysis of high-dimensional datasets, such as smart city data streams, by capturing local structures and providing a meaningful 2D representation. The architectural view of the proposed approach is presented in Figure 3, where the approach begins with K-means clustering, followed by the Hyperseed algorithm and the incremental Hyperseed algorithm that allows the model to adapt as new data are introduced. The algorithmic flow of this approach is presented in Algorithm 1.

3.1. K-Means Clustering

To manage the complexity and structure of high-dimensional data, we first apply the K-means clustering algorithm to partition the data into K clusters. Each cluster represents a subset of the data that is homogeneously distributed in the high-dimensional space. K-means clustering is a widely used method for partitioning a dataset into a set of distinct, non-overlapping groups. It operates by iteratively assigning each data point to the nearest cluster centre and then updating the cluster centres as the mean of the assigned points. Despite its simplicity, K-means can effectively handle clustering tasks in high-dimensional spaces, although several challenges and considerations arise in such contexts. The K-means algorithm is summarised in Algorithm 2.

Algorithm 1 Algorithmic flow of the proposed approach

1: Input: High-dimensional dataset $\mathbf{X}\in {\mathbb{C}}^d$, number of clusters K  2: Output: Clustered and approximated high-dimensional data in VSA space  3: Step 1: K-means Clustering  4: Initialize K cluster centers randomly  5: repeat  6:    for each data point ${\mathbf{x}}_i\in \mathbf{X}$ do  7:      Assign ${\mathbf{x}}_i$ to the nearest cluster center  8:    end for  9:    for each cluster $j\in \{1,2,\dots ,K\}$ do 10:      Update cluster center ${\mathbf{c}}_j$ as the mean of all points assigned to cluster j 11:    end for 12: until cluster centers converge 13: Step 2: Deploy Hyperseed on Clusters 14: for each cluster $j\in \{1,2,\dots ,K\}$ do 15:    Initialize HD-map ${\mathcal{P}}_j$ using FPE 16:    Apply Hyperseed algorithm to transform data hypervectors in cluster j to HD-map ${\mathcal{P}}_j$ 17: end for

Algorithm 2 K-means clustering

1: Input: Dataset $\mathbf{X}=\{{\mathbf{x}}_1,{\mathbf{x}}_2,\dots ,{\mathbf{x}}_n\}$, number of clusters K  2: Output: Cluster centers $\mathbf{C}=\{{\mathbf{c}}_1,{\mathbf{c}}_2,\dots ,{\mathbf{c}}_K\}$, cluster assignments $\mathbf{A}=\{a_1,a_2,\dots ,a_n\}$  3: Initialize K cluster centers randomly  4: repeat  5:    for each data point ${\mathbf{x}}_i\in \mathbf{X}$ do  6:      Assign ${\mathbf{x}}_i$ to the nearest cluster center $c_j$ using Euclidean distance  7:      Update cluster assignment $a_i\leftarrow j$  8:    end for  9:    for each cluster $j\in \{1,2,\dots ,K\}$ do 10:      Update cluster center ${\mathbf{c}}_j$ as the mean of all points assigned to cluster j 11:    end for 12: until cluster assignments no longer change significantly

3.2. Hypervector Mapping

To ensure that the high-dimensional data in each cluster can be effectively represented in a VSA space, the next step is to map the original data hypervectors $\mathcal{D}$ to an HD-map $\mathcal{P}$, as illustrated in Figure 4 using the unbinding operation, ensuring the preservation of the data’s internal similarity layout.

More formally, consider two vector spaces:

• $\mathcal{D}\in {\mathbb{C}}^d$: contains distributed representations of data sampled from an arbitrary manifold with unknown structure.
• $\mathcal{P}\in {\mathbb{C}}^d$: represents a subsymbolic data domain as an n-dimensional FPE tensor.

The goal is to construct $\mathcal{P}$ such that the transformation $\mathcal{F}\left(\mathcal{D}\right)$ maps vectors from $\mathcal{D}$ to vectors in $\mathcal{P}$, preserving the similarity layout. This ensures that the unknown similarity layout of $\mathcal{D}$ is accurately reflected in $\mathcal{P}$.

$$\mathcal{D}\oslash \mathbf{s}\Rightarrow \mathcal{P}$$

(1)

Here, $\mathbf{s}$ is a seed hypervector obtained through an unsupervised learning rule, allowing the similarity structure of the original data to be preserved in the transformed high-dimensional (HD)-map.

The factorisation of the unbinding result is efficiently solved using the Resonator network, which leverages the similarity preservation property of the (un)binding operation. This approach allows for the effective clustering and representation of high-dimensional data, facilitating more efficient data processing and analysis. By combining K-means clustering with the Hyperseed algorithm, high-dimensional data can be approximated and managed more effectively, preserving its intrinsic structure and enabling scalable analysis.

3.3. The Hyperseed Algorithm

Hyperseed [39] is an unsupervised learning algorithm that maps original data hypervectors to an HD-map using frequency holographic reduced representations (FHRRs) and fractional power encoding (FPE). The detailed steps of the Hyperseed algorithm are provided in Algorithm 3. The computational complexity of Hyperseed is $\mathcal{O}\left(INd\right|\mathcal{P}\left|\right)$, where I is the number of iterations, N is the number of training data hypervectors, d is the dimensionality of hypervectors and $\left|\mathcal{P}\right|$ is the size of the HD-map. When implemented on neuromorphic hardware, the search for the best matching vector (BMV) happens in constant time, further optimising the algorithm’s performance. Despite its low computational complexity, Hyperseed is primarily effective for datasets where the underlying data structure can be captured through relatively simple linear mappings. However, its performance degrades when dealing with more complex datasets that require mapping to non-linear manifolds. In these cases, where the data lie on or near a non-linear manifold with intricate feature interactions and high-dimensional correlations, Hyperseed may struggle to accurately capture the manifold’s structure, leading to reduced accuracy and robustness. This limitation impacts its applicability in scenarios where the data’s complexity is inherently non-linear and manifold-based.

3.4. Computational Complexity Analysis

The proposed approach integrates K-means clustering with the Hyperseed algorithm, aiming to enhance scalability and efficiency in handling high-dimensional data.

3.4.1. Complexity of the Proposed Approach

K-means clustering has a computational complexity of $\mathcal{O}(t·K·N·d)$, where t is the number of iterations, K the number of clusters, N the number of data points and d the dimensionality.

The Hyperseed algorithm operates with a complexity of $\mathcal{O}(I·{\displaystyle \frac{N}{K}}·d·|\mathcal{P}|)$ per cluster, where I is the number of iterations and $\left|\mathcal{P}\right|$ the size of the HD-map. On neuromorphic hardware, this reduces to $\mathcal{O}(I·N)·\tau$, with $\tau$ representing constant-time search operations.

Overall, the combined complexity is $\mathcal{O}(t·K·N·d)+\mathcal{O}(I·N·d·|\mathcal{P}\left|\right)$, demonstrating linear scalability with respect to N and d.

Algorithm 3 Hyperseed algorithm

1: Input: Set of data hypervectors $\mathcal{D}\in {\mathbb{C}}^d$  2: Output: Seed hypervector $\mathbf{s}$, HD-map $\mathcal{P}$  3: Initialization:  4: Generate initial random unit hypervectors ${\mathbf{x}}_0,{\mathbf{y}}_0\in {\mathbb{C}}^d$  5: Generate HD-map $\mathcal{P}$ using FPE:  6: for each grid coordinate $(i,j)$ do  7:    ${\mathbf{x}}_i={\mathbf{x}}_0^{ϵ·i}$  8:    ${\mathbf{y}}_i={\mathbf{y}}_0^{ϵ·i}$  9:    ${\mathbf{p}}_{(i,j)}={\mathbf{x}}_i\circ {\mathbf{y}}_j$ 10: end for 11: Initialize seed hypervector $\mathbf{s}\sim e^{j·2\pi ·U(0,1)}$ 12: Learning Phase: 13: for each iteration do 14:    Search Procedure: 15:    for each data hypervector ${\mathbf{d}}_i\in \mathcal{D}$ do 16:      Compute noisy vector ${\mathbf{p}}_i^{*}={\mathbf{d}}_i\oslash \mathbf{s}$ 17:      Find BMV in $\mathcal{P}$ with highest cosine similarity to ${\mathbf{p}}_i^{*}$ 18:    end for 19:    Weakest Match Search (WMS) Phase: 20:    Initialize lowest similarity variable $D_{min}=1$ 21:    for each ${\mathbf{d}}_i\in \mathcal{D}$ do 22:      Compute $p_i^{*}$ and find BMV in $\mathcal{P}$ 23:      Compute similarity $D_{BMV}$ 24:      if $D_{BMV}<D_{min}$ then 25:         Update $D_{min}=D_{BMV}$ 26:         Select ${\mathbf{d}}_i$ for update 27:      end if 28:    end for 29:    Update Phase: 30:    Select target hypervector ${\mathbf{p}}_{\mathtt{target}}$ using FFTR heuristic 31:    Compute perfect mapping hypervector ${\mathbf{d}}_i\circ {\mathbf{p}}_{\mathtt{target}}$ 32:    Update seed hypervector: $\mathbf{s}=\mathbf{s}+{\mathbf{d}}_i\circ {\mathbf{p}}_{\mathtt{target}}$ 33: end for

3.4.2. Comparison with Existing Manifold Learning Methods

Traditional manifold learning techniques such as t-SNE, UMAP and Isomap exhibit higher computational complexities, making them less scalable for large datasets. Specifically, t-SNE has a complexity of $\mathcal{O}(N^2·d)$, which poses scalability challenges due to its quadratic growth with the number of data points. UMAP improves upon this with a complexity of $\mathcal{O}(NlogN)$, offering better scalability than t-SNE but still higher than our proposed method for large N. Isometric Feature Mapping (Isomap), on the other hand, involves three main computational steps: K-nearest neighbour computation with a complexity of $\mathcal{O}\left(N^{2}D\right)$, shortest path computation using dynamic programming at $\mathcal{O}(N^{2}k+N^{2}logN)$, where k is the number of nearest neighbours, and multidimensional scaling (MDS) with a complexity of $\mathcal{O}\left(N^{2}d\right)$, where d is the dimensionality of the output space. Consequently, the overall complexity of Isomap simplifies to $\mathcal{O}\left(N^2·max(D,k,logN,d)\right)$, highlighting its quadratic dependence on N and thereby restricting its applicability to smaller datasets. In contrast, our approach maintains linear scalability, making it more suitable for large-scale datasets.

3.4.3. Advantages of the Proposed Approach

The integration of K-means and Hyperseed offers superior scalability, parallelisability and memory efficiency. Additionally, theoretical guarantees from K-means ensure convergence to a local optimum, while Hyperseed ensures stable transformation of hypervectors, especially when accelerated by neuromorphic hardware.

3.5. Incremental Hyperseed Algorithm

The proposed approach is further refined to accommodate new data samples in an online learning scenario, where dynamic updates drive clustering and representation mechanisms. This online learning process ensures that the system can adapt to new data samples by either updating existing clusters and HD-maps or creating new ones when necessary. This dynamic approach allows for continuous learning and representation of high-dimensional data while maintaining the efficiency and scalability of the system. The detailed steps of the incremental Hyperseed algorithm are provided in Algorithm 4.

For each new data sample, we perform the following steps outlined in Algorithm 1.

Algorithm 4 Incremental hyperseed algorithm

1: Input: New data sample ${\mathbf{x}}_{new}\in {\mathbb{C}}^d$, existing cluster centers $\{{\mathbf{c}}_1,{\mathbf{c}}_2,\dots ,{\mathbf{c}}_K\}$, threshold $\theta$  2: Output: Updated clusters and HD-maps  3: Step 1: Assign to Nearest Cluster  4: Assign ${\mathbf{x}}_{new}$ to the nearest cluster center ${\mathbf{c}}_k$  5: Step 2: Check Threshold Criterion  6: Compute the unbinding result $\mathbf{r}={\mathbf{x}}_{new}\oslash \mathbf{s}$, where $\mathbf{s}$ is the seed hypervector  7: Reconstruct ${\mathbf{x}}_{new}$ as $\widehat{r}$ using base vectors ${\mathbf{b}}_1,{\mathbf{b}}_2$ of the HD space  8: Compute the similarity $D({\mathbf{x}}_{new},\widehat{\mathbf{r}})$  9: if  $D({\mathbf{x}}_{new},\widehat{\mathbf{r}})\ge \theta$  then 10:    Update cluster ${\mathbf{c}}_k$ with ${\mathbf{x}}_{new}$ 11:    Update HD-map ${\mathcal{P}}_k$ using the Hyperseed algorithm 12: else 13:    Create a new cluster with ${\mathbf{x}}_{new}$ as the initial center 14:    Initialize a new HD-map ${\mathcal{P}}_{new}$ using FPE 15:    Apply Hyperseed algorithm to transform ${\mathbf{x}}_{new}$ to HD-map ${\mathcal{P}}_{new}$ 16: end if

4. Experiments

The empirical evaluation was conducted on data streams from the La Trobe Energy AI Platform (LEAP), part of La Trobe University’s Net Zero Carbon Emissions Program. Located in Victoria, Australia, La Trobe University is a multi-campus, multi-functional tertiary education institution. LEAP serves as the core AI technology stack for the Net Zero Carbon Emissions Program, which aims to reduce the university’s carbon footprint to net zero emissions by 2029, alongside improving energy efficiency and increasing resource utilisation. These experiments were conducted on the data streams of energy usage, energy generation and internal temperature sensor data collected at 15-min intervals from all buildings within the university campuses and includes post-COVID data from January 2023 to December 2023. For each experiment, we selected a single office building comprising 14 distinct zones, each equipped with temperature sensors. The temperature readings in these zones typically range between 15 °C and 30 °C. Data were collected at 15-min intervals, resulting in 96 readings per day per zone. Over the 12-month period, this setup generated approximately 490,560 temperature readings (96 readings/day × 14 zones × 365 days). The buildings are characterised by specific energy consumption patterns, where each day’s data are aggregated into a daily profile consisting of 96 readings (4 readings per hour).

4.1. Baseline Results with t-SNE

We utilised t-SNE [47] to establish baseline results for empirical comparison. The t-distributed stochastic neighbour embedding (t-SNE) algorithm is a widely accepted dimensionality reduction technique for visualising high-dimensional data. By reducing the data to two or three dimensions, t-SNE helps in understanding the structure and distribution of the data, identifying clusters and detecting patterns. Given that our approach employs a piecewise approximation technique, which differs fundamentally from traditional manifold learning methods, a direct comparison with other such methods is not directly applicable. Therefore, t-SNE serves as an appropriate baseline for evaluating the effectiveness of our proposed algorithm.

The t-SNE visualisation (Figure 5) provides insights into the overall distribution and clustering of the temperature sensor data. This visualisation also includes predefined profile vectors, which are described as follows:

• Low_Profile: a low consistent temperature profile, represented by a constant temperature of 15 °C.
• High_Profile: a high consistent temperature profile, represented by a constant temperature of 27 °C.
• Average_Profile: an average consistent temperature profile, represented by a constant temperature of 20 °C.
• Typical_8-5_Profile: a typical daily temperature profile, with lower temperatures before 8 a.m. and after 5 p.m. and slightly higher temperatures (21 °C) during the 8 a.m. to 5 p.m. working hours.

These predefined profiles serve as benchmarks for identifying typical patterns and deviations within the dataset. By comparing the data points to these reference profiles, we can more effectively assess the performance of the projection algorithms and identify anomalies or patterns that require further investigation.

4.2. Local Clusters with Hyperseed

In the first experiment, we apply the proposed algorithm to visualise local clusters within the temperature sensor data. This method is particularly effective in capturing the subtle patterns of the local clusters, which might be overlooked by traditional clustering algorithms. For K-means, we use 12 as the number of clusters to initially capture local neighbourhoods. We determined the optimal number of clusters using the elbow method, which indicated that 12 clusters provide a suitable balance between computational efficiency and cluster compactness. The Hyperseed algorithm is then applied with an epsilon value of $ϵ=0.01$ and a codebook size of 50 to refine these clusters.

The Hyperseed visualisations (Figure 6) reveal granular details about the local temperature variations within the sensor data. By providing a more detailed view, these visualisations help us understand intricate temperature patterns, allowing us to pinpoint specific locations with anomalous temperature settings and address potential issues in those regions. By following a given sensor profile over time, we can understand specific time-dependent patterns in different regions of the building. This dynamic analysis helps us track how temperature readings evolve, offering insights into temporal changes and trends. We demonstrate how different regions evolve over time between different Hyperseed HD planes. This temporal evolution visualisation helps us identify patterns such as gradual increases in temperature, periodic fluctuations or sudden anomalies that could indicate malfunctions or inefficiencies in the building’s climate control system.

The temporal evolution visualisation (Figure 7) highlights how specific regions of the building respond to various conditions over time. For example, during the first week of June, there is a noticeable deviation of almost all region profiles from their normal behavior, which could indicate an anomalous event in the building. By comparing these patterns across different Hyperseed HD planes, we can better understand the dynamics of the building’s internal environment and develop targeted strategies for improving energy efficiency and maintaining optimal temperature conditions.

4.3. Incremental Hyperseed

The second experiment evaluates the incremental extension of the algorithm using the remaining data. This involves continuously updating the model with new data from December 2023 onwards to evaluate its adaptability and performance over time.

The incremental learning results (Figure 8) demonstrate the model’s adaptability to new data. For instance, Region 2 shows convergence to a new HD plane, indicating the model’s ability to recognise and adapt to evolving data patterns, which may reflect a historically unforeseen event requiring attention for that region. Additionally, the changes consistently align with the historical patterns, emphasising the model’s capability to maintain performance and provide accurate predictions as new data become available. This validates the model’s robustness and suitability for dynamic, real-world environments. As demonstrated in the incremental learning experiment, the algorithm effectively adapts to outliers—instances that deviate from normal conditions—by creating new HD planes upon detection of anomalous data points. If these outliers do not persist over time, the corresponding HD planes are subsequently removed, thereby ensuring that the model maintains its integrity by not retaining transient or irrelevant anomalies. This dynamic adjustment underscores the robustness of our approach in handling outliers while preserving the overall stability and accuracy of the model.

5. Conclusions

This paper presents a novel AI approach for hypervector approximation of complex manifolds in AI digital twins for smart cities. Starting with the broad-based clustering capability of the K-means algorithm, the approach then leverages the Hyperseed algorithm to preserve internal structures and similarities of the non-linear manifolds, followed by incremental learning to accommodate online, continuous learning of the complex manifold. As reported in our experiments, the t-SNE algorithm provided a baseline comparator, while the proposed AI approach based on the Hyperseed algorithm offered a more detailed analysis, enabling us to detect finer details and specific anomalies within the temperature sensor data that reside on complex manifolds. The results confirm that by applying the proposed algorithm, we can uncover local variations within complex manifolds and recognise specific anomalous regions. It also demonstrates the temporal evolution of temperature profiles to detect historically unforeseen events and validates the model’s robustness through incremental learning, showcasing its adaptability to new data while maintaining performance. Additionally, the incremental learning approach demonstrated effectiveness in adapting to new data, showcasing the robustness and practicality of our methods in a dynamic, real-world environment. The integration of AI with digital twin technology offers a powerful approach to managing the complexities of urban systems. By continuously updating digital twins with real-time data, the proposed AI approach can also anticipate and respond to emerging challenges effectively. As urban areas continue to evolve, the interplay between AI, digital twins and the underlying complexities of urban systems will be crucial in shaping sustainable and resilient smart cities.