Reconstructing Geometric Models from the Point Clouds of Buildings Based on a Transformer Network

Abstract

Geometric building models are essential in BIM technology. The reconstruction results using current methods are usually represented using mesh, which is limited to visualization purposes and hard to directly import into BIM or modeling software for further application. In this paper, we propose a building model reconstruction method based on a transformer network (DeepBuilding). Instead of reconstructing the polyhedron model of buildings, we strive to recover the CAD modeling operation of constructing the building models from the building point cloud. By representing the building model with its modeling sequence, the reconstruction results can be imported into BIM software for further application. We first translate the procedure of constructing a building model into a command sequence that can be vectorized and processed by the transformer network. Then, we propose a transformer-based network that can convert input point clouds into the vectorized representation of the modeling sequences by decoding the geometry information encoded in the point features. A tool is developed to convert the vectorized modeling sequence into a 3D shape representation (such as mesh) or file format that other BIM software supports. Comprehensive experiments are conducted, and the evaluation results demonstrate that our method can produce competitive reconstruction results with high geometric fidelity while preserving more details of the building reconstruction.

1. Introduction

Building Information Modeling (BIM) technology represents a comprehensive virtual model that encapsulates all the relevant information of a building asset [1]. The use of three-dimensional geometric models enables the integration of various types of building data into a unified, parametric representation [2], facilitating efficient and seamless interactions within a Common Data Environment (CDE). In parallel, point clouds derived from advanced sensing technologies, such as 3D laser scanning and photogrammetry, are capable of capturing the surface geometries of target objects with high levels of accuracy and efficiency [3,4]. Consequently, point clouds have become an essential tool for reconstructing the geometric models of existing buildings [5], aiding in the creation of their digital twins [6]. However, current building reconstruction methods typically generate mesh models, which present challenges when it comes to importing them directly into BIM platforms like Revit [7]. As a result, there is a notable gap in the development of an automated solution capable of converting point clouds into editable geometric models that are easily integrated into BIM software for subsequent tasks.

Computer-aided design (CAD) is a cornerstone of modern industries, and is widely used in mechanical engineering, aerospace, architecture, and many other disciplines. CAD models are widely used as representations of parameterized designs, particularly in the field of civil engineering. During the engineering design phase, building design typically follows a top-down approach, where 3D CAD models of buildings are first created and subsequently utilized in various downstream tasks [8], including further design modifications and construction activities. In practice, building models are usually constructed through a sequence of modeling commands, such as drafting, sketching, extrusion, and Boolean operations. In contrast, establishing the Building Information Model (BIM) of an existing building can be viewed as the reverse process of the design workflow. If the original modeling commands are recoverable, both the geometric models and BIM data can be reconstructed. Additionally, these modeling commands can be reformatted into file formats compatible with BIM or modeling software, providing significant benefits for downstream applications like finite element analysis [9] and daily management [10]. Motivated by this concept, this paper explores the recovery of modeling commands used to construct a building’s geometry from input point clouds. These modeling commands can be regarded as a design language, akin to the language tokens in natural language processing [11], which encapsulate the operations and commands employed to generate the geometric model. In this study, we refer to this collection of commands as the modeling sequence. By recovering this modeling sequence, the corresponding building geometry can be recreated, and the generated sequence can be easily imported into BIM software for further use.

Recent advancements in Artificial Intelligence have led to significant breakthroughs, particularly in the field of Large Language Models (LLMs). Models such as BERT [11] and ChatGPT [12] have demonstrated exceptional capabilities in reasoning and multi-modal predictions [13]. The modeling sequence of civil structures can be interpreted as a design language, which can similarly be predicted using transformer-based language models. Moreover, the process of recovering the modeling sequence can be viewed as extracting the implicit geometric features encoded within the point cloud data. To address this, we propose an end-to-end network based on a transformer network designed to convert input point clouds into corresponding modeling sequences. By leveraging the transformer’s powerful ability of feature representation learning, our approach decodes the implicit geometric information embedded in the point cloud and transforms it into a modeling sequence.

In this paper, we propose an approach for reconstructing building models by recovering the modeling sequence from point cloud data. By retrieving the modeling sequence, we can achieve high geometric fidelity in building reconstruction, with results that can be seamlessly imported into BIM software for further applications. Our approach begins with the representation of building models as a vectorized modeling sequence, utilizing the widely adopted sketch–extrude modeling process. We then introduce an end-to-end network, based on the transformer architecture (DeepBuilding), which is capable of converting point clouds into the corresponding vectorized modeling sequence.

The main contributions of this paper can be summarized as follows:

• We propose a method for building model reconstruction by recovering the modeling sequence from input point clouds. In this approach, the building model is represented by its vectorized modeling sequence rather than a mesh. By representing the geometry through modeling commands, the reconstruction results can be directly imported into BIM software for further applications. Additionally, we develop a tool to convert the modeling sequence into file formats compatible with BIM software.
• We introduce an end-to-end network based on the transformer architecture that converts point clouds into vectorized modeling sequences. This network employs PointNet++ as the point tokenizer to extract point embeddings, after which a transformer network decodes the extracted features into corresponding command sequences.
• We conduct a comprehensive evaluation of the proposed building reconstruction method. The results show that our approach preserves more geometric details in the reconstruction while achieving competitive performance compared to existing methods.

2. Related Works

The reconstruction of building models has garnered significant attention due to its potential to integrate various types of data within Building Information Modeling (BIM) [14]. A wide range of approaches for urban building reconstruction has been proposed, including rule-based methods [15], image-based techniques [16], and 3D point cloud-based methods [17]. This paper specifically focuses on the reconstruction of geometric building models using point clouds. Furthermore, we propose a novel reconstruction method that predicts the modeling sequence directly from the input point cloud. This modeling sequence is formatted in accordance with CAD standards, ensuring seamless integration with BIM software such as Revit. Thus, both building model reconstruction techniques and CAD-based methods in reverse engineering are thoroughly discussed in the following section.

2.1. Building Model Reconstruction

Building model reconstruction methods based on point clouds can be primarily classified into two categories: data-driven and model-driven approaches [18]. Data-driven reconstruction begins by segmenting the point cloud using various data features. The predicted labeled point sets are then modeled individually, and different primitives are combined to obtain the final 3D building reconstruction. This approach includes several mainstream techniques for 3D building reconstruction from point clouds, such as region growing methods [19,20], feature clustering methods [21,22,23,24], model fitting methods [25,26,27,28], and global energy optimization methods [29].

Model-driven reconstruction, in contrast, is a top-down approach where the primitive library is first established, containing geometrical and topological information about the primitives. Optimization methods, like least square matching, are then used to fit the point cloud to these predefined primitives. Zhang and Weng [30] divided irregular building contours into non-overlapping quadrilaterals using 2D building contour data, with each corresponding roof primitive being identified via decision trees. The adjacent simple roof primitives were subsequently combined to form complex roof models. Similarly, Xiong et al. [31] developed roof topology graphs to identify the corresponding primitives for each roof, and through topological operations, they were able to generate complete building models. Nan et al. [32] proposed the PolyFit framework for reconstructing lightweight polygonal surfaces from point clouds by intersecting plane primitives and combining them to generate a manifold polygonal surface model without boundaries. Huang [33] introduced a fully automatic approach (City3D) for reconstructing compact 3D building models from large-scale airborne point clouds, directly inferring vertical walls from the data. Model-driven methods benefit from prior assumptions that ensure the correctness of topological relationships and aid in semantic information extraction. However, challenges persist, such as mismatches between the point cloud and primitive types, low model fitting accuracy, and the need for careful parameter tuning.

Additionally, some researchers have integrated deep learning algorithms into building model reconstruction. Huang et al. [34] designed 11 roof primitives and reconstructed complex roofs by merging them in the influence space. Li et al. [35] and Zhang et al. [36] proposed a model-driven parametric modeling method that uses the PointNet++ network model for high-precision classification of roof point cloud data. Chen et al. [37] proposed a framework called Points2poly for reconstructing building models from point clouds. This framework leverages a deep neural network for building occupancy estimation and uses a Markov random field to extract the outer surface of the building through combinatorial optimization. Chen et al. [38] also introduced PolyGNN, a polyhedron-based graph neural network for 3D building reconstruction, which learns to assemble primitives obtained by polyhedral decomposition via graph node classification to achieve a watertight and compact reconstruction.

While most point cloud-based 3D building reconstruction methods focus on geometric dimensions such as model accuracy and abstraction levels, they often lack semantic descriptions of the models. The output representations for these reconstructions are typically parametric models (e.g., model fitting or matching) and surface modeling (e.g., boundary representation, B-rep) [39]. The results are usually expressed using geometric representations like polygonal models (mesh), which are limited to visualization purposes. This restricts the ability to query, edit, and analyze building models and their components, thereby hindering deeper applications [40]. If the reconstruction results are programmatic or parametric, users can modify complex objects by altering their positions or performing logical operations. However, CSG-based representations for 3D building reconstruction are rarely employed, and no method currently uses programmatic CSG models to represent reconstruction results.

In this paper, we propose a novel method for building model reconstruction based on an end-to-end transformer network that converts point clouds into modeling sequences. By representing the geometric model using modeling commands, the reconstruction results can be easily imported into BIM software for further applications.

2.2. CAD Model Reconstruction

CAD reverse engineering has long been a sought-after goal in the CAD industry due to the significant time and resource savings it offers [41]. Fitting geometric primitives to 3D point clouds through algorithms such as RANSAC and the Hough transform has dominated this field for years [27,42,43,44]. However, these methods require meticulous parameter tuning for each shape type and often rely on user assistance.

With the recent availability of large boundary representation (B-Rep) datasets, a data-driven framework has emerged for reverse engineering in the CAD industry. SPFN [45] was the first end-to-end supervised primitive fitting network. PARS-ENET [46] expanded the range of primitives the network can handle, improving surface representation fidelity. CPFN [47] further enhanced SPFN’s performance on high-resolution point clouds by dynamically aggregating primitives across global and local scales. Despite these advances, these methods primarily segment primitive patches and overlook the edges that define the boundaries of the primitives, leading to inaccurate and incomplete reconstructions.

Beyond fitting primitives, methods such as ExtrudeNet represent sketches using signed distance fields (SDFs). Inspired by DeepCAD [48], Point2Cyl reconstructs 3D shapes from point clouds through extrusion segmentation and encodes the sketch using the latent embeddings of its SDF. Similarly, [49] advocates for the use of implicit fields for sketch representation and proposes SECAD-Net, which predicts 2D sketches and 3D extrusion parameters from raw shapes. However, these methods rely on implicit sketch representations, which introduce curved edges in the inferred sketches. Transforming from the implicit field to the sketch can lead to parameter errors, further complicating the process.

Some CAD generation methods [50,51] utilize generative models, such as diffusion models, to generate modeling sequences. CAD-Diffuser [50] is a generative model based on diffusion, but it is not designed for reconstruction purposes. While the diffusion model is inherently generative, our goal is to facilitate building reconstruction through parametric CAD modeling sequences for further application. By recovering the modeling sequence from input point clouds, the reconstruction results can be used in mechanical simulations or BIM reconstruction, moving beyond mere visualization.

To address this, we propose a reconstruction-based approach using a transformer network, similar to DETR [52] in image detection and PoinTr [53] in point cloud completion. Our method directly predicts or infers the modeling sequence, emphasizing editable and reusable building model reconstruction. The proposed geometric model reconstruction method explicitly represents the 3D shape of the point cloud through its CAD modeling sequence. The DeepBuilding network can directly produce vectors of the sketch command sequences and extrusion operations, which can then be converted and imported into BIM software to reconstruct editable 3D solids with sharp edges.

3. The Modeling Sequence of Buildings

Our goal is to generate the CAD modeling sequence of building models from input point clouds. To represent the modeling process of buildings, we introduce a command formation method, as illustrated in Figure 1. This modeling sequence is then vectorized to enable processing by the neural network. We leverage the widely used 3D modeling pattern, specifically the sketch–extrude modeling procedure, as the basis for the modeling sequence. This approach is capable of covering most modeling procedures, as demonstrated in the Fusion 360 Gallery Dataset [54]. In the following section, we will provide a detailed description of the sequence hierarchy and the definition of each command within the sequence.

3.1. Sequence Hierarchy

Figure 1 demonstrates a simple example of constructing a civil building in BIM or CAD software (such as AutoCAD 25.0.071.0) and the formulation of the modeling sequence that records the whole modeling procedure. It can be seen that the building modeling procedure can be seen as a repetitive sketch–extrude process. In this paper, only the sketch–extrude process is considered. More specifically, a sketch $S_i$ is extruded by an extrusion operation $E_i$ to form an extrusion solid. Multiple extrusions are grouped by Boolean operation and constitute the final geometrical model of the buildings. Thus, the modeling sequence M of a building can be formulated as $M=[\langle SOL\rangle ,S_1,E_1,S_2,E_2,...,S_i,E_i,\langle EOL\rangle ]$. The modeling sequence M can then be seen as a type of building modeling language and can be processed and reasoned using a language model.

3.2. Command Specification

The definition of each modeling command is shown in

Table 1. Modeling sequence definition. $\langle SOL\rangle$ indicates the start of a loop; $\langle EOS\rangle$ indicates the end of the whole extrusion sequence.

Command	Parameters
$\langle SOL\rangle$	∅
L (Line)	x, y: line endpoint
A (Arc)	x, y: arc endpoint
	$x_m$, $y_m$: arc midpoint
R (Circle)	x, y: circle center
	r: radius
E (Extrude)	$\theta ,\varphi ,\gamma$: sketch plane orientation
	$o_x,o_y,o_z$: sketch plane origin
	$e_1,e_2$: extrusion distances toward both sides
	b: Boolean type
	s: sketch scale
$\langle EOL\rangle$	∅

. The command sequence mainly consists of two types of modeling commands, drafting a sketch and extrusion operations, respectively.

Sketch. The sketch hierarchy consists of curves and loops. In this paper, the curve includes the three most widely used commands (draw a line, draw an arc, and draw a circle). A closed path can be defined by grouping one or multiple curves, namely a loop. A sketch profile is then formed by some outer loops as boundaries and inner loops as holes. A loop always starts with an indicator command, $\langle SOL\rangle$, and the whole modeling sequence ends with an $\langle EOL\rangle$ command as the indicator. Furthermore, if a curve is defined by its two endpoints, an extra matching procedure is needed to align its endpoint to the start point of the next curves, leading to accuracy cost. Therefore, we omit the starting points of each curve and leave the endpoint of the last curve as the starting point of each loop. The line is represented by its endpoint. We represent an arc using its start point, midpoint, and circle, as well as its center and radius. This simplification helps to uniformize the parameters. Therefore, a sketch $S_i$ is defined as $S_i=[t_i,x,y,x_m,y_m,r]$, where $t_i$ encodes the command type information.

Extrude. The extrusion operation consists of four types of parameters, including the sketch plane orientation, sketch plane origin, extrusion distances towards both sides, and Boolean operation. The sketch plane defines where the sketch is drafted, and the extrusion distance indicates how far the sketch is extruded. Each extrusion is grouped according to the type of Boolean operation. Thus, the extrusion operation can be defined as $E=[\theta ,\varphi ,\gamma ,o_x,o_y,o_z,e_1,e_2,b,s]$.

To sum up, for each command $m_i$, its parameters are stacked into a $1\times 15$ vector, which is $para{m}_i=[x,y,x_m,y_m,r,\theta ,\varphi ,\gamma ,o_x,o_y,o_z,e_1,e_2,b,s]$, the elements of which correspond to the collective parameters shown in Table 1. Unused parameters are padded with $-1$. Therefore, each command $C_i=(t_i,para{m}_i)=(t_i,x,y,x_m,y_m,r,\theta ,\varphi ,\gamma ,o_x,o_y,o_z,e_1,e_2,b,s)$ is specified by its command type $t_i$ and command parameters $para{m}_i$. In conclusion, we represent a geometrical model using a fixed total number $N_c$ of modeling commands, namely $M=[\langle SOL\rangle ,M_{E_1},M_{E_2},...,M_{E_i},\langle EOL\rangle ]$. In this paper, we choose $N_c=120$, a model’s maximal command sequence length in the training dataset.

4. Materials and Methods

4.1. Overview

We propose a building geometry model reconstruction method from point clouds, as shown in Figure 2. The point cloud of the buildings is first tokenized into point embeddings by the point tokenizer. The point embeddings are refined by the transformer encoder to extract fine-grained point features. The point features are then decoded into the vectorized modeling sequence $M=[\langle SOL\rangle ,M_{E_1},M_{E_2},...,M_{E_i},\langle EOL\rangle ]$ by the transformer decoder. A tool is developed based on the PythonOCC library [55] to convert the command vectors M into 3D representation, such as mesh, and the file formats that other BIM software support, such as ‘.step’ and. ‘JSON’. By importing the modeling sequence into this software, the final geometrical models can be further applied in simulation and daily management.

By recovering the modeling sequence from the building point cloud, the reconstruction results can be seamlessly imported into BIM software such as REVIT or CAD. To achieve this, we propose an end-to-end network based on the transformer architecture, namely DeepBuilding, as illustrated in Figure 3. The input point cloud is first tokenized into point embeddings using a point tokenizer. These point embeddings are then processed by the transformer encoder. By leveraging the long-range dependency modeling capabilities of the attention layer, the point features are refined. Following this, a transformer decoder decodes the refined point features into sequence features. Finally, two separate prediction heads—namely the type and parameter prediction heads—are employed to produce the final modeling sequence prediction.

To find the geometry information encoded in the point cloud of the buildings or infrastructure, we first develop a point tokenizer based on the PointNet++ used by Qi et al. [56] to extract point embeddings that the network can learn and further decode into modeling sequences, as shown in Figure 4a. After the repetitive sampling–grouping–PointNet operation, the final point embeddings are extracted. The point embeddings can be seen as a sequence of key point features which encode the latent geometry information contained in the input point cloud. The point embeddings are then input into the transformed network for modeling sequence prediction.

4.2. Transformer Encoder for Implicit Space Conversion

The procedure of decoding point embeddings into a modeling sequence can be viewed as a set-to-set conversion problem, which involves transforming the latent embeddings from the latent space of point features to the latent space of the modeling sequence. To achieve this transformation, a bridge is required. We leverage the long-range modeling capabilities of the transformer [57] and utilize the transformer encoder as this bridge to connect the two latent spaces. The model architecture of the transformer encoder is shown in Figure 4a. The transformer encoder consists of stacked self-attention layers, as depicted in Figure 4c. The embeddings are generated by the previously mentioned point tokenizer. This process can be seen as establishing deep connections between the point features $f_{pi}\in {\mathbb{R}}^{1\times C}$ through the repeated application of the self-attention layers. By leveraging the efficient long-range dependency modeling capabilities of the attention mechanism, the latent space of the point features is effectively connected to the latent space of the modeling sequence.

4.3. Transformer Decoder

The transformer decoder decodes the point features refined by the transformer encoder into a modeling sequence. As shown in Figure 4b, the transformer decoder has a symmetric architecture to the transformer encoder, except for the additional self-attention layers for the learnable queries. N layers of the self-attention and cross-attention layers constitute the transformer decoder. The point features $f_p\in {\mathbb{R}}^{N\times C}$ refined by the transformer encoder are input into the cross-attention layer, where a learnable query is used to query the corresponding information from the point features. We feed the positional queries (generated by the key points sampled by the FPS) and content queries (point features) into the decoder to probe the commands that correspond to the position encodings and have similar patterns to the content queries. Output from the last transformer block is fed into a linear layer to predict a modeling command sequence $M=[\langle SOL\rangle ,S_1,E_1,S_2,E_2,...,S_i,E_i,\langle EOL\rangle ]$, including both command type ${\widehat{t}}_i$ and command parameters ${\widehat{p}}_i$. Two separate Multi-Layer Perceptions (MLPs) are used to output the command type and command parameters, namely the type and parameter head. The prediction of our model can be factorized as

$$p\left(\widehat{M}\right|\widehat{f_p},\Theta )=\prod _{i=1}^{N_c}p({\widehat{t}}_i,{\widehat{p}}_i|\widehat{f_p},\Theta ),$$

(1)

where $\Theta$ denotes the network parameter of the transformer decoder, and $\widehat{f_p}$ denotes the point features refined by the transformer encoder. i denotes the i-th modeling command and $N_c$ denotes the number of predicted modeling commands.

4.4. Loss Function

The command parameter prediction is supervised using the L-1 loss, while the command type prediction is supervised using the focal loss [58]. The loss $\mathcal{L}$ for the training process can be formulated as

$$\mathcal{L}=\sum _{i=1}^{N_c}{l}_{focal}({\widehat{t}}_i,t_i)+\beta \sum _{i=1}^{N_c}\sum _{j=1}^{N_p}{l}_1({\widehat{p}}_{i,j},p_{i,j})$$

(2)

where $l_{focal}(·,·)$ represents the focal loss and $l_1(·,·)$ represents the L-1 loss, respectively. $N_P$ is the number of parameters and $N_c$ is the number of commands. $\beta$ is the weight to balance both terms ($\beta =2$ in our examples).

4.5. Importing into BIM Software

As for our method, the input of the model is the point cloud, and the output is a matrix $M=[\langle SOL\rangle ,S_1,E_1,\langle SOL\rangle ,S_2,E_2,...,\langle SOL\rangle ,S_i,E_i,\langle EOL\rangle ]$ that defines the modeling sequence. The matrix can be easily converted into ‘.step’ or ‘.JSON’ files using tools developed based on the PythonOCC [55] library. Each row of the output tensor $M\left[i\right]$ represents one modeling command. The first element $M\left[i\right]\left[0\right]$ of each row vector indicates the command type, and the other elements $M\left[i\right][1:]$ are the parameters. After parsing the output matrix into a sequence of the modeling commands $[S_i,E_i]$, a tool developed based on the PythonOCC library is used to construct the CAD shape from the modeling sequence. Then, the function ‘write_step_file()’ of OCC data API can be used to convert the reconstructed CAD shape into ‘.step’ files. The pseudocode of converting the command sequence into other file formats is presented in Appendix A. Therefore, the final output of our proposed reconstruction method can be imported into CAD software. Also, we provide a simple example of the output JSON file format in Appendix B to display the format of the exported file.

5. Results

We evaluate the proposed building reconstruction method on the widely used public modeling sequence ABC [59] Dataset and the collected building Dataset.

5.1. Experiment Setup

The proposed transformer-based network, DeepBuilding, requires a large amount of data for training from scratch. Given the limited availability of building modeling sequence datasets, we utilized the ABC dataset [59] to pre-train the network. The ABC dataset contains one million modeling sequence samples collected from CAD software, which provides an ample dataset for pre-training. The data in the ABC dataset includes modeling operations commonly used by humans to create and design 3D models. In total, we collected more than ten thousand training samples. Additionally, we created a smaller building dataset containing 1000 samples based on the OnShape API [60] to fine-tune the model. We report the performance of our method on these two datasets to evaluate the effectiveness of our proposed reconstruction method.

We implemented our network using the easily available $Pytorch$ library [61], which is prominent in this community. Our network was constructed using $Python$ 3.10 and $Pytorch$ 1.13.1. The transformer encoder and decoder contain six and eight transformer layers, respectively, with eight attention heads and latent model dimension 256. Moreover, we trained our model on a Linux server with two Nvidia V100 GPUs of 32 GB memory. The model weights pre-trained on the ABC database were used to initialize the model parameters during fine-tuning. The proposed network was trained for 400 epochs with a total batch size of 64 and a learning rate (lr) of $1\times {10}^{-5}$ with a linear warmup. $AdamW$ [62] with a $weightdecay$ of $1\times {10}^{-5}$ was used to optimize our model for back propagation. The learning rate strategy starts with a 10-epoch linear warmup from $1\times {10}^{-7}$ and is continued by an $ExponentialLR$ strategy with an initial learning rate of $1\times {10}^{-5}$ and a decay factor of $0.9\times (1/50)$. The learning rate is adjusted by the strategy at each epoch.

5.2. Evaluation Metrics

Following Vitruvion [63], we used the command type and parameter accuracy as the metric for evaluating the performance of the proposed reconstruction method. For each modeling command, we calculated the command type accuracy using

$$AC{C}_{type}={\displaystyle \frac{1}{N_c}}\sum _{i=1}^{N_c}\Psi [C_i={\widehat{C}}_i]$$

(3)

where $N_c$ denotes the total number of the CAD commands; $C_i$ and ${\widehat{C}}_i$ are the ground truth command type and predicted command type, respectively. $\Psi \left[i\right]\in \{0,1\}$ is the indicator function. The command parameter accuracy is calculated by

$$AC{C}_{param}={\displaystyle \frac{1}{K}}\sum _{i=1}^{N_c}\sum _{j=1}^{\left|{\widehat{p}}_i\right|}\Phi [|p_{i,j}-{\widehat{p}}_{i,j}|<ϵ]\Psi [C_i={\widehat{C}}_i]$$

(4)

where $K={\sum }_{i=1}^{N_c}\Psi [C_i={\widehat{C}}_i]\left|p_i\right|$ is the total number of parameters in all correctly recovered commands. $p_{i,j}$ and ${\widehat{p}}_{i,j}$ are ground truth command parameters and predicted command parameters. $ϵ$ is the tolerance threshold accounting for the parameter accuracy. In practice, we used $ϵ=0.01$.

After reconstructing the modeling command sequences of the input point clouds, the 3D geometry can be reconstructed using our developed tools based on PythonOCC [55]. To measure the quality of the recovered 3D geometry, we used chamfer distance (CD) and edge chamfer distance (ECD) following previous models [64]. We evaluated CD by uniformly sampling 2000 points on the surfaces of the reconstructed geometry shape. Furthermore, following [64], we also report chamfer distance ($CD$), edge chamfer distance ($ECD$), normal consistency ($NC$), and the number of generated primitives ($\Delta \#P$) to measure the quality of the recovered 3D geometry. Also, following [38], we evaluated the building reconstruction using the Hausdorff distance.

5.3. Results on Public Modeling Sequence Dataset

The proposed network was first pre-trained on the public ABC dataset [59], so we first report the quantitative and visualized results on this dataset to evaluate the effectiveness of our proposed reconstruction method.

Figure 5 demonstrates the validation loss curve during pre-training. It can be seen from Figure 5a that the type errors stably decrease along the training process and reach a minimum value of $4.1\%$, which indicates that the network is fully converged and the type prediction accuracy is $95.9\%$. Furthermore, Figure 5b demonstrates the parameter loss during validation. The graph shows that the L-1 parameter prediction errors for all command types (line, arc, and circle) decrease stably during the training process. The final parameter errors are below 0.039 for the line, 0.0009 for the circle, and 0.0127 for the arc. From both the type and parameter validation loss curves, we can see that the proposed network performs well when predicting modeling command type and parameter.

To demonstrate the effectiveness of our method, we compare the proposed model with several existing approaches. These include generation-based methods such as DeepCAD [48] and HNC-CAD [51], as well as methods that represent sketches using SDF encoding, including SECAD-Net [49], Point2Cyl [65], and ExtrudeNet [66].

Table 2. Evaluation results on the public ABC dataset.

Methods	${\mathit{Acc}}_{\mathit{t}}$	${\mathit{Acc}}_{\mathit{p}}$	$\mathit{CD}$	$\mathit{ECD}$	$\mathit{NC}$	$\#\mathbf{\Delta }\mathit{P}$
Point2Cyl [65]	31.98%	27.94%	0.518	1.065	0.791	27.96
ExtrudeNet [66]	24.46%	21.83%	0.614	1.117	0.776	36.14
SECAD-Net [49]	33.18%	28.81%	0.437	1.079	0.806	34.18
DeepCAD [48]	79.49%	70.19%	0.898	1.883	0.708	7.16
HNC-CAD [51]	81.27%	73.41%	0.827	1.064	0.711	6.56
Our Method	93.27%	83.97%	0.410	0.607	0.819	4.97

presents the quantitative comparison results. From the command type and parameter accuracy, we observe that our method achieves the best performance in modeling sequence prediction. The type and parameter accuracies are 93.27% and 83.97%, respectively. Additionally, our method achieves a lower chamfer distance (CD) of 0.41 and an edge chamfer distance (ECD) of 0.607, indicating that it produces reconstruction results with higher geometric fidelity. The number of primitives $\#\Delta P$ reflects the closeness of the reconstruction to human design. Our method achieves the lowest value of 4.97, suggesting that the generated modeling sequence is closer to human design. Given that our goal is to recover the modeling operations used to build the 3D model, these results validate the effectiveness of our proposed method.

For a more comprehensive comparison, we visualized the reconstruction results from all the methods, as shown in Figure 6. From Figure 6, it is evident that the geometric models produced by our method are the most complete and accurate. The edges are sharper than those produced by the other methods, and the reconstructed models fit the input point cloud perfectly. These model samples represent typical operations used in real-world design or modeling practices in industry. Therefore, these results demonstrate that our proposed method can effectively recover the modeling operations used to design real-world objects.

5.4. Results on Building Dataset

After being trained on the public ABC dataset, the proposed network was fine-tuned on a building dataset to empower the proposed network to reconstruct building models from the input point. Several widely used building reconstruction methods were selected for comparison, including PolyFit [32], City3D [33], Point2Poly [37], and PolyGNN [38]. Point2Poly [37] and PolyGNN [38] are learning-based methods, while PolyFit [32] and City3D [33] are non-learning methods.

Table 3. Evaluation results on the building dataset.

Methods	Hausdorff Distance	Chamfer Distance
City3D [33]	0.238	0.719
PolyFit [32]	0.191	0.675
Point2Poly [37]	0.160	0.516
PolyGNN [38]	0.107	0.218
Ours	0.096	0.213

demonstrates the quantitative results of the building dataset. From the results, we can see that our method achieves comparable results to PolyGNN [38]. The traditional optimization-based approach, City3D, is designed for airborne point clouds with more priors, so the results are in less detail, and the merits are relatively higher than the others.

Figure 7 showcases some reconstruction examples of the comparison experiment. All the methods exhibit compact reconstructions. Compared to the traditional optimization-based approach, City3D [33], our method demonstrates the capability to handle more complex buildings commonly found in urban scenes. City3D [33] adopts exhaustive partitioning, resulting in inferior reconstruction accuracy as measured by the Hausdorff distance. In contrast, our approach directly recovers the modeling sequence, resulting in a more detailed reconstruction.

Figure 8 demonstrates the reconstruction results from real-world point clouds. As for the small cities, we first normalized each building point set and stored the transformation used for the normalization (shift and scale). Then, the point sets of each building were converted into a modeling sequence using our proposed method. By applying the stored transformation for normalization to these reconstructions, the small cities were rebuilt. From the results, we can see that the geometrical models by all the methods fit the point cloud. There are some failures in the results of PolyFit [32] and City3D [33], while the completeness of other methods is better. The results demonstrate that our method can produce satisfying reconstruction results.

The experimental results prove the effectiveness of the proposed method. By recovering the modeling sequence from the point cloud, accurate geometrical models can be reconstructed. The quantitative performance is competitive with the SOTA reconstruction method. Also, Figure 9 demonstrates some examples of the close look at the visualized reconstruction results. It can be seen that more details of the building reconstruction are preserved by our method because the recovered modeling sequence describes the whole design process of the buildings. Also, since the modeling sequence can be easily converted into file formats (such as ‘.JSON’ and ‘.step’), the reconstruction results can be imported into BIM software such as AutoCAD or Revit (the converting process is detailed in Appendix B).

6. Discussion

6.1. Robustness Analysis

We randomly dropped the input points to analyze the robustness of our method against variations in point cloud density.

Table 4. Quantitative results of the robustness analysis.

Dropping Rate	Hausdorff Distance	Chamfer Distance
0%	0.107	0.218
5%	0.153	0.273
10%	0.201	0.389
15%	0.307	0.417
20%	0.607	0.961
25%	0.607	0.961

demonstrates the quantitative results of the robustness analysis. When uniformly dropping 25% of the points, the overall shape can still be maintained. The method fails to produce reconstruction results when the dropping rate is bigger than 25%, so the results after 25% dropping rate are omitted.

Figure 10 demonstrates some examples of building reconstruction under different dropping rates. It can be seen that most of the main structure can be obtained when the dropping rate is less than 10%. Details of the building reconstruction will be lost if the dropping rate is larger than 15%.

6.2. Limitations

We acknowledge that there are some limitations to our proposed method. The modeling sequence utilized in our approach currently supports only the sketch–extrude modeling procedure. While this sketch–extrude process is capable of describing most of the models, especially those commonly used in BIM, it may not cover all possible 3D shapes. Additionally, the sketch command in our method currently only supports basic curves (such as lines, arcs, and circles), which may restrict the expressive capacity of the building reconstruction. Models that cannot be constructed using the sketch-and-extrude commands, such as roofs with conical shapes, cannot be reconstructed using the proposed method. Furthermore, our approach requires a relatively complete point cloud input for accurate reconstruction. Incorporating point cloud completion techniques can address this limitation.

7. Conclusions

We propose a method for reconstructing a building model by recovering the modeling sequence from the input point cloud. By representing the building model through its modeling sequence, the reconstruction results can be directly imported into BIM software for further applications. We introduce a transformer-based network that converts the point cloud into a vectorized modeling sequence. Additionally, a tool based on PythonOCC has been developed to transform the modeling sequence into a 3D shape representation (e.g., mesh) and file formats supported by BIM software. The experimental results demonstrate that the proposed building reconstruction method achieves competitive performance compared to state-of-the-art methods while preserving more geometric details.