Distributed High-Speed Videogrammetry for Real-Time 3D Displacement Monitoring of Large Structure on Shaking Table

Abstract

The accurate and timely acquisition of high-frequency three-dimensional (3D) displacement responses of large structures is crucial for evaluating their condition during seismic excitation on shaking tables. This paper presents a distributed high-speed videogrammetric method designed to rapidly measure the 3D displacement of large shaking table structures at high sampling frequencies. The method uses non-coded circular targets affixed to key points on the structure and an automatic correspondence approach to efficiently estimate the extrinsic parameters of multiple cameras with large fields of view. This process eliminates the need for large calibration boards or manual visual adjustments. A distributed computation and reconstruction strategy, employing the alternating direction method of multipliers, enables the global reconstruction of time-sequenced 3D coordinates for all points of interest across multiple devices simultaneously. The accuracy and efficiency of this method were validated through comparisons with total stations, contact sensors, and conventional approaches in shaking table tests involving large structures with RCBs. Additionally, the proposed method demonstrated a speed increase of at least six times compared to the advanced commercial photogrammetric software. It could acquire 3D displacement responses of large structures at high sampling frequencies in real time without requiring a high-performance computing cluster.

1. Introduction

Innovative structures with replaceable coupling beams (RCBs) and dissipative seismic devices have recently emerged to mitigate damage and expedite repairs in buildings that need to remain operational after extreme seismic events, such as hospitals, fire stations, and government buildings [1,2]. Large shaking table tests are essential for investigating structural seismic performance and optimizing designs [3]. Contact sensors, such as displacement meters, accelerometers, and strain gauges, are frequently used in shaking table tests to record high-frequency displacement responses and other dynamic behaviors of structures in real time. These data are crucial not only for post-test seismic performance analysis but also for assessing the current state of structures to ensure the safety and success of subsequent tests [4]. However, installing numerous contact sensors is cumbersome, time-consuming, and expensive. Additionally, the reliance on wired networks for signal transmission increases the weight of the measured structures, potentially impacting their physical characteristics. Therefore, vision-based measurement methods are increasingly accepted in structural displacement measurement due to their flexibility and accuracy compared to contact sensors [5,6].

Ri et al. [7] presented a framework utilizing drone cameras for high-precision displacement measurement, achieving sub-millimeter accuracy. This method combines phase-based sampling moiré techniques with four-degrees-of-freedom geometric modeling to accurately distinguish bridge displacements from those induced by camera motion. Wen et al. [8] introduced the deep optical flow method RAFT-GOCor for extracting structural displacements and proposed Bayesian RAFT-GOCor, incorporating Monte Carlo dropout, to analyze displacement measurement uncertainty. Weng et al. [9] proposed a complementary approach that integrates phase-based optical flow with template matching, leveraging optical flow to capture high-frequency structural movement components and reducing drift error through template-matching measurements. Their results demonstrate sub-pixel accuracy for large-scale, drift-free displacement measurement and robustness against background clutter. Shi et al. [10] developed a videogrammetric method for monitoring the 3D deformation of critical internal nodes in large-scale suspended dome structures, even under camera instability. Experimental validation confirmed the method’s effectiveness, improving the positioning accuracy for internal nodes from 47.24 mm to 1.62 mm following dynamic parameter correction. Zhang et al. [11] proposed a non-overlapping dual camera measurement model with the aid of a global navigation satellite system (GNSS) to sense the 3D displacements of high-rise structures. A simulation and experiment demonstrate the feasibility and correctness of the proposed method.

However, to the best of our knowledge, there are few reports on vision techniques that can measure high-frequency 3D dynamic responses of large-scale structures in real time or near real time during seismic excitation on shaking tables. Due to the high cost and complexity of large-scale shaking tables, it is generally necessary to intermittently load diverse types and magnitudes of seismic waves onto the structure within a limited period. Long standby times for such large apparatuses pose risks, as they must operate under safe conditions [12]. Failure to promptly acquire structural displacement response data following each seismic wave and relying only on simple visual inspection renders it impossible to accurately evaluate the current state of the structure, complicating the safe and successful conduction of subsequent experiments. This issue is particularly critical for structures incorporating RCBs with dissipative seismic devices, as it is essential to study not only the seismic performance of these components under various seismic events but also their performance after a replacement following damage [13]. Evaluating the status of the structures through commonly used simple visual inspection increases the probability of experiment failure.

The efficient and accurate calibration of the camera parameters with large fields of view (FOVs) is crucial for enhancing measurement efficiency in large shaking table tests. These tests usually require stereo cameras to create multiple FOVs encompassing the entire structural multi-façade [14]. In vision-based structural health monitoring (SHM), the extrinsic calibration of cameras with large FOVs is typically achieved using artificial markers uniformly distributed on the measured structure or background. These markers can be categorized into coded and non-coded markers [15]. Coded markers are designed to ensure uniqueness in appearance, which primarily reduces the time-consuming process of target correspondence in different images [16,17]. However, using coded targets introduces a decoding process that can fail under poor image conditions, occlusions, and significant perspective variations [18]. In complex scenarios with cameras having wide baselines and limited target sizes, non-coded markers (e.g., circular targets and spheres) are preferred due to their scale invariance and robustness to image distortions, occlusions, and significant perspective variations. Zhang et al. [19] devised a stereoscopic calibration object comprising multiple high-precision sphere targets and proposed a separate parameter calibration method for cameras with large FOVs. Tong et al. [20] used non-coded circular targets to calibrate a stereo camera and measure the deformation of laminated rubber bearings on a shaking table. However, these methods typically require manual recognition and selection to establish correspondences among non-coded targets between different views and across 2D and 3D space (measured by other 3D sensors). Although non-coded circular targets are widely used in SHM [21], there remains a lack of reports about automatically establishing correspondences for numerous similar and randomly distributed non-coded circular targets. Even state-of-the-art commercial videogrammetric software, such as PhotoModeler (Version 2024), has yet to provide procedures to automate the determination of these correspondences. Excessive manual intervention can substantially reduce computational efficiency, particularly during the calibration of multiple cameras.

Besides camera calibration, target tracking and time sequence 3D coordinate reconstruction are crucial for enhancing measurement efficiency. Many high-speed tracking methods have been proposed to meet specific measurement requirements [22]. A rapid sub-pixel tracking method based on normalized cross-correlation (NCC) is used in this paper to extract time sequence 2D coordinates of dynamic structure from image sequences [23]. Separated bundle adjustment (BA) involves conducting 3D reconstruction for interest points in partial areas of the structure (the FOV of a pair of cameras in a multi-camera system) and then combining the 3D measurements of the entire structure through physical stitching [24,25]. However, these methods overlook the motion consistency of the entire structure and the potential improvement in measurement accuracy from points in the overlapping regions between each stereo FOV [26]. Global BA is more accurate and suitable, as it can integrally reconstruct the time sequence 3D coordinates of all interest points on the structure [27]. However, conventional global BA generally only supports optimization on a single computing device, posing challenges to the performance of the computing device due to the massive parameters requiring simultaneous optimization [28]. Some reports presented high-performance computing (HPC) clusters for the real-time calculation of structural dynamic parameters [29]. However, HPC clusters are extremely expensive to purchase and maintain, which can significantly increase experimental costs. Majchrowicz et al. [30] proposed a new parallel approach for 3D ECT image reconstruction, which is based on the application of multi-GPU, multi-node algorithms in a heterogeneous distributed system. The application of the framework with a new network communication layer reduced data transfer times significantly and improved the overall system efficiency. Xu et al. [31] presented an innovative image-based 3D reconstruction pipeline for the precise and efficient geometry measurement of bridge structures. By decomposing the large-scale reconstruction task into distributed sub-models using sub-image sets, the method significantly improves computational efficiency without compromising accuracy compared to conventional approaches. These studies illustrate that distributed systems and computational strategies can effectively boost efficiency without requiring HPC resources. However, the currently proposed distributed strategies or systems are unsuitable for real-time shaking table test processing.

This paper presents a distributed high-speed videogrammetric method with rapid calibration to efficiently and accurately measure the dynamic response of a large shaking table structure. A seven-story structure with RCBs equipped with hybrid devices was secured onto a shaking table, employing a videogrammetric network constructed by six high-speed cameras to monitor structural 3D displacement responses in real time. The primary contributions of this paper are as follows:

(1)

An efficient dynamic measurement method for large shaking table structures based on distributed high-speed videogrammetry was developed, which can obtain the high-frequency 3D displacement responses of all the interest points of large structures in real time without necessitating an HPC cluster. The efficiency and accuracy of the proposed method were verified through a shaking table test of a large structure with RBCs.

(2)

A fast calibration method for multiple cameras with large FOVs was proposed, which automatically and accurately estimates extrinsic parameters using non-coded circular targets fixed on the structural interest points. This method eliminates the need for artificial visual assistance to determine the target correspondence across different camera views and 2D-3D space.

(3)

A distributed computation and reconstruction strategy based on the Alternating Direction Method of Multipliers (ADMM) was proposed to fully exploit the computing resources of the conventional high-speed videogrammetric network. This strategy circumvents the time-consuming transmission of large-volume image data and achieves global reconstruction across different computing devices without compromising measurement precision.

2. Methodology

Figure 1 illustrates the framework of the proposed method for the real-time, high-frequency 3D displacement measurement of a large-scale structure on a shaking table. The method consists of four key components: (1) the establishment of a distributed videogrammetric network that integrates a synchronized multi-camera system, high-speed camera workstations, and a distributed measurement network; (2) the rapid calibration of the multi-camera system with large fields of view (FOVs) through non-coded circular targets affixed to key structural points; (3) the high-speed tracking of these circular targets using an efficient NCC method; and (4) the implementation of a distributed computation and reconstruction strategy, including a distributed computation strategy and the ADMM-based distributed reconstruction method.

2.1. Construction of Distributed Videogrammetric Network

Multiple high-speed cameras are essential for the 3D dynamic measurement of large-scale shaking table structures with high sampling frequencies. Since high-speed cameras can capture a significant amount of image data quickly, they typically require the following equipment to ensure proper functionality [32,33]: (1) a high-speed image capture card to prevent image loss; (2) a large-volume data storage device for lossless image storage; and (3) a multicore CPU for the real-time operation and control of the camera. Therefore, a corresponding minicomputer (workstation) with sufficient computational power was necessary to integrate these functions.

In pursuit of efficient 3D dynamic measurement for large-scale shaking table structures, this study established a general distributed videogrammetric network, illustrated in Figure 2. The network comprises the following components: (1) Multiple camera system: multiple high-speed cameras are dispersed throughout the network, each equipped with a minicomputer (workstation). (2) Synchronous control module: each camera is connected to the synchronous controller via a CamLink cable, ensuring synchronized image capture. (3) Principal workstation: this workstation oversees the synchronous control module and the multi-camera network, managing camera parameter adjustments, as well as data acquisition, transmission, and storage. (4) Distributed calculation module: this connects the primary and subordinate workstations with network cables and switches, facilitating signal and data transmission. OpenMPI v4.1 was employed for inter-device communication.

2.2. Fast Calibration of Multi-Camera System

2.2.1. Stereo Correspondence of Circular Targets

Since points of interest on the shaking table structure often lack distinctive texture, artificial markers are typically affixed to key points of the structure to enhance the accuracy of dynamic measurements [34]. These markers can serve both as tracking points for capturing the dynamic behavior of the structure and as control points for determining the position and orientation of the cameras.

The non-coded targets used in this study were simple white circular markers on a black background, as shown in Figure 3. A reflective sheet was affixed to the center of each marker to assist the auxiliary measuring equipment in acquiring the 3D coordinates of the target’s center. In practical testing, various complex environments hinder the precise identification of circular targets. For example, low-light indoor environments reduce the contrast of circular marks, overexposure in outdoor scenes obscures certain target outlines, and numerous circular-like interferences increase the likelihood of false detections. This paper employed CMNet to accurately recognize and locate the centers of circular targets [35]. Specifically, circular marks were first detected using an improved YOLOv4 model to restrict the search region of the circular contour. Next, the BASNet saliency object detection model extracted the contours of these circular marks. Finally, least squares fitting was applied to calculate the central pixel coordinates of the identified contours on the saliency map. The method was trained on a large dataset of circular mark images captured in diverse environments, achieving a high recognition rate and detection accuracy within a relatively short processing time.

To automatically establish stereo correspondences for numerous similar circular targets across different camera views without manual intervention, we proposed a stereo-matching method for circular targets based on affine invariance and epipolar constraints, as illustrated in Figure 3. In the illumination, $s_1$ and $s_2$ denote the optical center of a camera pair; $p_1$ and $p_2$ signify the center points of the detected circular targets from different views; $l_1$ and $l_2$ represent the polar lines; $e_1$ and $e_2$ denote the epipoles, while $l_1$ and $l_2$ are the epipolar lines; and red points represent the matched feature points of different images.

An affine map offers a local approximation of the apparent deformations induced by changes in camera viewpoints. This capability proves valuable in addressing the matching challenges posed by cameras with a wide baseline.

Any affine map $A$ is uniquely decomposed as Equation (1).

$$A=\lambda R_1(\psi )T_t{R}_2(\varphi )=\lambda \left[\begin{array}{cc}\cos \psi & -\sin \psi \\ \sin \psi & \cos \psi \end{array}\right]\left[\begin{array}{cc}t& 0\\ 0& 1\end{array}\right]\left[\begin{array}{cc}\cos \varphi & -\sin \varphi \\ \sin \varphi & \cos \varphi \end{array}\right]$$

(1)

where $\lambda >0,t>0$, $\psi \in [0,2\pi ],\varphi \in [0,\pi ]$, $R_1$, and $R_2$ represent rotations; $T_t$ represents a translation matrix that shifts the position in one direction; $\psi$ parameterizes the camera spin; $\lambda$ corresponds to the camera zoom; and the longitude $\varphi$ and latitude $\theta =\arccos (1/t)$ parameterize the camera’s tilt [36].

The conventional method implements affine matching for different views by simulating images with a range of optical tilts and employing brute force matching. To reduce the processing time, a near-optimal discrete set of affine transformations strategy was used, minimizing the number of simulation angles for the camera axis.

From Equation (1), if $T_{t(A)}{R}_{\varphi (A)}=T_{t(B)}{R}_{\varphi (B)}$, two classes [A] and [B] in the space of tilts are equal. Thus, the space of tilts can be parametrized by picking representative affine maps for each class.

$$\Omega =[Id]\cup \left\{\underset{(t,\varphi )\in ]1,\infty [\times [0,\pi [}{\cup }[T_t{R}_{\varphi }]\right\}$$

(2)

$I$ in $[Id]$ represents the identity matrix, and $d$ is a metric acting on the space of tilts that measures the affine distortion from a fixed affine viewpoint to surrounding affine viewpoints.

$$d:\left\{\begin{array}{ccc}\begin{array}{l}\Omega \times \Omega \\ ([A],[B])\end{array}& \begin{array}{l}\to \\ \mapsto \end{array}& \begin{array}{l}{ℝ}_{+}\\ \log (t(B{A}^{-1}))\end{array}\end{array}\right\}$$

(3)

The root scale-invariant feature transform (Root-SIFT) was selected as the descriptor for matching image pairs due to its improved robustness to viewpoint changes compared to the SIFT [37]. This improved process involved taking the square root of a SIFT descriptor after normalization. A state-of-the-art estimator was then utilized to recover the fundamental matrix from these matched feature points [38]. Furthermore, the epipolar constraint was utilized to narrow the search area for the circular target from different views, and affine template matching was applied to establish the correspondence of circular targets along the polar line, further reducing mismatches [39].

2.2.2. Cross-Modal Correspondence of Circular Targets

Auxiliary devices, such as the total station, were employed to measure the 3D coordinates of the targets fixed on the key points of the structure. These data serve as control and check points to ensure consistency between videogrammetric and structural measurement coordinate systems. However, a cross-modal discrepancy arose between the 3D coordinates measured by the total station and the 2D coordinates of the targets. The conventional method for establishing this correspondence relies on manual visual selection, as exemplified by commercial photogrammetry software like PhotoModeler 2024. However, numerous correspondences are required for large-scale structure measurement when utilizing multiple cameras. The manual correspondence procedure may increase the likelihood of obtaining erroneous correspondences and is time-consuming.

In this study, 2D and 3D correspondences of circular targets were implemented in 3D space through multi-view geometry. The specific process was as follows: With the assumption that $X_i\leftrightarrow {X_i}^{\prime }$ was a set of corresponding circular targets between different images and the quantity was over 5, the fundamental matrix $F$ was estimated using these matched circular targets [40]. The intrinsic parameter matrix $K$ was obtained by using Zhang’s calibration method in advance. The distance aspect ratio between the reconstructed space and real 3D space was consistent.

The essential matrix can be calculated as

$$E=K^{\prime T}FK$$

(4)

The $3\times 3$ essential matrix has two equal singular values, and a singular value of zero. Suppose the first camera normalization matrix is $P=[I|0]$, and the singular value decomposition (SVD) of the essential matrix is $U\mathrm{diag}(1,1,0)V^T$. The second camera normalization matrix $P^{\prime }$ has four possible choices:

$$\left\{\begin{array}{c}{P}^{\prime }=\left[UW{V}^T\mid +u_3\right]\\ P^{\prime }=\left[UW{V}^T\mid -u_3\right]\\ P^{\prime }=\left[U{W}^T{V}^T\mid +u_3\right]\\ P^{\prime }=\left[U{W}^T{V}^T\mid -u_3\right]\end{array}\right.$$

(5)

where the translation vector $t=\pm u_3=U{(0,0,1)}^T$, the rotations matrices $R=U{W}^T{V}^T \mathrm{or} UW{V}^T$, and

$$W=\left[\begin{array}{ccc}0& -1& 0\\ 1& 0& 0\\ 0& 0& 1\end{array}\right]$$

(6)

To determine the correct solution for $P^{\prime }$ among the four possibilities, one can derive a single world point using these four potential extrinsic matrices and choose the reconstructed world point located in front of both cameras. Then, the center points of circular targets can be metrically reconstructed using the selected camera parameters $P^{\prime }$.

Then, we can achieve cross-modal correspondence by establishing the correspondence between the metrically reconstructed circular target and the circular target obtained by the measurement sensor (e.g., total station) in 3D space ($P_i^2\leftrightarrow P_i^1$). Since the circular targets in both point sets have a consistent quantity and scale, we can identify four pairs of potential corresponding points by selecting the maximum and minimum spatial distances between points in each set. The two 3D point sets follow a rigid transformation:

$$P_i^2=R{P}_i^1+T=T_{3\times 4}{P}_i^1$$

(7)

where $T_{3\times 4}$ represents the combined transformation matrix (a 3 × 4 matrix), which includes both the rotation and translation, to map $P_i^1$ directly to $P_i^2$.

Thus, we were able to obtain four potential aligning rigid transformations. The 4PCS algorithm could then be used to determine the optimal rigid transformation, thereby achieving registration between the two 3D point sets [41]. Once the correspondences between the circular targets in 2D and 3D were established, the PnP algorithm was able to be used to obtain the external parameters of the cameras in the world coordinate system.

2.3. Sub-Pixel Tracking of Circular Targets

Normalized cross-correlation is a robust measure for assessing the similarity between corresponding object areas in image sequences. Since the shaking table test is conducted indoors, the structure benefits from stable and consistent lighting provided by professional indoor equipment. Additionally, the non-coded targets remain unobstructed during movement, allowing for clear imaging throughout the sequence. Thus, this study employed a fast NCC method for target tracking [23]. Using pre-calculated sum tables for the numerator and denominator in the NCC definition eliminated redundancy in repeated calculations. Since NCC only provides integer pixel accuracy, a sub-pixel localization algorithm was used to enhance the tracking precision. This algorithm involves extracting a 3 × 3 region around the matched point to fit a 2D quadratic surface, whose extremum is considered the sub-pixel position $c_1$ of the circular target [42]. Although using an area larger than 3 × 3 could result in a more accurate estimation, it also increases the computational complexity and time. The equation for the quadratic surface is shown in Equation (8), and the cross-correlation coefficient matrix $M$ is described in Equation (9).

$$f(x,y)=a_0{x}^2+a_1{y}^2+a_{2}xy+a_{3}x+a_{4}y+a_5$$

(8)

$$M=\left[\begin{array}{ccc}{m}_1& m_2& m_3\\ m_4& m_5& m_6\\ m_7& m_8& m_9\end{array}\right]$$

(9)

The coefficients $a_i(i=0,1,\cdots ,5)$ can be estimated by substituting Equation (9) into Equation (8), and $c_1$ can be calculated using the following equations:

$$\left\{\begin{array}{l}x={\displaystyle \frac{2a_1{a}_3-a_2{a}_4}{{a_2}^2-4a_0{a}_1}}\\ y={\displaystyle \frac{2a_0{a}_4-a_3{a}_2}{{a_2}^2-4a_0{a}_1}}\end{array}\right.$$

(10)

2.4. Distributed Computation and Reconstruction

2.4.1. Distributed Computation Strategy

The videogrammetry system recorded dynamic information of structures in image sequences, which were stored at individual workstations. Global reconstruction of the 3D displacement response of the entire structure required centralizing images from different workstations. Traditionally, this involves transmitting large images to a single workstation, hindering the real-time or near-real-time 3D dynamic response measurement due to the extensive transmission time. Ngeljaratan et al. [43] explored image compression techniques, such as non-adaptive linear interpolation and wavelet transform algorithms, to facilitate image storage and data transfer. Although this method reduces the data volume while preserving structural motion information, it requires additional image pre-processing time.

The proposed distributed computation and reconstruction strategy, illustrated in Figure 4, harnesses the computational capabilities of each subordinate workstation within the videogrammetric network to parallelly track circular targets. In that case, only the transmission of text data is required, and the data transmission time is minimal. Massive parameters are globally optimized across various computing devices for simultaneous solving using the ADMM. This computation strategy eliminates the need for extensive image data transmission within the local network, facilitating the rapid measurement of the structural 3D dynamic response.

2.4.2. Distributed 3D Reconstruction Based on ADMM

After acquiring the initial intrinsic and extrinsic parameters of multiple cameras and the sequential image coordinates of all structural points of interest, the BA is commonly used to obtain accurate time sequence 3D coordinates. Bundle adjustment minimizes the reprojection error between the 2D locations of observed and predicted image points, where the predicted points are reprojected from 3D points using camera parameters [44]. The BA problem can be described by Equation (11).

$$f(Cam,X)={\displaystyle \sum _{j=1}^n{\displaystyle \sum _{i=1}^m{\left(u_{ij}-\pi \left(Ca{m}_i,X_j\right)\right)}^2}}$$

(11)

where $u_{ij}$ is the 2D observation point, which represents the 3D point, $X_j$, which is observed by the $i\mathrm{th}$ image $Ca{m}_i$; $\pi \left(Ca{m}_i,X_j\right)$ is the nonlinear operation, which represents the reprojection pixel coordinates obtained by the 3D point $X_j$ and estimated image parameters $Ca{m}_i$; and $Ca{m}_i$ includes extrinsic and intrinsic parameters.

As shown in Equation (11), the conventional global BA only supports computation and optimizing all parameters integrally. It poses challenges to computational efficiency within a computing device due to the simultaneous optimization of a substantial number of parameters if the high-performance computation device is lacking.

Thus, considering multiple computer devices in the multiple-camera measurement network and the consensus characteristic of the ADMM, the global BA was divided into multiple sub-blocks using the ADMM to achieve a distributed computation [45]. The general form of the consensus problem is Equation (12) if the ADMM algorithm is used to solve it.

$$\min {\displaystyle \sum _{i=1}^{k}f(x_i)}{ \mathrm{subject} \mathrm{to} \mathrm{x}}_i=z,i=1,\cdots ,k$$

(12)

The parameter space is divided into $k$ blocks. Each sub-objective function’s parameter $f_i(x_i)$ dimensions are different, which are called local variables. Mapping a part of $x_i$ to a part of the global variable $z$ as $z_i$, the iteration expression using the ADMM can be derived as

$$x_i^{t+1}=\underset{x_i}{\arg \min }(f_i(x_i^t)+{\displaystyle \frac{\rho }{2}}{‖x_i^t-(z^t-y_i^t)‖}_2^2)$$

(13)

$$z^{t+1}={\displaystyle \frac{1}{k}}{\displaystyle \sum _{i=1}^k{x}_i^{t+1}}$$

(14)

$$y_i^{t+1}=y_i^t+x_i^{t+1}-z^{t+1}$$

(15)

The penalty parameter $\rho >0$ and Equation (14) are the average of the optimization results of different blocks.

The global BA problem is separated into multiple blocks, as shown in Figure 4 (left side). Supposing the sub-block number is $K$, the camera parameters $Ca{m}_i^K$ and the corresponding 3D points $X_j^K$ are estimated in each block, and Equation (12) can be written as

$$\min {\displaystyle \sum _{k=1}^{K}f}(Ca{m}^k,X^k) \mathrm{subject} \mathrm{to} \begin{array}{c}Ca{m}_i^k=Ca{m}_i,i=1,\cdots ,n,k=1,\cdots ,K\\ X_j^k=X_j,j=1,\cdots ,m,k=1,\cdots ,K\end{array}$$

(16)

Leveraging the ADMM algorithm, the iterations are given by

$${(Ca{m}^k,X^k)}^{t+1}=\underset{\left\{Ca{m}^k\right\},\left\{X^k\right\}}{\arg \min }(f(Ca{m}^{k(t)},X^{k(t)})+{\displaystyle \frac{\rho }{2}}‖X^{k(t)}-(X^t-{\tilde{X}}^{k(t)}‖,\forall k$$

(17)

$$X_j^{(t+1)}={\displaystyle \frac{1}{K}}{\displaystyle \sum _{k=1}^K{(X_j^k)}^{t+1}},\forall j$$

(18)

$${({\tilde{X}}_j^k)}^{t+1}={\tilde{X}}_j^{k(t)}+(X_j^{k(t+1)}-X_j^{(t+1)}),\forall j,k,$$

(19)

where ${\tilde{X}}_j^k$ drives the iterative process of Equation (17). The Schur complement was used to increase the computation efficiency. This approach simplified the block diagonal matrix, allowing for the subsequent optimization of the 3D points. During the optimization process, the Huber loss function was applied. The initial values of the 3D points were able to be calculated using the initial camera parameters and the triangulation, allowing Equation (16) to be easily distributed across multiple processors and solved using the Gauss–Newton method. In this study, the number of sub-blocks equaled the number of CPU cores, and the termination tolerance for the reduction in the reprojection error between iterations was set to 1 × 10⁻⁵.

Based on the reconstructed high-frequency time sequence 3D coordinates of tracking targets, the dynamic response and properties of the structural interest area were able to be easily acquired through numerical analysis and calculations [46]. These can be converted into physical response indicators for safety assessments and seismic performance research.

3. Experimental Results and Analysis

A large shaking table test on a structure with RCBs was conducted to verify the accuracy and efficiency of the proposed method. First, details regarding the structural model, ground motion, and measurement setup are presented. Next, the accuracy and efficiency of the proposed method are verified and analyzed by a comparison with various approaches, including total station measurements, contact sensors, and conventional methods. Finally, the 3D dynamic response measurements under different seismic wave loads are presented and analyzed.

3.1. Structure Model and Experiment Setup

3.1.1. Structure Model and Ground Motion

The seven-story structural model with RCBs is depicted in Figure 5a and has dimensions of 1.8 m in length, 1.3 m in width, and 6.3 m in height. Details of the used shaking table are shown in

Table 1. Simulated earthquake vibration table.

Performance	Experiment No.
Maximum Specimen Mass	25 tons
Table Size	4 m × 4 m
Vibration Direction	X, Y, Z axes
Degrees of Freedom	Six degrees of freedom
Frequency Range	0.1–100 Hz

.

Figure 5a also details the non-replaceable parts of the RCB, which connect to the hybrid device through end-plate bolted connections, facilitating easy removal and replacement after major earthquakes. Three pairs of ground motion records and six experiments were selected from the shaking table test, as shown in

Table 2. Selected six ground motion experiments on the shaking table.

Seismic Waves	Experiment No.	PGA (g)
		X Direction	Y Directions
SHW2	1	1.02	0
	2	0	1.02
El Centro	3	1.02	0.867
	4	0.867	1.02
Northridge	5	1.02	0.867
	6	0.867	1.02

. These records include: (1) Shanghai—an artificial seismic record from the appendix of the Shanghai Code for Seismic Design of Buildings 57; (2) El Centro—the 1940 Imperial Valley earthquake recorded at the El Centro station in California; and (3) Northridge—the 1994 Northridge earthquake recorded at the Sylmar Olive View FF station in Sylmar, California. The vertical component of the ground motion records was neglected.

3.1.2. Measurement Setup

Based on the field test environment, a video measurement network was established using six high-speed cameras to ensure that the stereo FOV covered both elevations of the building while minimizing the number of cameras required. The critical parameters of the camera are shown in

Table 3. Critical parameters of the high-speed camera.

Parameter	Configuration
Sensor resolution	1280 × 1024 pixels
Capture frame rate	300 fps
Image sensor	LUPA1300 Global Shutter CMOS
Pixel size	14 μm
Lens	20 mm

. Figure 5b illustrates the camera layout and the corresponding measurement areas. Specifically, Cameras A and B, Cameras B and C, Cameras D and E, and Cameras E and F establish four stereoscopic FOVs, each responsible for the measurement areas [A-B], [B-C], [D-E], and [E-F], respectively.

The videogrammetric network comprised six workstations (standard computers): five subordinate workstations, each equipped with quad-core 3.6 GHz CPUs and 16 GB RAM, and one primary workstation equipped with a six-core 3.2 GHz CPU, 16 GB RAM, and a GeForce GTX 1650. CPU and GPU algorithms were implemented in C++ and CUDA, with inter-workstation communications handled via OpenMPI v4.1.

Non-coded circular targets were affixed to the structural points of interest, as illustrated in Figure 5c. Blue $B_i$ numerals indicate measurement points on the structural coupling beams, while red numerals $R_i$ denote measurement points on the shear wall. A displacement meter in the X direction was installed at point $R_3$, and an accelerometer in the Y direction was installed at point $R_{18}$. The local spatial coordinate system was established using high-precision total stations, aligning the XY directions with the shaking table load orientation and completing the right-hand coordinate system with the Z direction. The 3D coordinates of all measurement points were determined using the total station before and after each shaking table load when the structure remained stationary.

3.2. Accuracy Verification and Analyzation

3.2.1. Accuracy of Calibration

Twenty-six circular targets were selected as checkpoints to assess the extrinsic calibration method’s accuracy. The extrinsic parameters of the six cameras were estimated using the remaining circular targets and the proposed method, with global BA employed to compute the 3D coordinates of the checkpoints. Figure 6 shows the measurement errors of checkpoints between the videogrammetry and the total station in the X, Y, and Z directions. The root mean square error (RMSE) of the 26 checkpoints in the X, Y, and Z directions was 0.58 mm, 0.60 mm, and 0.57 mm, respectively. The 3D positioning error of the checkpoint was calculated by taking the result of the RMSE values in all three directions, which equaled 1.02 mm. These results demonstrate the accuracy of the proposed calibration method.

3.2.2. Accuracy of Distributed Computation and Reconstruction

The sub-pixel tracking method was employed to track all circular targets during the six ground motions. The time sequence 3D coordinates of all points of interest were reconstructed using separated BA (BA performed separately for each measurement area), global BA (BA integrated for all measurement areas on one computing device), and the proposed method (BA integrated for all measurement areas across different computing devices). Since the total station measured the 3D coordinates of the checkpoints after each seismic wave load when the structure was stationary, the 3D positioning errors of the checkpoints of different methods after each seismic wave load were calculated and are depicted in Figure 7.

The overall positioning error of the checkpoint of separated BA, centralized global BA, and the proposed method was determined by averaging the positioning errors after each seismic wave load, resulting in values of 1.67 mm (X direction: 0.66 mm; Y direction: 0.83 mm; Z direction: 0.74 mm), 1.04 mm (X direction: 0.54 mm; Y direction: 0.63 mm; Z direction: 0.59 mm), and 1.01 mm (X direction: 0.57 mm; Y direction: 0.64 mm; Z direction: 0.52 mm), respectively. The separated BA has a relatively high overall positioning error due to the method neglecting the entire structure’s motion consistency and the potential measurement accuracy improvement from control points in the overlapping regions between each stereo FOV. The overall positioning error of the proposed method is comparable to those of conventional global BA, demonstrating its effectiveness and precision in time sequence 3D reconstruction.

3.2.3. Accuracy of Displacement and Acceleration Responses

Figure 8 compares the 3D displacement and acceleration response histories obtained from the contact sensor and videogrammetry under different seismic excitations. The RMSEs of displacement measurements in the X direction between the videogrammetry and the displacement sensor during experiments 1, 3, and 5 were 0.72 mm, 0.95 mm, and 1.22 mm, respectively. The range of values and movement trends for the acceleration response curves acquired by the accelerometer closely align with those from videogrammetric measurements. These results further demonstrate the efficacy of the proposed videogrammetric method for measuring the 3D dynamic response of the structure.

3.3. Efficiency Verification and Analyzation

3.3.1. Efficiency of Fast Calibration

The initial extrinsic parameters of a camera can be calculated using at least four 3D points in the world and their corresponding 2D projections in the image (2D-3D correspondences). Ideally, the more control points are available within the camera FOV, the higher the calibration precision will be. For the system with six cameras, at least 24 2D-3D correspondences are necessary to achieve extrinsic parameter calibration. Additionally, obtaining 3D measurements of the 52 structural interest points requires establishing image correspondences from at least two camera views for each point. Thus, we included the time consumption for this procedure in the extrinsic calibration step. In the conventional extrinsic calibration method (PhotoModoler 2024), these correspondences are manually acquired. We recruited five testers, recorded the time each took to determine these correspondences for the four camera pairs, and calculated the average time.

Table 4. Runtime of different calibration methods to calibrate the multi-camera system.

Extrinsic Calibration	Runtime (s)
	2D to 3D Correspondence	Stereo Correspondence
Proposed	1.2	1.6
Conventional	30.7	55.2

presents the time consumption of the conventional and proposed extrinsic calibration methods to calibrate the multi-camera system. The runtime for circular target detection and PnP is not counted, as they only take milliseconds. The fast calibration method is approximately sixteen times faster than the conventional method, with the reduction in time becoming increasingly significant as the number of cameras increases. In actual shaking table tests, the camera needs to be recalibrated after each ground motion to avoid possible slight changes in the camera’s extrinsic parameters due to the effects of vibration from a large shaking table on the surrounding environment.

3.3.2. Efficiency of Distributed Computation and Reconstruction

In addition to camera parameter estimation, image transmission, target tracking, and trajectory reconstruction are time-consuming steps in actual videogrammetric measurement tasks. In the proposed computation strategy, the time consumption of data transmission in the local network can be ignored. The runtime of target tracking is inversely proportional to the number of workstations in the primary replica system. The shaking table dataset, comprising 1,747,200 world points and 4,166,400 observations, was obtained from the six ground motion experiments to evaluate the efficiency of time sequence 3D reconstruction. This dataset underwent reconstruction using the sparse global BA (serial solution) [47], centralized global BA (parallel solution) [28], and the proposed distributed method, respectively. Each workstation utilized only CPUs, and the communication time was considered.

As depicted in Figure 9, the time consumed for sparse global BA was 1339.8 s, for centralized global BA was 268.1 s, and for distributed BA was 76.4 s. The distributed BA was approximately three times faster than the centralized BA and about sixteen times faster than the sparse BA. Post-optimization, the mean reprojection errors of the different methods were around 0.04 pixels, indicating that the three methods achieved convergence and acquired accurate optimization results in this case. These results demonstrate the effectiveness and efficiency of the proposed distributed computation and reconstruction strategy.

Additionally, we varied the number of workstations to investigate the relationship between the number of CPU cores and the time consumed for dataset reconstruction using the proposed method.

Table 5. Time consumption, mean reprojection error, and number of iterations for shaking table dataset reconstruction using different numbers of workstations.

Workstation Number	CPU Cores	Time (s)	Mean Reprojection Error (Pixels)	Iterations
1	6	272.7	0.036	6
2	10	198.1	0.036	7
3	14	154.6	0.035	7
4	18	101.3	0.037	8
5	22	81.9	0.038	8
6	26	76.4	0.040	8

illustrates that the time consumption for reconstructing the dataset decreases with increased CPU cores while the mean reprojection error remains stable. Although the increase in computational efficiency is not significant when a certain number of CPU cores is reached, the distributed reconstruction strategy fully utilizes the computational power of the different devices of the videogrammetric network. This significantly improves the onsite computation efficiency for the 3D dynamic measurement of large shaking table structures.

3.3.3. Efficiency of Videogrammetric Method

To further demonstrate the efficiency of the proposed videogrammetry method, we compared the time consumed for each procedure in reconstructing the same shaking table dataset with that of PhotoModeler, an advanced commercial videogrammetry software. PhotoModeler supports 3D reconstruction for time sequence interest points within only a single stereo field of view. For a fair comparison, PhotoModeler was installed on each workstation, and four professionals simultaneously reconstructed the time sequence 3D coordinates of the interest points in each measurement area. Since the target tracking procedure (least squares matching) in PhotoModeler is time-consuming, we assumed the runtime for this procedure to be the same as the method employed in this paper. Additionally, only the necessary runtime was included in the time consumed by PhotoModeler. The overall 3D positioning error for the checkpoints measured by PhotoModeler compared with the total station was approximately 1.61 mm, which is lower than the value obtained using the proposed videogrammetry method.

Figure 10 compares the time consumption between the proposed videogrammetric method and PhotoModeler in reconstructing the shaking table dataset. The intrinsic parameters of the industrial camera are pre-determined and therefore not included in the total time consumption. PhotoModeler took approximately 1128.1 s, while the proposed videogrammetric method took only 161.1 s. This result includes all time-consuming processes except for the preparatory work during camera calibration, which can be ignored as it can be performed before the shaking table work. The duration of the six seismic excitations was approximately 112 s. These results demonstrate the potential for the real-time 3D displacement response measurement of large-scale shaking table structures at 300 Hz, corresponding to the camera capture speed and system sampling frequency. Furthermore, by reducing the sequence image frame rate from 300 to 200 fps through frame extraction, the overall consumed time was reduced to 102.2 s, indicating that the method achieves high-frequency real-time measurement.

The time statistics for the proposed method were from an experiment conducted in a laboratory environment where computing resources were idle. In our laboratory setup, we used Cat8 Ethernet cables to support a maximum transmission rate of 40 Gbps, which meets the real-time processing requirements for this study. To enhance robustness in real-world applications, we recommend using optical fiber cables to establish wired connections across distributed devices whenever possible. If hardware upgrades alone cannot meet data and signal exchange demands, adopting edge computing solutions to decentralize processing tasks may reduce network dependency and improve robustness.

During field measurements, other programs (e.g., data acquisition and control) may use part of the computational resources of the distributed computation network, potentially preventing the real-time measurement frequency of the 3D displacement response from reaching 200 Hz. However, the above analysis and discussion demonstrate the effectiveness and efficiency of the proposed method. The method provides a non-contact, high-frequency, real-time measurement tool for shaking table tests. The real-time dynamic monitoring of large shaking table structures can be achieved using standard computing equipment and configuration strategies without the need for high-performance clusters.

3.4. Measurement Results

After evaluating the performance of the proposed method, we calculated the 3D dynamic response of all measurement points under different seismic wave loading using the proposed method. Figure 11 displays the 3D displacement response histories of measurement points distributed along the coupling beams in experiments 1, 3, and 5. Since the input peak ground acceleration (PGA) is greater in the X direction than in the Y direction, the displacement response of measurement points in the X direction exceeds that in the Y direction, which is consistent with structural vibration principles. The maximum displacements under different wave loads are as follows: for SHW2, the X direction: −91.2 mm at position $B_{19}$; the Y direction: −18.3 mm at position $B_{19}$; and the Z direction: 5.6 mm at position $B_{22}$; for El Centro, the X direction: 70.6 mm at position $B_1$; the Y direction: 65.5 mm at position $B_{19}$; and the Z direction: 4.6 mm at position $B_{21}$; and finally, for the Northridge wave load, the X direction: −49.3 mm at position $B_{19}$; the Y direction: −44.4 mm at position $B_{13}$; and the Z direction: 4.1 mm at position $B_3$.

The highest displacements in the X and Y directions are observed at measurement points $B_1$, $B_{13}$, and $B_{19}$, which are located on the top story of the structure. In addition, the structural vibration amplitude gradually decreases as the height of the measuring point decreases, aligning with the physical characteristics of high-story structures. These findings further demonstrate the accuracy of the 3D dynamic response measured using the proposed videogrammetric method.

The earthquake-damaged structure with replaceable components can partially restore its original integrity by substituting the damaged parts. Analyzing the amplitude of the deformation response and permanent deformation from previous experiments can effectively determine the current state of these components. Suppose the 3D dynamic response data of the structural area of interest from prior experiments are not promptly obtained. In that case, it becomes challenging to assess whether the hybrid device needs replacement or if the PGA values for the next seismic event need to be increased. This highlights the efficient advantage in 3D dynamic measurements facilitated by the proposed videogrammetric method.

3.5. Discussion

The proposed distributed high-speed videogrammetric method successfully achieved the real-time 3D displacement measurement of replaceable coupling beam structures on a large shaking table, validating its effectiveness and accuracy for dynamic monitoring applications. The study demonstrated significant improvements in the calibration speed without compromising accuracy, underscoring the utility of non-coded circular targets for rapid extrinsic parameter estimation in large field-of-view setups. Additionally, the adoption of a distributed computation strategy based on the ADMM enabled efficient resource utilization within a local network, reducing dependence on high-performance computing clusters.

Despite these advantages, certain limitations remain. Although this method reduces reliance on high-performance computing clusters, it still requires multiple computing devices and network communication support. The high-frequency data acquisition required for 3D dynamic response reconstruction necessitates substantial data storage capacity and transmission bandwidth, particularly for real-world applications. Future research could investigate adaptive algorithms to manage fluctuating network conditions or optimize data transmission protocols to reduce latency. Additionally, while the proposed method enables the high-frequency, dynamic 3D displacement measurement of key points on large shaking table structures, it does not currently support full-field real-time displacement measurement. Expanding the system’s scalability to accommodate more complex structural configurations or environments may be a focus of future work. This could involve integrating hybrid systems that utilize additional data sources, such as satellite imagery or ground-based sensors, to provide a more comprehensive monitoring solution.

As demand for the real-time, high-frequency 3D displacement measurement of large-scale civil structures continues to grow, the successful application of this videogrammetric method in large replaceable coupling beam shaking table tests highlights its potential as a practical and real-time displacement monitoring solution for large-scale civil structures in real-world scenarios.

4. Conclusions

To overcome the limitations of conventional multi-vision measurement methods, which fail to promptly acquire high-frequency 3D dynamic responses of large-scale structures onsite, we propose a distributed high-speed videogrammetry method with rapid calibration. This method achieved high-frequency 3D displacement response monitoring for large shaking table structures using a general system of multiple high-speed cameras to evaluate structure health under seismic excitations rapidly. The efficiency and accuracy of the proposed method were verified through large shaking table tests on a seven-story structure with RCBs. The main conclusions drawn from the experimental results are as follows:

(1)

The 3D displacement responses of all points of interest on a large structure with RCBs (1.8 m in length, 1.3 m in width, and 6.3 m in height) were measured using the proposed videogrammetric method in real time. The RMSE of the proposed method in the X, Y, and Z directions was 0.57 mm, 0.64 mm, and 0.52 mm, respectively, compared to the measurements of high-precision total stations. The RMSE of displacement responses in a single direction was approximately 0.96 mm compared to the contact displacement sensor, and the acceleration response range and trend were consistent with accelerometer measurements.

(2)

The proposed automatic correspondence method for the non-coded circular targets facilitates the extrinsic calibration of multiple cameras with FOVs. This method provides an alternative for calibrating multi-camera systems with large FOVs in SHM instead of using coded targets. In a scene with four large stereo FOVs comprising six cameras, the calibration time was reduced from 85.9 s to 2.8 s compared to the conventional calibration method.

(3)

The distributed computation and reconstruction strategy fully exploits the computing resources of different devices within the videogrammetric network. In laboratory conditions (computing resources idle), the method achieves the real-time high-frequency (200 Hz) 3D displacement response measurement of all points of interest on the large shaking table structure using only standard computing equipment and configurations without requiring a costly HPC cluster.

This paper provides a practical and efficient vision-based 3D monitoring solution for large shaking table structures, significantly facilitating the study of the seismic performance of shaking table structures. Future research will focus on full-field deformation measurements for large structures with texture features at high sampling frequencies.