Achieving 3D Beamforming by Non-Synchronous Microphone Array Measurements

Abstract

Beamforming technology is an essential method in acoustic imaging or reconstruction, which has been widely used in sound source localization and noise reduction. The beamforming algorithm can be described as all microphones in a plane simultaneously recording the source signal. The source position is then localized by maximizing the result of the beamformer. Evidence has shown that the accuracy of the sound source localization in a 2D plane can be improved by the non-synchronous measurements of moving the microphone array. In this paper, non-synchronous measurements are applied to 3D beamforming, in which the measurement array envelops the 3D sound source space to improve the resolution of the 3D space. The entire radiated object is covered better by a virtualized large or high-density microphone array, and the range of beamforming frequency is also expanded. The 3D imaging results are achieved in different ways: the conventional beamforming with a planar array, the non-synchronous measurements with orthogonal moving arrays, and the non-synchronous measurements with non-orthogonal moving arrays. The imaging results of the non-synchronous measurements are compared with the synchronous measurements and analyzed in detail. The number of microphones required for measurement is reduced compared with the synchronous measurement. The non-synchronous measurements with non-orthogonal moving arrays also have a good resolution in 3D source localization. The proposed approach is validated with a simulation and experiment.

1. Introduction

Sound source localization has high demand and exceptional value in applications such as automobiles [1], submarines [2], aircrafts [3,4], etc. Several methods have been proposed, such as beamforming and inverse methods [5,6], in which beamforming has evolved into one of the most important methods of sound source localization. The basic idea of beamforming is that by weighting each array element’s outputs, the expected maximum output power of the signal is directed to the position of the sound source. Different representations of the basis function also derive other classical methods, such as (1) near-field acoustical holography (NAH) [7,8,9]; (2) the inverse boundary element method (IBEM) [10,11,12]; (3) the equivalent source methods (ESM) [13,14,15]; (4) statistically optimal near-field acoustical holography (SONAH) [16,17]; and (5) the Helmholtz equation—least-squares method (HELS) [18]. The beamformer has a good resolution when the plane of the beamformer is parallel to the microphone plane. The spatial resolution of the planar beamformer deteriorated sharply when the plane is perpendicular to the microphone array. Most beamforming research is conducted on 2-dimentional (2D) because its model is typical and easy to explain the principle, which has been extensively studied in recent decades. Although most 2D beamforming algorithms can be directly extended to the 3-dimentional (3D) space domain for acoustic imaging [19,20,21], this simple extension method will cause problems, such as having a high computational complexity and difficulty in 2D parameter matching. On the other hand, 2D beamforming generally assumes that the distance between the source plane and the measurement plane is known. In contrast, 3D beamforming assumes that the source is distributed in a 3D space, and the distribution of the entire 3D sound source space is obtained by iterative scanning of the measured distance. In most studies, the implementation of 3D beamforming still uses planar array measurements, which leads to low spatial resolution problems. The 3D beamforming research is indispensable, achieving a high spatial resolution in both the lateral and normal directions of the planar array. The 3D model is more practical because the ability of great 3D sound source localization [22,23,24] is required in many industrial and national defense fields, especially in the aeroacoustics.

The research into 3D sound localization with a planar microphone array has been fully developed in recent years. A deconvolution approach of 3D DAMAS was developed to locate the distribution of gear noise sources, by Brooks et al. [20], which can improve the performance of the larger arrays at a higher frequency. A Fourier-based deconvolution with coordinate transformation and scanning technology were combined for 3D acoustic imaging by Xenaki et al. [22], which improves the normal resolution of the planar array but brings some side lobe pollutions. A compressive sensing algorithm was applied to 3D sound localization for obtaining a high-resolution source map by Ning et al. [25], which can provide more accurate results than conventional beamforming. The inverse solution strategies were exploited in the study of 3D acoustic mapping using a planar array, by Battista et al. [26], consisting of the equivalent source method and covariance matrix fitting method. Compared to the conventional beamforming approach, inverse strategies produce more accurate results. Sub-microphone arrays are added to the existing planar array to improve the 3D beamforming source map quality in some research. In 2013, four sub-microphone arrays, arranged around a wind tunnel, were used to detect aeroacoustic sources [27]. Padois and Berry [28] have compared the effects of various microphone array configurations on the 2D and 3D sound source localization. Two sub-arrays placed vertically were used to detect acoustic dipole sources in a 3D space [29]. These studies have found that adding a vertical sub-microphone array helps improve the planar array’s normal spatial resolution. There were 192 microphones in four arrays collecting sound signals at the same time in Padios et al.’s research [27]. The number of microphones in the combined array is larger when the microphone is dense in a single sub-array. Ping et al. proposed a 3D source localization model using a rigid spherical microphone array with spherical wave propagation [30], in which sparse Bayesian learning is used to perform the localization in 3D space. The above methods suffer from the problem of too many data acquisition channels or the high complexity of the algorithm. High-resolution 3D beamforming is still an important challenge and needs further study.

Non-synchronous measurements [31,32] can virtualize the microphone array into a large or high-density array by moving an array sequentially to record the signal at several locations. A sufficient number of fixed reference points are required to encode the phase relationships between the microphones in non-synchronous measurements. Antoni et al. [33] proposed a non-synchronous measurements method that does not require a reference point, in which the Bayesian probabilistic approach and the Expected-Maximization algorithm are used to reconstruct the source fields iteratively. The results of the non-synchronous measurements and the synchronous measurement have a similar accuracy. For this non-synchronous measurements method, Yu et al. [34] proposed fast iteration algorithms—including the augmented Lagrange multiplier (ALM) algorithm and alternative direction method of multipliers (ADMM) algorithm—to improve the iteration speed. These fast iteration algorithms can effectively recover the missing data in the cross-spectral matrix due to the non-synchronous measurements. The problem of non-synchronous measurements in 3D beamforming has received limited attention in the research literature, as most studies focused on 2D beamforming. Therefore, an interesting question arises: Is it possible to effectively improve the array’s spatial resolution by moving the planar microphone array in space?

In this paper, the performance of 3D beamforming with the non-synchronous measurements is studied systematically. The prototype array configuration is optimized by moving a planar array in space when the sound sources are distributed in 3D space. Compared to conventional 3D beamforming, the number of microphones required for the measurement is reduced. The spherical basis is selected on the basis function instead of the Fourier basis when the beamforming expands to being 3D. The algorithm flowchart of this paper is shown in Figure 1. It is noted that conventional beamforming is just one convenient choice here and all the beamformers (i.e., MVDR and MUSIC [35,36]) that are based on the cross-spectral matrix can be applied in the proposed methods. The current paper is organized as follows. In Section 2, the non-synchronous measurements are developed in the context of 3D beamforming. The simulation results between a single synchronous measurement (i.e., one planar array) and non-synchronous measurements (i.e., a non-synchronous moving array) are compared in Section 3. In Section 4, the performance of the non-synchronous measurements and the synchronous measurement are discussed. In Section 5, an experiment is made to validate the performance of the non-synchronous measurements in 3D imaging. The main conclusions of this study are summarized in Section 6.

2. Forward Model of Acoustic Imaging and Acoustic Measurement

2.1. Conventional Beamforming

Denoting the sound pressure measured by the microphone at position $\mathit{r}$ as $p(\mathit{r})$, $p(\mathit{r})$ is the sum of the particular pressure $p_{\mathrm{P}}(\mathit{r})$ and the random pressure $p_{\mathrm{N}}(\mathit{r})$. $p_{\mathrm{P}}(\mathit{r})$ is the sound pressure transmitted from the sound sources $\mathit{s}$ at ${\mathit{r}}^{\prime }$ to the microphone at $\mathit{r}$, which can be obtained from the Green’s function $G(\mathit{r},{\mathit{r}}^{\prime })$. $p_{\mathrm{N}}(\mathit{r})$ is the measurement noise, which is usually assumed to be a Gaussian distribution function [37,38]. For a given frequency $\omega$, $p(\mathit{r})$ can be expressed as

$$p(\mathit{r},\omega )=p_{\mathrm{P}}(\mathit{r},\omega )+p_{\mathrm{N}}(\omega )$$

(1)

where

$$p_{\mathrm{P}}(\mathit{r},\omega )={\displaystyle {\int }_{{\mathit{r}}^{\prime }\in \Gamma }G(\mathit{r},\omega |{\mathit{r}}^{\prime })}\mathit{s}({\mathit{r}}^{\prime },\omega )d{\mathit{r}}^{\prime },G(\mathit{r},\omega |{\mathit{r}}^{\prime })=\frac{e^{jk{\Vert \mathit{r}-{\mathit{r}}^{\prime }\Vert }_2}}{4\pi {\Vert \mathit{r}-{\mathit{r}}^{\prime }\Vert }_2}$$

(2)

$G(\mathit{r},\omega |{\mathit{r}}^{\prime })$ is the free-field Green’s function that describes the acoustic propagation between the sources at ${\mathit{r}}^{\prime }$ and the microphones at $\mathit{r}$. $k=\omega /c$ is the wavenumber that represents the number of radians per unit distance, and $c$ is the wave velocity. The ${\ell }_2\text{-}\mathrm{norm}$ is defined as ${\Vert •\Vert }_2$.

Substituting Equation (2) into Equation (1), and re-writing the measured pressure signal in the form of the matrix, we can get

$$p(\mathit{r},\omega )=G(\mathit{r},\omega |{\mathit{r}}^{\prime })\mathit{s}({\mathit{r}}^{\prime },\omega )+p_{\mathrm{N}}(\omega ).$$

(3)

Denoting the number of microphones as $M$ and the number of the sources as $S$, the sizes are $p(\mathit{r},\omega )\in {ℂ}^{M\times 1}$, $G(\mathit{r},\omega |{\mathit{r}}^{\prime })\in {ℂ}^{M\times S},G=({\mathrm{g}}_1,\dots ,{\mathrm{g}}_{\mathit{s}},\dots ,{\mathrm{g}}_S)$, $\mathit{s}({\mathit{r}}^{\prime },\omega )\in {ℂ}^{S\times 1}$, and $p_{\mathrm{N}}(\omega )\in {ℂ}^{M\times 1}$. The cross-spectral matrix $\mathit{C}(\omega )$ can be obtained as

$$\mathit{C}(\omega )=\mathbb{E}\{p(\mathit{r},\omega )p^H(\mathit{r},\omega )\}=\frac{1}{I}{\displaystyle \sum _{i=1}^I{p}_i(\mathit{r},\omega )p_i^H(\mathit{r},\omega )}$$

(4)

where $\mathbb{E}\{•\}$ represent the mathematical expectation, ${(•)}^H$ represents the Hermitian transpose, $I$ is the number of pressure snapshots, and the size is $\mathit{C}(\omega )\in {ℂ}^{M\times M}$.

Conventional beamforming is designed to locate sound sources by compensating for the time delay and amplitude attenuation between the microphones and the virtual source. The outcome of the beamformer ${\mathit{Q}}_{\mathrm{BF}}$ is calculated in terms of the cross-spectral matrix as

$${\mathit{Q}}_{\mathrm{BF}}={\mathit{h}}_n^H\mathit{C}(\omega ){\mathit{h}}_n$$

(5)

The weights ${\mathit{h}}_n$ are designed to be independent of the array data or data statistics, which compensate for the time delays and amplitude attenuation of the forward propagation. It can be obtained as

$${\mathit{h}}_n=\frac{{\mathrm{g}}_n}{{\Vert {\mathrm{g}}_n\Vert }_2^2},n=1,\dots ,S$$

(6)

The beamformer is designed to search for the source location by steering the microphone array. When the virtual source location coincides with the real source position, the outcome of the conventional beamforming gets the maximum. A more detailed description of the conventional beamforming can be found in Chu’s work [35].

2.2. Non-Synchronous Measurements Theory

Unlike synchronous measurement, in which all microphones acquire data simultaneously, non-synchronous measurements receive data by moving the microphone array under the assumption of a stable sound field. There is evidence that non-synchronous measurements are effective in 2D beamforming. For non-synchronous measurements, a denser or larger virtual array can be obtained by moving the microphone array in a 2D space, as shown in Figure 2. In denser virtual arrays, the working frequency of the beamforming algorithms is expanded; in larger virtual arrays, the entire radiating object is also better covered [39].

The sound source localization is based on the beamforming method, whether the synchronous or non-synchronous measurements. The cross-spectral matrix $\mathit{C}(\omega )$ is the main difference between the synchronous and non-synchronous measurements. For synchronous measurements, $\mathit{C}(\omega )$ is a $M\times M$ full matrix. The rank of $\mathit{C}(\omega )$ is equal to the number of uncorrelated sources $S$. For the non-synchronous measurements, a complete cross-spectral matrix is obtained, no longer due to the moving microphone array. The obtained matrix ${\widehat{\mathit{C}}}^P(\omega )$ will lose the non-diagonal block elements’ information when the cross-spectral matrix obtained from each measurement ${\mathit{C}}^{(i)}(\omega )$ is placed on the diagonal block of a larger matrix. The superscript $P$ refers to the total number of the non-synchronous measurements and the superscript $i$ refers to the $i$-th measurement. ${\widehat{\mathit{C}}}^P(\omega )$ is a $MP\times MP$ matrix. When the elements at the non-diagonal position of ${\widehat{\mathit{C}}}^P(\omega )$ are set to zero in the absence of references, the rank of ${\widehat{\mathit{C}}}^P(\omega )$, $r({\widehat{\mathit{C}}}^P(\omega ))$, is equal to the product of the number of the sources and the number of moves, $SP$. The rank of the cross-spectral matrix is considered to be the number of sources. The non-synchronous measurements method is located in the source position by supplementing the missing data in ${\widehat{\mathit{C}}}^P(\omega )$. The accomplished cross-spectral matrix ${\tilde{\mathit{C}}}^P(\omega )$ has the same rank as ${\mathit{C}}^{(i)}(\omega )$ in one measurement. There is an example to clarify the spectral matrix issues in the non-synchronous measurements: assuming that the microphone array includes 25 microphones and the scanning plane contains three sound sources. Figure 3a shows the cross-spectral matrix ${\mathit{C}}^{(1)}(\omega )$ in one measurement. It is a $25\times 25$ full matrix, and the rank of ${\mathit{C}}^{(i)}(\omega )$, $r({\mathit{C}}^{(i)})$, is equal to 3. If the prototype array moves four times, ${\widehat{\mathit{C}}}^4(\omega )$ (as shown in Figure 3b) is obtained by rearranging the ${\mathit{C}}^{(i)}(\omega ),i=1,2,3,4$ of the single measurement in the diagonal block position. ${\widehat{\mathit{C}}}^4(\omega )$ is a $(25\times 4)\times (25\times 4)$ matrix, with data missing at the non-diagonal positions. The rank of ${\widehat{\mathit{C}}}^4(\omega )$, $r({\widehat{\mathit{C}}}^4)$, is equal to 12 (i.e., $3\times 4$). Figure 3c shows the target spectral matrix, ${\tilde{\mathit{C}}}^4(\omega )$. The missing data were supplemented by the non-synchronous measurements and the rank of ${\tilde{\mathit{C}}}^4(\omega )$, $r({\tilde{\mathit{C}}}^4)$, was restored to 3.

The core problem in the non-synchronous measurements is the data-missing spectral matrix completion problem. The process of finding a full cross-spectral matrix can be described as

$$\begin{array}{l}\mathrm{find}\tilde{\mathit{C}}\in {ℂ}^{MP\times MP}suchthat\{\begin{array}{l}r(\tilde{\mathit{C}})=S\hfill \\ {\Vert \mathbf{A}(\tilde{\mathit{C}})-\widehat{\mathit{C}}\Vert }_F\le {\epsilon }_1\hfill \\ \Vert \mathit{\Psi }\tilde{\mathit{C}}{\mathit{\Psi }}^H-\tilde{\mathit{C}}{\Vert }_F\le {\epsilon }_2\hfill \\ {\tilde{\mathit{C}}}^H=\tilde{\mathit{C}}\ge 0\hfill \end{array}\\ \end{array}$$

(7)

The four additional constraints are explained as follows:

• The 1st constraint: $r(\tilde{\mathit{C}})=S$ ensures that the rank of the reconstructed cross-spectral matrix is still equal to the number of the sources $S$.
• The 2nd constraint: $\mathbf{A}(•)$ denotes the sampling operator that gets the elements in the diagonal block of a matrix, and the size is $\mathbf{A}(\tilde{\mathit{C}})\in {ℂ}^{MP\times MP}$, which is identical to $\tilde{\mathit{C}}\in {ℂ}^{MP\times MP}$. $||\mathbf{A}(\tilde{\mathit{C}})-\widehat{\mathit{C}}|{|}_F\le {\epsilon }_1$ ensures that the difference between $\mathbf{A}(\tilde{\mathit{C}})$ and $\widehat{\mathit{C}}$ in the Frobenius norm is less than a given tolerance ${\epsilon }_1$.
• The 3rd constraint: $\mathit{\Psi }$ is a projection matrix. $\mathit{\Psi }\tilde{\mathit{C}}{\mathit{\Psi }}^H=\mathbb{E}\{\mathit{\Psi }\tilde{\mathit{P}}{\tilde{\mathit{P}}}^H{\mathit{\Psi }}^H\}$ is the cross-spectral matrix of $\mathit{\Psi }\tilde{\mathit{P}}$, where the projected matrix $\mathit{\Psi }\tilde{\mathit{P}}$ can be the smooth pressure [40] of the non-synchronous measured pressure $\tilde{\mathit{P}}$. To ensure the acoustic field’s spatial continuity, $||\mathit{\Psi }\tilde{\mathit{C}}{\mathit{\Psi }}^H-\tilde{\mathit{C}}|{|}_F=||\mathbb{E}\{\mathit{\Psi }\tilde{\mathit{P}}{\tilde{\mathit{P}}}^H{\mathit{\Psi }}^H\}-\mathbb{E}\{\tilde{\mathit{P}}{\tilde{\mathit{P}}}^H\}|{|}_F\le {\epsilon }_2$ is added here, and the difference in the cross-spectral matrix between $\mathit{\Psi }\tilde{\mathit{P}}$ and $\tilde{\mathit{P}}$ is required to be smaller than a given tolerance ${\epsilon }_2$. The detailed discussion on this constraint is addressed in Section 2.2.
• The 4th constraint: ${\tilde{\mathit{C}}}^H=\tilde{\mathit{C}}\ge 0$ ensures that both ${\tilde{\mathit{C}}}^H$ and $\tilde{\mathit{C}}$ are positive semi-definite matrixes.

It can be seen from the 1st constraint, the eigenvalue vector of the cross-spectral matrix should be $S$-sparse. The number of non-zero eigenvalues is $S$, equal to the number of sound sources. The rank estimation of a matrix is still a difficult problem. An alternative model has been proposed to solve this problem: the cross-spectral matrix is considered to be a full rank matrix, with a few dominant eigenvalues. This method considers the eigenvalue spectrum to be “weak sparse”, in which the sorted eigenvalue elements of the spectral matrix decay rapidly according to the power law, rather than “complete $S$-sparse”. Equation (7) can be realized in another form of constrained optimization as

$$\begin{array}{ll}\underset{\tilde{\mathit{C}}}{\mathrm{minimize}}& ||\tilde{\mathit{C}}|{|}_{*}\\ \mathrm{subject}\mathrm{to}& ||\mathbf{A}(\tilde{\mathit{C}})-{\widehat{\mathit{C}}}^m|{|}_F\le {\epsilon }_1\\ & ||\mathit{\Psi }\tilde{\mathit{C}}{\mathit{\Psi }}^H-\tilde{\mathit{C}}|{|}_F\le {\epsilon }_2\\ & {\tilde{\mathit{C}}}^H=\tilde{\mathit{C}}\ge 0\end{array}$$

(8)

where $||•|{|}_{*}$ denotes the nuclear norm of a matrix, which is defined as the sum of the eigenvalues ${\lambda }_i$, so $||•|{|}_{*}={\displaystyle \sum _{i=1}^{i=MP}{\lambda }_i(•)}$. The objective function $\underset{\tilde{\mathit{C}}}{\mathrm{minimize}}||\tilde{\mathit{C}}|{|}_{*}$ is to seek $\tilde{\mathit{C}}$ with the minimum nuclear norm.

2D beamforming generally assumes that the distance between the source plane and the measurement plane is known, and the non-synchronous measurements move the microphone array on one plane, making the virtual microphone array denser or larger (seen in Figure 2). 3D beamforming assumes that the source is distributed in a 3D space, and the distribution of the entire 3D sound source space is obtained by iterative scanning of the measured distance. In most studies, the implementation of 3D beamforming still uses planar array measurements, which leads to low spatial resolution problems. In this paper, the non-synchronous measurements are used to measure 3D space, in which the measurement array envelops the 3D sound source space to improve the resolution of the 3D space. As shown in Figure 4, the original array is located in the x–y plane, and then the array can be moved sequentially to the y–z plane and x–z plane, such that the scattering object is surrounded by the virtual microphone array in three planes. This study aims to extend the application of non-synchronous measurements in 3D beamforming.

2.3. Spatial Basis and Spatial Continuity of the Acoustic Field

The sound pressure signal captured by the microphone $\mathit{P}$ can be expressed using a set of spatial bases as

$$\mathit{P}={\displaystyle \sum _{i=1}^n{\varphi }_i}(\mathit{r}){\vartheta }_i=\mathit{\Phi }\mathit{\vartheta }$$

(9)

where $\mathit{\Phi }$ is the spatial basis vector and $\mathit{\vartheta }$ is the coefficients vector. ${\varphi }_i(\mathit{r})$ and ${\vartheta }_i$ are the $i$-th basis and the corresponding coefficient, respectively. The Fourier basis $\mathit{\Phi }(x,y)=e^{i(k_{x}x+k_{y}y)}$ is chosen for 2D beamforming, where $x$ and $y$ are the microphones’ coordinates, and $k_x$ and $k_y$ are the wavenumbers along the $x$ and $y$ directions. For 3D beamforming, a 3D spatial basis should be considered. In this paper, spherical harmonics are chosen (as shown in Figure 5), and its complete orthonormal form in polar coordinates ($\mathit{r},\theta ,\phi$), $Y_l^m(\theta ,\phi )$ of order $l$ and degree $m$, can be written as

$$Y_l^m(\theta ,\phi )={(i)}^{m+|m|}\sqrt{\frac{(2l+1)}{4\pi }\frac{(l-|m|)!}{(l+|m|)!}}{P}_l^m(\cos \theta )e^{im\phi }$$

(10)

where $l\in {\mathrm{R}}^{+}$, $|m|\le l$. $P_l^m(•)$ is the associated Legendre function:

$$P_l^m(x)={(1-x^2)}^{|m|/2}\frac{d^{|m|}}{d{x}^{|m|}}{P}_l(x)$$

(11)

$P_l(•)$ is the Legendre polynomial of degree $l$:

$$P_l(x)=\frac{1}{2^{l}l!}\frac{d^l}{d{x}^l}{(x^2-1)}^l$$

(12)

The negative order spherical harmonics $Y_l^{-m}(\theta ,\phi )$ can be obtained by rotating ${90}^{\mathrm{o}}/m$ around the z-axis relative to the positive harmonics. The pressure signals at one microphone $P(r,\theta ,\phi )$ can be re-written in forms of spherical harmonics as

$$P(r,\theta ,\phi )={\displaystyle \sum _{l=0}^n{\displaystyle \sum _{m=-l}^l{\theta }_l^m(r)Y_l^m(\theta ,\phi )}}$$

(13)

where ${\theta }_l^m(r)$ is the coefficient.

Assuming the acoustic field’s spatial continuity, the reconstructed acoustic field should be smooth, and the sound pressures measured by the two adjacent microphones should have comparable levels. Such constraints should be included in the optimization model. Since the best approximation of vector $\mathit{P}$ in subspace $R(\mathit{\Phi })$ is its projection vector ${\mathrm{Proj}}_{R(\mathit{\Phi })}\mathit{P}$, we need to get the orthogonal projection matrix on the subspace $R(\mathit{\Phi })$. According to Equation (9), the coefficient $\mathit{\vartheta }$ can be obtained from the measured pressure $\mathit{P}$ as

$$\mathit{\vartheta }={\mathit{\Phi }}^{†}\mathit{P}$$

(14)

where $†$ denotes the pseudo-inverse of a matrix based on the fact that $\mathit{\Phi }$ is not generally invertible. The smoothed pressure $\tilde{\mathit{P}}$ can be obtained by

$$\tilde{\mathit{P}}=\mathit{\Phi }{\mathit{\Phi }}^{†}\mathit{P}=\mathit{\Psi }\mathit{P},$$

(15)

where $\mathit{\Psi }=\mathit{\Phi }{\mathit{\Phi }}^{†}$ is the orthogonal projection matrix, and $\tilde{\mathit{P}}={\mathrm{Proj}}_{R(\mathit{\Phi })}\mathit{P}=\mathit{\Phi }{\mathit{\Phi }}^{†}\mathit{P}$ is the projection of $\mathit{P}$ on space $R(\mathit{\Phi })$. The cross-spectral matrix of the smoothed pressure $\tilde{\mathit{C}}$ is given by

$$\tilde{\mathit{C}}=\mathbb{E}(\tilde{\mathit{P}}{\tilde{\mathit{P}}}^H)=\mathbb{E}(\mathit{\Psi }\mathit{P}{\mathit{P}}^H{\mathit{\Psi }}^H)=\mathit{\Psi }\mathbb{E}(\mathit{P}{\mathit{P}}^H){\mathit{\Psi }}^H=\mathit{\Psi }\mathit{C}{\mathit{\Psi }}^H$$

(16)

If the smoothed pressure $\tilde{\mathit{P}}$ is already in the space $R(\mathit{\Phi })$, the projection of $\tilde{\mathit{P}}$ in space $R(\mathit{\Phi })$ should be itself. The constraint $||\mathit{\Psi }\tilde{\mathit{C}}{\mathit{\Psi }}^H-\tilde{\mathit{C}}|{|}_F\le {\epsilon }_2$ is included in the optimization model, where $\mathit{\Psi }\tilde{\mathit{C}}{\mathit{\Psi }}^H$ denotes the re-projection result and $\tilde{\mathit{C}}$ denotes the smoothed result. $\mathit{\Psi }=\mathit{\Phi }{\mathit{\Phi }}^{†}$ has encoded the microphone position information into the matrix $\tilde{\mathit{C}}$.

3. Simulation Results

For conventional 2D beamforming, the plane where the sound sources are located is discretized into N grids with known positions. For 3D acoustic imaging, the observation zone will be extended from one plane to a rectangular box. In the current numerical experiments, the rectangular box’s center to be scanned is at the origin, and the size is $0.4\mathrm{m}\times 0.4\mathrm{m}\times 0.4\mathrm{m}$. The scanned box is discretized uniformly by a $41\times 41\times 41$ grid with a distance of 0.01 m. The six-point sources with equivalent unit magnitudes are located at (−0.1, −0.1, −0.1), (−0.1, 0.1, −0.1), (0.1, −0.1, −0.1), (−0.1, 0.1, 0.1), (0.1, 0.1, 0.1), and (0.1, −0.1, 0.1) (m). These sound sources are located at the vertex of an imaginary cube with the center at the origin and the side length of 0.2 m. The signal of the source is generated by random white noise. The prototype planar array with 56 microphones distributed by the Archimedes spiral is initially located in the z = −0.5 m plane, and the center of the planar array is located at (0, 0, −0.5) (m). The frequency of beamforming is set to 4000 Hz, and the signal-to-noise ratio (SNR) is set to 20 dB. To illustrate the advantages of the 3D beamforming with non-synchronous measurements, the results of 3D imaging by the conventional beamforming and the non-synchronous measurements are compared in the following sections. The results of 3D imaging by the conventional beamforming with the original planar array are given in Section 3.1. As a comparison, the results of the 3D imaging by the non-synchronous measurements with orthogonal and non-orthogonal moving arrays are shown, respectively, in Section 3.2 and Section 3.3. The orthogonal moving arrays mean that the initial microphone array and the moving microphone array are perpendicular to each other. The non-orthogonal moving arrays mean a certain angle between the microphone array before and after moving.

3.1. 3D Imaging by the Conventional Beamforming with a Planar Array

Figure 6a shows the microphone array’s geometry and the location of the sound sources in the numerical simulation. The microphone array is set to be z = −0.5 m, and the center of the array is set to (x = 0 m, y = 0 m, z = −0.5 m). The scanned box is shown as partially transparent to allow visualization of the sound sources. Figure 6b shows the source of power maps in the observation zone, which has been normalized by the maximum value. Six slices are presented on this figure to get a better view of the sound sources location. The simulated sound sources are located at the intersection of these slices. Figure 6c–e respectively show the sources power maps at three typical slices (x = 0.1 m, y = 0.1 m, z = −0.1 m). The positions of the simulated sound sources are marked with “+” on the figures. Figure 6e shows, on the z = −0.1 m slice, that, parallel to the microphone array, conventional beamforming locates the sound sources well, and has a good spatial resolution in both the x and y directions. As shown in Figure 6c,d, on x = 0.1 m and y = 0.1 m, for the slices perpendicular to the microphone array, it is difficult to distinguish the sound source’s location by conventional beamforming. The main lobes of these sound sources are merged into the perpendicular directions, which leads to poor spatial resolution so that the sound source’s position cannot be accurately located.

3.2. 3D Imaging by the Non-Synchronous Measurements with Orthogonal Moving Arrays

The orthogonal moving arrays mean that the moving microphone array is perpendicular to the plane of the previously tested microphone. Figure 7a shows the microphone array configuration when the prototype array is moved once. The initial microphone array is located on the z = −0.5 m plane, which is consistent with the array position in Section 3.1, and the moving microphone array is located on the y = −0.5 m plane. The geometric center of the moving microphone array is located on (x = 0 m, y = −0.5 m, z = 0 m). The results in the 3D zone and typical slices are shown in Figure 7b–e. Comparing the results in Figure 6d and Figure 7d, the orthogonally moving the microphone array one time can improve the resolution in the z-direction of the x-z plane of the initial prototype array. The positions of the sound sources in the Z direction are not distinguished well due to the merging of the main lobes in Figure 6d, while the two sound source positions in the Z direction are located accurately in Figure 7d. By comparing the results in Figure 6c and Figure 7c, the y–z plane’s resolution is slightly improved when the microphone array is moved orthogonally once, the resolution in the Z direction is not enough to locate the positions of the sound sources.

In this simulation case, the microphone array moves orthogonally twice. It moves vertically again based on the simulation case in Figure 7. In Figure 8a, the “third” microphone array is located on the x = 0.5 m plane, and the microphone array’s center is located at (x = 0.5 m, y = 0 m, z = 0 m). The non-synchronous measurements of using three orthogonal moving microphone arrays can achieve good spatial resolution in the x, y, and z directions. Figure 6, Figure 7 and Figure 8 show that the non-synchronized measurement by orthogonally moving the microphone array can overcome the lack of spatial resolution in the normal direction of the planar array. The planar microphone array can only guarantee the spatial resolution on the plane parallel to the microphone array. The microphone array is moved orthogonally and continuously twice, where a high spatial resolution can be obtained in all directions during 3D acoustic imaging.

3.3. 3D Imaging by the Non-Synchronous Measurements with Non-Orthogonal Moving Arrays

In the previous simulation configuration, the microphone array is always moving orthogonally. Due to limitations in measurement space or movement error, it is not always guaranteed that the microphone array remains orthogonal. Figure 9a shows the microphone array configuration when the prototype array is rotated 45° along the x-axis. The miniatures in the upper-right and lower-right corners are the x–z and y–z views of the microphone array configuration, respectively. The initial microphone array is located at the z = −0.5 m plane. The “second” microphone array is at an angle of 45° to the z = −0.5 m plane, and its center is 0.5 m away from the origin point (0, 0, 0). Figure 9 shows that the “second” microphone array still improves the resolution in the Z direction in the y = 0.1 m plane. Compared to the results of the orthogonally moving microphone array in Figure 7, the main lobes of the sources at y = 0.1 is relatively larger. As shown in Figure 9c, the extension direction of the main lobes in the x = 0.1 plane changes a bit. The resolution in the x = 0.1 m plane needs to be improved by further moving the microphone array.

The microphone array is moved again based on the previous analysis. The configuration of the microphone array after moving is shown in Figure 10a. The miniatures in the upper-right and lower-right corners are the x–z and y–z views of the microphone array configuration. The “third” microphone array rotates the initial microphone array by −45° on the y-axis. Its center still keeps 0.5 m away from the origin point (0, 0, 0). The results show that the resolution in the x = 0.1 m plane is improved. The main lobes of sound sources can be distinguished in the Z direction, and the sound sources localization effect is relatively better than that in Figure 6 and Figure 9. We can conclude that the non-synchronous measurements in the 3D space can improve the spatial resolution. Compared with Figure 8e, Figure 10e shows the more accurate result of the sound source localization because in the x–y slice the density of the microphone array increases.

4. Comparison between the Synchronous Measurements and Non-Synchronous Measurements

The results of 3D imaging between the synchronous and non-synchronous measurements are compared in this section. The microphone array is moved orthogonally twice for the non-synchronous measurements. The corresponding synchronous measurement is that the three microphone arrays capture data simultaneously. Figure 11a shows the cross-spectral matrix from three independent measurements, which has been arranged on the matrix’s diagonal in the form of blocks. The elements in the non-diagonal block positions of the measured spectral matrix are missing, and they have been filled with zeros in this figure to visualize these unknown parts. Figure 11b shows the spectral matrix completion by the proposed non-synchronous measurements method. All elements in the current spectral matrix are known in both diagonal and non-diagonal blocks. Figure 11c shows the full cross-spectral matrix in the case of all microphones in three arrays, capturing data simultaneously (i.e., the synchronous measurements). Let ${\mathit{D}}^{\mathrm{non}}$ denote the position of the sound source obtained by the non-synchronous measurements. The relative error can quantify the difference between the real sound source position ${\mathit{D}}^{\mathrm{real}}$ and ${\mathit{D}}^{\mathrm{non}}$ by the ${\ell }_2\text{-}\mathrm{norm}$, which is obtained as ${\mathrm{Error}}_{\mathrm{position}}=({\Vert {\mathit{D}}^{\mathrm{non}}\Vert }_2-{\Vert {\mathit{D}}^{\mathrm{real}}\Vert }_2)/{\Vert {\mathit{D}}^{\mathrm{real}}\Vert }_2$. The frequency of beamforming is set to 4000 Hz, and the signal-to-noise ratio is set to 20 dB. In this case, the maximum relative error ${\mathrm{Error}}_{\mathrm{position}}$ of the six source positions is 1.926%. Since the beamforming algorithm is suitable for higher working frequencies, we selected frequencies of 4000 Hz (Figure 12), 6000 Hz (Figure 13), and 8000 Hz (Figure 14). The beamforming map slices of the synchronous measurement are shown in Figure 12a–c, Figure 13a–c, and Figure 14a–c, and the beamforming map slices of the non-synchronous measurements are shown in Figure 12d–f, Figure 13d–f, and Figure 14d–f. The beamforming results of Figure 12, Figure 13 and Figure 14 show (1) both the synchronous measurements and the non-synchronous measurements can accurately locate the position of the sound source and have a good spatial resolution; and (2) compared to the synchronous measurements, the non-synchronous measurements increase the sidelobes slightly, but the number of microphones required for measurement is reduced significantly in the localization of the spatial sound sources.

The relative error can quantify the difference between the spectral matrix completion (${\mathrm{S}}^{\mathrm{com}}$) and the full spectral matrix (${\mathrm{S}}^{\mathrm{Full}}$) by the Frobenius norm, which is obtained as ${\mathrm{Error}}_{\mathrm{CSM}}=({\Vert {\mathrm{S}}^{\mathrm{com}}\Vert }_{\mathrm{F}}-{\Vert {\mathrm{S}}^{\mathrm{full}}\Vert }_{\mathrm{F}})/{\Vert {\mathrm{S}}^{\mathrm{full}}\Vert }_{\mathrm{F}}$. Figure 15 shows the errors of the spectral matrix completion (${\mathrm{S}}^{\mathrm{com}}$) and the full spectral matrix (${\mathrm{S}}^{\mathrm{Full}}$) with different frequencies and different SNRs. Although the performance of the non-synchronous measurements is influenced by noise, the errors of ${\mathrm{S}}^{\mathrm{com}}$ and ${\mathrm{S}}^{\mathrm{Full}}$ decrease with the SNR. When the SNR is relatively low (for example, SNR = 20 dB and 10 dB), the noise plays a key role in the results of acoustic reconstruction. As the SNR increases and frequency increase, the difference between the spectral matrix completion and the full spectral matrix gradually decreases.

5. Experimental Verification

5.1. Experimental Setup

An experiment of applying the non-synchronous measurements to 3D acoustic imaging has been done in a full-anechoic chamber with a cut-off frequency of 40 Hz and background noise of −1 dB(A). The acoustic vibration measurement platform was provided by the Zhejiang Institute of Metrology (ZJIM). The microphone model was MPA416 (provided by Beijing Prestige Technology Co., Beijing, China). The acquisition instrument model was DH5922D (provided by Jiangsu Donghua Testing Technology Co., Ltd., Jingjiang, China). Figure 16a shows the photograph of the on-site experiment setup. To display the spatial locations of the sound sources, the schematic diagram of the sound source locations and the geometry of the microphone array are drawn in Figure 16b. Four speakers are not in one plane in the experiment. The sound source was driven by Bluetooth speakers. Similar to previous simulations, these four speakers were located at the four vertices (A, C, D and B1) of an imaginary cube ABCD-A1B1C1D1. The side length of this imaginary cube was about 0.25 m. To more conveniently describe the spatial position of the sources and the array in the following, this imaginary cube’s faces and vertices were used as the reference for positioning. The four speakers were supported by four tripods and placed on the marked locations on the ground. The microphone array has 56 elements and has the same geometry as the previous simulation. Taking the center of the imaginary cube (O) as the origin, the space Cartesian coordinate system is established in Figure 16b. When the microphone array was initially placed, the array was placed parallel to the plane ABB1A1 with a distance of 0.6 m. The microphone array was located on the x = −0.725 m plane. The microphone array position was adjusted so that the position of the array center (O1) is the same as the imaginary cube center (O). The strategy for moving the microphone array is shown in Figure 17. It is the top view. The 1st position of the microphone array was located at the x = −0.725 m plane. In the second measurement, the microphone array was rotated 90 degrees counterclockwise. The 2nd position of the microphone array was located at the y = −0.725 m. During the measurement, four speakers simultaneously emitted 4000 Hz pure tone. The acquisition instrument was provided by Jiangsu Donghua Testing Technology Co., Ltd., and its model was DH5922D. When there were more than 32 channels, the maximum sampling rate was 128 kHz/channel. So, a sampling frequency of 50 KHz was selected. The sampling time was 30 s.

5.2. Experimental Results

The 3D beamforming results between a single planar microphone array and the non-synchronous measurements are compared in Figure 18. A significant difference is visible in three cases. Only with the planar microphone array in the 1st position (Figure 18a) or 2nd position (Figure 18b), there are one or two distinct maximum peaks in the field of view, and there is a large sidelobe in the normal direction of each position. It is impossible to locate the spatial source sources accurately. With the non-synchronous measurements (Figure 18c), three distinct maximums appear in the field of vision, and the side lobes in the normal direction are reduced significantly. The 4th sound source is located at the corner on the other side, which was blocked by the slices of the 3D beamforming. The spatial localization accuracy of the sound sources is significantly improved when using the non-synchronous measurements when moving the arrays in 3D space, especially when the microphone array is perpendicular to the sound source plane.

Three slices (x = −0.13 m, y = −0.13 m, z = −0.13 m) of the 3D beamforming results are plotted in Figure 19. With the single microphone array placed in the 1st position (i.e., parallel to the y-z plane), two sound sources can be located at the upper-left and lower-right quadrants at the x = −0.13 m plane. At the y = −0.13 m and z = −0.13 m plane, beamforming maps are not used to determine the sources’ positions due to the larger extension of the lobes. With the single microphone array placed in the 2nd position (i.e., parallel to the x-z plane), the beamforming map at the y = −0.13 m plane exhibits two major lobes that correspond to the positions of the two sound sources. At the x = −0.13 m and z = −0.13 m plane, the main lobes extend so badly that it is impossible to find the sound source positions exactly. When the non-synchronous measurements are adopted, the spatial resolution in both the x = −0.13 m and y = −0.13 m planes is improved relative to the single microphone array. For the z = −0.13 m plane, the beamforming map is improved slightly with the non-synchronous measurements than with the single microphone array. Comparing the beamforming between the non-synchronous measurements and the single microphone array suggests that the non-synchronous measurements improve the spatial positioning resolution of the planar microphone array.

6. Conclusions

Due to the scanning area being a plane in the 2D beamforming method, the distance between the microphone and the sound source needs to be known in advance. The sound sources of industrial production are randomly distributed in space, instead of in a plane. The application of conventional beamforming from 2D to 3D is expanded in this paper. By moving the microphone array in space, the non-synchronous measurements can significantly improve the planar array’s positioning resolution in the normal direction. The 3D sound source localization is achieved by increasing the number of microphone arrays in most current studies, which have the disadvantage of requiring more microphones and acquisition channels. The non-synchronous measurements overcome this shortcoming and can also achieve a high-resolution in 3D positioning. The method proposed in this paper has extensive application potential in the field of 3D sound localization.