A Study of Correction to the Point Cloud Distortion Based on MEMS LiDAR System

Abstract

Active imaging technology can perceive the surrounding environment and obtain three-dimensional information of the target. Among them, light detection and ranging (LiDAR) imaging systems are one of the hottest topics in the field of photoelectric active imaging. Due to the small size, fast scanning speed, low power consumption, low price and strong anti-interference, a micro-electro-mechanical system (MEMS) based micro-scanning LiDAR is widely used in LiDAR imaging systems. However, the imaging point cloud will be distorted, which affects the accurate acquisition of target information. Therefore, in this article, we analyzed the causes of distortion initially, and then introduced a novel coordinate correction method, which can correct the point cloud distortion of the micro-scanning LiDAR system based on MEMS. We implemented our coordinate correction method in a two-dimensional MEMS LiDAR system to verify the feasibility. Experiments show that the point cloud distortion is basically corrected and the distortion is reduced by almost 72.5%. This method can provide an effective reference for the correction of point cloud distortion.

1. Introduction

In the field of active optical imaging, a target was exposed and the backscattered light was detected to extract the depth and spatial information for 3D image formation. Light detection and ranging (LiDAR) systems are one of the most important sensors for 3D optical imaging technology [1,2], which has shown enormous potential in different applications including target recognition, ground surface estimation, robotics, and driving on autonomous vehicles [3,4,5,6,7,8]. Therefore, in order to obtain high-quality target information, it is necessary to select appropriate scanning components to improve the imaging performance of LiDAR, which includes, e.g., a galvanometer scanner [9] or Risley prisms [10]. With the rapid development of laser technology, photoelectric control technology, and image processing technology, 3D imaging LiDAR is tending towards miniaturization, high resolution, high-speed, low cost, and so on. The traditional mechanical scanners (such as vlp-16 [11], LMS511 and Lux-4L/8L) are the earliest developed and the most mature technologies, and have been applied to autonomous driving concept vehicles such as Google and Baidu or industrial applications [12,13,14]. The core technology is to place the linear array laser and linear array detector on the speed-regulating motor, which can rotate at a high speed in one dimension to achieve panoramic imaging. Generally, the volume of mechanical scanners is relatively large and bulky, they need rather high-power consumption (from 6 W in vlp-16 to 22 W in Alpha prime), and it is placed on the roof of the car, so the accuracy is very unstable; Solid-state laser 3D image sensor, the core technology is by waveguide or liquid crystal optical phased array device, through the electronic control technology to guide the laser pulse in any direction. However, the optical phased array technology is essentially based on the diffraction effect, optical transmittance is the bottleneck of its further development. The core technology of the hybrid solid-state laser 3D image sensor is to replace the traditional one-dimensional mechanical rotation device with a micro-electro-mechanical system (MEMS), which is placed inside the panoramic optical system to solve the problem of solid-state. The MEMS mirror has been widely used in the LiDAR system due to its advantage including small size, fast scanning speed, low power consumption, low price, and strong anti-interference. The MEMS scanning LiDAR has become the main application direction of today’s LiDAR imaging system [15,16]. However, in the actual detection process, the obtained point cloud has obvious distortion, which seriously affects the application performance of the MEMS LiDAR, reduces the coverage of the detected target, and causes the loss of target information. Hence, aiming at point cloud distortion phenomenon, we have analyzed the reason for distortion initially, proposed and experimentally demonstrated a de-distortion method based on an improved coordinate correction.

2. Methods

2.1. Principle

Figure 1 shows a schematic diagram of a MEMS LiDAR system. The pulsed laser beam generated by the laser is collimated by a collimator, and after passing through a beam splitter, a laser beam directly passes through a plane mirror deflection direction and hits the MEMS scanning mirror to illuminate the target. The MEMS mirror is a two-dimensional galvanometer that is driven by static electricity and can be deflected in the x and y directions at the same time. It can emit pulsed light into space to actively illuminate the object, thereby scanning the target. The other laser beam is directly transmitted to the photodetector (PD) in the processing module as a reference signal for pulsed laser emission. The backscattered light signal from the target surface is received by the non-imaging objective lens of the receiving system and transmitted to the photosensitive surface of the avalanche photodiode (APD). The detector converts the light signal into an electrical signal and then obtains the return signal of the laser after the subsequent processing module. Combining the reference signal recorded by the previous PD, the time of flight (TOF) of the pulsed laser can be calculated, and obtained the z-coordinate of the distance between the MEMS LiDAR system and the target. Then fused with the two-dimensional spatial coordinates (x, y) formed by the scanning trajectory of the MEMS mirror, the three-dimensional spatial coordinates (x, y, z) of the object can be obtained, and the three-dimensional coordinates (x, y, z) can be displayed the 3D point cloud images in real-time on the PC through the signal transmission.

There are two main scanning methods for MEMS mirrors: point-to-point scanning and continuous scanning [17,18,19]. The frequency of point-to-point scanning is relatively low. In this way, the MEMS mirror must go through the process of starting, accelerating, and decelerating whenever it scans a point, so high-speed scanning cannot be performed; while the frequency of continuous scanning is relatively faster. In this paper, the continuous scanning method is selected, and the scanning trajectory of the MEMS mirror is simulated as shown in Figure 2, where x and y are normalized coordinates, and the range of value is −1 to 1. Each point in the normalized coordinates will have a corresponding z coordinate, forming the three-dimensional coordinates of the target.

2.2. Analysis of the Causes of Distortion and Correction Methods

The MEMS mirror performs two-dimensional scanning of the target in the x and y directions, the receiving optical system is responsible for detecting the echo signal of each point in the scan field of view (FOV), and the time measurement system calculates the TOF. The distance between the target and the receiver can be obtained by the formula r = c × t/2, which is the z coordinate of the target in the three-dimensional space (where r is the range to the target, c is the velocity of light in the free space and t is the time it takes for the pulse of energy to travel from its emitter to the observed object and then back to the receiver). However, the distance r corresponding to the TOF detected during the experiment is not the true z coordinate of the target surface point, but the radial distance r from the point to the receiving objective lens. As shown in Figure 3, the radial distance between the target and the receiving objective lens is r_i. When scanning a plane, the radial distance r_i between different positions of the plane and the objective lens is different. If r_i is directly regarded as the z coordinate of the object in the three-dimensional rectangular coordinate system. It will form a distorted point cloud surface as shown in Figure 3, but it is a plane. Additionally, the degree of distortion is related to the angle of the scanning FOV. The bigger the FOV, the more obvious the distortion.

During the experiment, we converted the scanning coordinates (x, y) of the MEMS mirror into normalized coordinates with a value range of −1 to 1. Therefore, there is a scale factor between the real coordinates (X, Y) and the normalized coordinates (x, y), which is related to the distance information of the target and the maximum deflection angle of the galvanometer. Assuming that the normalized coordinates of the point i obtained by the MEMS scanner are (x_0i, y_0i), the coordinates after fusing the distance information are (x_0i, y_0i, r_i), and the corresponding real coordinates under the LiDAR system are (X_i, Y_i, Z_i). The deflection angles of the MEMS galvanometer scanning along the x-axis and y-axis are θ_x and θ_y, respectively. The maximum deflection angle is set as θ_xmax, θ_ymax. As shown in Figure 4.

The real coordinate (X_i, Y_i, Z_i) in the LiDAR system of the point i in the normalized coordinate with the target distance information r_i and the MEMS deflection angle θ_x, θ_y has the following relationship:

$$\begin{array}{l}{X}_i=r_i\cdot \sin ({\theta }_x)\cdot \cos ({\theta }_y)\\ Y_i=r_i\cdot \cos ({\theta }_x)\cdot \sin ({\theta }_y)\\ Z_i=r_i\cdot \cos ({\theta }_x)\cdot \cos ({\theta }_y)\end{array}$$

(1)

Assuming that the n is a proportional coefficient between the normalized coordinate and the real coordinate under the LiDAR system. $1\cdot n=z\cdot \tan ({\theta }_{x\max })$ with x_0i equal to 1. Therefore, the following equation is satisfied for any points (x_0i, y_0i) in the normalized coordinate:

$$\left\{\begin{array}{c}{x}_{0i}\cdot n=z\cdot \tan ({\theta }_x)\\ y_{0i}\cdot n=z\cdot \tan ({\theta }_y)\end{array}\right.$$

(2)

From Equation (2), we solve the coordinates x_0i and y_0i with the deflection angle θ as:

$$\begin{array}{l}{x}_{0i}=\tan ({\theta }_x)/\tan ({\theta }_{x\max })\\ y_{0i}=\tan ({\theta }_y)/\tan ({\theta }_{y\max })\end{array}$$

(3)

It can be seen that x_0i and y_0i respectively represent the ratio of the scanning angle in the horizontal and vertical direction to the maximum scanning angle. The following equations can be obtained from Equations (2) and (3):

$$\begin{array}{l}\cos ({\theta }_x)=\frac{1}{\sqrt{1+x_{0i}^2\cdot {\tan }^2({\theta }_{x\max })}}\\ \sin ({\theta }_x)=\frac{x_{0i}\cdot \tan ({\theta }_{x\max })}{\sqrt{1+x_{0i}^2\cdot {\tan }^2({\theta }_{x\max })}}\\ \cos ({\theta }_y)=\frac{1}{\sqrt{1+y_{0i}^2\cdot {\tan }^2({\theta }_{y\max })}}\\ \sin ({\theta }_y)=\frac{y_{0i}\cdot \tan ({\theta }_{y\max })}{\sqrt{1+y_{0i}^2\cdot {\tan }^2({\theta }_{y\max })}}\end{array}$$

(4)

According to the range of the values for x_0i and y_0i are between −1 to 1 in the normalized coordinate system, and tanθ_xmax and tanθ_ymax are less than 1. Naturally, we applied Taylor expansion to the formula $\sqrt{1+x_{0i}^2\cdot {\tan }^2({\theta }_{x\max })}$ and $\sqrt{1+y_{0i}^2\cdot {\tan }^2({\theta }_{y\max })}$, respectively. Omit higher-order terms above the second order from $x_{0i}^2\cdot {\tan }^2({\theta }_{x\max })$ and $y_{0i}^2\cdot {\tan }^2({\theta }_{y\max })$, Equation (4) is expressed as follows:

$$\begin{array}{l}\cos ({\theta }_x)=1-x_{{}_{0i}}^2\cdot {\tan }^2({\theta }_{x\max })/2\\ \sin ({\theta }_x)=x_{0i}\cdot \tan ({\theta }_{x\max })\cdot \left(1-x_{{}_{0i}}^2\cdot {\tan }^2({\theta }_{x\max })/2\right)\\ \cos ({\theta }_y)=1-y_{0i}^2\cdot {\tan }^2({\theta }_{y\max })/2\\ \sin ({\theta }_x)=y_{0i}\cdot \tan ({\theta }_{y\max })\cdot \left(1-y_{0i}^2\cdot {\tan }^2({\theta }_{y\max })/2\right)\end{array}$$

(5)

Bring Equation (5) into Equation (1), we have:

$$\begin{array}{l}{X}_i=r_i\cdot \left\{\left[1-\frac{1}{2}\left(x_{0i}^2\cdot {\tan }^2({\theta }_{x\max })+y_{0i}^2\cdot {\tan }^2({\theta }_{y\max })\right)\right]\cdot \tan ({\theta }_{x\max })\right\}{x}_{0i}\\ Y_i=r_i\cdot \left\{\left[1-\frac{1}{2}\left(x_{0i}^2\cdot {\tan }^2({\theta }_{x\max })+y_{0i}^2\cdot {\tan }^2({\theta }_{y\max })\right)\right]\cdot \tan ({\theta }_{y\max })\right\}{y}_{0i}\\ Z_i=r_i\cdot \left\{\left[1-\frac{1}{2}\left(x_{0i}^2\cdot {\tan }^2({\theta }_{x\max })+y_{0i}^2\cdot {\tan }^2({\theta }_{y\max })\right)\right]\right\}\end{array}$$

(6)

Let

$$M=1-\frac{1}{2}\left(x_{0i}^2\cdot {\tan }^2({\theta }_{x\max })+y_{0i}^2\cdot {\tan }^2({\theta }_{y\max })\right)$$

(7)

Then the Equation (6) can be expressed in the form of a matrix as:

$$\left(\begin{array}{c}{X}_i\\ Y_i\\ Z_i\end{array}\right)=\left(\begin{array}{c}{x}_{0i}\cdot M_{Xi}\\ y_{0i}\cdot M_{Yi}\\ 1\cdot M_{Zi}\end{array}\right)\cdot r_i$$

(8)

where

$$\begin{array}{l}{M}_{X\mathrm{i}}=M\cdot \tan ({\theta }_{x\max })\\ M_{Yi}=M\cdot \tan ({\theta }_{y\max })\\ M_{Zi}=M\end{array}$$

(9)

M_Xi, M_Yi and M_Zi represent the conversion factors related to normalizing coordinates (x_0i, y_0i, r_i) to the Cartesian coordinates (X_i, Y_i, Z_i) in the LiDAR system, respectively.

Theoretically speaking, the three-dimensional coordinates (x, y, z) of each point are related to the radial distance r_i between the target and the receiver. In Equation (7), since the two-dimensional scan coordinates (x_0i, y_0i) of each point are preset, the maximum scan angle 2θ_max is also known, thus, the conversion factor for each point in the scan can be determined, the actual coordinates of each point in the FOV satisfy the relationship of Equation (7). The coefficients before r_i on the right side of the equal sign are all known. In the actual operation of data processing, after determining the maximum angle of the scan, we can calculate the coordinate conversion factor M for each scan normalized position point according to the coordinate sequence (x_0i, y_0i) in the MEMS scan track M_xi, M_yi, and M_zi, after the time measurement system obtains the distance coordinate r_i, they can be converted into the required (X_i, Y_i, Z_i) according to Equation (8).

3. Experiment

In order to verify the above theory, we conducted the following experiments. For reference, the specifications of the MEMS LiDAR system are summarized in

Table 1. Specifications of the MEMS LiDAR system.

	1 laser
	60° (azimuth) by 60° (vertical) FOV
Sensors	Range: Up To 50 m
	13 cm range accuracy
	100 kHz
	0.48 Watts
	Class 1
Laser	1064 nm wavelength
	4 ns pulse width

and applied the correction method to the system’s 3D imaging experiment. The experimental parameters are shown in

Table 2. Parameter values used in the experiments.

Parameters	Values
Transmitter
Beam diameter	3.0 mm
Pulse energy	4.8 uJ
Angle of divergence	0.51 mrad
Transmitting system optical aperture	32 mm
Receiver
Receiving system optical aperture	50 mm
APD photosensitive surface size	3.0 mm
APD response frequency band	250 MHz
Spectral response range (minimum)	600 to 1150 nm
ADC sampling rate	5 GSa/s
ADC bandwidth	1 GHz
ADC sampling digit	14 bit

. The scanning parameters of the MEMS mirror are the following: the sampling rate is 50 kHz and each frame scans 64 lines, each line contains 256 points, so a frame contains 64 × 256 = 32,768 points, the frame frequency is about 3Hz, and the scanning trajectory adopts raster scanning.

Combine the two-dimensional coordinates (x, y) of the MEMS scanner in a certain frame with the z-coordinates converted from time measurement, and analyze the changes in the data, and use the correction method mentioned in this article to process the data to obtain the three-dimensional imaging invert the image.

We first measured a simple whiteboard plane. The experimental results are shown in Figure 5. Figure 5a is the point cloud image of the whiteboard before calibration. It can be seen that there is an obvious distortion. Figure 5b is the point cloud after applying the method of this paper to correct. In order to quantitatively illustrate the effectiveness of the method in this paper, we respectively analyze the edge contour part of the whiteboard, as shown in Figure 6. The four pictures in Figure 6 are the plane contour point clouds after the boundary contours in the x and y directions are superimposed with the coordinates in the z-direction. We define the deflection angle of the boundary point of the plane contour centerline relative to the horizontal axis as an index of the degree of distortion of the point cloud image. That is, the degree of point cloud distortion is judged by the size of α and β the Figure 6. The specific data is shown in

Table 3. Comparison of point cloud distortion before and after correction.

	Before Correction/°	After Correction/°	Distortion Reduction
XZ plane	15.255	3.901	74.428%
YZ plane	17.650	5.194	70.572%

.

It can be seen from Table 3 that the distortion degree of the XZ plane profile before correction reaches 15.255°, and after correction, it is reduced to 3.901°. Our method reduces the distortion by 74.428%. The contour distortion of the YZ plane is also reduced by more than 70%. Therefore, our method almost reduces the point cloud distortion by 72.5%. It can be seen that our method works well.

In order to prove the robustness of the method proposed in this paper, we further designed an experiment as shown in Figure 7. Using a LiDAR system to scan complex targets. The experimental results are shown in Figure 8. Figure 8a shows the imaging inversion obtained by combining the scanning coordinates (x, y) and the ranging coordinates z. This is the imaging effect before calibration. Obviously, the inverted point cloud image has obvious distortion. Figure 8c shows the corrected point cloud image, and in order to show the results more clearly, the singular points in the point cloud model and the large-scale outliers of the relative point cloud data are eliminated, that is, the laser 3D scanning imaging point cloud data is directly processed for de-noising [20], the point cloud images after de-noising are shown in Figure 8b,d, respectively.

Obviously, as we can see from the Figure 8 that after using the correction method, the point cloud image has almost no distortion, which further proves the robustness of the proposed method. Therefore, the method proposed in this paper can fully meet future vision imaging requirements.

4. Conclusions

Aiming at the phenomenon of distortion in the point cloud of MEMS micro-scan imaging LiDAR, the causes of the distortion are analyzed and the MEMS scan trajectory is simulated, and a novel coordinate correction method is proposed to correct the distorted point cloud. Applying the method proposed in this article to our experimental equipment, the point cloud distortion of simple and complex targets obtained by scanning are all well corrected. In the experiment, we found that the distortion degree of the point cloud is approximately 16.45° before correction, and only 4.7° after correction. Therefore, the degree of distortion is reduced by nearly 72.5%. It is fully proved that our method has good de-distortion ability.