Next Article in Journal
Scattering Intensity Analysis and Classification of Two Types of Rice Based on Multi-Temporal and Multi-Mode Simulated Compact Polarimetric SAR Data
Previous Article in Journal
Mapping Soil Properties with Fixed Rank Kriging of Proximally Sensed Soil Data Fused with Sentinel-2 Biophysical Parameter
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Technical Note

Feed-Forward Neural Network Denoising Applied to Goldstone Solar System Radar Images

by
Nereida Rodriguez-Alvarez
1,*,
Joseph S. Jao
1,
Joan Francesc Munoz-Martin
1,
Clement G. Lee
1 and
Kamal Oudrhiri
2
1
Planetary Radar and Radio Science Group, Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109, USA
2
Communication Architectures and Research Section, Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109, USA
*
Author to whom correspondence should be addressed.
Remote Sens. 2022, 14(7), 1643; https://doi.org/10.3390/rs14071643
Submission received: 1 February 2022 / Revised: 21 March 2022 / Accepted: 24 March 2022 / Published: 29 March 2022

Abstract

:
The study of Near-Earth Asteroids (NEA) is crucial for human safety. Small hazardous asteroids with small radar cross sections are not easy to detect, track, and characterize due to the small signal-to-noise ratio (SNR) of the radar echo. This manuscript describes the results obtained for the application of a feed-forward neural network (FFNN) denoising methodology to NEA data obtained from the Goldstone Solar System Radar (GSSR). We demonstrate an increase in the signal level of up to ×4 the original value—in terms of sigma above the mean noise—when applying the FFNN denoising technique to radar Z-score normalized Binary Phase Code (BPC) images. This improvement benefits better radar detection of NEAs in general. Reducing the noise background level for antennas that have lower aperture, e.g., 34 m dishes, enables the use of FFNN denoising to improve visual detections on those noisier conditions. In addition, reducing noise level benefits shorter integration times of the data to obtain adequate signal levels. When talking about detection of small bodies crossing the antenna beam, since the asteroids or debris can move across the beam quite fast, it is relevant to reduce the integration time to allow for an increased number of independent pieces of information crossing the target through the antenna beam. The increased distance between the signal level and the noise level enables a better detection of the small-bodies at shorter integration times and therefore would be very useful for the detection of objects in the cis-lunar space.

1. Introduction

The Goldstone Solar System Radar (GSSR) has contributed to a vast number of near-earth asteroid (NEA) observations. The GSSR together with Arecibo, until its collapse, and recently the Southern Hemisphere Radar (SHR), have accomplished observations of over 1000 different NEAs [1]. The 1000th NEA, 2021 PJ1, was observed by GSSR by the principal investigators from the Jet Propulsion Laboratory (JPL), with the support of the Planetary Radar and Radio Science group from JPL, on 14 August 2021 [2]. From those ~1000 asteroids, the GSSR has contributed to the observation of 377 asteroids. Therefore, over the years, the GSSR has proven to be essential in tracking NEAs. The United States annual expense in the search for NEAs is close to $4 million [3], and the main goal has been to keep track of those NEAs that pose a hazard because of their potential to impact the Earth.
GSSR has also been essential in providing NASA’s Office of Safety and Mission Assurance with exclusive orbital debris data in Low Earth Orbit (LEO) via Goldstone’s Orbital Debris Radar (ODR) [4], ensuring the safety of astronauts and spacecraft operating in that region of space. With the new focus on sending humans to the Moon, there is a need to extend the radar range to monitor the cis-lunar space, the region between the Moon and the Earth. In addition to spacecraft, it is known that there are small asteroids crossing the cis-lunar space that may endanger missions going to or coming from the Moon [5]. The development and assessment of new detection and processing techniques for a better characterization of small spacecraft in the cis-lunar space as well as the improvement in the characterization of NEA, both in general and particularly in the cis-lunar space, are therefore strategic. This manuscript covers the denoising technique developed to further advance the current signal processing capabilities allowing for better detection of the observed objects. Modernizing the current signal processing is key to project the GSSR into the future, making it capable of more advanced detection of small bodies in the cis-lunar space, and also further away from the Moon, while positioning GSSR and the DSN as indispensable elements to protect Earth, Moon, and the cis-lunar space and beyond.
This document is organized as follows: Section 2 describes the data collected by the GSSR; Section 3 goes over the characteristics of datasets selected for the study; Section 4 is dedicated to introducing the feed-forward neural network (FFNN) denoising technique; and Section 5 analyzes the performance of the FFNN denoised data versus the original data obtained from conventional signal processing methods. In Section 6, we discuss the relevance of the denoised data on: (1) the reduction of noise levels enabling the use of smaller antenna dishes; and (2) the reduction of noise levels allowing for higher angular resolution images at an adequate sigma level for a successful detection. Section 7 provides the conclusions.

2. GSSR Radar Images

The GSSR is capable of measuring in different configurations. As described in [6], the GSSR is capable to configure different waveforms that are tailored to the target characteristics and the type of measurement needed for the particular investigation. The most widely used waveforms by the scientific community to assess target characteristics are the continuous wave (CW), the chirp, and the binary phase-coded (BPC). CW produces a 1-dimension power spectrum signal, while BPC and chirp produce 2-dimension power distribution in delay and Doppler domain. In this study, for the initial development of the FFNN denoising technique we selected BPC images. The technique can be extended to chirp data, as those correspond to 2-dimensions as well. The BPC mode provides ranging measurements, by modulating a pseudo noise (PN) code on the carrier. Code lengths are set to ensure they can cover the delay depth of the target. To obtain the distance to the target, the radar system measures the round-trip travel time of the radar signal, from which the distance can be calculated. This produces a delay power distribution along with the Doppler power distribution, resulting in delay-Doppler images such as the one shown in Figure 1.
The delay-Doppler image in Figure 1 shows the results obtained for NEA 2018 DH1. GSSR configured to BPC waveform with 0.25 us baud rate and a sampling frequency of 20 MHz with a resolution of 1 Hz. The image has been Z-score normalized following Equation (1).
B P C   Z s c o r e = B P C μ n o i s e σ n o i s e
Consecutive radar measurements are coherently integrated to ultimately increase the sigma levels. The obtained BPC image is Z-score normalized following Equation (1), i.e., by computing the noise floor mean ( μ n o i s e ) and its standard deviation ( σ n o i s e ) for the areas clean of signal and then subtracting μ n o i s e from the signal pixels and performing the ratio over σ n o i s e [7]. Figure 1 corresponds to the integration of 60 radar files of 60 s each, which results in a maximum Z-score value of 34.15-sigma above the mean noise, with an original σ n o i s e = 2.49 (before normalization). In the following sections, we denoise images from a smaller number of radar files, integrating just enough to be able to see the signal, denoising those and then denormalizing using the original μ n o i s e and σ n o i s e , and again Z-score normalizing to the denoised levels. This is further explained in Section 4. While this denoised data is beneficial to improve detection chances (with the same or smaller antenna dishes), it could potentially be used for scientific purposes given a set of conditions. We discuss this in Section 6.

3. Selected Datasets

A number of NEAs were selected to develop and analyze the results of the FFNN denoising technique. Those are all Apollo NEAs and are classified as Potentially Hazardous Asteroids (PHAs). Selected NEAs are shown in Table 1.
The selected pool of NEAs therefore covers asteroids with H between 18 and 22, diameters between ~200 m to ~1400 m, and rotation periods from ~1 h to ~20 h. In addition, one of them (2017 YE5) is a binary asteroid, whose BPC images show signatures different from the typical BPC image of a non-binary NEA. We have included 2017 YE5 in an attempt to make the database more comprehensive; we show the results in Section 5.

4. Denoising Methodology

Within the frame of the project that funded this research, we investigated various denoising techniques. Regardless of the denoising methodology, in order to denoise the images we conducted the following approach:
  • Process BPC image from a number of radar files. The total number of radar files available differs from track to track as it depends on the round-trip-time to the object.
  • Apply Z-score normalization to the BPC image, following Equation (1). Store original μ n o i s e and σ n o i s e values.
  • Apply denoising technique.
  • Denormalize denoised BPC image using original μ n o i s e and σ n o i s e values.
  • Apply Z-score normalization to the denoised BPC image, following Equation (1) using denoised μ d e n o i s e d and σ d e n o i s e d values.
We studied wavelet transform (WT) denoising (Daubechies 1, Daubechies 4, Coiflet 1, and Coiflet 4) [8], principal components analysis (PCA) denoising [9], block-matching 3D (BM3D) denoising [10], and feed-forward neural network (FFNN) denoising [11] techniques. Here we summarize those methods:
  • Daubechies: The Daubechies wavelet transform is a generalization of the Haar transform, which is implemented as a succession of decompositions. Daubechies allow for a filter length (N) bigger than 2, corresponding to the Haar transform, which provides more localization and smoothing effect, see Figure 1 for some examples. Daubechies transform are described by the number of vanishing moments, i.e., N/2 and are based on the use of compactly supported orthonormal wavelets, which makes discrete wavelet analysis feasible.
  • Coiflet: The Coiflet wavelet transform is also based on scaling functions, similar to Daubechies wavelet, that exhibit vanishing moments.
  • PCA: The PCA is a statistical technique that consists of simplifying a dataset by reducing it to a dataset composed of a lower number of dimensions. It uses orthogonal properties to transform a set of observations of possibly correlated variables into a set of values of uncorrelated variables. The denoising application intends to remove the noise while keeping the signal information. Basically, PCA is used to find the principal components that describe the maximum variance of the matrix (in this case an image), eliminating therefore those that contain residual information, expected to be related to the noise of the image. The image is then formed back from the reduced number of principal components which contain most of the information, and the noise is reduced.
  • BM3D: Block-matching 3D (BM3D) filtering is a two-stage non-locally collaborative filtering method in the transform domain. This method is based on selecting 2D image fragments, known as blocks, and creating 3D arrays with those blocks that share similarities, known as groups, i.e., block matching. The 3D groups are then transformed into another domain, and a collaborative filtering is applied, that by attenuating the noise, reveals more fine details shared by grouped blocks and at the same time preserves the essential unique features of each individual block. This filtering can be a simple thresholding filter or something in the lines of a Wiener filter. After filtering the inverse transform is applied, and all the grouped images are aggregated to reconstruct the image. It is known that when the noise level is high, the BM3D performance decreases and can create undesired artifacts.
Table 2 provides a summary of the preliminary study results from denoising BPC images of 2018 DH1 and Figure 2 shows how Figure 1 is affected by the denoising approach.
Some WT denoising and FFNN denoising methods showed promising results comparable in terms of noise reduction, but the effect on signal levels and smoothing of image edges led to the rejection of the WT, PCA, and BM3D methods. We selected FFNN to be the technique shown in this study.

Feed-Forward Neural Network (FFNN)

An FFNN [11] is generally described as an artificial neural network where the connections between neurons do not form a cycle, i.e., there is no loop or back-fed information from neurons of the next layer to the previous layer. Figure 3 shows the architecture for a basic FFNN with M hidden layers and N neurons per layer.
In our case, N, in Figure 3, is the same as the number of inputs (Xi=1:N) in the input layer and the number of outputs (Yi=1:N) in the output layer, but it can be set to a different value. The architecture is therefore the most basic with the information moving only in one direction: from the input to the output, passing a number of hidden layers, with a number of nodes. This type of architectures can be used to accomplish many goals, and recently FFNN have been used to denoise images in a variety of fields [12,13], including astronomical images. Note that X corresponds to the vectorized version of the original Z-score normalized BPC images. Y corresponds to the vectorized denoised images. In other words, the denoising scheme in Figure 3 corresponds to step 3 in the approach defined at the beginning of Section 4.

5. FFNN Applied to Selected NEA

The total number of files recorded by GSSR for each NEA varies from track to track, as it is dependent on the round-trip time (RTT) of the asteroid during its observing run. For example, for 2018 DH1 there are 60 individual radar files. Individual files may not show sufficient signal to be able to apply any denoising methods, but we found that coherently averaging together four files was sufficient to have enough sigma level for the asteroids considered in this study. Therefore, we generally defined a SET as the BPC image resulting from averaging four consecutive radar files for a particular asteroid track. Our goal was to demonstrate that the FFNN denoising technique significantly reduces the noise of the BPC images obtained after a few coherent integrations, thus producing increased sigma levels (once we denormalize using the original μ n o i s e and σ n o i s e values and Z-score normalize again to the denoised μ d e n o i s e d and σ d e n o i s e d values).

5.1. FFNN Set-Up: Data and Net

For this study we employ the BPC images of the pool of the nine NEAs described in Section 3. For each one of the NEAs a total of 36 radar files is considered, regardless of the total amount of data available. Figure 4 shows the processing of the GSSR radar files into vectorized Z-score normalized BPC images ready to be ingested by the FFNN.
As can be seen in Figure 4, each NEA undergoes the same preparation process to generate a total of nine BPC images. The 36 radar data files for each NEA are selected and grouped in SETs of four and processed independently into BPC images. In order to process the files into BPC we use standard GSSR signal processing algorithms, [14]. Table 3 provides the processing information applied to each asteroid to obtain BPC images.
This algorithm basically consists of cross-correlating the received signal with the PN code of the transmitted radar waveform for different delay and Doppler frequency values. This produces nine BPC images per asteroid. A Z-score normalization is applied to the nine BPC images. Then those are vectorized and inputted as vectors into the FFNN. The FFNN denoises the inputted vectors, and the output vectors are then reshaped into denoised images. As explained at the beginning of Section 4, once we obtain denoised images from the FFNN, we apply steps 4 and 5 in the approach, i.e., we denormalize the denoised BPC image using the original μ n o i s e and σ n o i s e values, and apply a new Z-score normalization using denoised μ d e n o i s e d and σ d e n o i s e d values.
In order to train the FFNN we need target denoised images for the images selected for training. The targeted denoised images are obtained for each SET by characterizing the noise of the original BPC images. This is done by computing its mean and setting a threshold based on this mean. We allowed three times its mean value to fall into the noise category, then selected the pixels that meet the noise threshold and reduced those samples to 20% of their initial value. Figure 5 shows an example of a noisy BPC image and a target denoised BPC image for one SET processed for NEA 2018 DH1.
As can be seen, in the targeted images the noise is reduced and the shape of the signal is preserved. Therefore, the noise characterization done for this particular image works for this particular asteroid. We wanted to generalize the process by training our FFNN to do this regardless of the NEA being processed. For this same reason, it is very important to the application to test the FFNN with completely independent asteroids. We use the BPC images for four NEA for the training and validation and the BPC images of the other five NEA for the testing. To accomplish this, we built a simple network like the one shown in Figure 2 with M = 3 hidden layers and N = 12 neurons per layer. Normally, three hidden layers can solve non-linear problems, if the neurons per layer are properly set. Up to date, there is no rule to select the number of neurons. Most of the applications run in a combination of 1–3 layers with the number of neurons not exceeding 7–15 neurons. As a general rule, the number of neurons should not be larger than the number of input features for simple problems. In our case, since our 2D images are 61 (on the shortest axis), the total number of neurons should not exceed this number; hence, 12 × 3 = 36, which is half the number of expected features. Here, the feature would be a noise realization inside our image. Note that, the number of neurons is selected to avoid any kind of over-fitting in our signal. Increasing the number of neurons beyond the limit (e.g., using 100 neurons) would cause over-fitting, and the network would not actually “learn” to denoise a signal, but to directly copy the training dataset output. Conversely, a fewer number of neurons may cause under-fitting and no meaningful output. The runtime to train the net is below 10 s. Then, applying the neural network to denoise the image takes an extra 2 s per set of radar files.
The next sub-sections provide the training and validation and testing steps in this FFNN. In summary our pool of data corresponds to a total of nine NEA i.e., 81 BPC images following the diagram in Figure 4. In order to develop the FFNN, 12 of those BPC images were selected as training set and 24 of them as a validation set. Finally, the 45 remaining were set aside for testing.

5.2. FFNN Training and Validation

The training and validation are applied to NEAs 2018 DH1, 2017 YE5, 2010 NY65, and 2014 SR339. Figure 6 shows a diagram of this process.
As shown in Figure 6, the 36 vectorized BPC images processed from four NEAs are split in 12/24 for training/validation. Then the FFNN of three hidden layers/12 neurons is trained using the Levenberg–Marquardt algorithm [15,16], employed to solve non-linear least squares problems. As testing, the developed neural network is applied to the rest of 2018 DH1, 2017 YE5, 2010 NY65 and 2014 SR339 sets (i.e., the 24 sets not used for training). Figure 7 shows the noise standard deviation, both σ n o i s e and σ d e n o i s e d , for all the SETs of asteroids 2010 NY65 and 2014 SR339. We also include the training sets in the Figure 7 (red circles) to show that the FFNN works very similarly for both of them.
Figure 8 shows the resulting BPC images corresponding to the integration of all data in their Z-score normalized original and Z-score normalized denoised versions.
As can be seen there is a substantial sigma level increase, which is proportional to the noise reduction, when comparing noisy and denoised images of ×3.6 and ×3.23, respectively for asteroids 2014 SR339 and 2010 NY65. The next section is dedicated to the FFNN testing. Note that the testing of this FFNN is performed over completely independent asteroid SETs. In addition, it is important to note that, comparing Figure 8a,b, some signal within the echo has been interpreted as noise by the procedure. This results in loss of some shape and scattering information.

5.3. FFNN Testing

To prove that the FFNN is able to generalize to other NEAs, the pre-trained FFNN is applied to a set of completely independent NEAs: 2018 EB, 2017 VR12, 2017 WX12, 2016 AJ193, and 2006 UK. The testing is therefore applied to all SETs for those NEAs, i.e., 45° BPC images. We decided to keep those five NEAs out of the training and validation pool, to ensure the testing was completely independent from the training. The final performance is assessed by comparing the sigma values of the original images to the sigma values of the denoised images. Figure 9 and Figure 10 show the final integrated Z-score normalized BPC images of the original and denoised data for NEA 2016 AJ193 and 2017 WX12, respectively.
Table 4 shows the performance evaluation for the pool of selected asteroids.
Overall, the FFNN performs really well for all the targets and obtains denoised images with σ d e n o i s e d up to four times lower than the original σ n o i s e .
The pool of asteroids used for training and validation include 2017 YE5, which is a binary object. The results for this particular object are not as good as for conventional asteroids. The resulting denoised image is shown in Figure 11.
The main issue is attributed to the FFNN not being properly trained for those type of objects given the pool of asteroids that was used is primarily based on conventional objects. For 2017 YE5, the FFNN does not destroy the signal in terms of edges, but it does affect the sigma levels considerably, smoothing them over a larger area around the peak. To include binary with comparable size moons, binary with smaller moon, and contact binary asteroids, we first need to increase the size of the pool of asteroids we are using for training, in order to make those statistically representative. If a single network is not able to handle the difference between the expected signal, there is still the possibility of including those types of asteroids by generating a pre-decision tree that classifies the original image into its category: conventional, binary, contact binary, or simply having the radar astronomer conducting the investigation for each track select the appropriate classification for the asteroid. Then three FFNN denoising chains can be built by training independent FFNN with asteroids from one type only.

6. Discussion: The Relevance of Denoised Radar Images

The main purpose of the work presented here was to demonstrate that signal processing strategies can help mitigate the noise of the GSSR radar data and produce, in this case, Z-score normalized BPC images with increased sigma levels—with respect to the noise. The main motivation for the denoising of these images is to help towards improving the detection of targets in the cis-lunar space. By reducing the noise level, radar detections become more evident. In such a way we can reduce the integration time needed to obtain adequate sigma levels to claim a detection. Therefore, in the typical scenario when GSSR is staring at a given location, we could potentially identify targets crossing the antenna beam with a certain speed more easily because we can resolve them faster. Even though the normal procedure is to employ continuous wave for detection purposes, the FFNN denoising techniques would enable detection to be pursued with ranging images. In ranging experiments, it is sometimes difficult to distinguish the object against the noise and it is necessary to explore a long-code to find a collection of a few pixels corresponding to an object. In that sense the FFNN would help by enhancing the signal with respect to the noise floor and can facilitate the detection.
The FFNN denoising technique can be understood as an enhancing tool in situations where a DSS-14 70 m dish antenna is not available (i.e., downtime, scheduling conflicts). This risk can be mitigated using other 34 m dish antennas (such as DSS-13) in combination with FFNN denoising techniques to enhance weak detections, i.e., reducing the noise level with respect to the signal level, an easing of the capability to detect objects, for example in the cis-lunar space. Fundamentally, though, the FFNN technique is limited by the true signal-to-noise ratio of the observed object, and improving detectability is not improving SNR for radar astronomy purposes.
The BPC images are generally used for improving the orbits and measuring the near surface radar scattering properties, leading to the characterization of the physical properties of the NEAs, such as size, spin state, mass and/or density. BPC images are also used in shape models, when the signal levels allow and the observation time covers the rotation periods of the NEA. Here we provide a number of analyses to facilitate the assessment by a radar astronomer or asteroid researcher, of whether or not the resulting images can be used for scientific purposes and not only detection purposes.
In order to be able to use denoised images for scientific purposes, a number of features must be preserved from the original images, such as:
  • Preservation of the radar scattering function after denoising is applied.
  • Preservation of echo pixels close to the noise level.
  • Preservation of the center of mass of the echo after denoising is applied.
  • Denoised images to produce similar shape models for the asteroid, as compared to original images.
Based on these premises, for the purpose of providing information towards the evaluation of the usefulness of the FFNN technique for asteroid near-surface regolith characterization, we looked into the radar scattering function. Figure 12 analyses the radar scattering function, obtained by summing the power in the range axis to create a power vs range plot.
Note that in Figure 12, the blue line corresponds to the radar scattering function of the original images and orange corresponds to the radar scattering function of the denoised images. Based on the different radar scattering functions obtained for the different asteroids, the way power decreases as a function of range into the echo is preserved to some degree, potentially for some asteroids more than others (such as 2010 NY65 or 20017 VR12). We cannot guarantee that the information on the radar scattering properties of the asteroid’s near-surface regolith remains for all asteroids, such as 2014 SR339 in Figure 12a. In addition, from the plots in Figure 12, the preservation of echo pixels close to the noise levels is difficult to assess by a non-expert. If we were to make the FFNN denoising tool available to asteroid researchers this would have to be validated for each asteroid. Table 5 provides a measure of the deviation between the radar scattering functions for the original and denoised images in Figure 12.
The Root Mean Square Error (RMSE) corresponds to the error between the normalized to the maximum original radar scattering function and the normalized to the maximum denoised radar scattering function. The correlation coefficient corresponds to the Pearson correlation coefficient computed as the covariance of the two radar scattering functions (original and denoised) over the standard deviation of each radar scattering function. Based on those numbers a possible metric is that the RMSE is <0.5% and the correlation coefficient is >98%. This will classify the denoising obtained in Figure 12c,g as preserving the radar scattering function.
Additionally, for the purpose of providing information towards the evaluation of the usefulness of the FFNN technique for shape modelling, we looked into the images noise statistics. By employing SHAPE software [17,18], it could be validated whether or not the FFNN denoised images from a well-known, high signal-to-noise ratio object, with a known shape model, produce similar outputs. For example, the study in [19] shows the radar imaging and physical characterization of NEA 2000 ET70 using data from Arecibo and Goldstone. In the study the authors used range-Doppler images from chirp measurements to feed a shape model software to produce the shape model for 2000 ET70. The authors in [19] fit all the radar images and CW spectra obtained from the collected data on February 2012 for 2000 ET70. One check that can be done is to examine the statistics of the noise of the denoised images. Figure 13 shows the results for 2000 ET70.
The noise statistics are modified from a normal distribution to a Lognormal distribution. If the SHAPE software tool can ingest those images, a shape model could be derived. This team has no access to the SHAPE software tool, but by providing noise statistics information it may help radar astronomers to decide on the usability of those images for the specific purpose of shape modelling.
It is also important to note that, for example, for the denoised image of NEA 2014 SR339, Figure 8b clearly shows the loss of signal caused by the FFNN denoising technique. Although this is not as evident in the other NEA denoised images, it must be carefully examined by radar astronomers. In the case of NEA 2014 SR339 (Figure 8b) signal pixels in the middle of the echo were designated as noise and were consequently denoised, this change in the signal echo will certainly impact the shape modeling of the object.
If the BPC images could be used by the SHAPE model, the reduced integration times required to obtain the same sigma-levels can be an additional asset. In [19] the authors combined every 8 images from Arecibo data and every 14, 9, 13, 13, 14, 12, 6, and 12 images for February 15, 16, 17, 18, 19, 20, 22, and 23, respectively, for the Goldstone data. Then the authors minimized an objective function that consists of the sum of squares of residuals between model and actual images and other techniques to ensure a reasonably smooth surface. We processed the data for 2000 ET70 in different ways to prove the sigma level increases with respect to the denoised noise floor can benefit shorter integrations. First of all, we obtained Z-score normalized BPC images from the integration of 15 radar files of 15 s each, i.e., 225 s. We choose 15 instead of 14 just to have more flexibility in grouping data for the denoising. Figure 14a shows the original Z-score normalized BPC image of 2000 ET70 for an integration of 15 files. This image is denoised in Figure 14b.
Similarly, we plotted the results of the integration of five radar files (Figure 14c,d), and three radar files (Figure 14e,f), showing both the original Z-score normalized BPC image of 2000 ET70 and its denoised version. Note that even for the shorter integration time of the three radar files (Figure 14f), the FFNN denoising obtains images with sigma levels on the same order as the 15 radar files with integrated Z-score normalized BPC image of 2000 ET70 (Figure 14a). This means that the FFNN denoising images could be provided at an integration of three radar files equivalent to 45 s of data. As the rotation of the asteroid was 9 h, the available information would have improved from a resolution of 2.5° per image to a 0.5° per image. If the asteroid is very slow, the ability to have more frequent measurements may not have a real impact on the final asteroid shape model, but if the asteroid rotates quickly, compared to the data collection time frame, the less images that are integrated the better in order to capture the asteroid features, avoiding averaging them away. The minimum number of radar tracks that needs to be integrated varies and is a trade-off between the required signal levels for the shape model and the asteroid rotation period. Therefore, if the images were suitable for the SHAPE modelling tool, the use of the FFNN denoising technique has the potential to increase the temporal resolution of measurements, therefore increasing the spatial information when fitting the model to the measurements.
Next, we analyzed if the FFNN denoising technique distorted the power distribution, i.e., if the FFNN denoised image changes the distribution of the power with respect to the original image. In order to check this, we computed the center of mass (CoM) of the signal distribution in the image, not to confuse with the CoM of the object as this would have required the estimation of the spin-axis to estimate for astrometry where the CoM is positioned. We applied the same code to both the original and FFNN denoised images and summarized the calculations in Table 6.
As can be seen in Table 6, the CoM moves in some cases by one delay bin. Particularly for 2014 SR339 and 2010 NY65 it moves one bin in delay and 2008 EB and 2017 WX12 move one bin in Doppler at the resolution. In the same line of the conclusions for the radar scattering function analysis, while some denoised images preserve the CoM of the power distribution, we cannot guarantee that this is true for all asteroid denoised images.
We added as part of the discussion of this Technical Note some tools for the evaluation of the scientific value of the denoised data, proper verification is to be conducted by an expert. As future work we will contact radar astronomers to help the team validate or discard the approach for scientific purposes. Regardless, the main motivation for the denoising of these images is to ultimately improve the detection of targets in the cis-lunar space. With the new upcoming developments to set base at the Moon and the increase in space traffic, developing techniques towards enhancing GSSR detection capabilities becomes relevant.

7. Conclusions

The presented FFNN denoising techniques shows promising performance improvement towards increasing the small-body detection capabilities of the GSSR. We demonstrated that the FFNN denoising technique leads to more clearly distinguishable echoes as the noise in the images is reduced. This can be understood as enabling radar tracks with smaller dish antennas, such as 34 m antennas, enhancing the measurements by identifying the echo and reducing the noise, therefore improves the chances to claim a detection. As part of the discussion section, we included a number of analyses that would allow radar astronomers to evaluate the usefulness of the FFNN denoising technique for scientific purposes. Regardless of the scientific value, the improved detection capability is key to the GSSR and other antenna complexes as it becomes more and more important to detect small-bodies in the cis-lunar space. Since small-bodies can move across the antenna beam quite fast, it is relevant to reduce the integration time allowing for an increased number of independent pieces of information crossing the target through the antenna beam. In that regard, we developed an FFNN denoising approach and validated it on a pool of asteroids independent of the training set.

Author Contributions

Conceptualization, N.R.-A., C.G.L. and K.O.; methodology, N.R.-A.; software, N.R.-A.; validation, N.R.-A.; formal analysis, N.R.-A. and J.F.M.-M.; investigation, N.R.-A.; resources, J.S.J.; data curation, N.R.-A. and J.S.J.; writing—original draft preparation, N.R.-A.; writing—review and editing, N.R.-A. and J.F.M.-M.; visualization, N.R.-A.; funding acquisition, C.G.L. and K.O. All authors have read and agreed to the published version of the manuscript.

Funding

This work was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration. The research was funded by the Jet Propulsion Laboratory under the project with grant number 01STRS-R.20.285.045 © 2022. California Institute of Technology. Government sponsorship acknowledged.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data used in this study is available under request.

Acknowledgments

The authors would like to acknowledge the PIs of the investigations that collected the data for the NEAs used in this study: Lance Benner, Marina Brozovic, and Shantanu Naidu. We would also like to acknowledge the support and effort of the operators and personnel at the DSN site in Goldstone: Lawrence Snedeker and his team.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Asteroid Radar History Website. Available online: https://echo.jpl.nasa.gov/asteroids/PDS.asteroid.radar.history.html (accessed on 18 October 2021).
  2. Press Release on 3 September 2021: Planetary Radar Observes 1000th Near-Earth Asteroid Since 1968. Available online: https://www.jpl.nasa.gov/news/planetary-radar-observes-1000th-near-earth-asteroid-since-1968 (accessed on 18 October 2021).
  3. National Research Council. Defending Planet Earth: Near-Earth-Object Surveys and Hazard Mitigation Strategies; The National Academies Press: Washington, DC, USA, 2010. [Google Scholar]
  4. Lee, C.; Jao, J.; Slade, M.A.; Rodriguez-Alvarez, N. Micro-Meteoroid and Orbital Debris Radar from Goldstone Radar Observations. J. Space Saf. Eng. 2020, 7, 242–248. [Google Scholar] [CrossRef]
  5. McGuire, M.L.; Burke, L.M.; Hack, K.J.; Strange, N.J.; McElrath, T.P.; Landau, D.F.; Lantoine, G.; Lopez, P.; McDonald, M.A. Potential cislunar and interplanetary proving ground excursion trajectory concepts. In Proceedings of the 2016 IEEE Aerospace Conference, Big Sky, MT, USA, 5–12 March 2016; pp. 1–11. [Google Scholar] [CrossRef]
  6. Rodriguez-Alvarez, N.; Jao, J.S.; Lee, C.G.; Slade, M.A.; Lazio, J.; Oudrhiri, K.; Andrews, K.S.; Snedeker, L.G.; Liou, R.R.; Stanchfield, K.A. The Improved Capabilities of the Goldstone Solar System Radar Observatory. IEEE Trans. Geosci. Remote Sens. 2021, 60, 5103415. [Google Scholar] [CrossRef]
  7. Schroeder, D.J. Astronomical Optics, 2nd ed.; Academic Press: San Diego, CA, USA, 1999; p. 278. ISBN 978-0-12-629810-9. [Google Scholar]
  8. Daubechies, I. Ten Lectures on Wavelets; SIAM: Philadelphia, PA, USA, 1992; ISBN 978-0-89871-274-2. [Google Scholar]
  9. Zhang, L.; Dong, W.; Zhang, D.; Shi, G. Two-stage Image Denoising by Principal Component Analysis with Local Pixel Grouping. Pattern Recognit. 2010, 43, 1531–1549. [Google Scholar] [CrossRef] [Green Version]
  10. Dabov, K.; Foi, A.; Katkovnik, V.; Egiazarian, K. Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering. IEEE Trans. Image Process. 2007, 16, 2080–2095. [Google Scholar] [CrossRef] [PubMed]
  11. Thakur, R.S.; Yadav, R.N.; Gupta, L. State-of-art analysis of image denoising methods using convolutional neural networks. IET Image Process. 2019, 13, 2367–2380. [Google Scholar] [CrossRef]
  12. Fan, L.; Zhang, F.; Fan, H.; Zhang, C. Brief review of image denoising techniques. Vis. Comput. Ind. Biomed. Art 2019, 2, 7. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Vojtekova, A.; Lieu, M.; Valtchanov, I.; Altieri, B.; Old, L.; Chen, Q.; Hroch, F. Learning to denoise astronomical images with U-nets. Mon. Not. R. Astron. Soc. 2021, 503, 3204–3215. [Google Scholar] [CrossRef]
  14. Slade, M.A.; Benner, L.A.M.; Silva, A. Goldstone Solar System Radar Observatory: Earth-Based Planetary Mission Support and Unique Science Results. Proc. IEEE 2010, 99, 757–769. [Google Scholar] [CrossRef]
  15. Levenberg, K. A Method for the Solution of Certain Problems in Least Squares. Q. Appl. Math. 1944, 2, 164–168. [Google Scholar] [CrossRef] [Green Version]
  16. Marquardt, D. An Algorithm for Least-Squares Estimation of Nonlinear Parameters. J. Soc. Ind. Appl. Math. 1963, 11, 431–441. [Google Scholar] [CrossRef]
  17. Hudson, S. Three-dimensional reconstruction of asteroids from radar observations. Remote Sens. Rev. 1993, 8, 195–203. [Google Scholar] [CrossRef]
  18. Magri, C.; Ostro, S.J.; Scheeres, D.; Nolan, M.; Giorgini, J.D.; Benner, L.A.; Margot, J.-L. Radar observations and a physical model of Asteroid 1580 Betulia. Icarus 2007, 186, 152–177. [Google Scholar] [CrossRef]
  19. Naidu, S.P.; Margot, J.-L.; Busch, M.W.; Taylor, P.A.; Nolan, M.C.; Brozovic, M.; Benner, L.A.M.; Giorgini, J.D.; Magri, C. Radar imaging and physical characterization of near-Earth Asteroid (162421) 2000 ET70. Icarus 2013, 226, 323–335. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Z-score Normalized BPC image for NEA 2018 DH1. The scale indicates the sigma level above noise, in this case 34.5. The standard deviation of the noise samples is σ n o i s e = 2.49 . Image scale is clipped to values > −1.
Figure 1. Z-score Normalized BPC image for NEA 2018 DH1. The scale indicates the sigma level above noise, in this case 34.5. The standard deviation of the noise samples is σ n o i s e = 2.49 . Image scale is clipped to values > −1.
Remotesensing 14 01643 g001
Figure 2. Denoising methods: (a) Daubechies 1 WT, (b) Daubechies 4 WT, (c) Coiflet 1 WT, (d) Coiflet 4 WT, (e) PCA, (f) BM3D, and (g) FFNN. Image scales are clipped to values > −1.
Figure 2. Denoising methods: (a) Daubechies 1 WT, (b) Daubechies 4 WT, (c) Coiflet 1 WT, (d) Coiflet 4 WT, (e) PCA, (f) BM3D, and (g) FFNN. Image scales are clipped to values > −1.
Remotesensing 14 01643 g002aRemotesensing 14 01643 g002b
Figure 3. Feed-forward neural network architecture with M hidden layers and N neurons per layer. Xi corresponds to the vectorized version of the original Z-score BPC images. Yi corresponds to the vectorized denoised images, then reshaped into the original dimensions.
Figure 3. Feed-forward neural network architecture with M hidden layers and N neurons per layer. Xi corresponds to the vectorized version of the original Z-score BPC images. Yi corresponds to the vectorized denoised images, then reshaped into the original dimensions.
Remotesensing 14 01643 g003
Figure 4. Diagram of the preparation process of the GSSR data to be inputted to the FFNN.
Figure 4. Diagram of the preparation process of the GSSR data to be inputted to the FFNN.
Remotesensing 14 01643 g004
Figure 5. Example of (a) original BPC image and (b) targeted denoised BPC image, computed for one SET for NEA 2018 DH1. The samples in the black region correspond to samples that were identified as noise and reduced to 20% of their initial value (not set to 0).
Figure 5. Example of (a) original BPC image and (b) targeted denoised BPC image, computed for one SET for NEA 2018 DH1. The samples in the black region correspond to samples that were identified as noise and reduced to 20% of their initial value (not set to 0).
Remotesensing 14 01643 g005
Figure 6. Diagram of the preparation process of the GSSR data to be input to the FFNN.
Figure 6. Diagram of the preparation process of the GSSR data to be input to the FFNN.
Remotesensing 14 01643 g006
Figure 7. Noise standard deviation observed for the images corresponding to each one of the sets for asteroid (a) 2014 SR339 and (b) 2010 NY65. Training sets are marked with a red circle.
Figure 7. Noise standard deviation observed for the images corresponding to each one of the sets for asteroid (a) 2014 SR339 and (b) 2010 NY65. Training sets are marked with a red circle.
Remotesensing 14 01643 g007
Figure 8. Z-score normalized BPC images corresponding to the incoherent averaging of all SETs for NEA 2014 SR339 (a) original (noisy) and (b) denoised; and NEA 2010 NY65 (c) original (noisy) and (d) denoised. Image scales are clipped to values > −1.
Figure 8. Z-score normalized BPC images corresponding to the incoherent averaging of all SETs for NEA 2014 SR339 (a) original (noisy) and (b) denoised; and NEA 2010 NY65 (c) original (noisy) and (d) denoised. Image scales are clipped to values > −1.
Remotesensing 14 01643 g008
Figure 9. Results for asteroid 2016 AJ193: (a) Noise standard deviation observed for the images corresponding to each one of the SETs, (b) Z-score normalized original (noisy) BPC image, and (c) Z-score normalized denoised BPC images. Image scales are clipped to values > −1.
Figure 9. Results for asteroid 2016 AJ193: (a) Noise standard deviation observed for the images corresponding to each one of the SETs, (b) Z-score normalized original (noisy) BPC image, and (c) Z-score normalized denoised BPC images. Image scales are clipped to values > −1.
Remotesensing 14 01643 g009
Figure 10. Results for asteroid 2017 WX12: (a) Noise standard deviation observed for the images corresponding to each one of the sets, (b) original BPC image, and (c) denoised BPC images. Image scales are clipped to values > −1.
Figure 10. Results for asteroid 2017 WX12: (a) Noise standard deviation observed for the images corresponding to each one of the sets, (b) original BPC image, and (c) denoised BPC images. Image scales are clipped to values > −1.
Remotesensing 14 01643 g010
Figure 11. Results for binary asteroid 2017 YE5: (a) Z-score normalized BPC image and (b) denoised Z-score normalized BPC images. Image scales are clipped to values > −1.
Figure 11. Results for binary asteroid 2017 YE5: (a) Z-score normalized BPC image and (b) denoised Z-score normalized BPC images. Image scales are clipped to values > −1.
Remotesensing 14 01643 g011
Figure 12. Radar Scattering Function (power vs range) asteroids, in blue for original images and orange for denoised images, for: (a) 2014 SR339, (b) 2006 UK, (c) 2017 VR12, (d) 2018 EB, (e) 2017 WX12, (f) 2016 AJ193, (g) 2010 NY65, and (h) 2018 DH1.
Figure 12. Radar Scattering Function (power vs range) asteroids, in blue for original images and orange for denoised images, for: (a) 2014 SR339, (b) 2006 UK, (c) 2017 VR12, (d) 2018 EB, (e) 2017 WX12, (f) 2016 AJ193, (g) 2010 NY65, and (h) 2018 DH1.
Remotesensing 14 01643 g012aRemotesensing 14 01643 g012b
Figure 13. Histogram of the noise samples for 2000 ET70: (a) Noise statistics of the BPC image and (b) noise statistics of the denoised BPC image.
Figure 13. Histogram of the noise samples for 2000 ET70: (a) Noise statistics of the BPC image and (b) noise statistics of the denoised BPC image.
Remotesensing 14 01643 g013
Figure 14. Denoising processing for 2000 ET70: (a) image for 15 BPC raw data files coherently integrated and (b) denoised image for 15 BPC raw data files coherently integrated (×2.56 noise reduction); (c) image for 5 BPC raw data files coherently integrated and (d) denoised image for 5 BPC raw data files coherently integrated (×3.28 noise reduction); (e) image for 3 BPC raw data files coherently integrated and (f) denoised image for 3 BPC raw data files coherently integrated (×2.83 noise reduction). Image scales are clipped to values > −1.
Figure 14. Denoising processing for 2000 ET70: (a) image for 15 BPC raw data files coherently integrated and (b) denoised image for 15 BPC raw data files coherently integrated (×2.56 noise reduction); (c) image for 5 BPC raw data files coherently integrated and (d) denoised image for 5 BPC raw data files coherently integrated (×3.28 noise reduction); (e) image for 3 BPC raw data files coherently integrated and (f) denoised image for 3 BPC raw data files coherently integrated (×2.83 noise reduction). Image scales are clipped to values > −1.
Remotesensing 14 01643 g014aRemotesensing 14 01643 g014b
Table 1. List of selected NEA and physical description, including absolute magnitude (H), rotation period (P), albedo ( α ) and diameter (D). Information collected from the Jet Propulsion Laboratory Small Body Database (JPL SMD) accessible at https://ssd.jpl.nasa.gov/tools/sbdb_lookup.html (accessed on 31 January 2022) and/or the Asteroid Radar Research accessible at https://echo.jpl.nasa.gov/ (accessed on 31 January 2022).
Table 1. List of selected NEA and physical description, including absolute magnitude (H), rotation period (P), albedo ( α ) and diameter (D). Information collected from the Jet Propulsion Laboratory Small Body Database (JPL SMD) accessible at https://ssd.jpl.nasa.gov/tools/sbdb_lookup.html (accessed on 31 January 2022) and/or the Asteroid Radar Research accessible at https://echo.jpl.nasa.gov/ (accessed on 31 January 2022).
NameHP [h] α D [m]
2018 DH121.135unknown200 *
(505657) 2014 SR33918.548.710.068971 ^
(441987) 2010 NY6521.394.970.071228 ^
2017 YE5 119.220.6dark860 *
2018 EB21.93.160.07 *240 ^
2017 VR1220.51.38bright250 *
2017 WX1222.116.23unknown100 *
2016 AJ19318.99unknown0.0321374 +
(444584) 2006 UK20.25.72unknown300 *
1 Binary. * Estimated from H. ^ Estimated from thermal observations from NEOWISE. + Estimated from radar.
Table 2. Summary of results for the different denoising techniques applied to BPC images from 2018 DH1.
Table 2. Summary of results for the different denoising techniques applied to BPC images from 2018 DH1.
Method σ n o i s e Reduction Image Alteration
Daubechies 1 WT×4.53Yes-smoothing
Daubechies 4 WT×3.19Yes-smoothing
Coiflet 1 WT×3.32Yes-smoothing
Coiflet 4 WT×1.58Yes-smoothing
PCAincreasedYes-shape
BM3D×1.25Yes-shape
FFNN×3.66Not evident
Table 3. Processing information for each asteroid to obtain BPC images from radar collected data. Note FFT stands for Fast Fourier Transform.
Table 3. Processing information for each asteroid to obtain BPC images from radar collected data. Note FFT stands for Fast Fourier Transform.
ObjectProcessing Information
2018 DH1FFT length = 2048, frequency = 0.5 Hz, code length = 511, baud rate = 0.125 us
2014 SR339FFT length = 2048, frequency = 1 Hz, code length = 255, baud rate = 0.5 us
2010 NY65FFT length = 1024, frequency = 0.1 Hz, code length = 127, baud rate = 0.125 us
2017 YE5FFT length = 2048, frequency = 0.025 Hz, code length = 255, baud rate = 0.25 us
1988 XBFFT length = 512, frequency = 0.5 Hz, code length = 127, baud rate = 0.5 us
2016 AJ193FFT length = 254, frequency = 3 Hz, code length = 255, baud rate = 0.25 us
2008 EBFFT length = 256, frequency = 0.5 Hz, code length = 127, baud rate = 0.25 us
2017 VR12FFT length = 1024, frequency = 1 Hz, code length = 511, baud rate = 0.125 us
2017 WX12FFT length = 256, frequency = 0.04 Hz, code length = 127, baud rate = 0.25 us
2006 UKFFT length = 511, frequency = 0.25 Hz, code length = 255, baud rate = 0.125 us
2000 ET70FFT length = 512, frequency = 1 Hz, code length = 255, baud rate = 0.25 us
Table 4. Performance Assessment of the FFNN denoising technique applied to a variety of NEAs. The original image σ n o i s e is compared to the denoised image σ d e n o i s e d together with its corresponding noise reduction.
Table 4. Performance Assessment of the FFNN denoising technique applied to a variety of NEAs. The original image σ n o i s e is compared to the denoised image σ d e n o i s e d together with its corresponding noise reduction.
Asteroid Noise Reduction
2016 AJ1930.040.01×4
2018 EB0.70.22×3.2
2017 VR1211.143.93×2.83
2017 WX120.040.01×4
2006 UK0.630.22×2.86
Table 5. Deviation between the radar scattering function of the original images and the denoised images in terms of RMSE and correlation coefficient.
Table 5. Deviation between the radar scattering function of the original images and the denoised images in terms of RMSE and correlation coefficient.
ObjectRMSECorrelation Coefficient
2014 SR339 (Figure 12a)3.690%90.5%
2006 UK (Figure 12b)1.670%72.9%
2017 VR12 (Figure 12c)0.062%99.7%
2008 EB (Figure 12d)0.610%84.9%
2017 WX12 (Figure 12e)1.260%88.5%
2016 AJ193 (Figure 12f)1.310%98.1%
2010 NY65 (Figure 12g)0.061%98.2%
2018 DH1 (Figure 12h)1.210%99.1%
Table 6. Center of Mass calculation for the signal power distribution in the images of the pool of asteroids—all images are matrices of [61 × 121] where the axes are vectors of 1 × 121 in the x-axis and 1 × 61 in the y-axis.
Table 6. Center of Mass calculation for the signal power distribution in the images of the pool of asteroids—all images are matrices of [61 × 121] where the axes are vectors of 1 × 121 in the x-axis and 1 × 61 in the y-axis.
ObjectOriginal image
[Doppler, Delay]
[Hz,    Bin]
FFNN Denoised
[Doppler, Delay]
[Hz,    Bin]
Difference
[Doppler, Delay]
[Hz,   Bin]
2018 DH1[101,    477][101,    477][0,   0]
2014 SR339[500,    267][500,    266][0,   1]
2010 NY65[20.9,   250][20.9,   249][0,     1]
2017 YE5[10.1,   253][10.1,   253][0,     0]
1988 XB[20,      256][20,      255][0,     1]
2016 AJ193[98.05,    252][98.05,    251][0,     1]
2008 EB[10.25,    271][10.75,    271][−0.5,   0]
2017 VR12[20.05,    328][20.05,    328][0,     0]
2017 WX12[−0.28,    242][−0.24,    242][−0.04, 0]
2006 UK[−35.25,  239][−35.25,  239][0,     0]
2000 ET70[−80.68,  290][−80.68,  287][0,     3]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Rodriguez-Alvarez, N.; Jao, J.S.; Munoz-Martin, J.F.; Lee, C.G.; Oudrhiri, K. Feed-Forward Neural Network Denoising Applied to Goldstone Solar System Radar Images. Remote Sens. 2022, 14, 1643. https://doi.org/10.3390/rs14071643

AMA Style

Rodriguez-Alvarez N, Jao JS, Munoz-Martin JF, Lee CG, Oudrhiri K. Feed-Forward Neural Network Denoising Applied to Goldstone Solar System Radar Images. Remote Sensing. 2022; 14(7):1643. https://doi.org/10.3390/rs14071643

Chicago/Turabian Style

Rodriguez-Alvarez, Nereida, Joseph S. Jao, Joan Francesc Munoz-Martin, Clement G. Lee, and Kamal Oudrhiri. 2022. "Feed-Forward Neural Network Denoising Applied to Goldstone Solar System Radar Images" Remote Sensing 14, no. 7: 1643. https://doi.org/10.3390/rs14071643

APA Style

Rodriguez-Alvarez, N., Jao, J. S., Munoz-Martin, J. F., Lee, C. G., & Oudrhiri, K. (2022). Feed-Forward Neural Network Denoising Applied to Goldstone Solar System Radar Images. Remote Sensing, 14(7), 1643. https://doi.org/10.3390/rs14071643

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop