Pilot Study of Low-Light Enhanced Terrain Mapping for Robotic Exploration in Lunar PSRs

Abstract

The recent discovery of water ice in the lunar polar shadowed regions (PSRs) has driven interest in robotic exploration, due to its potential utilization to generate water, oxygen, and hydrogen that would enable sustainable human exploration in the future. However, the absence of direct sunlight in the PSRs poses a significant challenge for the robotic operation to obtain clear images, consequently impacting crucial tasks such as obstacle avoidance, pathfinding, and scientific investigation. In this regard, this study proposes a visual simultaneous localization and mapping (SLAM)-based robotic mapping approach that combines dense mapping and low-light image enhancement (LLIE) methods. The proposed approach was experimentally examined and validated in an environment that simulated the lighting conditions of the PSRs. The mapping results show that the LLIE method leverages scattered low light to enhance the quality and clarity of terrain images, resulting in an overall improvement of the rover’s perception and mapping capabilities in low-light environments.

1. Introduction

The discovery of water ice in the lunar permanently shadowed regions (PSRs) has driven international space agencies to conduct robotic explorations [1,2,3]. The PSRs are inside impact craters near the lunar south and north poles. As the Moon’s rotation axis is nearly perpendicular to its orbital plane around the Sun, these crater interiors did not receive direct sunlight for geologically long periods of time. Cold and dark environments in the PSRs trapped volatile materials, especially water ice [4,5,6]. In the context of in-situ resource utilization (ISRU) for producing consumable products from native resources, water ice can be utilized to produce oxygen and water for life support or hydrogen for fuel and propellant [7,8]. Thus, water ice is a key resource for enabling long-term sustainable human exploration and habitation.

Planetary rovers have provided in-situ ground imagery for scientific investigation. Future robotic missions in the lunar PSRs are planned to carry out comprehensive surveys to characterize the presence and behavior of volatile resources and to eventually create a resource map for future human exploration [9,10]. However, the dark environment in the PSRs affects rovers’ vision systems for detecting hazardous obstacles and building 3D topographic maps of unknown environments. Although Lunar Orbiter Laser Altimeter (LOLA)-based digital elevation models (DEM) definitely help in understanding surface elevations and topographical properties of polar regions, their low spatial resolution, which ranges from 5 m to 240 m [11], is insufficient for a rover to determine an obstacle-free path covering the shortest distance and consuming the least energy. Highly detailed and accurate 3D topographic maps are essential for planning safe and efficient traverse routes, identifying in-situ resources, and establishing infrastructure for human settlements and activities [12,13]. Future robotic missions require rovers to autonomously navigate and map uncertain environments. For example, a rover might need to identify hazardous obstacles in a timely manner. Therefore, there have been active research efforts on simultaneous location and mapping (SLAM)-based robotic mapping systems employing a variety of image sensors. The image sensors are generally classified into active (e.g., LiDAR, radar, and time-of-flight camera) and passive (e.g., optical camera) [14,15,16]. Active sensors directly obtain 3D topographic data regardless of dark illumination conditions. However, for a solar-panel- and battery-powered rover, active sensors are heavy, large, and have high energy consumption. Therefore, most rovers for Moon and Mars explorations carry optical cameras due to their lighter weight, smaller size, and lower power consumption [17,18,19]. Adopting supplementary light sources such as headlights can be practical under low illumination conditions [2,20]. However, as the rover moves, the position and orientation of headlights continuously changes relative to the surrounding environment. Variations in light intensity and the creation of shadows on nearby terrain features can cause inaccuracies in the rover trajectory estimation and inconsistencies in RGB colors in 3D point clouds.

This research is motivated by recent research results indicating that the PSRs are not completely dark. While direct sunlight does not reach the PSRs, most areas receive reflected light from the crater rim and Earth and faint light from stars [21,22,23]. For a comprehensive understanding of illumination conditions, a number of simulation models were developed to provide insights into how light is scattered and distributed within the PSRs [23,24,25]. Mazarico et al. [23] reported that the LCROSS impact site in the Cabeus PSR is expected to have an average flux of 0.025 mWm⁻² and a maximum flux of 0.172 mWm⁻². In addition, the ShadowCam onboard the Korean Pathfinder Lunar Orbiter (KPLO) utilized scattered light to acquire high-resolution and high-signal-to-noise-ratio images within the PSRs [26,27]. In this regard, this research presents a visual SLAM-based robotic mapping method that incorporates the low-light image enhancement (LLIE) method. The LLIE method improves the brightness and contrast of images captured in poorly exposed regions, enhancing the capability of the robotic mapping method to estimate the location of a rover and to construct a topographic map of its surroundings in the PSRs. This paper is organized as follows: Section 2 provides an overview of related research on planetary robotic mapping and LLIE methods. The robotic mapping method is presented in Section 3. Section 4 describes the test bed and presents test results for examining and validating the proposed method. Finally, the conclusion is presented in Section 5.

2. Related Works

2.1. Planetary Robotic Mapping Method

In robotic exploration, localization and mapping is a fundamental task for supporting scientific and engineering operations. In the early stages of Mars exploration, the Sojourner rover was deployed to explore a landing site within a 10 m × 10 m area. Iterative feature matching on lander imagery and dead reckoning with wheel encoders and a turn-rate sensor were used for localization [28]. For more robust localization over longer distances, close-range high-resolution images were used for topographic mapping and rover localization. The Opportunity, Spirit, and Curiosity rovers used visual odometry and bundle adjustment methods to correct positional errors caused by wheel slippage and azimuthal angle drift. Topographic products such as local DEMs and digital orthophotos were then generated around the landing site (or along traversed areas) [29,30]. In the recent Moon explorations, the Yutu 1 and 2 rovers were localized by applying feature matching and bundle adjustment methods to adjacent images along traverses. Alternatively, when the rovers were near distinctive landmarks, feature matching between orthophotos derived from rover and orbital (or lander) images was used for localization [31].

Although significant achievements have been made in planetary exploration, future robotic missions will require rovers to traverse longer distances at higher speeds than in past missions [32,33]. Prospective lunar missions are expected to involve tasks such as sample returns or the establishment of a base [1,13,34], for which SLAM is considered to be a fundamental method for enabling autonomous operation when locating, navigating, and mapping in unknown environments. Extensive research has been conducted on planetary robotic SLAM, in which the perception of terrain and mapping outcomes primarily rely on sensor types. Monocular SLAM methods were presented to address the challenges of locating unconstrained motion of a rover [35,36]. However, a single camera without inertial and range sensors is limited to unevenly distributed and sparse features, causing scale ambiguity and measurement drift. The RGB-D SLAM was proposed as an alternative sensor that takes advantage of both depth per pixel and color images [37,38]. However, the RGB-D sensor has a limited range and is sensitive to changes in lighting conditions, resulting in noisy depth measurements. LiDAR SLAM, which enables high-resolution and long-range measurement, has showed potential for enhancing the autonomous navigation capabilities of rovers. Tong and Barfoot [15] demonstrated the effectiveness of a LiDAR mounted on a rover for building a globally consistent map for lunar base construction. However, planetary rovers are constrained by limited power, computational resources, and data storage [16]. Moreover, the LiDAR sensor has not been employed in contemporary rovers.

Therefore, in this study we selected a stereo camera for developing a visual SLAM-based robotic mapping method. In comparison to a monocular camera, a stereo camera can directly measure bearing and range. Point clouds with color information facilitate the identification and detection of objects of interests. Camera systems are relatively lighter and consume less power per unit than LiDAR. In addition, stereo cameras will be compulsory payloads on current and future rovers (e.g., the Yutu rovers in the Chang-E 3 and 4 missions, the Perseverance rover in the Mars 2020 mission, and the VIPER rover in the Artemis mission) [39,40,41,42]. However, adapting visual SLAM to the PSRs poses substantial technical challenges, such as recognizing homogeneous and repetitive terrain features in low-light environments. Therefore, to increase the visual perception and mapping capabilities of visual SLAM, an investigation of LLIE methods is described in the following subsection.

2.2. Low-Light Enhancement Method

The LLIE method is categorized as a conventional deep-learning-based method. Conventional methods generally build models that characterize intensity values in a given image, refining illumination to enhance the image. Histogram Equalization (HE) improves the global contrast of an image by redistributing the histogram evenly across the intensity range. A contrast-limited adaptive histogram (CLAHE) was proposed to overcome the limitation of HE of overamplifying noise, especially in areas with low contrast [43]. The entire image is divided into overlapping patches, and HE, which is adapted to each patch, is then independently applied with a predefined threshold. The divided images tend to improve the overall quality of an image. Dehazing was developed to mitigate the haze effect in images [44]. However, when a low-light image is inverted, the visual effect is similar to that of a daytime image in fog. Therefore, a transmission map using a dark channel is first used to process the inverted low-light images. The re-inverted image offer improved brightness and enhanced visual details.

The deep-learning-based method relies on feature extraction and a deep neural network as the architecture for representing learning of data. Retinex algorithms such as SSR [45] and MSR [46], which are inspired by the human visual system, enhance the image by separately adjusting illumination and reflectance components in the scene [47]. A global illumination-aware and detail-preserving network (GladNet) first uses an encoder-decoder network to estimate global illumination distribution of low-light images [48]. A convolution network is then followed to enhance the images while preserving their details through concatenation of the original low-light image. A multi-branch low-light enhancement network (MBLLEN), which consists of a feature extraction model, an enhancement module, and a fusion module, extracts and enhances feature maps at different scales [49]. These feature maps are then fused to generate an enhanced image. A Kindling the Darkness network (KinD) is a convolution network based on retinex theory [50]. An illumination network enhances low light in an image, and a reflectance network then removes degradations. These deep-learning-based LLIE methods require paired datasets of low- and normal-light images for training purposes. However, the Moon’s surface includes landforms and features that are topographically and geologically distinct from those common on the Earth. Therefore, in this research, emulated PSR terrain is utilized to examine the proposed robotic mapping method described in Section 3.

3. Methodology

The visual SLAM-based robotic mapping method is designed for building a highly detailed 3D point cloud map in the lunar PSRs. Figure 1 shows the overall flow of the proposed method consisting of three main threads: preprocessing, localization, and dense mapping. Unlike other planetary robotic missions during daytime, a low-illumination condition in the PSRs can degrade the quality of image pairs from the stereo camera mounted on a rover. Low visibility, color distortion, and increased noise can lead to erroneous rover trajectories and mapping results. Therefore, in the preprocessing thread, the LLIE method is additionally utilized to enhance mapping capabilities (Section 3.1) and visual perception and tracking (Section 3.2).

3.1. Low-Light Enhanced Image for Dense Mapping

In the mapping thread, semi-global block matching (SGBM) [51], which involves an enhanced image pair from the preprocessing thread, is used to create a disparity map for dense 3D mapping (Figure 1). However, the enhanced images inevitably have color alterations and information loss, which can be considered as potential error sources that create noise in a 3D point cloud. Thus, structural dissimilarity (DSSIM), which is a distance metric derived from the structural similarity index measure (SSIM) [52], is used as a filter to examine a pixel-wise correspondence between stereo pairs and to increase the mapping accuracy (1):

$$\mathrm{DSSIM}=\frac{1-\mathrm{SSIM}(\mathrm{x},\mathrm{y})}{2}$$

(1)

where x and y are patches extracted from the left and right images, respectively, with a kernel size of 15. A disparity estimate below the predefined DSSIM is used to create a 3D point cloud as follows:

$$\left[\begin{array}{c}{\mathrm{X}}_{3\mathrm{D}}\\ {\mathrm{Y}}_{3\mathrm{D}}\\ {\mathrm{Z}}_{3\mathrm{D}}\end{array}\right]={\mathrm{K}}^{-1}\left[\begin{array}{c}{\mathrm{X}}_{2\mathrm{D}}\cdot {\mathrm{Z}}_{3\mathrm{D}}\\ {\mathrm{Y}}_{2\mathrm{D}}\cdot {\mathrm{Z}}_{3\mathrm{D}}\\ {\mathrm{Z}}_{3\mathrm{D}}\end{array}\right]$$

(2)

$${\mathrm{Z}}_{3\mathrm{D}}=\frac{\mathrm{f}\cdot \mathrm{b}}{{\mathrm{D}(\mathrm{x}}_{2\mathrm{d}}{,\mathrm{y}}_{2\mathrm{d}})}$$

(3)

where K, f, and b are the intrinsic matrix, the focal length, and the baseline between two cameras, respectively. D(x_2d, y_2d) is the disparity estimate at 2D coordinates (x_2d, y_2d). Also, since the accuracy of a disparity estimate is lower at longer distances, only 3D points closer than a predefined distance (e.g., 5 m) are mapped. In the mapping process, all point clouds from every key frame are re-projected onto a coordinate system from the first frame in an image sequence. The mapping space is divided into a set of 3D grids with a predefined resolution. Each grid referenced in a 3D coordinate system registers a number of 3D points and their RGB pixel color information. The 3D grid helps in filtering and managing point clouds. For example, when the minimal number constraint of points is set as 10 in a grid size of 5 × 5 × 5 cm³, grids with numbers of points below the predefined threshold are considered as noisy measurements to filter out, and other grids are recorded as unoccupied. The RGB color associated with an occupied grid is computed simply as the mean RGB. The occupancy and color information for each 3D grid are used to maintain consistent RGB color information for the 3D point clouds.

3.2. Low-Light Enhanced Image for Localization

The localization thread adopts stereo parallel tracking and mapping (S-PTAM) [53] as a base framework for estimating a camera’s pose and trajectory by matching the correspondences between a terrain feature and identical features on the stereo image keyframe (Figure 1). The following keyframe, which is selected if the number of feature matches is less than 90% of that in the previous keyframe, is used to update the camera pose by triangulating between feature matches of neighboring frames. Although low-light enhanced images from the preprocessing thread can increase the quality and quantity of feature matches [54], the feature extraction and matching method is limited to homogeneous (textureless) terrains in the PSRs. Therefore, in addition to the nearest neighbor distance ratio (NNDR), the disparity map based on the mapping thread is used as another constraint to increase accurate feature matches. In a sequence of stereo image pairs, the nearest neighbor feature matches are typically obtained by searching along the epipolar line. The disparity between feature matches is then computed and compared with the corresponding position on the disparity map. When the difference between two disparities is below a threshold, the feature matches are considered to be valid. Also, the locational error is unavoidably propagated and consequently accumulated into the camera trajectory while a rover is traversing rugged and homogeneous terrains. The bundle adjustment, which involves a series of keyframes from the beginning, is used to locally optimize the camera poses. Also, when a rover returns to a previously visited place, loop closure using a Bag of Words (BoW) model [55] is used to globally minimize the accumulated drift along the rover’s trajectory.

4. Application and Results

4.1. Test Environment

The robotic mapping approach in the previous section was applied to the emulated lunar terrain (hereafter referred to as the testbed) at the Korea Institute of Civil Engineering and Building Technology. The test bed, which consists of a soil bin (Figure 2a) and an adjustable-light-level LED lamp (Figure 2b), was designed to allow the rover to simulate the mapping process under dark environments of PSRs. The soil bin has horizontal dimensions of 6.45 m × 3.86 m. Soil is used to shape small craters and rounded piles on the flat ground, to which a layer of the lunar simulant (KLS-1) [56] is added, and rocks and pebbles are irregularly placed to emulate various terrain features on the lunar surface. The LED lamp, covered with diffuser films, is utilized for simulating sunlight. The rover is a four wheeled mobile platform equipped with two cameras (FLIR Blackfly) at a 20 cm baseline (Figure 2c). In the experiments, the camera parameters were automatically adjusted using FLIR SpinView. Gain, gamma, and exposure time were fixed at 17.9 db, 0.7, and 49,991 μs, respectively. Also, during rover movement terrain images were collected at a frame rate of 15 frames per second (fps). CLAHE and Dehaze were selected as representative conventional LLIE methods. Due to the absence of on-site images of the lunar PSRs, it would not be feasible to develop a new deep-learning-based LLIE method. Therefore, well-known LLIE methods like GladNet, KIND, and MBLLEN were selected based on preliminary experiments using outdoor images at sunset. The deep-learning-based LLIE methods were trained with the Low-Light (LOL) dataset, which contains 500 low-light and normal-light image pairs [57]. These methods were further fine-tuned with 400 image sets from emulated illumination conditions in the testbed. In the experiment, the LLIE methods were empirically applied to the test image set under varying illumination conditions from 0.008 mWm⁻² to 0.025 mWm⁻² (Figure 3), which were then compared to the images taken in a normal light condition (Figure 2c).

4.2. Experiments under Dark Illumination Conditions

The first experiment was designed to examine the image enhancement results from the darkest illumination condition in the test image set. Terrain images in Figure 2c and Figure 3a were taken at the same pose and orientation of the rover-mounted camera but under normal (1896 mWm⁻²) and low (7.74 mWm⁻²) lighting conditions. The enhanced images were then quantitatively and qualitatively inspected.

Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Map (SSIM) [58], and Delta-E were used to quantify the enhanced image qualities, using the normal light image as the reference image. PSNR is the ratio of the maximum value of the pixel to the noise. A higher PSNR value implies that a better dark image has been restored to match the normal light image. SSIM analyzes the similarity of brightness, contrast, and structural changes within an image pair in terms of human perception. A SSIM value closer to 1 means that the enhanced image is visually similar to the normal light image, preserving the identical structure. Delta-E is a color difference measure between normal-light and enhanced images. In contrast to PNSR and SSIM, a lower value indicates that it is harder for humans to distinguish between the colors in two images. In

Table 1. Performance of LLIE Methods on Low-Light Image.

LLIE	PSNR (dB)	SSIM	Delta-E
Low-Light Image	7.48	0.16	45.68
CLAHE	14.86	0.29	17.57
Dehaze	19.12	0.36	16.61
GladNet	19.70	0.45	12.38
KIND	16.59	0.38	17.41
MBLLEN	14.96	0.38	22.14

, the evaluation metrics of PSNR, SSIM, Delta-E indicate that all LLIE methods significantly improve the perceptual quality of the low-light image. The overall performance of GladNet is obviously better than the others. However, the enhanced images are inconsistently evaluated because each metric treats different aspects of image quality. For example, Dehaze and KIND achieve the second and third best measures in PSNR and Delta-E, respectively, but for SSIM, KIND slightly outperforms Dehaze.

Quantitative measures do not always correspond to subjective human perceptions of image quality. For example, the visual inspection shown in Figure 4 confirms that hidden terrain objects (e.g., rocks and pebbles) in the dark (Figure 4a) are visualized with improved brightness. However, in Figure 4b, CLAHE causes severe chromatic distortion, resulting in an abnormal appearance. In MBLLEN (Figure 4f), the terrain objects and details are made distinct by the improved color and contrast, but an excessive vignetting effect degrades the image quality. In Figure 4e, KIND preserves the clarity, but chromatic artifacts, mainly blue, are randomly introduced over the entire image. Dehaze (Figure 4c) improves the color and contrast without severely degrading the image quality, but the color of the entire image deviates from that of the normal light image. GladNet, in Figure 4d, improves the brightness best, maintaining clarity and color in the darkest condition illustrated in Figure 3a.

Although the PSRs are known to have low-light environments, the lighting condition varies depending on the landforms and topological features. Therefore, we devised a second experiment to investigate the consistency of restoration in various illumination conditions. Dehaze and GladNet were selected due to their stable quantitative and qualitative performance in the first experiment. Figure 5 shows the image enhancement results for the low-light images in Figure 3. In Figure 5a, the image brightness of Dehaze is obviously proportional to the illumination level. However, in Figure 5b, GladNet tends to demonstrate relatively consistent color enhancements of ground and terrain objects, maintaining image quality. In addition, when enhanced images in 7.74 mWm⁻² and 23.54 mWm⁻² are compared, GladNet enhances low-light images with less noise. For this reason, we ultimately selected GladNet for the proposed robotic mapping method to reconstruct the emulated terrain in the test bed.

4.3. 3D Mapping Results

In the experiment for 3D mapping, the rover moved along a clockwise path around the soil bin and returned to its starting position. In the test bed, the LED lamp on the ceiling is not located at the center above the soil bin. The six locations from A to F show different illumination measures varying from 3.318 mWm⁻² to 22.199 mWm⁻² (Figure 6a). Under such dark conditions, without LLIE methods the proposed method failed to reconstruct a 3D terrain due to the insufficient number of feature matches. However, when combined with GladNet, the proposed method sequentially enhanced the stereo image pairs from the rover-mounted camera, resulting in a significant improvement in both quantity and quality of feature matches. Moreover, the restored color information combined with the depth value in the disparity map produced point clouds with consistent brightness and color (Figure 6b). The point cloud map has sparse density in the middle, and occlusions exist at nearby craters and rocks due to the low tilting angle of the stereo camera mounted on the rover’s mast. However, the colorized 3D points enable an enhanced understanding of morphological characteristics such as craters and mounds. Furthermore, rocks and pebbles with different colors are clearly identified, along with their respective sizes and shapes.

Also, for the accuracy assessment, the referenced point cloud from the 3D laser scanner (Trimble X7) was compared to the 3D point clouds from the proposed method. The Iterative Closest Point (ICP) method was employed to align the two sets of point clouds, and the Root Mean Square Error (RMSE) measure was then computed as 0.034 m. Meanwhile, the positional error distribution over the test bed (Figure 7a) was computed along with the positional error histogram (Figure 7b), using identical magnitude of a positional error ranging from blue to yellow-green. The RMSE measure and the positional error distribution provide a good indication of the overall performance of the proposed method, demonstrating its ability to obtain accurate and reliable point clouds in low illumination conditions. The positional error histogram also confirms that 85% of positional errors are less than 0.03 m.

5. Discussion and Conclusions

The discovery of water ice in the lunar PSRs has greatly increased their scientific importance and interest, leading to new plans for a robotic exploration mission to collect data on the distribution and abundance of water ice and other volatile compounds in the lunar soil. Although the PSRs are exposed to sunlight or starlight reflected diffusely from adjacent sunlit regions, the low illumination poses technical challenges for robotic mapping due to the restricted visibility range and the accumulation of errors over long-term operations. Although artificial lights could be helpful for the robotic mapping, uneven illumination and shadows nearby terrain features may have adverse effects on rover trajectory estimation and mapping results.

This research presents a robotic mapping method that leverages the LLIE method to improve the brightness and color of low-light terrain images. However, the lighting conditions in the PSRs can vary depending on the landforms and topological features. Therefore, in the experiment, three terrain images under different illumination conditions were obtained from an emulated PSR terrain. The performance of both conventional (CLAHE, Dehaze) and deep-learning-based (GladNet, KinD, MBLLEN) LLIE methods was evaluated to ensure image enhancement results in the darkest conditions as well as consistent color restoration across terrain images at different degrees of illumination. The experimental results demonstrated that all LLIE methods reveal hidden details and outlines of terrain objects in low-light images. However, the predefined mathematical models in the conventional methods have limitations in scalability for diverse low-lighting conditions. The excessive contrast enhancement in the CLAHE leads to the amplification of image details and intensity beyond the desired level. Although Dehaze enhances both color and contrast without significant degradation of image quality, the brightness of enhanced images varies depending on the illumination level. In contrast, the deep-learning-based methods are data-driven, requiring a paired training dataset to derive a learning process to find unknown structures or patterns in diverse illumination environments. In particular, GladNet significantly improves image quality metrics and flexibly enhances structural consistency, brightness, and color consistency of low-light images. In the experiment for 3D mapping, the proposed method without LLIE methods failed to reconstruct a 3D terrain. However, when combined with GladNet, the dense point cloud map was reconstructed with a more accurate natural appearance and inherent colors.

The experimental results show that the proposed method has potential for the robotic exploration of the lunar PSRs. However, since robotic exploration of the lunar PSRs has not yet been achieved, there are no direct measurements of illumination available. It is expected that the actual illuminance levels will vary significantly depending on factors such as topography, nearby terrain features, and the specific location within the lunar PSRs. Also, the illumination and terrain conditions in the testbed are limited to emulate the large-scale and complex environment of the PSRs. Therefore, as the knowledge of the lunar PSRs advances, the proposed method should be continually improved.