A 5K Efficient Low-Light Enhancement Model by Estimating Increment between Dark Image and Transmission Map Based on Local Maximum Color Value Prior

Abstract

Low-light enhancement (LLE) has seen significant advancements over decades, leading to substantial improvements in image quality that even surpass ground truth. However, these advancements have come with a downside as the models grew in size and complexity, losing their lightweight and real-time capabilities crucial for applications like surveillance, autonomous driving, smartphones, and unmanned aerial vehicles (UAVs). To address this challenge, we propose an exceptionally lightweight model with just around 5K parameters, which is capable of delivering high-quality LLE results. Our method focuses on estimating the incremental changes from dark images to transmission maps based on the low maximum color value prior, and we introduce a novel three-channel transmission map to capture more details and information compared to the traditional one-channel transmission map. This innovative design allows for more effective matching of incremental estimation results, enabling distinct transmission adjustments to be applied to the R, G, and B channels of the image. This streamlined approach ensures that our model remains lightweight, making it suitable for deployment on low-performance devices without compromising real-time performance. Our experiments confirm the effectiveness of our model, achieving high-quality LLE comparable to the IAT (local) model. Impressively, our model achieves this level of performance while utilizing only 0.512 GFLOPs and 4.7K parameters, representing just 39.1% of the GFLOPs and 23.5% of the parameters used by the IAT (local) model.

1. Introduction

From early humans seeking to detect predators in the darkness to modern-day applications like surveillance systems, autonomous vehicles, and smartphone cameras, the ability to see clearly in low-light environments remains a crucial technological challenge.

Despite significant progress in the development of LLE techniques, including classical algorithms [1,2,3], deep learning methods [4,5,6], and image generation techniques [7,8,9,10,11], several key challenges persist in current methodologies. These challenges revolve around achieving a balance between enhancement quality and real-time processing, optimizing models for lightweight deployment on resource-constrained devices, and ensuring the reliability of image enhancement results.

Classical algorithms are known for their exceptional reliability due to logical design principles. In contrast, deep learning models excel in efficiency and accuracy but may suffer from design redundancy. Our objective is to leverage the strengths of both approaches by developing a hybrid deep learning model that is efficient, lightweight, and produces high-quality results while ensuring reliability.

To achieve this goal, we focus on deconstructing the classical local maximum color value (LMCV) prior-based LLE algorithm [3]. We extract critical functions such as transmission map estimation, transmission value correction using guided filters, atmospheric light calculation, and adaptive exposure adjustment, which are all suitable for replacement with deep learning models.

We introduce a transmission estimation module (TEM) to estimate the increment between the dark image and the transmission map, thereby replacing the transmission map estimation function. The three-channel transmission map produced by TEM can preserve more details and intelligence while avoiding checkerboard artifacts, thus eliminating the need for the transmission value correction function. Additionally, a correction module is introduced to replace atmospheric light calculation and adaptive exposure adjustment functions. These functions were redefined as color correction and denoise layers within the correction module.

Finally, we created a novel and remarkably compact LLE model, and the experimental results show that our model achieves high-quality enhancements while utilizing only around 5K parameters and 0.512 GFLOPs. These numbers represent a significant reduction compared to other existing models, showcasing the efficiency and effectiveness of our proposed method.

Our main contributions can be summarized as follows:

• A novel method for estimating a three-channel transmission map by estimating the incremental values from the dark image to the transmission map.
• Introducing a lightweight U-Net structure transmission estimation module.
• Creation of a lightweight correction module for color correction and denoise.
• Proposing a joint loss function specifically designed for optimizing the training process of the low-light enhancement model.

2. Related Work

In the development of LLE, numerous physical models and algorithms have been proposed. The most fundamental physical theories include the Atmospheric Scattering Model (ASM) [12] and the Retinex model [13]. As for the latest deep learning algorithms, transformers [14] and AI-generated content (AIGC) [15] have been widely utilized.

2.1. Atmospheric Scattering-Based Model

The ASM is a fundamental physical framework for the LLE and defogging models. It elucidates the correlations between scene radiance $J\left(x\right)$, transmission map $t\left(x\right)$ and atmospheric light A in the source images $I\left(x\right)$ as depicted in Equation (1); x donates the pixel coordinates.

$$I\left(x\right)=J\left(x\right)t\left(x\right)+(1-t(x\left)\right)A$$

(1)

Several studies have integrated the ASM approach to developing the LLE model, including those by Wang et al. [1], Liu et al. [16], Jeon et al. [17] and Makwana et al. [18]. These endeavors aimed to estimate scene radiance $J\left(x\right)$ by determining the values of transmission $t\left(x\right)$ and atmospheric light A, resulting in satisfactory performance.

2.2. Retinex-Based Model

The Retinex-based model was proposed by Land et al. [13] in 1963. It represents another classical theory for the LLE model. The Retinex model decomposes the source image $I\left(x\right)$ into reflectance $R\left(x\right)$ and illumination $L\left(x\right)$, as shown in Equation (2).

$$I\left(x\right)=R\left(x\right)\circ L\left(x\right)$$

(2)

where ∘ denotes element-wise multiplication.

Consequently, to derive the reflectance $R\left(x\right)$ result, we can estimate the illumination $L\left(x\right)$. The Retinex can be viewed as a simplified ASM that integrates the estimation for atmospheric light A and transmitted light $t\left(x\right)$ into an illumination $L\left(x\right)$. The latest Retinex-based models include Zero-DCE++ [19], CSDNet [20], DeepUPE [21], MBLLEN [22], RetinexNet [23], RUAS [24], TBEFN [25], SGM [26], DRBN [27], and KinD++ [28].

2.3. Transformer-Based Model

The Transformer model has an exceptional capability to extract both global and local information, rendering it suitable for various tasks as a plug-and-play module. Transformer-based LLEs, such as Retinexformer [29], Uformer [30], Restormer [31], LLFormer [32], and LYT-Net [33] have demonstrated significant accuracy advantages. However, Transformers demand substantial computational and memory resources, which is unsuitable for low-performance devices. Further efforts are required to address the computational and memory requirements of Transformer-based models and make progress in lightweight models.

2.4. AI-Generated Content-Based Model

In 2022, AI-generated content [15] emerged as a prominent topic worldwide, leading to significant advancements in image generation. Numerous image generation models for LLE have been proposed, including LPDM [7], PyDiff [8], DiffLIE [9], L²DM [10], and Diff-Retinex [11]. These models achieved LLE by generating normal light images from dark images. They rely on a probabilistic distribution-based random mapping to mitigate limitations such as noise, color shifts, and information loss from the source image, resulting in a perfect enhancement of low-light images to ground truth (GT) images. However, it is essential to note that the generated images may not always be reliable, as generative models might introduce content not present in the source image. It posed challenges for tasks requiring content authenticity.

2.5. Challenges

LLE methods leverage physical models like the Atmospheric Scattering Model (ASM) and Retinex model, or deep learning techniques such as Transformer-based models and AI-generated content (AIGC) methods. ASM and Retinex models form the basis of many algorithms but suffer from complex calculations regarding transmission maps and atmospheric light. Transformer-based models offer high accuracy but demand significant computational resources, limiting their deployment on low-performance devices. AIGC methods show promise in image enhancement but require substantial computational overhead and may lack content authenticity, posing challenges for practical applications. Future research needs to address these challenges to develop efficient and reliable low-light image enhancement solutions suitable for various computing environments.

3. Methodology

3.1. Motivation

The Local Maximum Color Value (LMCV) algorithm was proposed by Dong et al. [3] in 2016. The LMCV algorithm stands as a remarkable classical LLE method due to its introduction of the LMCV prior concept based on well-exposed images. By combining the ASM model with the LMCV prior, the algorithm effectively enhances low-light images while also offering adaptive exposure adjustment for under-exposed regions. This innovative approach not only improves image quality but also demonstrates the adaptability of LLE methods to varying lighting conditions, making it a significant contribution to the field of image processing.

Firstly, they introduce a new term LMCV map $J^{LMCV}$ that represents a map where each pixel’s value is the local maximum color value within its neighborhood. Specifically, in an image processed using the LMCV algorithm, each pixel’s value in the LMCV map is derived from the highest color value found in its local area, which is represented by Equation (3).

$$J^{LMCV}\left(x\right)=\underset{c\in \{r,g,b\}}{max}\underset{y\in \Omega \left(x\right)}{max}{J}^c\left(y\right)$$

(3)

where x represents the pixel coordinates, J represents the scene radiance, c is the RGB channel and $\Omega \left(x\right)$ represents the points within a certain region centered at x. Specifically, in the LMCV algorithm, the region size is typically set to 15 × 15 pixels.

Moreover, through their statistical study, they observed that in an image with sufficient exposure, each pixel’s value in its LMCV map tends to approximate the highest value within the pixel’s range (see Figure 1). This observation is formally denoted as the LMCV prior, which is represented by Equation (4).

$$J^{LMCV}\left(x\right)\to 1$$

(4)

Therefore, when we apply the LMCV map operation to both sides of the ASM Equation (1), we obtain the modified Equation (5),

$$I^{LMCV}\left(x\right)=t\left(x\right)J^{LMCV}\left(x\right)+(1-t\left(x\right))\underset{c\in \{r,g,b\}}{max}{A}^c$$

(5)

where $I^{LMCV}\left(x\right)$ represents the LMCV map of the source image, $J^{LMCV}\left(x\right)$ represents the LMCV map of the scene radiance, and $t\left(x\right)$ represents the transmission map.

Based on the LMCV prior, we assume that the scene radiance $J\left(x\right)$ represents the well-exposed image we desire; then, we can set its LMCV map as $J^{LMCV}\left(x\right)=1$. Subsequently, Equation (5) can be simplified to Equation (6).

$$I^{LMCV}\left(x\right)=t\left(x\right)\times 1+(1-t\left(x\right))\underset{c\in \{r,g,b\}}{max}{A}^c$$

(6)

Consequently, as shown in Equation (7), the transmission map $t\left(x\right)$ can be considered as composed of $I^{LMCV}\left(x\right)$ and the atmospheric light A.

$$t\left(x\right)=\frac{I^{LMCV}\left(x\right)-{max}_{c\in \{r,g,b\}}{A}^c}{1-{max}_{c\in \{r,g,b\}}{A}^c}$$

(7)

Once we obtain the transmission map $t\left(x\right)$, we can solve for the scene radiance $J\left(x\right)$ based on the ASM model, as shown in Equation (8).

$$J\left(x\right)=\frac{I\left(x\right)-A}{t\left(x\right)}+A$$

(8)

In the original LMCV algorithm, a 15 × 15 kernel operation is employed to generate the LMCV map $I^{LMCV}$, while complex matrix element lookups are used to determine the atmospheric light A. Additionally, the algorithm relies on the highly computationally intensive guided filter to mitigate checkerboard artifacts resulting from the 15 × 15 kernel operation (as shown in Figure 2, middle). These operations contribute significantly to the overall complexity of computations within the LMCV algorithm. To streamline the algorithm and improve computational efficiency, it is imperative to simplify these operations or replace them with more efficient deep learning techniques.

In this paper, we introduce a novel method for estimating the transmission map called Dark Image Increment Estimation (DIIE). Furthermore, we propose two deep learning modules: the Transmission Estimation Module (TEM) and the Correction Module (CM). These methods and modules are specifically designed to replace the computationally intensive functions present in the original LMCV algorithm. By leveraging DIIE and these modules, we aim to enhance efficiency while either maintaining or improving accuracy in low-light image enhancement tasks.

3.2. Dark Image Increment Estimation (DIIE) for Transmission Map

In Section 3.1, we derive the transmission map $t\left(x\right)$ and scene radiance $J\left(x\right)$ from the dark source image $I\left(x\right)$, its LMCV map $I^{LMCV}\left(x\right)$, and the atmospheric light A, as shown in Equations (7) and (8). If we assume that any missing influence of atmospheric light A can be addressed through deep learning model adaptivity, by ignoring A in Equations (7) and (8), we can simplify them to Equations (9) and (10).

$$t\left(x\right)=I^{LMCV}\left(x\right)$$

(9)

$$J\left(x\right)=\frac{I\left(x\right)}{t\left(x\right)}$$

(10)

where $t\left(x\right)$ represents the simplified estimation of the transmission map and $J\left(x\right)$ refers to the scene radiance derived from our simplified LMCV algorithm.

Combining Equations (9) and (10) yields Equation (11), indicating that the scene radiance $J\left(x\right)$ depends solely on the dark source image $I\left(x\right)$ and its LMCV map $I^{LMCV}\left(x\right)$.

$$J\left(x\right)=\frac{I\left(x\right)}{I^{LMCV}\left(x\right)}$$

(11)

Additionally, considering the definition of the LMCV map as having pixel values representing maximum values in local areas, $I^{LMCV}\left(x\right)$ must be greater than or equal to $I\left(x\right)$ at any pixel point x, as per Equation (12).

$$I\left(x\right)\le I^{LMCV}\left(x\right)$$

(12)

Thus, we decompose $I^{LMCV}\left(x\right)$ into adding a positive increment $f\left(x\right)$ to $I\left(x\right)$ in Equation (13). Notably, we redefine both the LMCV map $I^{LMCV}\left(x\right)$ and the transmission map $t\left(x\right)$ as three-channel maps from Equation (13) onwards, aligning better with deep learning models and preserving image details and color information.

$$t\left(x\right)=I^{LMCV}\left(x\right)=I\left(x\right)+f\left(x\right)$$

(13)

Therefore, solving for scene radiance $J\left(x\right)$ involves finding the correct increment $f\left(x\right)$ as shown in Equation (14), where $J^{\prime }\left(x\right)$ is an intermediate variable for $J\left(x\right)$.

$$J^{\prime }\left(x\right)=\frac{I\left(x\right)}{I\left(x\right)+f\left(x\right)}$$

(14)

To address biases from removing atmospheric light A, we need to predict a correction $g(·)$ to obtain the final scene radiance $J\left(x\right)$, as shown in Equation (15).

$$J\left(x\right)=g\left(J^{\prime }\left(x\right)\right)$$

(15)

In summary, our proposed DIIE (Dark Image Increment Estimation) method simplifies the LMCV algorithm by focusing on predicting the increment $f\left(x\right)$ and performing atmospheric light correction $g(·)$. To achieve this, we introduce two deep learning modules: the Transmission Estimation Module (TEM) and the Correction Module (CM) to estimate them, respectively. Pseudocode Algorithm 1 provides an overview of our proposed model.

Algorithm 1 Overview of our proposed model
1: Input: I: Dark image: [C × H × W]
2: Output: E Enhanced image: [C × H × W]
3: procedure Model(I)
4:       $f\leftarrow TEM\left(I\right)$	▹ Estimates the increment f
5:       $t\leftarrow I+f$	▹ Gets 3-channel transmission map t
6:       $J\leftarrow I÷t$	▹ Gets scene radiance image J
7:       $E\leftarrow CM(I,J)$	▹ Generates the final enahnced image E
8:       return E
9: end procedure

3.3. Model Architecture: TEM and CM Module

We propose an LLE model based on our proposed DIIE method, comprising approximately 5K learnable parameters, as illustrated in Figure 3. The Transmission Estimation Module (TEM) is structured as a U-Net [34] model, focusing on estimating the increment $f\left(x\right)$ between the dark source image $I\left(x\right)$ and the three-channel transmission map $t\left(x\right)$ shown in Figure 2 (Right). On the other hand, the Correction Module (CM) adopts the skip connection [35] structure, which is utilized for color correction and noise reduction, compensating for the lack of atmospheric light A. Notably, our model maintains a consistent three-channel matrix output for most layers without increasing the number of channels in the matrices. This design choice significantly contributes to the model’s compactness and efficiency in terms of parameters.

3.3.1. Transmission Estimation Module

The Transmission Estimation Module (TEM) takes the dark source image I as input and is responsible for estimating the increment f. The sum of the increment f and the dark source image I yields the transmission map t. To meet this requirement, we propose a U-Net structure module for our TEM, comprising three parts: Encoder, Skip Block, and Decoder as shown in Figure 4.

• Encoder: does not contain any trainable parameters but includes three AvgPooling layers. These layers generate pooling images at 1/2, 1/4, and 1/8 scales of the original image. These pooling images will be fed into the Decoder.
• Skip Block: has a structure of conv3×3 + ResBlock + ResBlock + ResBlock + conv3×3 with inputs and outputs being 3-channel features. It is primarily responsible for extracting global features from the smallest 1/8 scale pooling image, which will be fed into the first layer of the Decoder.
• Decoder: contains most of our trainable parameters and consists of four T_UP layers. The structure of the T_UP layer is similar to the Skip Block, but we have added a Deconvolution layer for upscaling the pooling images from the Encoder, as shown in the T_UP diagram in Figure 5.

Based on the outputs of the Skip Block and Decoder in Figure 5, we find that the output of the T_UP 4 layer is the estimated increment f. At the fluorescent tube lights’ position in the image $x^{\prime }$, the increment value $f\left(x^{\prime }\right)$ tends to 0, while in the dark source image $I\left(x^{\prime }\right)$, the color at position $x^{\prime }$ should tend to 1. According to Equation (14), we find that the scene radiance $J\left(x^{\prime }\right)$ at this position is equal to $1/(1+0)=1$, maintaining the original brightness, which aligns with our desire.

Compared to the original transmission map, our estimated 3-channel transmission map addresses the checkerboard artifacts problem while also offering enhanced detail and color information, as depicted in Figure 2.

3.3.2. Correction Module

While the TEM can somewhat reduce the influence of atmospheric light due to the adaptability of deep learning models, the scene radiance image J outputted by TEM may still suffer from color biases and noise introduced by image enhancement. Therefore, we introduce the Correction Module (CM), which includes a color correction block and a denoising block, as shown in Figure 6.

• Color Correction Block: To adjust color biases, we concatenate the dark source image I and the scene radiance image J, forming a 6-channel feature metric $I\_J$. This combined feature metric $I\_J$ is then fed to the Color Correction block. The output of the Color Correction block is a 3-channel matrix c in the range of 0–1, and the result of $J\times c$ represents the color-corrected result.
• Denoise Block: Inspired by the Zero-Shot Noise2Noise model [36], we utilize two convolutional layers to estimate and subtract noise from the image, thereby producing a denoised image or, in other words, the final enhanced image E.

In summary, our proposed model, including Transmission Estimation Module (TEM) and Correction Module (CM), leverages deep learning techniques to estimate the increment $f\left(x\right)$ for the transmission map based on our DIIE method. Furthermore, it performs color correction and noise reduction for the final enhanced image E. This streamlined approach offers computational efficiency while maintaining high quality in low-light image enhancement tasks.

3.4. Joint Loss

To train the proposed model to achieve the best performance, we combine multiple loss functions, which include both full-reference and no-reference loss functions.

(1) 

Full-Reference Loss Functions

• Mean Absolute (L1) Loss. L1 Loss is a common loss function for various image-to-image tasks such as image reconstruction, image super-resolution and image denoise. It is an effective loss function as it directly computes the mean distance between the enhanced result and the ground truth. We propose L1 loss ${\mathcal{L}}_{{\ell }_1}$ to compare the final enhanced image and the ground truth data.

  $${\mathcal{L}}_{{\ell }_1}=\frac{1}{N}\sum _{x\in p}|E\left(x\right)-GT\left(x\right)|$$

  (16)

  where E denotes the final enhanced image, $GT$ denotes the ground truth data and x denote the pixel’s coordinate in p space. N is the total number of pixels.

The L1 loss measures the pixel-level difference between the enhanced image and the ground truth, offering a highly accurate match to the ground truth data. However, its high precision can sometimes lead to overfitting issues. To address this, we introduce the Root Mean Squared Log Error (RMSLE) loss.

• Root Mean Squared Log Error (RMSLE) loss. RMSLE loss utilizes the logarithm function based on the root mean squared error, which can reduce the impact of large differences between a few values and the ground truth in the overall error calculation. Thus, RMSLE loss ${\mathcal{L}}_{RMSLE}$ allows for localized small errors. We use RMSLE loss to measure the difference between the scene radiance $J\left(x\right)$ and the ground truth because scene radiance $J\left(x\right)$ is not the final enhanced image processed by the correction module. Therefore, we tolerate some errors in this comparison to overcome the overfitting.

  $${\mathcal{L}}_{RMSLE}=\sqrt{\frac{1}{N}{\displaystyle \sum _{x\in p}}{((logJ\left(x\right)+1)-log(GT\left(x\right)+1))}^2}$$

  (17)

While both the L1 and RMSLE loss functions operate at the pixel level and aim to minimize differences between pixels, they do not account for whether the enhanced image aligns with human visual perception. To enhance the visual quality of the enhanced image, we introduce the Structural Similarity (SSIM) loss.

• Structural Similarity (SSIM) loss. SSIM loss compares two images’ brightness, contrast, and structure, providing a better metric of human visual perception of image differences. We propose the SSIM loss ${\mathcal{L}}_{SSIM}$ to measure the loss between the enhanced image and the ground truth data.

  $${\mathcal{L}}_{SSIM}=1-SSIM(E,GT)$$

  (18)

(2) 

No-Reference Loss Functions

Additionally, to suppress noise and mitigate color biases, we introduce two no-reference functions.

• Illumination Smoothness Loss. The Illumination Smoothness Loss ${\mathcal{L}}_{tv}$ calculates the noise level of the enhanced image by identifying prominent points on smooth surfaces using gradient operations in both horizontal and vertical directions. Through statistical analysis of these prominent points, the loss function quantifies the noise level present in the enhanced image.

  $${\mathcal{L}}_{tv}=\frac{1}{N}\sum _{c\in \xi }\left(\right|{\nabla }_h{E}^c|+|{\nabla }_v{E}^c{\left|\right)}^2,\xi =\{R,G,B\}$$

  (19)

  where c denotes the color channel, while ${\nabla }_h$ and ${\nabla }_v$ denote the horizontal and vertical gradient operations.
  ${\mathcal{L}}_{tv}$ loss aims to minimize noise and irregularities in the output image, resulting in smoother images with fewer prominent noise points.

• Color Constancy Loss, based on the Gray-World color constancy hypothesis [37], states that the average intensity of any channel among the RGB channels should be gray. Therefore, we propose Color Constancy Loss ${\mathcal{L}}_{col}$ as a measure of whether the overall color of the enhanced image is correct.

  $${\mathcal{L}}_{col}=\sum _{\mathrm{v}(k,l)\in \epsilon }{(E_{avg}^k-E_{avg}^l)}^2,\epsilon =\{(R,G),(R,B),(G,B)\}$$

  (20)

  where $E_{avg}^k$ denotes the average intensity value of the k channel in the enhanced image and $E_{avg}^l$ denotes the average intensity value of the l channel in the enhanced image.
  ${\mathcal{L}}_{col}$ loss focuses on maintaining color balance across RGB channels. It ensures overall enhancement without color shifts, thereby improving the overall visual quality of the enhanced images.

The total loss in this paper can be represented as Equation (21); it is important to note that no special hyperparameters need to be set for the loss functions in the equation. The outcome of each loss function is solely dependent on the enhanced image and the ground truth image.

$${\mathcal{L}}_{total}={\mathcal{L}}_{{\ell }_1}+{\mathcal{L}}_{RMSLE}+{\mathcal{L}}_{SSIM}+{\mathcal{L}}_{tv}+{\mathcal{L}}_{col}$$

(21)

Within the total loss function, we utilize three full-reference loss functions (${\mathcal{L}}_{{\ell }_1}$, ${\mathcal{L}}_{RMSLE}$, ${\mathcal{L}}_{SSIM}$) and two reference-independent loss functions (${\mathcal{L}}_{tv}$, ${\mathcal{L}}_{col}$). ${\mathcal{L}}_{{\ell }_1}$ efficiently calculates the error of the enhanced image at the pixel level but may lead to overfitting due to its high precision. To address this, we introduce ${\mathcal{L}}_{RMSLE}$, which can tolerate local large errors and help mitigate overfitting issues. Furthermore, to ensure that the enhanced images align with high-quality human perception, we use ${\mathcal{L}}_{SSIM}$ to encourage brightness, contrast, and structural similarities between the enhanced image and high-quality ground truth images. To suppress noise amplification and correct color biases from post-enhancement, we employ ${\mathcal{L}}_{tv}$ and ${\mathcal{L}}_{col}$ functions in our final total loss function. The combination of these five loss functions contributes significantly to producing visually high-quality enhanced images while ensuring robustness. The individual impact of each loss function can be observed in Figure 7.

4. Experiments

4.1. Implementation Details

• Experimentation Platform: We implement our model on a Ubuntu 22.04 system equipped with an Intel Core i5-4690 3.50 GHz CPU from Intel corporation, Santa Clara, California, United States, an Nvidia GeForce RTX 3090 graphics card from Nvidia corporation, Santa Clara, California, United States, and 16 GB of memory from Samsung, Suwon-si, South Korea. The deep learning training framework used for the model was PyTorch 1.13.1.
• Training Hyperparameters: We employed the Adam optimizer with exponential decay rate parameters ${\beta }_1=0.9$ and ${\beta }_2=0.999$, setting the initial learning rate to $5\times {10}^{-5}$. Additionally, we employed the OneCycleLR learning rate scheduler to dynamically adjust the learning rate during training, and the training duration for each experiment consisted of 20 epochs.
• Datasets: Three different datasets were used to demonstrate the performance of our model in various low-light enhancement (LLE) tasks.

For indoor LLE tasks, the Low-Light (LOL) [23,27] dataset is used in this paper to prove our model’s performance. LOL has two versions: LOLV1 and LOLV2. LOLV1 consists of 485 training data and 15 test data with 400 × 600 resolution images, while LOLV2 comprises the real captured dataset and synthetic dataset. The real captured dataset includes 689 training data and 100 test data with 400 × 600 resolution. The synthetic dataset consists of 900 training data and 100 test data with 384 × 384 resolution.

For outdoor LLE tasks, we apply the MIT-Adobe FiveK (MIT5K) [38] dataset to test our model. The MIT5K dataset collected 5000 images, and each image was enhanced by A, B, C, D, and E experts using Adobe Lightroom. In our training, we only used expert C enhanced images as ground truth images. Following the settings in the paper [21], we split 4500 images for training and 500 images for testing.

For high-contrast image LLE tasks, we utilized the local color distributions prior (LCDP) [39] dataset. This dataset comprises 1415 training images, 100 validation images, and 218 test images. The LCDP dataset is particularly well suited for assessing a model’s ability to enhance images containing both overexposed and underexposed regions.

4.2. Performance Evaluation

Compared with real-time LLE state-of-the-art (SOTA) methods such as LIME [2], Zero-DCE [5] and IAT [6] on the LOL dataset, our method has a highly competitive performance in both enhancement quality and efficiency, as shown in

Table 1. Experimental results on LOL-V1 and LOL-V2-real datasets with other SOTA models in PSNR, SSIM, and efficiency metrics. The results highlight that our model achieves comparable performance to the state-of-the-art model while leveraging significantly fewer computational resources (0.512 GFLOPs) and parameters (4.7K), showcasing its efficiency.

Methods	LOL-V1		LOL-V2-Real		Efficiency
	PSNR ↑	SSIM ↑	PSNR ↑	SSIM ↑	FLOPs (G) ↓	#Params (M) ↓
LIME [2]	16.67	0.560	15.24	0.470	-	-
Zero-DCE [5]	14.83	0.531	14.32	0.511	2.53	0.08
RetiNexNet [23]	16.77	0.562	18.37	0.723	587.47	0.84
MBLLEN [22]	17.90	0.702	18.00	0.715	19.95	20.47
DRBN [27]	19.55	0.746	20.13	0.820	37.79	0.58
3D-LUT [40]	16.35	0.585	17.59	0.721	7.67	0.6
KIND [4]	20.86	0.790	19.74	0.761	356.72	8.16
UFormer [30]	16.36	0.771	18.82	0.771	12.00	5.29
IPT [41]	16.27	0.504	19.80	0.813	2087.35	115.63
MAXIM [42]	23.43	0.863	22.86	0.818	216.00	14.14
IAT [6]	23.38	0.809	23.50	0.824	1.44	0.09
IAT (local) [6]	20.20	0.782	20.30	0.789	1.31	0.02
Ours	19.4635	0.6144	20.6379	0.6878	0.512	0.0047

The first, second, and third best results are highlighted in red, green, and blue, respectively. The input image size is 400 × 600. ↑ represents higher is better. ↓ represents lower is better. # represents the number of.

. Our method achieves a high level of capability (high PSNR and SSIM) of LLE while just utilizing the minimum number of parameters and least FLOPs. While our method does not achieve the highest PSNR and SSIM values among real-time LLE models, its performance is comparable to SOTA models in terms of human perceptual experience, as shown in Figure 8.

From Table 1, we can observe that our method only utilizes 5K parameters to achieve the same performance for which some models require over a million parameters. Moreover, our model’s FLOPs is only 0.512 G, indicating that our model can execute real-time LLE of high-resolution images even on low-performance devices.

The LOL dataset is a well-known basic dataset for LLE tasks, but most of its scenes are indoors. In practical applications, we also need to enhance low-light images from outdoor scenes. The MIT-Adobe FiveK dataset includes a variety of dark images from outdoor environments. When we trained our model on this dataset, we achieved the results shown in

Table 2. Experimental results on MIT-Adobe FiveK [38] dataset. Quantitative comparison with other SOTA methods on outdoor LLE dataset.

Metric	White-Box [43]	DPED [44]	D-UPE [21]	3D-LUT [40]	IAT [6]	Ours
PSNR ↑	18.57	21.76	23.04	25.21	25.32	20.68
SSIM ↑	0.701	0.871	0.893	0.922	0.920	0.832
#Params (M) ↓	-	-	1.0	0.6	0.09	0.0047

The best results are highlighted in red. ↑ represents higher is better. ↓ represents lower is better. # represents the number of.

, and the results demonstrate that even for the outdoor dark images, our model can achieve a decent enhancement performance.

Enhancing the high-contrast images containing both overexposed and underexposed regions poses another significant challenge in LLE tasks. The LLE model must enhance the underexposed regions while reducing the brightness in overexposed areas. Remarkably, our model also achieved outstanding performance on the LCDP dataset (a dataset created specifically for such tasks), as shown in

Table 3. Experimental results on LCDP [39] dataset. Quantitative comparison with other SOTA methods on high-contrast image LLE dataset.

Metric	LIME [2]	RetinexNet [23]	Zero-DCE [5]	D-UPE [21]	LCDP [39]	Ours
PSNR ↑	17.335	19.250	12.587	20.970	23.239	20.016
SSIM ↑	0.686	0.704	0.653	0.818	0.842	0.793
#Params (M) ↓	-	0.84	0.08	1.0	0.28	0.0047

The best results are highlighted in red. ↑ represents higher is better. ↓ represents lower is better. # represents the number of.

.

In our performance evaluation, we compared our proposed method with SOTA LLE models on the LOL-V1 and LOL-V2-real datasets. Despite utilizing only 4.7K parameters and 0.512 G FLOPs, our method achieved competitive performance in terms of PSNR and SSIM metrics, showcasing its efficiency in real-time LLE tasks. Although our model did not achieve the highest PSNR and SSIM values among real-time LLE models, it demonstrated comparable performance to SOTA models in terms of human perceptual experience. Additionally, we evaluated our method on the MIT-Adobe FiveK dataset for outdoor scenes and observed promising results, highlighting the versatility of our model. Furthermore, our model showed excellent performance on the LCDP dataset for handling high-contrast images, surpassing some SOTA models in terms of PSNR and SSIM metrics while maintaining a significantly lower number of parameters (<0.005 M) compared to other methods. Overall, our method excels in achieving a balance between enhancement quality, computational efficiency, and reliability across various LLE tasks and datasets.

4.3. Ablation Study

In our study, we deployed five distinct loss functions and a correction module to enhance our model’s performance. Each component addresses specific aspects crucial for image enhancement, such as image smoothness, color balance, error tolerance, human perceptual alignment, and basic error measurement. Our goal was to evaluate the impact of each loss function and the correction module on the overall model performance through ablation experiments, as summarized in

Table 4. Ablation experiment results: impact of removing correction module or various loss functions on model performance. We observed that excluding either the correction module or individual loss functions leads to a degradation in our model’s performance.

Methods	LOL-V1		LOL-V2-Real
	PSNR ↑	SSIM ↑	PSNR ↑	SSIM ↑
w/o CM	17.8738	0.5816	18.6065	0.5916
w/o ${\mathcal{L}}_{{\ell }_1}$	18.8969	0.6440	19.5882	0.6877
w/o ${\mathcal{L}}_{SSIM}$	18.3100	0.5754	19.8165	0.6646
w/o ${\mathcal{L}}_{RMSLE}$	18.8289	0.6354	19.5118	0.6850
w/o ${\mathcal{L}}_{tv}$	19.2315	0.6125	18.7706	0.6238
w/o ${\mathcal{L}}_{col}$	18.3861	0.6227	20.0880	0.7022
Ours	19.4635	0.6144	20.6379	0.6878

The best results are highlighted in red. CM is Correction module. ↑ represents higher is better. ↓ represents lower is better.

.

Observing Table 4, we found that excluding either the correction module or individual loss functions led to a noticeable degradation in model performance across both LOL-V1 and LOL-V2-real versions. $\mathcal{L}\ell 1$, $\mathcal{L}SSIM$, $\mathcal{L}RMSLE$, and ${\mathcal{L}}_{col}$ contributed significantly to improving the PSNR and SSIM metrics in both datasets, showcasing their importance in enhancing image quality. A visual difference is displayed in Figure 9.

Our ablation study underscores the importance of selected loss functions and integrated correction module in our model. They are crucial in improving image enhancement quality, contributing collectively to overall model performance. However, further research and refinement in addressing persistent challenges like color bias and noise amplification are essential for advancing low-light image enhancement techniques.

5. Conclusions

To develop a lightweight, real-time, and efficient low-light enhancement (LLE) model applicable to low-performance devices such as surveillance systems, autonomous vehicles, UAVs, and smartphones, we innovatively combined traditional LMCV algorithms with deep learning techniques. We proposed the Dark Image Increment Estimation (DIIE) method, which estimates the incremental changes between dark images and the three-channel transmission map. Based on this method, we constructed a deep learning model with the Transmission Enhancement Module (TEM) and Correction Module (CM). Our model seamlessly integrates the lightweight and interpretability of traditional algorithms with the efficiency, adaptability, and robustness of deep learning methods.

Despite having only 39.1% and 23% of the parameters and computational load of the latest real-time SOTA model IAT (local), our model achieves a significant LLE performance. This advancement contributes significantly to providing high-quality real-time LLE models on performance-constrained devices.

While our contribution is substantial, our method still has some limitations. For instance, we only conducted qualitative analysis of the loss functions without quantitative analysis. Although there are no hyperparameters to adjust within each loss function, we can introduce weights for each loss function in the total loss function equation such as:

$${\mathcal{L}}_{total}=w_1{\mathcal{L}}_{{\ell }_1}+w_2{\mathcal{L}}_{RMSLE}+w_3{\mathcal{L}}_{SSIM}+w_4{\mathcal{L}}_{tv}+w_5{\mathcal{L}}_{col}$$

(22)

In our future work, we will perform quantitative analysis to determine the optimal combination of weights $w_1,w_2,w_3,w_4,$ and $w_5$.

In addition, our model still encounters notable challenges in terms of image color correction and denoising. Further refinement of the TEM and CM is necessary. Moreover, we have observed mutual impacts on learning efficiency between the TEM and CM modules during initial model training. In future work, we intend to train these modules independently to alleviate their negative interactions and enhance the overall model performance.