1. Introduction
Remote sensing plays a crucial role in understanding the Earth and its various systems. It provides a unique perspective, allowing us to observe and gather information about the planet from a distance. This technology has a wide range of applications, including environmental monitoring [
1], resource management, disaster response [
2], cropland expansion [
3], crop yield estimation [
4], and urban planning. The importance of remote sensing cannot be overstated as it provides crucial information for farmers to make decisions and plans, and address global challenges such as sustainability.
Additionally, Deep Convolutional Neural Networks (DCNNs) are a pivotal component of modern computer visions and have revolutionized the field with their remarkable ability to learn hierarchical representations of visual data. These networks consist of multiple convolutional and pooling layers, which are trained end-to-end on large datasets to extract high-level features from raw input images. The effectiveness of DCNNs lies in their capacity to learn and encode rich, discriminative features that capture both low-level and high-level information, thereby enabling robust and accurate performances in various vision tasks, such as image classification, object detection, and semantic segmentation. The importance of DCNNs stems from their wide range of applications in multiple domains, such as autonomous driving, medical imaging, image processing, and biometrics, among others [
5].
Remote sensing and DCNNs have been widely applied together in a variety of fields, and electrical energy is no exception. In recent years, the combination of these two technologies has shown great potential for improving the accuracy and efficiency of electrical energy monitoring and management [
6]. The use of DCNNs in the electrical energy field allows for improved automation and scalability, reducing the need for manual inspections and enabling more efficient large-scale monitoring.
Manual inspections for detecting non-technical losses in energy consumption on farms can be a challenging and time-consuming process. Non-technical losses refer to energy losses that are due to theft, meter tampering, billing errors, and other factors that are not related to the technical performance of the electrical system. To detect these losses, manual inspections typically involve physically visiting each farm, checking meters and electrical equipment, and interviewing farmers and other stakeholders. However, this process can be difficult and prone to errors, particularly in large and geographically dispersed farming operations, as in Brazil. Inspectors may face challenges such as harsh weather conditions, limited access to electrical equipment, and resistance by farmers. Furthermore, manual inspections are often labor-intensive and require significant resources, making them an expensive and inefficient method for detecting non-technical losses. As a result, there is a growing interest in exploring alternative methods for detecting non-technical losses, such as remote sensing and machine learning techniques.
The electrical energy consumption of crops is an important aspect of modern agriculture that can significantly impact the sustainability and efficiency of food production. Electricity is used for a variety of tasks on farms, such as powering irrigation pumping systems, heating and cooling greenhouses, operating farm machinery, and providing lighting. Especially in rice planting, irrigation plays a crucial role in plant growth and productivity. Therefore, understanding and managing the electrical energy consumption of crops is crucial for ensuring the long-term sustainability and competitiveness of agriculture [
7]. In particular, the development of energy-efficient technologies and practices, as well as the optimization of energy use through monitoring and control systems, can help to decrease energy costs, improve energy efficiency, and reduce the carbon footprint of agriculture [
8,
9,
10].
Irrigated rice crops play a vital role in the agroindustry of southern Brazil, especially in the Rio Grande do Sul State, where it accounts for 42.6% of the total production [
11]. With an increasing demand for rice, it is crucial for a power distribution company to gather precise and up-to-date information about the planted area to effectively plan and allocate resources. This ensures a reliable supply of energy to farmers for optimal crop growth and production. Moreover, accurate data on rice crop areas can offer valuable insights to optimize energy usage of pumping systems, minimize waste, identify non-technical losses, and support the transition towards a more sustainable agriculture industry [
12]. However, the development of specific methodologies that consider regional variations in rice cultivation practices and limited research on using remote sensing for monitoring electrical energy usage in irrigated rice crops indicate a need for further investigation in this field.
Remote sensing images collected from satellite-based platforms have become a valuable source of information for various applications in agriculture, including crop monitoring and analysis. The mapping of irrigated areas through remote sensing can essentially be divided into two approaches. The first one involves research that utilizes optical sensors, such as [
13,
14,
15,
16]. The second approach utilizes microwave sensors, which include [
17,
18,
19]. However, the majority of crop mapping studies are applied to areas that are small or have a very low resolution, as in the case of [
20].
The Sentinel-2 mission provides multi-spectral images with a high spatial and temporal resolution. Hence, this paper aims to explore these satellite images to estimate the electrical energy consumption in irrigated rice production. The study focuses on the processing and analysis of Sentinel-2 images using U-Net to extract relevant information for electrical consumption, and the development of algorithms to accurately estimate energy based on this information. The results of this study have implications for improving the efficiency of agricultural practices, reducing the carbon footprint, and supporting sustainable agriculture practices.
This paper is organized as follows: the first section describes the method, the second section presents the results and discussions, and the last section brings conclusions.
2. Materials and Methods
2.1. Study Area
The pilot study area, where the method was applied in a real-world scenario, is the municipality of Uruguaiana, highlighted in
Figure 1. It is a city located in the western region of the state of Rio Grande do Sul, Brazil. The municipality has an area of approximately 5700 square kilometers and a population of around 126,800 people [
21]. The region has a predominantly flat relief, with a few hills and valleys in the surrounding areas. The local economy is based on agriculture, with large plantations of rice, soybeans, and corn.
This area was chosen because it is relevant to the study, as it is the largest rice producer in the state of Rio Grande do Sul. In addition, the power energy distribution company provided all the necessary data from electricity customers. These data were crucial to enable the development and implementation of the proposed method and allowed a more accurate assessment of the electricity consumption patterns of the different rice production units, allowing the identification of areas of improvement and opportunities for energy savings.
2.2. Images
Sentinel-2 is an Earth observation mission developed by the European Space Agency (ESA) as part of the Copernicus Programme. Launched in 2015, Sentinel-2 is equipped with a multispectral imager that provides high-resolution optical images of the Earth’s land surfaces, coastal zones, and inland waterways. The main objective of the mission is to support a wide range of applications, such as land cover mapping, agricultural monitoring, forest management, and disaster response. The satellite captures images in 13 spectral bands, which allows for the identification and analysis of different types of vegetation, water bodies, and urban areas [
22].
The images captured by Sentinel-2 have a high resolution, with a spatial resolution of 10, 20, or 60 m, depending on the spectral band. This level of detail enables the detection of small changes in land cover and the identification of individual objects on the ground. The multispectral imager of the satellite captures data in the visible, near-infrared, and shortwave infrared regions of the electromagnetic spectrum, which makes it possible to distinguish between different types of vegetation, such as crops, forests, and grasslands. The images by Sentinel-2 are also used for monitoring water quality, tracking the spread of wildfires, and assessing the impact of natural disasters, such as floods and landslides.
The data generated by Sentinel-2 are freely available to the public, which makes it an important resource for scientific research and environmental monitoring. The mission is expected to have a significant impact on global efforts to address climate change, by providing accurate information on land use, deforestation, and carbon storage. The images captured by Sentinel-2 are also used for urban planning, infrastructure development, and natural resource management. As the mission continues to collect data, its images will provide valuable insights into the changes taking place on our planet, and help us to make more informed decisions about how to manage and protect the Earth’s resources.
Sentinel-2 has twin satellites in the same orbit but are phased as 180°, which allows 5 days of revisit frequency at the Equator. The data from Sentinel-2 is split along fixed-size scenes, 100 × 100 km
2, which are ortho-images in a UTM/WGS84 projection. The study region is covered by four scenes.
Figure 2 shows all four scenes. Images 1C and 2A are two of the data products generated by the mission, with 1C providing orthorectified top-of-atmosphere reflectance data and 2A providing atmospheric correction and bottom-of-atmosphere reflectance data. In this work, images 2A were used because they performed better with the neural network.
In the analyzed area, the cycle of the flooded rice fields usually starts in September and ends in March. It consists of four main stages: land preparation, seeding, growing, and harvesting. In the land preparation phase, the fields are plowed, leveled, and flooded with water, creating an environment suitable for the growth of rice plants. Then, the seeds are sown and the plants begins to grow; they are irrigated and fertilized throughout the growing phase. Harvest, usually performed by machines, takes place when the grains reach the ideal degree of ripeness and humidity, and rice is then processed and stored for commercialization.
To locate rice fields, January is considered the best month to acquire the images. This is because rice fields are at the vegetative stage during this month, which means that they can be better distinguished from other crops or land cover types [
23].
In the municipality of Uruguaiana, there are 445 electricity customers classified as irrigators. The local power distribution company uses this classification because these customers have a different tariff for irrigation activity [
24].
2.3. U-Net Topology
U-Net is a highly effective deep-learning architecture primarily used for image segmentation tasks [
25]. It was first introduced in 2015 and has since become a popular choice among researchers in various fields, including medical imaging, satellite imaging, and robotics. The architecture is symmetrical, with a contracting path and an expansive path that allow for both high-level context and low-level detail to be captured and processed. The U-Net structure is made up of multiple layers of convolutions, pooling, and up-sampling, with skip connections between the contracting and expansive paths to preserve spatial information from early layers.
One of the key advantages of the U-Net model is its ability to handle complex and highly variable images. By incorporating skip connections, the model is able to preserve fine-grained details that are often lost during the downsampling process of traditional CNNs. This makes it particularly useful for medical imaging applications, where accurate segmentation of tumors, organs, and other structures are critical for diagnosis and treatment planning.
The U-Net model was chosen for this study because it is a highly effective and versatile tool for image segmentation, with a fully convolutional network architecture, in which the input image and the output mask have a one-to-one correspondence. This facilitates the segmentation of satellite images, and consequently the identification of rice crops. Another characteristic is the encoder–decoder topology, as illustrated in
Figure 3, where z is a vector representing what the neural network identifies when compressing the input image x. Meanwhile, the decoding function y = f(z) represents the output y based on z. The vector z stores the semantic information that the neural network deems most crucial for predicting the output y. This feature reduces computational efforts as the image is reduced multiple times in the contraction stage to capture the context, achieved through convolutional and pooling layers.
The drawback of this topology is the loss of information owing to the loss of resolution in this stage. However, for this study, this is not relevant since there is no need to identify small objects. During the contraction stage, the neural network can identify patterns in the images but loses information about location. Reconstruction of the area where the main features were located is left to the expansion stage, which uses information from the previous stage and is concatenated with the same level between the encoder and decoder.
In the expansion stage, transposed convolution layers are used for image reconstruction. These characteristics make the U-Net a suitable topology for this study because a very large training dataset is not required. The ability of the U-Net to accurately segment images of rice crops using a limited number of training samples is a significant advantage in practice; therefore, it is a promising tool for the rice crop analysis and management.
To implement the U-Net topology for this study, the Python programming language was used along with the TensorFlow [
26] and Keras [
27] libraries. These libraries are popular for deep learning applications, as they provide a high-level interface for building and training neural networks. TensorFlow is a widely-used open-source platform for numerical computation and machine learning, while Keras is a user-friendly neural network library written in Python. The combination of these two libraries allowed for the efficient implementation of the U-Net topology, making it easier to build, train, and evaluate the model for rice crop segmentation. Additionally, Python’s extensive ecosystem of scientific computing libraries, such as NumPy and Matplotlib, enabled data manipulation and visualization tasks, providing a comprehensive framework for the study.
2.4. Images Preprocessing
Before inserting the images into the neural network, some preprocessing steps need to be performed. In this study, the images were first resized from 10,980 × 10,980 pixels to smaller images of 512 × 512 pixels. This resizing step divided each scene of 100 square kilometers into 1849 smaller images of 0.054 square kilometers. The size of each image was chosen based on the number of layers used by the U-Net neural network topology and the fact that it works with images of exponential multiples of two dimensions. In total, 11,094 images were used in this study, which were randomly divided into training and validation datasets before being fed into the neural network. The images used for testing the neural network consisted of 1849 images, which were manually selected to represent both classes proportionally.
To balance the dataset, the classification of each pixel was analyzed using the generated labels. It was found that the final dataset consisted of approximately 20% of pixels representing rice crops and 80% representing areas where no crops were present. Therefore, training methods that took into account this class imbalance were necessary.
2.5. Labels
One of the main challenges posed by Artificial Intelligence— AI applications—is the construction of the training dataset. The lack of available datasets also affects applications that utilize satellite images, as this is a recent area of AI utilization. The construction of a quality dataset requires significant effort, as it involves the collection, annotation, and preparation of large amounts of data. This process is further complicated in the case of satellite imagery owing to factors such as cloud cover, changing lighting conditions, and the presence of shadows. Addressing these challenges requires novel approaches to data collection and preparation, as well as the development of advanced algorithms capable of handling complex and diverse datasets [
28,
29].
In this study, the maximum likelihood classification method [
30] was employed in combination with visual photo interpretation by experts. The first method considers the weighting of average distances using static parameters. The training sets define the scatter plot of classes and their probability distributions, considering the normal probability distribution for each training class. Thus, the probability of each pixel belonging to a specific cultivar type is generated. Subsequently, experts classified the crops through photo interpretation.
Figure 4 displays the classified crops in two scenes in the year 2021. The yellow polygons represent rice crops, the purple ones show soybean crops, and the blue polygons indicate water bodies. These pre-labeled scenes were divided into 512 × 512 images and used to train the U-Net model.
These two scenes were selected for this study owing to the abundance of rice crops, which provided a substantial number of labels for training purposes. By selecting regions with a high concentration of rice cultivation, the dataset can be enriched with a greater diversity of rice plant types, growth stages, and environmental conditions, which are all important factors for accurately training machine learning algorithms to recognize and classify rice crops. As a result, these scenes were deemed suitable for acquiring a comprehensive and diverse dataset to aid in the development of more robust and accurate computer vision models for the rice crop analysis.
2.6. Metric
Selecting the appropriate evaluation metric to measure the performance of a neural network is essential [
31]. In this study, due to the imbalanced nature of the dataset, the choice of metric is even more critical. For example, in an image with only 10% rice fields, if the neural network fails to detect any of them, it would still have an accuracy of 90% if the metric were based on the number of pixels. In refs. [
32,
33], the authors presented a review of several popular metrics, including Precision, Recall, F-Measure, Area Under Curve, Intersection Over Union (IoU), Dice Coefficient, and Warping Error. Among these metrics, IoU emerged as the most reliable and precise for diverse applications.
Therefore, this study used the IoU [
33] method, which involves calculating the ratio of the intersection of the predicted segmentation and the ground truth label to the union of the predicted segmentation and the ground truth label. This index ranges from 0 to 100% and compensates for the unbalanced distribution between the rice fields and other areas in an image. The IoU metric has become increasingly popular in computer vision applications, particularly in object detection and segmentation tasks. It provides a more comprehensive evaluation of the network’s performance by taking into account both false positives and false negatives and it is robust to class imbalance issues.
2.7. Crop Allocation
The regional Power Distribution Company responsible for the study area has provided the geographic coordinates of the electricity meters of all farms and their corresponding billed energy. The region of study encompasses a total of 524 customers’ farms.
Allocating each crop to its corresponding meter posed a challenge since the metric of the smaller distance between them was not feasible, as illustrated in
Figure 5. The blue markers represent meters, and the light green regions represent the detected crops. The two crops shown in the figure belong to the meter highlighted in red. Notably, the distance between these crops is not smaller than the distance between the crops and the other meters. Furthermore, 15.07% of the meters have the same geographic coordinates as at least one other meter. This phenomenon mainly arises from the extensive distances between farms; as a result costumers share costs for energy infrastructure and metering points that are located at the same site.
To address the challenge of allocating crops to their corresponding meters, clusters had to be created. This was achieved by using the k-means method [
34], which involves partitioning data points into
k distinct clusters based on their proximity to the centroid of the cluster. Additionally, the database of the Rural Environmental Registry, known as the CAR in Brazil, was used. This is a nationwide electronic public record, mandatory for all rural properties, with the purpose of integrating environmental information from rural properties and possessions for control, monitoring, environmental and economic planning, and combat against deforestation.
The CAR requires farmers and landowners to provide detailed information about their properties, including location, area size, land use, and environmental conservation measures, among others. This database contains the geographical boundaries of rural properties, which are illustrated in
Figure 6, depicting the farms within the study region. The system enables better monitoring and enforcement of environmental regulations, and it serves as a tool for developing public policies for sustainable land use and rural development. Overall, the CAR is an essential instrument for advancing Brazil’s environmental and social sustainability goals.
2.8. Estimation of Energy Consumption
The methodology used in this work to estimate energy consumption in irrigated rice crops is presented in [
35]; the author studied the area and presents the average value for all the necessary parameters. The energy consumption is calculated using Equation (
1).
where
q is the Flow Rate, i.e., the quantity of water per time that is pumped to the crop. For this work, 1.5 L per second per hectare is used.
A is the Irrigated Area (m
2) and accounts for the total irrigated area of rice crops;
H (Head) represents the necessary energy to raise the water from the reservoir to the higher point of the crops. According to [
35], the average head in Uruguaiana is 10 m. The System Efficiency
η is defined as 65% by [
35], and lastly,
t corresponds to the time during which the pump remains working on average, which is 21 h per day and 100 days per crop period.
4. Discussion
The two groups of rice farms in
Table 3 and
Table 4 provide a comprehensive evaluation of the electricity consumption on the farms. The average of the study area is 578.86 kWh/ha. The results demonstrate that 26.5% of the clusters presented significant differences between the estimated energy and the amount that was billed. This indicates the need for further investigation.
These discrepancies in energy consumption may have occurred for various reasons, including the use of data provided by malfunctioning energy meters, theft of electricity, crops that have distributed energy generation, human error in data collection or entry, or variations in weather conditions affecting crop growth and water needs. Some farms in the dataset may have implemented energy-saving measures or crops that use gravity-based irrigation systems, resulting in a lower energy consumption than expected. Nonetheless, these potential factors only highlight the complexity of accurately estimating energy consumption in the agricultural sector.
Addressing these factors will be crucial to ensure the development of more reliable and accurate methods for estimating energy consumption, which can have significant implications for energy planning and sustainability in the agriculture industry. As such, future studies should consider a wider range of variables and potential factors that could impact energy consumption in agricultural settings. By utilizing these findings, electricity consumption can also be compared across clusters, thereby generating efficiency indicators that assist farmers in making informed decisions regarding the upgrade of their electrical equipment.
However, these results already provide valuable insights into the sustainable management of rice production systems and the electricity grid in southern Brazil. They can be used as input for an AI network in conjunction with other variables, such as meteorological data, soil characteristics, and productivity, in future studies.
The methodology developed in this study not only provides valuable insights for optimizing resource management in the context of the specific crop and region analyzed, but it also holds the potential for broader applications across various other crops and regions. By adapting and tailoring the framework to suit the unique characteristics and requirements of different agricultural systems, this methodology could serve as a versatile tool to enhance more efficient food production systems worldwide.
5. Conclusions
It was shown that the U-Net model can effectively segment satellite images to identify rice plantations, with a precision of 90% and an IoU of 68.84%. Then, the k-means algorithm was utilized to group the meters of customers farms and estimate electrical energy usage for irrigation purposes.
Comparing the estimated consumption with the energy billed by the utility, properties with very different patterns could be distinguished: in particular, five clusters with more than 500% less consumption than expected. This method can be a powerful tool for promoting more efficient electricity usage and reducing inspection costs, with applications in other fields as well. Additionally, these techniques can be applied for energy management and planning.
The findings of this study can be used to feed another AI system, along with new data such as meteorological information, soil data, and other relevant factors. This integration of additional data can further enhance the accuracy and refinement of the findings.