Self-Calibration of UAV Thermal Imagery Using Gradient Descent Algorithm
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThe authors propose self-calibration of UAV thermal images by minimizing the bias between overlapping thermal images using gradient descent optimization.
The topic is of interest and the results are promising. However, the paper must be improved before publication, as it seems to be submitted in a hurry, unfinished. The quality decrease as you advance reading and there are a lot of cross-references errors.
About the method, one of the major concerns is that it does not consider the elevation of the terrain. The flight altitude included in the EXIF metadata is relative to the take-off point, which is not the real altitude if the terrain is not flat. This can contribute to a significant drift error in the mosaic generation for most scenarios.
It also does not take into account the tilt of the camera. They assume nadir orientation, but, apart for the effect of wind, thermal flights often require some tilt to reduce reflections, depending on the time of the day.
Also, in the evaluated case, the temperature is constant along the river. But what if the temperature varies throughout the flight? The authors should discuss if this would affect the results, and how.
Certain statements should be justified by the relevant literature or eliminated, such as that the frequency of image acquisition is often 1 second, which actually depends on the application and many other factors. The limitations of current thermal libraries should also be referenced.
About the structure of the document, I think it would be clearer if the results were discussed at the time they are presented, not in a separate section. And then adding a more elaborated "Conclusions" (or "Discussion and conclusions") section.
The nomenclature is fine, but the description is not. See for example "p1st" in line 212 and "p1st^" in line 232, they are described exactly the same, but they are not the same. Different expressions are used to refer to the values estimated by computing vision and the values estimated by georeferencing. I suggest choosing (and clearly defining) a term and always using the same words to refer to the same concepts. The authors should avoid naming them differently in each paragraph, because it is very confusing.
I would suggest simplifying "The root of the mean of the square..." in line 20 to just "The root mean square error (RMSE)".
I would also recommend the use of "about" instead of "ca."
Author Response
The authors would like to thank all reviewers for their valuable and detailed comments, which were taken into account in the new version of the manuscript.
We believe that they allowed us to take a critical look at the content of the article again and improve its quality.
Detailed responses to individual comments are provided in attached document.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsDear Sir/Madam
I hereby give the Author/Authors some comments and suggestions on “Self-calibration of UAV Thermal Imagery Using Gradient Descent Algorithm.”This paper presents an appropriate method to minimize unstable temperature variations that negatively affect UAV thermal imagery. In this regard, the paper is very useful to utilize the idea and result in UAV thermal image applications.
However it is not suitable for publication in its present format due to critical faults: missing conclusion, the unsystematic structure of the paper, and insufficient results analysis, etc. These must be corrected before the article is published. To help this, I give some revision recommendations attached on separate pages.
Best regards.
Reviewer
General comments:
1. Conclusion is missed Please, add chapter 5. Conclusion after discussion. 2. Editing errors in Lines 277 and 339 Please, make sentences completely. Additional comments:
Line 28: Definition of self-calibration I think it would be greatly helpful in understanding the paper if the definition and meaning of self-calibration were explained somewhere in Chapter 1.
Line 327: Recognition of river in image One cannot easily recognize the river on the left edge in Figure 5.
Line 343: 3.2 Visual assessment, Fig. 7-11 1) There is no title on the abscissa in all pictures. 2) What did you mean the sentence: Where the images overlap, …?. I think this means that the georeferenced images were simply connected in order. 3) A more detailed explanation of the images, including the characteristics and differences between these pictures, should be added. Line 364: Table 2 The unit is missed in Table 2. Line 372: Figure 12 What does the thick dashed line in the middle of Figures represent? Please explain in more detail the characteristics of the results shown in Figures.
Line 427: Appendix A Since Formula A1 is redundant, it would be a good idea to consider merging the appendix into the main text. Sub-title georeferencing is described twice.
–The end-
Comments on the Quality of English Language
Author Response
The authors would like to thank all reviewers for their valuable and detailed comments, which were taken into account in the new version of the manuscript.
We believe that they allowed us to take a critical look at the content of the article again and improve its quality.
Detailed responses to individual comments are provided in attached document.
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for AuthorsThe authors present an interesting methodological proposal for the calibration of thermal images obtained with UAV. The research proposes a methodology for post-processing of UAV thermal images to eliminate unwanted effects without reference or complementary data. The paper allows to demonstrate the calibration of the images accuracy by removing the vignette effect, favoring the image visualization, and its possible subsequent application for the generation of orthoimages. The results obtained allow demonstrating the goodness of the proposed methodology.
The article begins with an entertaining and easy to read introduction, however, the structure of the article makes us lose the thread of the research. The research presented lacks depth, in addition to not validating the methodology by means of an objective and real contrast to verify the hypothesis. On the other hand, the calibration seems only focused on eliminating the vignette effect, not correcting the temperature measurement errors, which as the paper points out for UAV thermal images is ±5ºC. Therefore, the title presented seems ambitious in comparison with the scope of the research carried out. As it stands, the manuscript fits more as a technical note than as an article. The idea to be developed shows potency, however, the document requires certain modifications to be published.
The structure of the article should be revised, as it lacks fundamental headings. First of all, the introduction with subdivisions does not allow following the research thread. The Gradient descent algorithm is introduced for the first time as an epigraph, without any relation or justification of its suitability to solve the problem posed. On the other hand, the paper lacks conclusions. The discussion is aimed at debating and analyzing the results, but does not allow highlighting the main contributions of the research. It would be advisable to add a section on conclusions.
The introduction is correctly developed to contextualize the uncooled thermal sensors for UAVs supported by references, however the Gradient descent algorithm (Lines 82-98) requires references.
"We assume that images with vignette effect occurring contain correct temperature in their central part." (Lines 155-156). This statement is controversial. The document states that the temperature can have deviations of ±5ºC, so the statement cannot be assumed. It would be more appropriate to state that images with vignette effect occurring contain more accurate temperature in their central part than in the periphery of the TIR image.
Georeferencing" may not be the best alternative when using dual sensors (RGB+TIR) or multisensors such as those proposed and used in the document. The application of image correlation algorithms such as SIFT with thermal images has lower accuracy than when using RGB images, due to the lower resolution, contrast and sharpness of thermal images. It may be possible to obtain more accurate results using Ground Control Points (GCP) such as those used in photogrammetry with RGB imagery, although benchmarking seems to be outside the scope of the research. This type of problems related to the georeferencing of thermal images is addressed in other researches, which propose to orient the thermal images based on the RGB images taken at the same instant by the sensor. In the case of the present investigation, this alternative could be implemented by using a multisensor with an RGB camera with a resolution of 5184×3888 px.
The expression "well-defined emissivity value of 0.95" (Lines 312) should be justified because it is wel-defined, this is the emissivity of the water sheet, of the vegetation of the channel or the average emissivity. It may help to reference it to other studies that define the emissivity of rivers or to articles that use thermal images of rivers and use the same value.
The expression "well-defined emissivity value of 0.95" (Lines 312) should be justified because it is wel-defined, this is the emissivity of the water sheet, of the vegetation of the riverbed or the average emissivity. It may help to reference it to other studies that define the emissivity of rivers or to articles that use thermal images of rivers and use the same value.
The idea of correcting or calibrating the temperature of rivers in the ortho mosaic is interesting. This adjustment is done before or after generating the orthomosaic, since it allows not only to eliminate the vignette effect of the images but also to improve the thermal precision and reduce the error of ±5ºC that is indicated for this type of images. Depending on when it is applied, it may or may not be included as part of the proposed methodology for UAV thermal image calibration. Lines 313-318.
The results obtained are validated by visual analysis of thermal image mosaics showing apparent and colorful results, which indicates the suitability of the methodology for pre-processing thermal images for photogrammetric applications, by eliminating the vignette effect.
The item "3.3. Waterbody temperature" related to the accuracy of the sensor measurements does not clearly represent the results of the pressure enhancement in the temperature averaging. What is the final error or accuracy of the calibrated images? Perhaps the results need to be validated in a more objective and easier to understand way than the proposal made in point 3.3.
The number of references is limited. A greater number of references would provide context and support the veracity of the research
Other more specific commentaries to take into account:
Line 164: "in our method". Avoid writing in the third person. It would be more appropriate: "in the proposed method".
Line 167: what means EXIF, define acronyms.
The aliases defined in table 1, such as "20211215_kocinka_rybna" are difficult to identify or memorize in order to understand the subsequent results. Renaming the aliases of the different surveys would allow a better understanding by the reader.
Line 405: "our proposed solution". Avoid writing in the third person. It would be more appropriate: "the solution proposed in this research".
In view of the whole article, the proposed method presents promising results to calibrate images, but the results seem to be focused on the elimination of the vignette effect and not on the global calibration of the thermal images in the temperature measurement. The objective and title may be misleading based on the way the manuscript is written.
Author Response
The authors would like to thank all reviewers for their valuable and detailed comments, which were taken into account in the new version of the manuscript.
We believe that they allowed us to take a critical look at the content of the article again and improve its quality.
Detailed responses to individual comments are provided in attached document.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsThe paper has been significantly improved.
Author Response
Dear Reviewer,
Thank you very much for the positive final decision.
with best regards,
Authors
Reviewer 2 Report
Comments and Suggestions for AuthorsDear authors,
I appreciate your extensive revision of the paper. I recommend that you discuss the following matter with editors before publication.
Please discuss with editors why an additional author was added during the review and whether this does not violate publishing regulations.
Best regards,
Reviewer
Author Response
Dear Reviewer,
Before the decision related to change in the list of authors, first we discussed the possibility with the editor. We found that there is an official procedure for that, and according to the information received, we prepared the 'Journal-Change of Authorship Form' available in the MDPI system which was accepted and signed by all co-authors. We also specified the reason of the extension and described the contribution of each co-author in the form and manuscript.
with best regards,
Authors
Reviewer 3 Report
Comments and Suggestions for AuthorsDear authors;
The authors have well revised and enhanced the quality of the article. In general, the structure of the manuscript has improved, as well as the definition of the research conducted and the presentation of the results. Compared with the previous one, this resubmitted version made some improvements but a couple of points should be clarified to improve the potential publication.
Regarding the "Landmark referencing" point, the proposal is very simple and a more robust method needs to be established. It is indicated that "at least one ground based reference point" (Line 129) and "at least one temperature reference point 158 measured directly during the flight mission" (Line 132). It would be recommended to take the temperature in at least 3 points as in other cases [1] [2]. In this way it can be corrected avoiding errors in the ground truth temperature measurement. Based on the interpretation of the paper "The water temperature of the rivers was measured directly using thermocouple along their course in each case" (Lines151-152), these data are available. As the river temperature is uniform, it does not affect the results, but it may be relevant in other cases of application.
[1] Kelly, J.; Kljun, N.; Olsson, P.O.; Mihai, L.; Liljeblad, B.;Weslien, P.; Klemedtsson, L.; Eklundh, L. Challenges and Best Practices for Deriving Temperature Data from an Uncalibrated UAV Thermal Infrared Camera. Remote Sens. 2019, 11, 567.
[2] J. Sedano-Cibrián, R. Pérez-Álvarez, J. M. de Luis-Ruiz, R. Pereda-García, and B. R. Salas-Menocal, “Thermal Water Prospection with UAV, Low-Cost Sensors and GIS. Application to the Case of La Hermida” Sensors, vol. 22, no. 18, 2022, doi: 10.3390/s22186756.
In Table 4, the flight time of case E is missing to homogenize the data.
In section 2.2.1. Debignetting algorithm, it is indicated that the value of each pixel is converted to kelvin to eliminate negative values. Sometimes TIR images store for each pixel conversion values related to temperature (K). Since these values are not usually negative there would not be such a problem, however, they are high values above the conventional 255. Does the algorithm consider that kind of data? Todos el estuido areas planas
The more general Conclusions do not focus on the concrete results of the case studies since it seems to lose meaning and instead of an article it is a research note.
Line 343 a point to be deleted.
Finally, with regard to the previous replies. The document states "One of the intentions of the developed method was that it should be accessible to the widest possible range of users, even those who do not have a dual camera system".
However, it must be taken into account that the vast majority of commercial thermal, multispectral or hyperspectral sensors currently have a visual sensor to help interpret the images. In spite of this, the proposal developed is interesting as it does not require such information and can simplify the process, although it does not achieve such accurate results.
Author Response
Dear Reviewer,
Thank you very much for the positive feedback and valuable comments in both steps. We went trough your remarks and modified the text in in the indicated places. We also rearranged the conclusions section to highlight more general conclusions of the article which are not directly concerning the study site.
Regarding the "Landmark referencing" point, the proposal is very simple and a more robust method needs to be established. It is indicated that "at least one ground based reference point" (Line 129) and "at least one temperature reference point 158 measured directly during the flight mission" (Line 132). It would be recommended to take the temperature in at least 3 points as in other cases [1] [2]. In this way it can be corrected avoiding errors in the ground truth temperature measurement. Based on the interpretation of the paper "The water temperature of the rivers was measured directly using thermocouple along their course in each case" (Lines151-152), these data are available. As the river temperature is uniform, it does not affect the results, but it may be relevant in other cases of application.
[1] Kelly, J.; Kljun, N.; Olsson, P.O.; Mihai, L.; Liljeblad, B.;Weslien, P.; Klemedtsson, L.; Eklundh, L. Challenges and Best Practices for Deriving Temperature Data from an Uncalibrated UAV Thermal Infrared Camera. Remote Sens. 2019, 11, 567.
[2] J. Sedano-Cibrián, R. Pérez-Álvarez, J. M. de Luis-Ruiz, R. Pereda-García, and B. R. Salas-Menocal, “Thermal Water Prospection with UAV, Low-Cost Sensors and GIS. Application to the Case of La Hermida” Sensors, vol. 22, no. 18, 2022, doi: 10.3390/s22186756.
Ans: We agree with the reviewer's comment on “landmark referencing”. We have included the proposed citations and updated the text around the lines 130 and 350.
In Table 4, the flight time of case E is missing to homogenize the data.
Ans:The table has been updated.
In section 2.2.1. Debignetting algorithm, it is indicated that the value of each pixel is converted to kelvin to eliminate negative values. Sometimes TIR images store for each pixel conversion values related to temperature (K). Since these values are not usually negative there would not be such a problem, however, they are high values above the conventional 255. Does the algorithm consider that kind of data? Todos el estuido areas planas
Ans: This is one of the problems related to selected camera brands. In case of DJI there is no clear description of R-JPEG contents, and the only way is to use DJI SDK library to convert raw R-JPEG to TIFF containing the temperatures assigned to each pixel. In such case the conversion of the temperatures to Kelvin scale eliminates the problem with negative values. The method proposed in the article is working with any floating point positive values (also >255).
The more general Conclusions do not focus on the concrete results of the case studies since it seems to lose meaning and instead of an article it is a research note.
Ans:The conclusions have been rewritten according to the suggestion.
Line 343 a point to be deleted.
Ans: Commas have been removed from all equations (in accordance with English punctuation).
Finally, with regard to the previous replies. The document states "One of the intentions of the developed method was that it should be accessible to the widest possible range of users, even those who do not have a dual camera system".
However, it must be taken into account that the vast majority of commercial thermal, multispectral or hyperspectral sensors currently have a visual sensor to help interpret the images. In spite of this, the proposal developed is interesting as it does not require such information and can simplify the process, although it does not achieve such accurate results.
Ans: We agree with the reviewer's comment, however we would like to stress that not all UAV users have dual cameras. For those who have, use of higher resolution RGB pictures can help in better positioning of the pictures and additional landmark referencing serves to enhance the temperature reading accuracy. The text has been modified.
