The Uncertainty of SNO Cross-Calibration for Satellite Infrared Channels

Round 1

Reviewer 1 Report (New Reviewer)

General comments:

 The manuscript titled "The Uncertainty of SNO Cross-Calibration for Satellite Infrared 2 Channels" addresses an important topic in the field of infrared Earth observation (EO) instruments. The authors tackle the challenging task of evaluating uncertainty within the SNO approach. While the study offers valuable insights into this area, I have some concerns regarding the readability and structure of the manuscript that need to be addressed. It would be beneficial for the authors to structure their ideas in a more coherent and accessible manner. The authors should consider reorganizing the content, ensuring that each section seamlessly leads to the next

 

Specific comments:

 

·       Abstract: Abstract is difficult to follow. Please rewrite. More specifically:

o   “corresponding matching thresholds are selected to introduce errors into the calibration results.“ > thresholds are not used to introduce errors ? Please reformulate

 

o   “In this study, we propose to analysis from two aspects, namely, sample matching error and sample fitting  method error.” Sentence is not correct

 

o   “error transmission” Is error propagation not a better term ?

 

·       Section 3.2: This section is very difficult to follow. It is not clear why this discussion on PSF is needed. What is the purpose of this section? Please introduce section with a clear explanation of the purpose

·       Line 72-73 “using simultaneous observations over common targets,  such as the sun or Mars.“  Why Sun, Mars ?? For EO satellites SNO is done over earth target.

 

·       Line 125-129 “Finally, there are uncertainties in the reference data used for cross-calibration. Reference data include ground-based measurements, laboratory experiments, and radiative  transfer models. Each of these has its own sources of uncertainty, which can propagate to the cross-calibration process” These are needed for SNO. Why do they influence the SNO uncertainty?

·       Line 154-155:  All tests need to be explained as they are an essential part of the uncertainty analysis

·       Figure 1 “Spectral conversion” is not mentioned in the text. Please explain. Why is it not part of the uncertainty?

·       Line 167 “The above analysis shows that” Which analysis? No analysis has been done in previous section?

·       Line 167 “sample matching and fitting methods… major sources’ This is not clear from text/figure . “Fitting method” not used in figure. Use same terminology.

·       Line 193 “The uncertainty of the cross-calibration results consists of two parts. One part is the  calibration accuracy or measurement noise of the reference instrument, and the other part is the error of the SNO cross-calibration method itself”  This is in contradiction/not in line with  with line 167 :“The above analysis shows that the sample matching and fitting methods during the  cross-calibration process are major sources of errors”

·       Table 2 Difficult to compare both satellites if different units are used. Please convert to micrometer.

·       Figure 3 legend should be clearer, or explained

·       Line 302 Please explain what is meant with target area and background area

·       Line 381 “Analysis of the data matching uncertainty was conducted mainly from temporal

·       matching, spatial matching, angular matching, and spectral matching, where the sensitivity analysis with respect to these four factors was important for the uncertainty assessment.” Sentence is unclear.

·       Line 445 “hyperspectrum instrument” > Hyperspectral instrument

Author Response

Thank you for reviewing our manuscript. We understand your concerns about the readability and structure of your manuscript. We agree that it is critical to present the content in a coherent and understandable manner. In response to your feedback, we have made changes to the phrasing and section articulation sections of the manuscript. In addition, this paper has been linguistically repaired through the official MDPI language editing service. Please see the highlighted section of the manuscript for specific details of the changes.

Author Response File: Author Response.docx

Reviewer 2 Report (New Reviewer)

This manuscript focuses on uncertainty analysis of the cross-calibration. Uncertainty is the key to realize the traceability, which is relating the result to a stated metrological reference through an unbroken chain of calibrations of a measuring system or comparisons. The premise of the uncertainty analysis is to construct a complete cross-calibration chain. There are still some technical drawbacks.

Firstly, the article can focus mainly on the uncertainty of the cross-calibration model, but the measurement noise of the reference instrument should be taken into account, and this is also a factor that cannot be ignored.

Secondly, the methodology requires further consideration, such as that whether there is a correlation between different factors, whether the same factor affects differently in different wavelengths, and how to modeling it mathematically. So far as I know, atmospheric conditions change over time and also affect the observed values from different angles. The method described in the article is not innovative enough.

Thirdly, the article uses FY-3D\HIRAS, FY-3D\MERSI as cases, if there could be some cross-calibration results to demonstrate the correctness of the method.

Lastly, some of the descriptions in the article are not scientifically rigorous. In line 89-97. In my opinion, the noise of the measurements should be a kind of random uncertainty, and the method uncertainty that always shows biases may be more likely a kind of systematic uncertainty. And I do not understand what’s the meaning that the proposed method “can avoid the measurement noise”. For the noise, I think, is inherent in all of the measurement. In line 119. Generally, the signal-to-noise ratio is higher at high temperature.

Furthermore, there are still lots of linguistic errors in the article. For example, in line 50. If the “high points” refers to the “GaoFen (High Resolution)” series satellites? In line 110-111. Is the abbreviation “SNO” also for “Satellite Observations in Support of Climate Monitoring”? In line 332, there is an incomplete sentence.

• Language needs to be improved.

Author Response

Thank you for reviewing our manuscript. The SNO cross-calibration method is currently widely applied to evaluate the radiometric calibrations accuracy for the infrared channel of meteorological satellites. However, the uncertainty of the SNO cross-calibration method itself has rarely been assessed. In this manuscript, a set of generalized models and methods are proposed which are applicable for the uncertainty assessment of commonly used meteorological satellite remote sensing instruments using the SNO cross-calibration method. This study can provide reliable basic data support for climate application work. The satellite cross-calibration method is important for climate studies, where long-term observations of subtle changes in the Earth's atmosphere, oceans, and land surface are needed. It also helps to identify and reduce instrumental biases and uncertainties, thus improving the accuracy and reliability of the data. We have made changes to the phrasing and section articulation sections of the manuscript. In addition, this paper has been linguistically repaired through the official MDPI language editing service. Please see the highlighted section of the manuscript for specific details of the changes.

Author Response File: Author Response.docx

Reviewer 3 Report (New Reviewer)

This paper intends to analyse the uncertainty of the SNO cross-calibration method, briefly giving a description of the relevant method and experimental results, but the version is poorly readable and logical, the advantages and characteristics of the relevant method are not elaborated, but only a general talk about the importance, and it is recommended that it be re-invested after a major overhaul.

Some minor specific comments are as follows:

1. onboard generally refers to on-board calibrations, but on-orbit calibrations are generally used on-orbit;

2, SNO cross-calibration error transfer model, tested with HIRAS and MERSI, why suddenly appeared a "two-dimensional Gaussian" simulation of PSF?

3、in-orbit onboard calibration  in-orbit vicarious calibration,  the 'in-orbit' can be removed.

4. The last two sentences of the first paragraph are nonsense, and the logic is completely irrelevant to the descriptions before and after, so delete them.

5. lines 69-78 This paragraph is useless and the logic is totally incompatible; suggest deleting it.

6、Lines 79-80 say that a number of studies have now evaluated the accuracy achieved by the SNO method, but from line 86 onwards it says that few studies have done an assessment of the uncertainty in the calibration of the SNO itself; from the conceptual theory the two should be the same, but the authors argue that they are not, but do not give a reason for this, suggesting a more detailed description.

 

7、Common sense dictates that the uncertainty of the method itself should already be included in the total uncertainty of the calibration results. What is the point of studying the uncertainty of the method itself when the uncertainty of the final result can already be given? If we have to study it, we suggest that the topic should be the SNO cross-calibration method itself, rather than cross-calibration.

 

8、Introduction, method Poorly written logic; for example, the introduction is generally written: what is the background (importance), what is the method, what is the problem, what is the impact of not solving, and finally what this paper is prepared to solve.

 

9、The passage in lines 167-170 is unfounded, and which analysis leads to this conclusion is too far-fetched and not clearly written.

 

10、The PSF is a very important part of the paper, and it is recommended that the overall approach be clearly explained.

minor editing of English language required

Author Response

First of all, thank you for reviewing our manuscript, we appreciate your feedback on the readability and logical structure of the manuscript. We apologize for the lack of clarity in the current version, which does not adequately highlight the advantages and features of assessing the SNO cross-calibration method's own uncertainty and generic model. We understand your suggestion for a major revision to address these issues. In order to improve the manuscript, we have carefully reevaluated and revised the content to improve readability, logical flow, and to provide a more comprehensive description of the method and its advantages. Your feedback has been valuable and we appreciate your suggestions for improvement. We are committed to providing a revised version of the manuscript that meets the highest standards of quality and clarity. Thank you for bringing these issues to our attention, and we have worked diligently to revise and address some of them. In addition, the paper has been revised to improve the language by using MDPI language editing services.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report (New Reviewer)

The authors have satisfactorily addressed most of my concerns. 

Author Response

Thank you for reviewing our manuscript.  We appreciate your positive evaluation of our work. Our deepest gratitude goes to you for your careful work and thoughtful suggestions that have helped improve this paper substantially. And we have reviewed the text and logical structure of the entire paper, the changes were highlighted in the revised manuscript.

Reviewer 2 Report (New Reviewer)

This paper brings some valuable informations on uncertainties in cross-calibration procedure. The main issues in this manuscript have been explained and revised. 

The author should still conduct a full text review to avoid errors in writting.

Author Response

Thank you for reviewing our manuscript.  We appreciate your positive evaluation of our work. Our deepest gratitude goes to you for your careful work and thoughtful suggestions that have helped improve this paper substantially. And we have reviewed the text and logical structure of the entire paper, the changes were highlighted in the revised manuscript.

Reviewer 3 Report (New Reviewer)

This paper intends to analyse or quantify the uncertainty of the SNO cross-calibration method in a more detailed way. This version is more logical and readable than the previous one, but there are still some issues that need attention, especially whether the PSF simulations are representative and whether the conclusions can give some meaningful quantified conclusions. The other commetns are as follows:

1. The order of the authors of the two editions has been adjusted, what are the considerations?

2. the first sentence of abstract ‘The on-orbit calibration of remote sensing ....’, change into ‘on-orbit radiometric calibration’  may be more appropriate. 

3.  “Since the spatial resolution of HIRAS is lower than that of MERSI, we introduced the point spread function (PSF) to simulate the MERSI simulation 31 data more realistically.”  and “Additionally, errors are introduced by the linear regression  method.” in the abstract  may be deleted.

 4.  The first sentences in line 66-68 can be deleted.  and the paragraph in line 68-71 can follow  after line 65.

5.  line 90-93 ‘In this paper, we aim to fill this gap by specifically focusing on the uncer-90 tainty assessment of the SNO cross-calibration method. By analyzing the main factors 91 contributing to uncertainty and deriving theoretical equations, we provide a detailed 92 description of the sources of uncertainty and their impacts on the calibration results.’   vs.  106-108 'In this study, the aim is to assess the uncertainty of the SNO cross-calibration 106 method itself for satellite IR channels. The analysis focuses on the calibration uncertain-107 ties arising from sample matching and regression analysis methods.'  Is there a conflict?     It is suggested that line 90-93 be blended within line 106-108 and that it is not appropriate to have multiple and inconsistent occurrences of what is to be studied in this paper.  The description of 90-93 breaks through the logical relationship with the preceding and following text. The previous statement that existing studies have not done a detailed analysis of the uncertainty of the SNO cross-calibration method in terms of influencing factors should be followed immediately by the statement that this is a topic worthy of study. This is followed by line 94, which describes the problems with detailed influence studies. 

6. The topic of this paper described in line 106-108,  should be placed before 'instead of ...' in line 100.  line100-105 should be integrated into paragraphs 106-115.

7.It is suggested that the penultimate 1, 2 and 3 paragraphs of the introduction be fused, while the penultimate 2 paragraphs be significantly reduced. The relevant descriptions in this section could be placed in the experimental section.

8.It is suggested that further explanation be given as to why simulated data is used instead of real data.

9. Equation 1 is the model for SNO cross-calibration? Cross-calibration establishes a regression relationship between the toa radiance of the reference sensor and the toa radiance of the sensor to be calibrated, which does not seem to be consistent with Equation 1.  Also, the appearance and interpretation of Equation 1 is not logically related to the preceding paragraphs.

10. lin81, 'From the introduction of the SNO cross-calibration method, it is clear that..' can be deleted. the relevant description in the second paragraph of the new addition 2.1 can already be stated.  

11. line81 'sample matching.' appears several times in the preceding and following text, but there is no explanation of what exactly it is, and it seems to mean 'temporal matching, spatial matching,  angular matching, spectral matching,'.?

12. Such descriptions are recommended to be removed, the logical relationship between exercise and statement is linked by natural statement descriptions that do not need to be artificially emphasized several times, but rather reduce the readability of the text. It is recommended that the whole text is combed and checked.

13.How PSF was applied to produce the experimental sample data is not described.

14.The PSF is the key parameter describing the remote sensing imaging system and is composed of the optical PSF, the detector PSF, the motion PSF, the electronics PSF and the atmospheric PSF.  Mathematically, the system PSF is the spatial convolution of the components’ PSF.  Among them, only the optical PSF can be simulated with a Gaussian function, while others are not, such as the detector PSF, the satellite-motion-related motion PSF, etc.  This paper is also innovative in modelling the data through PSFs with a view to verifying or assessing the uncertainty under the meticulous influence of the SNO cross-calibration method.  However, in the experimental key data, only the optical PSF  is modelled by Gaussian functions, while other PSFs are neglected. It can be argued that the experimental data in this paper may differ significantly from the real in-orbit data of the satellite. It is questionable whether the conclusions of this paper are representative and can be applied or are in line with the application reality of the real in-orbit satellite. It is suggested that the authors clarify this.

15.The core innovation of this paper is the detailed analysis of the uncertainties associated with the specific factors influencing the SNO cross-calibration method, which has also been verified through relevant experiments. However, the conclusions are only general descriptions, and there are no quantitative and specific conclusions, such as how many days difference in calibration is acceptable for temporal matching? Or what is the uncertainty associated with a difference of how many days, 1% or 3%?  What is the impact of a 1 pixel error in spatial matching on the uncertainty of the final calibration? etc., are not reflected in the conclusions.  

Author Response

Thank you for reviewing our manuscript. The valuable suggestions and comments have undoubtedly helped us to improve the article. In the following we reply to each of the comments (lines start with “Response”). Please find below the comments in blue italics and our responses in black, the changes were highlighted in the revised manuscript. 

It's great to hear that this version of the paper is more logical and readable. We have conducted a full text review to avoid errors in the writing. Based on your feedback, we have enhanced the description of the role of PSF in constructing the simulation data and the quantitative description of the conclusion section in the manuscript. 

Author Response File: Author Response.docx

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

Please, see attached file.

Comments for author File: Comments.pdf

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

General Comments:

Multiple missions are being planned by various international agencies to employ benchmark SI-traceable sensors into space to measure the Earth radiance with unprecedented calibration accuracy and to serve as the reference of other on-orbit sensors through on-orbit intercalibration. The characterization and the minimization of the uncertainties associated to the popular inter-calibration methodology thus becomes more and more critical. This manuscript demonstrates a thorough understanding of the issue and a clear pathway to develop a generalized model to assess the uncertainty through an in-depth analysis of the error propagation chain. The process can also be retroactively applied to optimize the SNO selection thresholds. The manuscript is well written. The results, particularly those of the sensitivity analysis of the error sources to the SNO matching, are promising.  Some assumptions made in the modeling process could be more scientifically rigorous, leaving room for further improvement to the model. I’d recommend with its publication if the following minor issues are addressed. The below JGR-atmosphere paper on the same subject can be a good reference.

Tobin,D., R.Holz, F.Nagle, and H. Revercomb (2016), Characterization of the Climate Absolute Radiance and Refractivity Observatory (CLARREO) ability to serve as an infrared satellite intercalibration reference, J. Geophys. Res. Atmos., 121, 4258–4271, doi:10.1002/2016JD024770.

P2L56: “other calibration…”->”some other calibration…” While on-orbit inter-sensor comparison is a proven approach to calibrate sensors of lesser calibration capability, it is not superior over qualitative calibrator-based on-orbit calibration, which is still needed at least for the reference sensor.

P2L66-71: The logic is a bit vague here. The uncertainty analysis naturally should cover both random and systematic uncertainties. The reason of not using actual data from the compared satellites is not explicitly provided then.

P3L98-99: “the two major sources” -> “major sources”. I can understand “sample matching” error and fitting error are the focus of the manuscript. Still, the analysis cannot validate they are “the two major” error sources to the calibration results. There are other potential sources such as the calibration of reference sensor itself and the instability of the monitored sensor, etc.

P3L108-110: Suggest linking the letters in the table to the physical quantities here.

P3L122: It is arguable that all input parameters are independent and have equal probabilities of occurrence. When several of the parameters are limited within their thresholds, it is often the case that the expected values of other parameters have unique, non-uniform distribution patterns determined by the orbits or scanning mechanism of the instrument pairs.

P6L198-200: I assume the size of the sample set is large. It is plausible to put the number here.  Also, Fig. 3 bring in an impression that only limited HIRAS spectra are used as the input of the simulation as truth. The impacts of matching parameters are so scene dependent, like spectral mismatching, that it is suggested to clarify this and clearly specify the input data set.

P7L268: The derivative of the Planck equation determines that the same dL leads to less dT at higher brightness temperature. The emphasis of this effect in the manuscript is optional.

P8Table6: The uncertainty attributed to zenith angle difference is extremely small in the table This is contradicting to the previous experience of others that the intercomparison results are sensitive to the zenith angle difference. The description in Sec 4.3 does not specify the zenith angle(s) used to derive the sensitivity coefficient. It is recommended to add this information.

P9L308: Presenting results of fitting error in the manuscript makes it more complete.

Comments for author File: Comments.pdf

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 3 Report

This paper is trying to construct a model to answer the uncertainty of SNO cross-calibration method for thermal-infrared band of remote sensors onboard meteorological satellites. Since each work on this area is usually required to report its uncertainty and the mathematics of the uncertainty estimation is well constructed, the model proposed in this study is not new at all. Furthermore, several questions need to be paid special attention.

(1) The maximums temporal difference, positioning difference, angular difference, and spectral difference for the need to be specified at the beginning of this manuscript before analyzing. However, the authors give several self-assumed values for analyzing and they didn't declare the reasonability of these values.

(2) for equation 8 and 9, the value of the following part is definitely negative, which may cause errors while applied by the readers. Therefore, the foundation of the whole deduction is wrong. It needs to be emphasized by the authors.

(3) 

Here, the interval of theta is 0.01°, but it is 1° elsewhere. 

(4) The purpose of this paper is to construct a model to estimate the uncertainty of SNO method; therefore, in order to declare the model is effective, the authors should evaluate several studies on SNO method, which have been done before. I believe this is more reasonable. 

(5) From the results of the study, I can conclude that the uncertainty induced by zenith angle and spectral setting is insignificant, because the order of magnitudes for both of them are much lower than those of temporal and spatial aspects. However, is this true? e.g., while the atmosphere is full of water vapor or very dry, the difference of brightness temperature will largely different. 

 

Author Response

Please see the attachment.

Author Response File: Author Response.docx