LPHOG: A Line Feature and Point Feature Combined Rotation Invariant Method for Heterologous Image Registration

Round 1

Reviewer 1 Report

The paper deals with rotation invariant matching of images from different and equal sources based on line feature points that are used to compute rotation invariant point features. All features are hand-crafted. An extensive comparison with other methods prove that the novel method has superior performance when rotation invariance is needed.

The paper is well written and the evaluation shows that the approach is useful. 

The rotation invariance is implemented by computing a line-based rotation angle. My main question is: Does there exist machine learning approaches that are able to automatically determine such features?

Detailed questions:

96: The definition of Harris scale space is unknown to me. The terminology is used in the cited paper, but is it common knowledge?

98: What is sigma?

153: Probably, multiple points have a largest gradient, not only one?

166: Difficult to read, please use vector representation instead of bitmap

221: Could you highlight the differences between feature points in c) and d)? Differences are difficult to see.

235, Formula (5): Is arctan the right function or is atan2 used? The denominator can be zero, and opposite directions are treated like one. Theta depends on x and y?

243 Is Theta the angle associated with the starting point?

250 What is gamma, and is the constant factor relevant?

289 Do you compare 2n+1 gradients  for s_j vertical lines for each line segment? s_j has not been mentioned before, and I do not understand (17).

291: Why are there boundary lines in image (b)?

309: What is the reason for exactly four bins?

311: Do you only compute a rotation matrix or did you compute some sort of affine or non-linear transformation when matching feature points?

415 variable n is used twice: It is the parameter in (15) and the number of matching points. This is extremely confusing!

425 Please explain the purpose of the checkerboard images: What can be seen with the help of the underlying board?

99: typo: keypionts

383 typo: on the

393 type: we

521 and 589 typo: show -> shows

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

This paper presents a new registration method for heterologous images. The idea of primary orientation is interesting and seems effective. Here are some suggestions.

 1. The quality of the Figures in the manuscript is low, especially, the Fig. 1, 2, 6, 8. Besides, I can barely see the detected lines in Fig. 5b. 

2. The line detection accuracy is significant for the registration results.  Have you tried the other line detection methods, like the Canny operator? 

3. The references are a bit old-fashioned. 

4. The resolution of the applied images has a similar resolution, I would be curious about the effect of scale change on the result. 

 

There are some grammar errors, such as on Page 6, Line 213, “for each pair of two optical images”.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 3 Report

The authors propose an image co-registration method which can be applied to various type of images (SAR, visible and IR images). The main innovation is the introduction of a specific keypoint extraction method which is designed to be rotationally invariant. The authors use 3 different datasets to demontrate the robustness of the method and its invariance by rotation.

The paper is reasonably well presented, although the English is not always clear and the quality of the figure is relatively poor.

The paper could be improved by adding a discusion section commenting on current limitations and future avenues of research. Run-time aspects, as well as the number of iterations of the FSC could be discussed. It seems to me that the keypoints extracted along a given line segments can be expected to be very similar (with very close HOG descriptors). I am wondering if this is not impacting the convergence of the FSC algorithm, and if it could not be better to match the line segments globally (which would provide a partial constraint on the transformation matrix).

I add below some other minor comments:

line 19 : I suggest to use "optical-visible" instead of "optical-optical"
line 190: Is it not possible to optimize the bilateral filter so that the first Gaussian fitler can be avoided ? More generally it could be useful to justify the need for these 2 filters.
line 199: "as with the line features of the optical image, the line features become more elongated" : the meaning of the sentence is not clear to me
line 276: please explain how the image rotation is performed (which interpolation method is used).
line 281 : As I understand it, the method is as follows: After rotation of the LSD rectangle, the center line along the horizontal (longer) dimension is considered. For each point along this line, the location of the point with maximum gradient along the vertical direction is extracted and recorded as a keypoint. I think it would be more clear to speak about each point x',y' along the horizontal line rather than an "arbitrary position x',y' ". Also please clarify in the text that the "n points on each side" are along the vertical direction in Fig 9.
chapter 3 : please clarify if the source images from the different datasets are plublicly availabl,e and if yes how (e.g. http://gpcv.whu.edu.cn/data/building_dataset.html for dataset 3).

 

See above.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

Some minor remarks:

line 167: I still do not understand why there can't be more than one point with a largest gradient length. Multiple points can't have the same gradient?

line 249: arctan can't deliver value pi/2 or -pi/2. For these values, (5) is not well defined. Also, you only get angles between -pi/2 and pi/2, and you can't distinguish between opposite directions. Therefore, I asked for atan2 or for discussing cases. 

 

typos: lines 72-73 combine -> combines?, improve -> improves?

lines 690/691: sampling?

For example, you can use the deepl.com website and select deepl.write to get free English editing.

Reviewer 2 Report

The authors have solved all my concerns. I recommend publication.

Reviewer 3 Report

The revised manuscript has been improved. The quality of the figures in particular is much better. I am a bit disappointed by the new Discussion section (section 5) which is very minimal. Anyway the paper is suitable for publication in my opinion.

Regarding my point 3 : "as with the line features of the optical image, the line features become more elongated"

from your reply I now understand what you mean.

I would propose the following formulation : "the line features of the SAR image become as elongated as those of the optical image".