Indoor Localization for Augmented Reality Devices Using BIM, Point Clouds, and Template Matching

Round 1

Reviewer 1 Report

This paper presents an indoor localization technique using the 2D floor map generated from a depth sensor and that from BIM. First, from a point cloud acquired with a depth sensor, 2D floor map is computed by estimating the ground floor plane, and projecting the point cloud onto the plane. Then, the map is also generated from BIM. To match them, the map from the sensor is first rotated by estimating dominant orientations, and then match with the map from BIM by using template matching. The proposed method is evaluated in one building.

 

Overall, this paper is well structured, and easy-to-read. The issue tackled in this paper is still an important for navigation tasks. However, the problem of this paper is insufficient survey on similar issues. Also the technical novelty is not so clear compared with existing methods. Finally, the effectiveness of the proposed method is not well evaluated.

 

1. In the title, the authors included “augmented reality”. If we follow the definition of augmented reality proposed by Ron Azuma, the system should be interactive and online.

Azuma, Ronald T. "A survey of augmented reality." Presence: Teleoperators & Virtual Environments 6.4 (1997): 355-385.

As describe in Figure 1, the proposed method works offline because it takes recorded data from the device, and then output the transformation from the local frame to the BIM frame. Additional description of how to use the proposed method for augmented reality systems should be necessary. Also, an example of the AR system with the proposed method should be introduced.

 

2. The authors provided the survey on localization without external sensor, but the solutions for robot kidnapped problem or relocalization are missing even though they are almost same as what the authors did. As the authors stated, in the field of robotics, this issue has been investigated for a long time. Actually, some are based on ICP, but many others are not. There are some online methods by using probabilistic filtering, which can be useful for AR systems. Therefore, the survey is insufficient.

 

Boniardi, Federico, et al. "Robust LiDAR-based localization in architectural floor plans." 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2017.

 

Desrochers, Benoît, Simon Lacroix, and Luc Jaulin. "Set-membership approach to the kidnapped robot problem." 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2015.

 

Jagbrant, Gustav, et al. "Lidar based tree and platform localisation in almond orchards." Field and Service Robotics. Springer, Cham, 2015.

 

Magnusson, Martin, et al. "Appearance-based loop detection from 3D laser data using the normal distributions transform." 2009 IEEE International Conference on Robotics and Automation. IEEE, 2009.

 

Magnusson, Martin, et al. "Appearance-based loop detection from 3D laser data using the normal distributions transform." 2009 IEEE International Conference on Robotics and Automation. IEEE, 2009.

 

The authors should survey the papers again in terms of robot kidnapped problem or relocalization, and clarify the difference and the advantage compared with existing methods. 

 

3. Additional comparison with some existing methods is required to show the advantage. For instance, it is not clear why inverse distance map from BIM is required. Just matching raster data from BIM with the map from sensor would be enough. The effectiveness or importance of each process in the proposed method should be clarified.

Author Response

Dear Reviewer,

Thank you very much for your contribution, as it provided valuable insight to our work from your field of expertise. Please see the attachment for our response.

Kind Regards

Author Response File: Author Response.docx

Reviewer 2 Report

The algorithm is very interesting.

The results are satisfactory.

Authors should do the further work to improve the navigation, in accordance with the plans presented in the conclusion.

Author Response

The algorithm is very interesting.

The results are satisfactory.

Authors should do the further work to improve the navigation, in accordance with the plans presented in the conclusion.

Thank you very much for your contribution and your kind comments. In our paper we present navigation as a possible application for our algorithm, but do not focus on navigation itself. Due to your comment, we have added a paragraph to our conclusion explaining how our research can support indoor navigation tasks in future work.

Round 2

Reviewer 1 Report

The authors revised the paper according to the reviewer comments. However, it is not completed. This is still not a paper on AR because there is no example of the AR system. Actually, the proposed method outputs 2D trajectory after moving for a moment. This means that the users cannot get the position at the beginning. For instance, the users cannot localize if they move only in one room. Therefore, it is not clear how to use the proposed method for AR systems in practice.

In the evaluation, the method is not evaluated in terms of AR context. This is related to my first comment. In Figure .5, it seems that the users can localize after they move for a moment. However, it is not clear how much they should move to localize. For instance, at the beginning of running the system, I can imagine that the users cannot localize. The localization accuracy should be evaluated according to the trajectory length from the beginning to the total length.

Author Response

We thank the reviewer for his comments and suggestions. Due to the reviewer’s feedback, we have decided that our collected data and evaluation was insufficient. We therefore acquired more data and reevaluated our findings. We now have recorded 20 point cloud frames along a path for each scenario. We have analysed data for distance walked and area covered, and show the point of first correct localization for each scenario. We expanded our Evaluation section with further insight into reliability based on the findings of our new data. We have reworked Table 1 to be more concise and to show our new findings. We have added two Figures showing the localization accuracy over time as positional and rotational error. We also state our chosen parameters for the experiments, which were optimized using a grid search.

Furthermore, we have added a paragraph to the Conclusion section detailing how an augmented reality supported workflow can benefit from our approach to localization and give another example for a possible application in the maintenance area.

All changes have been recorded in the attached PDF.

Author Response File: Author Response.pdf

Round 3

Reviewer 1 Report

The authors updated the results to clarify when the localization succeeds and how the accuracy is improved according to the movement. In terms of 2D localization issue, this evaluation is reasonable.

However, as pointed out many times, it is not clear how to use this technique for AR systems. To visualize BIM, it is necessary to compute 6DoF pose with respect to the environment. With the proposed method, only 3 DoF pose can be computed. In the conclusion, some descriptions are added, but it is not clear how to implement AR system with the proposed method in practice.

If the authors insist that this paper is about AR, the authors must present the example of AR systems.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 4

Reviewer 1 Report

This paper is well revised. So, I think this is ok for the acceptance.