Open-Source MQTT-Based End-to-End IoT System for Smart City Scenarios
Round 1
Reviewer 1 Report
Authors propose an IoT-ready architecture for data collection based on sensing units and on the MQTT communication protocol. The paper is clear and easy to follow, authors well organized the manuscript. The topic addressed also fits with the goals of the journal.
The architecture proposed is reasonable, but it is not clear how the design follows the requirements of Vulnerable Road User Scenarios. More specifically, the architecture seems not tailored on the specific scenario described in the introduction and detailed in section 2. Rather, such architecture is general-purpose, it might work for:
- heath care scenario
- environmental monitoring
- ambient assisted living scenarios
etc
Indeed, authors report: "[...] The sensed data can be human related data, (e.g., hearth monitor, position) as well vehicle related data (e.g., battery charge, vehicle position) or environmental data (e.g., humidity, air temperature, pressure).". Authors are required to strongly motivate how the architecture in Fig. 1 addresses the specific requirements of the considered scenario. Authors are also required to list such requirements.
Moreover, as the scenario considered refers to vehicles, some design choices have to be clarified:
- do authors assume that the gateway is always deployed in a vehicle? More specifically, does a user always carry a mobile device acting as sensor gateway?
- how to manage disconnections from the MQTT broker (with or without the retention policy)
- how to collect sensing data available from the vehicle. Do authors consider to interact with the CAN-bus or similar?
- what about the possibility of running an Android service acting as data publisher? Android OS prevents more and more to run long-lasting background services, there exist workaround to prevent this but they require to be clarified
- Finally, why assuming to have a mobile device acting as a Gateway rather than assuming using a micro-controller also for the purpose of publishing data (e.f Arduino nono IoT or similar)
Author Response
Answers to Reviewer 1
The authors wish to thank the anonymous reviewer whose comments have helped in improving the paper. Changes introduced in the paper are reported in purple color.
Authors propose an IoT-ready architecture for data collection based on sensing units and on the MQTT communication protocol. The paper is clear and easy to follow, authors well organized the manuscript. The topic addressed also fits with the goals of the journal.
Answer: The authors would like to thank the reviewer for having summarized the main aims of the paper.
The architecture proposed is reasonable, but it is not clear how the design follows the requirements of Vulnerable Road User Scenarios. More specifically, the architecture seems not tailored on the specific scenario described in the introduction and detailed in section 2. Rather, such architecture is general-purpose, it might work for:
- health care scenario
- environmental monitoring
- ambient assisted living scenarios
etc
Answer: The solution proposed is the demonstration of a complete workflow, having in mind Smart Cities applications, with a particular interest to those scenarios considering the use of wearable sensors; among others, this fits with applications for pedestrians and/or bikers that do not have to use vehicles with built-in technologies (e.g., cars). For this reason we have addressed the vulnerable user scenario which perfectly fits the demonstration. Usually motorbikes, cars or even sophisticated bikes have their own equipment on board, hence not strictly necessitating the proposed solution. It is true however that the methodology as such is useful for different application scenarios, as indicated by the reviewer and this is a value. The hardware and software chain described can be then tailored to specific users, user density and generated traffic. In any case in order to better outline the applicability of the approach to a wider range of scenarios, the title has been changed, using the reference to smart city instead then to vulnerable users, which is a more restricted case. The abstract has been changed accordingly.
Indeed, authors report: "[...] The sensed data can be human related data, (e.g., heart monitor, position) as well vehicle related data (e.g., battery charge, vehicle position) or environmental data (e.g., humidity, air temperature, pressure).". Authors are required to strongly motivate how the architecture in Fig. 1 addresses the specific requirements of the considered scenario. Authors are also required to list such requirements.
Answer: The proposed approach, based on the use of the MQTT protocol, that is a pub/sub protocol, allows to collect data once for many different subscribers. In our idea the publisher is installed on a smart device carried on by the user, while the MQTT brokers are disseminated within the urban scenario. This approach allows to appropriately send data filtered for the desiderata of each different subscriber. As an example, sensing the luminance of a certain area is useful for both safety action or road maintenance planning. Thanks to the MQTT topics this could be easily managed.
As far as the requirements they depend on the environment and user behavior and are beyond the scope of the paper, that is represented by the proof of concept of interconnected devices and open source software
Moreover, as the scenario considered refers to vehicles, some design choices have to be clarified:
- do authors assume that the gateway is always deployed in a vehicle? More specifically, does a user always carry a mobile device acting as sensor gateway?
Answer: As previously stated our idea is to focus on pedestrians and bikers, i.e., the vulnerable road users, that are supposed to always carry a smartphone device. Thanks to this, it is our idea to exploit the smartphone for implementing a gateway node between sensors and MQTT links to transmit any needed data. In more complex vehicles it could be instead considered as built-in in the vehicle itself.
- how to manage disconnections from the MQTT broker (with or without the retention policy)
Answer: We agree with the reviewer that reliability is an issue. However, MQTT itself provides different levels of quality of service for this purpose, hence, depending on the service request we aim at exploiting them.
- how to collect sensing data available from the vehicle. Do authors consider to interact with the CAN-bus or similar?
Answer: The target here is on road users non equipped with vehicles with built-in technologies. This is the reason why we envisage the possibility of using BLE between sensors and smart devices. We agree that in cars and engine based vehicles, the gateway could be installed on the vehicle itself and can interact with the vehicle's embedded bus. This latter is anyway out of the scope of the paper.
- what about the possibility of running an Android service acting as data publisher? Android OS prevents more and more to run long-lasting background services, there exist workaround to prevent this but they require to be clarified
Answer: The Android app has been designed for being activated by the user once they are willing to use the service. We do not think that the service should be always on, while activated upon user request every time they and to use it. This is the reason why in the actual implementation it is based on an app exploiting the Android system libraries. A possibility that could be considered is to trigger the application execution based on the automatic user behavior recognition, that is now part of the Google services.
- Finally, why assuming to have a mobile device acting as a Gateway rather than assuming using a micro-controller also for the purpose of publishing data (e.f Arduino nono IoT or similar)
Answer: As previously said, our idea is to use the proposed application for pedestrians and bikers. With this in mind we feel it is not proper to design a system where the gateway is implemented on SoC devices. Instead, it is much more convenient to implement it through an Android device usually carried on by any user. Anyway, the idea could be interesting for future developments especially for electric vehicles.
Reviewer 2 Report
Overall, it seems like a well-written paper. The referee was able to understand the proposed scheme well. However, it would be a good paper if the performance evaluation was poor and further strengthened.
First, please add the physical environment and actual photos for the experiment.
In addition, the referee would like for authors to diversify the performance experiments further and analyze the performance of this paper in various ways.
Finally, please describe the benefits users can get from the paper's contribution.
Author Response
Answers to Reviewer 2
The authors wish to thank the anonymous reviewer whose comments have helped in improving the paper. Changes introduced in the paper are reported in purple color.
Overall, it seems like a well-written paper. The referee was able to understand the proposed scheme well. However, it would be a good paper if the performance evaluation was poor and further strengthened.
Answer: The target of the paper is to show which hardware and software components can be used to implement a complete workflow in support of mobile user monitoring with functional assessment. Extensive performance evaluation is left for a better defined scenario from the application viewpoint.
First, please add the physical environment and actual photos for the experiment.
Answer: The photo of the experiment has been added showing a simple set-up with the wearable prototype and the other devices described in the paper.
In addition, the referee would like for authors to diversify the performance experiments further and analyze the performance of this paper in various ways.
Answer: A complete system design would require consideration on further aspects like user density and traffic behavior that are under study. Here, a proof-of-concept is provided, with focus on functional testing, that can further be considered in different contexts for more extensive design and performance evaluation.
Finally, please describe the benefits users can get from the paper's contribution.
Answer: The reader interested in system implementation can start the whole design from a choice of hardware and (open source) software components that are here shown to work properly. In addition to this, in the paper the link from which download the software used for the implementation is provided, giving the possibility to implement and/or extend the PoC to other scenarios. We think this is the most practical contribution of the paper.
Author Response File: 
Author Response.pdf
Round 2
Reviewer 1 Report
Looking at the review letters (major and minor) It seems that authors invested few effort on answering to the comments and detailing the design choices. The new version of the paper does not significantly improve the previous one, and some of the comments raised by reviewers still remain un-answered.
Few example:
- do authors assume that the gateway is always deployed in a vehicle? More specifically, does a user always carry a mobile device acting as sensor gateway?
- answer does to clarify if yes or not
- how to manage disconnections from the MQTT broker (with or without the retention policy)
- answer is not clear. How authors manage the disconnections? Did authors configured the broker, did authors ignore the problem?
- Finally, why assuming to have a mobile device acting as a Gateway rather than assuming using a micro-controller also for the purpose of publishing data (e.f Arduino nono IoT or similar)
- answer does not motivate why it is not convenient using a SoC. Motivations are required
-
Finally, please describe the benefits users can get from the paper's contribution.
- answer is vague
Moreover, from the cover letters it is not always possible to understand the edits in the paper answering to the specific comment.
Author Response
Dear Reviewer,
thank you for the time put in reviewing our paper. Please find the answers to each questions below.
---------
Looking at the review letters (major and minor) It seems that authors invested few effort on answering to the comments and detailing the design choices. The new version of the paper does not significantly improve the previous one, and some of the comments raised by reviewers still remain un-answered.
Answer: The authors wish to apologize if the effort put in answering didn’t meet what expected by the reviewer. We are happy to try to be clearer now.
do authors assume that the gateway is always deployed in a vehicle? More specifically, does a user always carry a mobile device acting as sensor gateway? answer does to clarify if yes or not
Answer: The aim is to find a solution to maintain the vehicle as simple as possible, e.g. a simple bicycle or a pedestrian. So the gateway needs to be deployed on a smart device carried by the road user. It has to be highlighted that nowadays we can safely assume that each user has a smartphone with her/him. Of course, in case of more complex vehicles, like sport bikes, motorbikes, cars the gateway can be directly implemented on the vehicle itself with no need of a mobile device to carry.
Going in the details of the two questions:
do authors assume that the gateway is always deployed in a vehicle? No, it could be
More specifically, does a user always carry a mobile device acting as sensor gateway? yes, in case of simple vehicles or in case of vehicles not equipped with the gateway
In order to clarify these points, we have included a sentence at page 1 line 74 and page 6 line 248
how to manage disconnections from the MQTT broker (with or without the retention policy)
answer is not clear. How authors manage the disconnections? Did authors configure the broker, did authors ignore the problem?
Answer: perhaps in the previous reply we didn’t catch exactly what the reviewer would know. Disconnection management is out of the scope of this paper. The value of the keep alive parameter has been left to the default value (60 s) but can be set to any value depending on the effectiveness target of the system. It has to be also highlighted that the choice of using the MQTT protocol is efficient from this point of view. As clarified at page 6 line 265, MQTT as the application layer protocol allows to decouple the transmitter, i.e., the publisher, and the receiver, i.e., the subscriber. A pub/sub paradigm allows indeed to implement an efficient way for delivering messages through an intermediate node even in those situations where one of the two parts may be disconnected or temporarily unavailable, allowing at the same time to provide an efficient way for delivering messages.
Finally, why assuming to have a mobile device acting as a Gateway rather than assuming using a micro-controller also for the purpose of publishing data (e.f Arduino nono IoT or similar) answer does not motivate why it is not convenient using a SoC. Motivations are required
Answer: as said above, although in principle this is possible and even more convenient from a cost and/or performance perspective (convenience should be in any case properly defined), the solution based on a smart mobile device, that is likely to be hold by the user in any case, can be applied to any road user even with very simple and cheap vehicles. A large-scale application of this approach can be in smart cities where this kind of system can remarkably improve mobility safety. This has been clarified at page 5 line 215.
Finally, please describe the benefits users can get from the paper's contribution answer is vague
Answer: main solutions for IoT are based on ad hoc data collection that lacks flexibility and scalability. The solution proposed in this paper shows the very simple elements that can enable the design of highly scalable and flexible IoT system, that, even though presented for road users, can be also applied, with the data collected by the same set of sensors, to other purposes, like road maintenance or crowd management and many other use cases.
We believe that having shown a complete workflow for this kind of application can foster the development of large scale IoT systems, being of interest not only for researchers but also for enterprises that can rise and spread in this field.
The above considerations have been added in the Discussion section at page 6 line 807.
Moreover, from the cover letters it is not always possible to understand the edits in the paper answering to the specific comment.
Answer: Perhaps it was not uploaded correctly. Sorry for that.
Author Response File: Author Response.pdf
Round 3
Reviewer 1 Report
The paper can now be accepted. Authors are encouraged to revise the manuscript for a final check.
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
The paper covers an interesting topic and authors provide a clear presentation of the work done. The description of the implemented architecture is clear and , as well as the background material. The proposed architecture is reasonable, however it is not clear the scientific problem addressed. At the current stage, the paper is more a technical report rather than a scientific paper. Authors are encouraged to revise the paper and to clarify the novelty of the proposed solution also with respect to the state of the art.
Author Response
See attached file
Author Response File: Author Response.pdf
Reviewer 2 Report
The authors present an end-to-end IoT solution for solving traffic and security issues for Vulnerable Road Users (VRU). The authors combine some of-the-shelf hardware and software solutions in order to provide the functionalities of a whole system.
Strengths: the authors offer an open-source pragmatic solution for a real and every-day problem.
Weaknesses: The proposed approach is not new; all the software/hardware/communication tools are well known, no innovation at any level; maybe their combination may be considered of some novelty.
In the feasibility analysis the authors didn't give enough information related to:
- the communication overhead and load required for a full-sized system (e.g. with thousands of participants in the traffic)
- the power consumption required for the hardware components
- where and how to place the MQTT brokers?
- evaluation of the volume of data generated in a common scenario
- what kind of sensors and actuators are typical for a user
- what kind of services (feedback) a user should get ? (not generic ones); what would I get if I wear it?
- what about information security and user's privacy?
Author Response
See attached file
Author Response File: Author Response.pdf
Reviewer 3 Report
First of all, I appreciate the effort of the authors towards sustainable E2E IoT systems, and I highly recommend the authors to enhance the paper and submit again, based on the following comment:
1. Contribution: Although as a Proof-of-Concept presentation, the paper contains limited contribution, scientifically and experimentally. Please clarify.
2. Methodology: the claimed difference in Section 2 cannot be supported clearly in Section 3, which only presents the generic architecture. I cannot find the corresponding challenges and solutions directly related to the claimed difference in detail.
3. Similarly, Section 4 also contains limited scientific soundness, which proposes basic implementation of the proof-of-concept.
4. To prove the performance, the evaluation in the paper is not sufficient, which only focuses on the feasibility, instead of quantitative analysis. More scenarios with detailed setting and numerical analysis should be included. Besides, I recommend some comparison between the proposed and the previous work to enhance the significance of the work.
Author Response
See attached file
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
The revised version of the paper partially answers to the comments provided by Reviewers. In my opinion, this paper still lacks of a clear scientific contribution even if authors stressed that the novel contribution is more related to a methodology.
Reviewer 3 Report
The authors have answered all my questions.
