Auction-Based Learning for Question Answering over Knowledge Graphs
Round 1
Reviewer 1 Report
The introduction provides a solid foundation on the challenges posed by knowledge graphs for question-answering tasks but could benefit from a few improvements. Firstly, it needs a clear thesis statement that concisely presents the paper's main contribution, which is currently lacking. Secondly, the relevance of auction algorithms should be introduced earlier and connected to the challenges faced by knowledge graphs, as the current text does not explain why auction algorithms are chosen as the solution. I believe the introduction should better address how the proposed auction-based approach deals with the challenges of explainability and interpretation, elaborating on how the auction algorithms help in providing interpretable answers and handling multi-hop reasoning tasks.
While the related work section gives a good overview of the existing approaches, it can be enhanced in several ways. First, the limitations of embedding-based models in terms of explainability and reasoning should be further detailed by providing specific examples of their struggles with certain types of queries or scenarios. Second, the section should compare path-based methods with the proposed auction-based approach in terms of performance, scalability, and adaptability to evolving knowledge graphs. This would help readers understand the advantages of auction algorithms over traditional methods. Third, the authors should delve deeper into the history and development of auction algorithms, discussing their previous applications in various domains to establish their relevance to the problem at hand. The related work section may be brief, but it should be organized into sub-sections based on different approaches (e.g., embedding-based, path-based, and auction algorithms) to enhance clarity and ease of understanding for the readers.
Additionally, the authors provide a detailed description of the APC algorithm, its background, terminology, and formal definition. One criticism is that the explanations may be overly complex and verbose, making it difficult for readers to grasp the main ideas quickly. The authors could improve this by providing a high-level overview before diving into the specific details and terminology.
Furthermore, the authors discuss an extension of the APC algorithm, the Auction Weighted Path Construction (AWPC) algorithm. However, this part of the section seems less developed in comparison to the APC description. It would be more beneficial to either provide a more thorough explanation of the AWPC algorithm or dedicate a separate section for it to avoid overwhelming the reader. Moreover, the significance and implications of the ϵ parameter could be emphasized more clearly, as it plays an important role in regulating the size of the price rise and affects the algorithm's convergence rate and path quality.
Author Response
Thank you for your comments and suggestions to improve the manuscript. Very helpful! We made the necessary changes and recorded our responses in attached file. Thanks!
Author Response File: 
Author Response.pdf
Reviewer 2 Report
This paper deals with the exploitation of techniques for online learning on dynamic knowledge graphs. My comments are summarised below:
* The paper is well-written and the passages are easy to read.
* It relies on online learning for learning on dynamic knowledge graphs.
* It utilises auction algorithms to advance the task of question-answering.
* The proposed algorithm pays particular attention to managing environments that change in real-time (i.e. dynamic environments)
* It is suitable for both offline and online training.
* The proposed approach is formulated in an understandable mathematical manner.
* The proposed approach is thoroughly evaluated via a list of experiments and different evaluation metrics.
* The evaluation results are extensively discussed.
* The advantages and disadvantages of the proposed approach are mentioned in the discussion section.
Suggested changes:
It is recommended that the authors highlight the main contribution of the paper (may be as a bullet list) at the end of the Introduction section.
Author Response
Thank you for your suggestions to improve the manuscript. Very helpful! We made the necessary changes and recorded our responses in attached file. Thanks!
Author Response File: Author Response.pdf
Reviewer 3 Report
The paper presents new auction algorithms for question answering over knowledge graphs. According to the Authors, the proposed approach aims to reduce search iterations and make the algorithm computationally efficient. The Authors’ approach was discussed over the examples where the method can be used. The topic is interesting and the paper well corresponds with the journal's aim and scope.
The paper is well structured. In the introduction, the Authors clearly stated the problem and the aim of their work.
The paper is very well prepared. The Authors thoroughly described the Related work and then the presented approach.
The evaluation was conducted to demonstrate the application of APC in knowledge graphs for question answering - this part has been done. An analysis of the results was provided.
In the Conclusions section, it is worth emphasizing the aim of the paper and highlights.
Overall, the manuscript looks good and it is complete. The topic considered by the Authors is up-to-date.
The figures are well prepared.
The list of references contains new items.
Author Response
Thank you for your comments and suggestions to improve the manuscript. Very helpful! We made the necessary changes and recorded our responses in attached file. Thanks!
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
The abstract provides an overview of a research work proposing the use of auction algorithms for question-answering over knowledge graphs, although it has room for improvement in several areas. It could better introduce knowledge graphs and the problem being addressed. Specifically, it should be more clear on whether the issue is integrating new information into the knowledge graph or the inability of question-answering systems to learn from new entities. The proposed solution, the auction algorithms, is described, but without sufficient detail or a practical example, leaving the reader unclear about its functionality and advantages.
While the authors claim computational efficiency and improved explainability as the advantages of their solution, they fail to provide any quantitative results or performance indicators to support these statements. A more explicit description of the method's contributions to the field would enhance the abstract's clarity and impact. More details on how the algorithm adapts to dynamic changes and an inclusion of some preliminary results could significantly improve the abstract.
Author Response
Hi
Thanks for your valuable feedback to improve our manuscript. We have made the suggested changes in the abstract. Copied below for your quick reference.
Revised Abstract:
"Knowledge graphs are a graph-based data model which can represent real-time data that is constantly growing with the addition of new information. The question-answering systems over knowledge graphs (KGQA) retrieve answers to a natural language question from the knowledge graph. Most existing KGQA systems use static knowledge bases for offline training. After deployment, they fail to learn from unseen new entities added to the graph. There is a need for dynamic algorithms which can adapt to the evolving graphs and give interpretable results. In this research work, we propose using new auction algorithms for question answering over knowledge graphs. These algorithms can adapt to changing environments in real-time, making them suitable for offline and online training. An auction algorithm computes paths connecting an origin node to one or more destination nodes in a directed graph and uses node prices to guide the search for the path. The prices are initially assigned arbitrarily and updated dynamically based on defined rules. The algorithm navigates the graph from the high-price to the low-price nodes. When new nodes and edges are dynamically added or removed in an evolving knowledge graph, the algorithm can adapt by reusing the prices of existing nodes and assigning arbitrary prices to the new nodes. For subsequent related searches, the ‘learned’ prices provide the means to ‘transfer knowledge’ and act as a ‘guide’ to steer it towards the lower-priced nodes. Our approach reduces the search computational effort by 60% in our experiments, thus making the algorithm computationally efficient. The resulting path given by the algorithm can be mapped to the attributes of entities and relations in knowledge graphs to provide an explainable answer to the query. We discuss some applications where our method can be used."
Thank you again!
