Reasoning Algorithms on Feature Modeling—A Systematic Mapping Study
Round 1
Reviewer 1 Report
The article is within the scope of the journal, and deals with an interesting topic.
It is well written and structured. It is easy to read.
Regarding the content, the described experiment has been correctly designed following an adequate methodology.
The results obtained constitute an advance in the area of knowledge.
The article can be accepted, however it should be improved:
a) It is suggested to extend the related work section.
b) In the conclusions section, the scientific contribution should be clarified and lines of future work should be identified.
c) In the discussion section, the explanation of the advances and limitations of the work presented should be improved.
Author Response
Please see the attachment
Author Response File: 
Author Response.pdf
Reviewer 2 Report
- The paper needs re-structuring. It is confusing. The text in few sections are present twice with different view point. PQ1 is not required and its corresponding figure. All the information can be presented in one table or figure. Similarly, section 6.2. It can be merged with earlier sections. Figures are not clear.
- The authors should provide the reader with context to understand the background to the survey, and the reasons for which the research has been conducted. In the present form, the paper requires improvement in highlighting motivation and need for carrying out this research. It should preferably be support with research objectives for better clarity.
- The survey paper should discuss the target source included in the survey and why? How the filtering criteria is applied and on what grounds for inclusion in this study. Individual subsections can be included to adequately cover all the details. Right now, the reader is clueless about the selection and rejection of literature and its source.
- The paper is informative but to further empower the quality of the survey, the survey should not only shed the light on the different techniques but also provide a detailed comparison between the techniques, highlighting the advantages and disadvantages of each.
- Finally, a survey should be concluded with details of key results from the survey. It should preferably be summarized according to the research objectives of the survey, and it should be ensured that all research objectives are covered. Further, it should highlight results that are of both statistical and practical significance. All sources of data in the findings should be referenced, so it is clear where they have come from, and so their credibility can be assessed
Author Response
Please see the attachment
Author Response File: Author Response.pdf
Reviewer 3 Report
The paper provides a systematic mapping study of the literature on reasoning algorithms for feature modelling for software product lines.
The topic is timely and the authors applied a sound methodology regarding literature selection and derivation of research questions. The added value to existing literature studies on the topic is pointed out clearly.
The paper can, however, improved in several ways.
The introduction to reasoning algorithms in Section 2.4 is a bit brief and describes the reasoning process on a very high level. It is, for example, never described what kind of information is exactly extracted from feature models. A more detailed description with some examples would greatly help to develop an idea about reasoning algorithms on feature models also for the unexperienced reader.
The methods used (RQ2) and the problems solved therewith (RQ6) are clearly related. The selection of methods is usually driven by the problem that is supposed to be solved. Section 6.3 addresses relationships between the research questions but only in a Sankey diagram, which is very hard to comprehend. It would be desirable to have a more extensive discussion on which methods are used for which purposes and why.
The analysis of the most relevant terms in Section 6.4.1 does not provide much information. The term network contains mostly evident terms and relations, and there is no discussion on it. This kind of analysis can be improved by tagging terms as method, problem, feature model type, etc. and by discarding general terms like “SPL”, “feature model”, “software”, “research”.
Minor remarks:
Line 29: LPS -> SPL
Line 75: Delete “Productivity increases” since enumeration already includes “increase in productivity”
Line 351: varaibility -> variability
Table 8: Duplicated entry SP9
Author Response
Please see the attachment
Author Response File: Author Response.pdf
Round 2
Reviewer 3 Report
My comments have been partially addressed. There are still some shortcomings, however:
Comment 2 (More precise description of the feature extraction process):
Section 2.4 has been extended and a few more details about the feature modelling process have been added. However, a specific example describing what kind of features are modelled is still missing. It is not clear which information is extracted from feature models and how this is further used to develop software product lines. The new Figure 3 is not helpful without describing how features are transformed, i.e. represented and how the analysis results look like.
Comment 4 (Improvement of the bibliographic content analysis): The added value of the bibliographic analysis is still not clear. The key terms and their connections are evident and very general. The thesaurus used could be further currated more focussing on specific methodological concepts.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 3
Reviewer 3 Report
My comments have now been fully taken into account. I have no further objections against publication.
