Quality Control Methods Using Quality Characteristics in Development and Operationsâ€
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsBased on the abstract, the work looks interesting and intriguing, which is an advantage. But, please do not use abbreviations in the abstract and title. Not everyone knows that the concept of DevOps was proposed in 2009 by Patrick Debois during the DevOps days in Ghent.
DevOps can be also understood directly as development and operations without quality assurance.
The content of the work corresponds to a short review publication and not to the publication of original own results. It might seem that section "5.Results of two practices" contains the results of our own research. However, in my opinion, these are the authors' free thoughts.
The literature included in the work would seem barely sufficient to publish my own results. However, adopting it in a review publication is too far-reaching a simplification. Moreover, in a review publication I would expect advanced assessment and research methods implemented on specific examples. This is not the case here.
My proposal is to classify the submitted publication as a tutorial for students of a master's course in the field of IT.
As such, the publication should be addressed to another (educational) publishing house or used to expand it into a classic review publication, or I should request analysis or quantitative and qualitative research in the area of new proposed methods.
Comments on the Quality of English LanguagePlease do not use abbreviations in the abstract and title.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for Authors1. The main question addressed by the study.
The object of the study is software produced using DevOps methodology.
The subject of the research is methods and means of software testing.
The purpose of the work is to improve the efficiency of software testing based on the application of quality characteristics used in the software product quality model.
2. Assessment of the originality and relevance of the topic of the article
The topic of research is relevant. The practical value of the article is that the authors have proposed a solution that allows increasing the efficiency of software developed with the help of DevOps methodology. This solution is based on the use of quality characteristics used in the software product quality model during software testing.
The research topic is an original one. Currently, classical quality metrics such as defect rate and test coverage rate are used in many DevOps projects in order to evaluate the effectiveness of software testing. However, these quality indicators are often difficult to use to judge the quality of software under development. The scientific hypothesis of the feasibility of quality characteristics used in the quality model is based on the fact that not only software developers, but also other stakeholders such as system integrators, acquirers, owners, maintenance specialists, contractors are associated with the quality of the created software product.
3. Contribution to the subject area compared to other published materials.
The novelty of the solution considered in the article is that the authors proposed a procedure of visualization of the testing process to improve the efficiency of testing in the CI/CD pipeline in terms of quality characteristics. To use quality attributes as KPIs, it is necessary to manage the test results of each pipeline and compare them with the results of previous builds. if the execution time of a test increases compared to the previous test, it is likely that some performance degradation is occurring and the impact of the added code should be suspected.
4. Assessment of the conformity of conclusions with the presented evidence and arguments.
Two projects were compared to confirm the theoretical conclusions about the feasibility of using qualitative characteristics in testing. The first project used the cascade model. And the second project used DevOps methodology, which proposed to conduct testing using the characteristics of the software product quality model. The conducted research has shown that during the implementation of the second project both unit and integration tests were able to produce the following benefits while adding performance efficiency assessments:
- increased development productivity by being able to find features and performance and load degrades in the build pipeline;
- increased testing efficiency through automation;
- increased coverage of automated tests to capture more quality data.
As a result, the use of quality characteristics increased the test density to 144% compared to the case where no quality characteristics were used, but the number of defects found was 32% of the predicted value, which means that not only was quality-conscious development carried out, but the task was also able to proceed as planned.
5. Evaluation of the sources of information presented in the article.
References to sources of information presented in the article are appropriate. Sources are available for study.
Remarks
1. The testing pyramid shown in Figure 6 should be discussed in Section 2. This would explain the relationship of the tests to the characteristics of the software product quality model.
2. In the abstract, the authors reported that they have developed a method for rapid and high-quality development of a software product. However, there is no description of the method in the text of the article.
3. To confirm the theoretical conclusions about the expediency of using qualitative characteristics in testing, two projects using the cascade model and DevOps technology were compared. I consider this comparison incorrect, as DevOps was created to eliminate the disadvantages of the cascade model of software development and replace it.
Comments on the Quality of English LanguageEnglish is mostly fine, but minor edits are still required
Author Response
Please see the attachment file.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsThe correction of the work was done in an exemplary way. Congratulations to the authors.
Reviewer 2 Report
Comments and Suggestions for AuthorsThe comments have been eliminated.
The article is relevant, has scientific and practical novelty, and is recommended for publication in the journal
Comments on the Quality of English LanguageEnglish is fine.
