Enhanced Soft Error Rate Estimation Technique for Aerospace Electronics Safety Design via Emulation Fault Injection
Round 1
Reviewer 1 Report
Comments and Suggestions for Authors
Automatic Configuration Memory Fault Injection (ACMFI) is developed as a tool that calculates the Architectural Vulnerability Factor (AVF) and Soft Error Rate (SER) by using the emulation fault injection technique. SER can be obtained by experiments (beam tests) , it performed in a beam facility equipped with high energy radiation facilities.
The propose with high energy radiation facilities.
The proposed method in this paper has the advantage of minimizing side effects by injecting faults into a dedicated SRAM area. By studing the Single Event Upsets on SRAM-based FPGAs and Fault Injection Methods for SEU Analisy, finally the Emulation based Fault Injection.
Advanced Techniques in Fault Injection and Error Rate Estimation have been developed.
The use of the ACMFI tool developed in this study allowed for a closer, alignment between the reliability analysis and the actual beam test results by effectively addressing the issue of side effects. The method in this paper has the advantage of minimizing side effects by injecting faults into a dedicated SRAM area. By studying the Single Event Upsets on SRAM-based FPGAs and Fault Injection Methods for SEU Analysis, finally the Emulation based Fault Injection.
Author Response
Thank you for your valuable comments.
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for Authorshe paper introduces Automatic Configuration Memory Fault Injection (ACMFI), a tool designed to calculate the Architectural Vulnerability Factor (AVF) and Soft Error Rate (SER) for aerospace electronics. The ACMFI tool offers a cost-effective and time-efficient alternative to traditional beam tests for calculating the Soft Error Rate (SER) in aerospace electronics. This is particularly advantageous in the conceptual design stage, allowing for quicker and more economical assessments during aerospace system development. The proposed method from authors of using emulation fault injection in SRAM-based FPGAs, with a focus on a dedicated SRAM area, minimizes side effects and allows for a more accurate estimation of the Soft Error Rate (SER) compared to existing methods. This accuracy is demonstrated through the close alignment of results obtained by ACMFI with those from beam facility tests. I recommend expanding the overview of the current state and used references, especially in the section of chapter 2. Background.
mages should be proportional, some images have no scientific meaning and do not bring new informational value, e.g. Fig.8, Fig.9, Fig. 10 and similarly Fig. 13, Fig. 14, Fig. 15 (a table of values would be more appropriate here).
I recommend expanding the conclusion of the article, as it only very briefly discusses the findings, and it is also necessary to clearly refrain from other advanced reference methods.
Author Response
Thank you for your valuable review.
We appreciate all your comments and have responded to all of them.
Please see the attached file.
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for Authors1. Introduction needs to be improved. It is not recommended to cite 4, 5 papers in one sentence without details (references 8-11, 12-16). Please summarize each reference and clarify why it is referred to in current work.
2. Many figures are screenshots from software which looks unprofessional. Please move these to an appendix if needed and reformat them as flow charts or similar diagrams in main text. 3. The citation under Table 2 (1 https://docs.xilinx.com/r/en-US/ug116/Failure-Rate-Summary) should be added in references. Only reference numbers should be used in the main text. Why are there several references from docs.xilinx.com? 4. More comparison between proposed approaches and existing methods should be provided.Author Response
Thank you for your valuable review.
We appreciate all your comments and have responded to all of them.
Please see the attached file.
Author Response File: Author Response.pdf
Reviewer 4 Report
Comments and Suggestions for AuthorsAccording to authors this paper presents a comprehensive analysis of the system error rate of FPGA configured memory security circuits, with a focus on soft errors. To achieve this goal, authors created test vectors with more than 90% fault coverage using Atalanta, an open-source ATPG tool. Based on the generated test vectors, it conducted a statistical fault injection test with a confidence interval of 95% (Error 1%) using the developed ACMFI tool for four types of target security circuits.
This comprehensive study is very difficult to follow and verify if the reader is not an expert in that profession and specific knowledge. The auditor can only take into account the qualitative and quantitative methodology of the indicated hypothesis.
Correct research was conducted with a detailed description of the problem and a well-marked reference literature. The research manuscript is very well structured, the research results are accompanied by pictorial, conceptual and graphic presentations.
The first important objection concerns the too long abstract. The abstract must briefly guide the reader to the essence, i.e. the hypothesis, the type of research and the obtained advantages of the procedure or technology.
Another objection is that authors should avoid writing a scientific research paper or manuscript in the first person singular. It is more acceptable for the reader to read a paper written in the third person singular or plural.
The list of literature is sufficient and acceptable.
Author Response
Thank you for your valuable review.
We appreciate all your comments and have responded to all of them.
Please see the attached file.
Author Response File: Author Response.pdf
Reviewer 5 Report
Comments and Suggestions for AuthorsTitle: Enhanced Soft Error Rate Estimation Technique for Aerospace Electronics Safety Design via Emulation Fault Injection
The paper introduces Automatic Configuration Memory Fault Injection (ACMFI), a tool for estimating the Soft Error Rate (SER) and Architectural Vulnerability Factor (AVF) in aerospace electronics. Traditionally, SER is determined using expensive and time-consuming beam tests in radiation facilities. ACMFI offers a cost-effective alternative by using emulation fault injection in SRAM-based FPGAs, commonly used in aerospace hardware. This method minimizes side effects by targeting a specific SRAM area, leading to more accurate SER estimations compared to previous methods. The accuracy of ACMFI is validated by comparing its results with those from beam facility tests. This tool aids in the safety design of aerospace systems by enabling more precise failure rate calculations, thereby contributing to the development of safer and more cost-effective aerospace electronic systems. However, the following comments should be addressed before the manuscript can be considered for publication.
1. What’s the purpose of section 2? From the reviewer’s perspective, background information should be provided in the introduction section.
2. The comparison presented in section 4.3 is insufficient. The authors limit their comparison to the results observed in reference to Ref 35 only. The authors should conduct a more thorough and wide-ranging comparison to effectively evaluate the proposed method.
3. How does the use of ACMFI influence the determination of development assurance levels in aerospace system safety design?
4. The authors mention the cost-effectiveness of ACMFI compared to traditional beam tests. The authors should provide a quantitative analysis or case study highlighting the cost savings in terms of time and resources.
5. The conclusion section should be revised. In conclusion, the authors should summarize their work and point out the results concisely. Also, the authors need to acknowledge any limitations of the proposed ACMFI tool. This will help readers better understand the tool's applicability and identify areas where further development and refinement may be needed.
Author Response
Thank you for your valuable review.
We appreciate all your comments and have responded to all of them.
Please see the attached file.
Author Response File: Author Response.pdf
Round 2
Reviewer 2 Report
Comments and Suggestions for AuthorsI have read the edited version of the article and I am glad that the requested changes have been incorporated. The article in its current form is clear and clearly presents the outputs of the authors. I refrain from publishing it in its current form.
Reviewer 3 Report
Comments and Suggestions for AuthorsComments are addressed properly.
