Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The presented manuscript shows an interesting research in the field of analyzing assembly code extracted from executable files and detecting functions within this code. The proposed method uses a dual approach, i.e. it performs the fusion of feature vectors coming from two independent streams. One is the extraction of the semantic feature vector of the assembly code from the semantic representation, while the other is the analysis of the function call graph using GAT. The first stream represents the modification of the PalmTree model with the application of BERT, which is followed by a Bi-LSTM for feature extraction. A simple concatenation of features is carried out, which are then passed to the LSTM and the features are classified.

Although the method may be interesting from the point of view of applying neural networks in identifying functions in assembly code, the context in which it was or should have been considered and the specific application are not well explained, making it difficult to assess its value. Let us now look at the main disadvantages:

1. Title - "RepFTI: Representation-fused Function Type Inference for Vehicular Secure Software Systems" - The title suggests that something has been proposed that will improve the security of software in vehicles. This of course sounds very interesting, but unfortunately the manuscript does not directly support these expectations. First, the question of how the proposed method supports software security should be answered. Secondly, what is so specific about the proposed method that it can be applied specifically in the automotive industry?

2. Security - The very first sentence of the abstract states: "For enhancing the security of vehicular software systems, inversely identifying function types of binary files play a key role in threat discovering." Why is the identification of functional types crucial for threat detection? RepFTI is then claimed to be a "framework for secure vehicular software systems". This raises the expectation that RepFTI can analyze the software and provide some level of security as well as point out potential security issues. Sections 2.1-2.4 indicate that security should be considered in all phases of software development (design, testing, deployment and maintenance) and that dynamic analysis is required. However, section 4 states that "RepFTI can improve the observability of the target software." In conclusion it is claimed that "we can observe that different functions are distinguishable on the T-SNE plot, which is crucial to improving the security of complex vehicular software systems." Why is this so important? How can the T-SNE plot be used to improve security? In short, the theme of security runs throughout the manuscript, but it is not clearly presented how the proposed method improves security.

3. Vehicular software - This term refers to a software system that is part of the vehicle or is used for its management and function. This includes software for engine management, brakes, safety functions such as stability control systems, and navigation and communication software. Why is the proposed solution suitable for vehicular software? I see a few points of contention here. Firstly, an assembler for the x86 architecture has been discussed. The registers and instructions used suggest this, if I am not mistaken. Frankly, the papers the current work is based on do the same, but nowhere is it mentioned that they are for the automotive industry. I am not saying that the x86 architecture is not used, but I do not think it is very typical for this application. Secondly, the code used to train and test the system does not belong to this software category either.

4. Missing ablation study - The presented system has many components and two independent streams to create feature vectors. It would be good - and this is common in systems based on neural networks - to show how turning off certain parts of the system affects the quality of the final decision. This would also validate the entire concept.

5. Comparison with other solutions - In section 4.2.2. the authors compared RepFTI (in 4 variants) with three other methods. First, the current method names in the diagrams are DeepFTI instead of RetFTI. Secondly, both diagrams in Figure 6 are identical. Thirdly, variants of RepFTI are shown for different data sets, while the other methods are shown with one curve each. On which data were these methods tested or were the values simply taken from other work and for the completely different data sets?

6. FTI - The term "function type inference" is not standard and it should be explained what it means in the context of the current work (or provide a reference). This is also necessary because none of the methods/papers mentioned in Table 1 mention FTI and they are compared as FTI strategies. This needs to be defined right at the beginning of the work along with the motivation for the work itself. The motivation only becomes clear in sections 2.1 and 2.5.1. This should be pointed out right at the beginning.

7. Additional explanations

- It is necessary to additionally explain the general application of the cosine distance and then the context shown in Figure 4.

- The meaning of the t-SNE diagram is known, but the color coding in Figure 5 is not clear, nor is what can be specifically inferred from this image.

- The statement that the experiments were carried out on a server with a Snapdragon 8 Gen 2 processor is also not clear. Qualcomm's Snapdragon 8 series of mobile processors is not a stand-alone server processor.

Other comments:

Fig.1 - Avoid slanting the text boxes as the content is difficult to read. Also, there is no reference to Fig.1 in the text.

Comments on the Quality of English Language

The English is generally good, although minor corrections are still desirable (e.g. threat behaviors -> threat behavior).

Author Response

THE REPLY LETTER

----------------------------------------------------------------------------------------------------------------------

Paper Number: applsci-2992867

Title: RepFTI: Representation-fused Function Type Inference for Vehicular Secure Software Systems

Authors: Xiaoyu Yi, Gaolei Li, Jianhua Li and Ao ding

----------------------------------------------------------------------------------------------------------------------

Dear Editor and Reviewers,

We would like to thank the editor and the reviewers for the insightful comments and helpful suggestions on our paper. We have given careful consideration to each comment and suggestion made by the reviewers and editor for us. We also revised the previous manuscript according to these suggestions and comments.

In addition, we have revised several ambiguous words/terms, or text segments in order to make the manuscript clearer and more coherent. We now would like to introduce the way of MARKING the revision/addition that we have made in the revised submission, as follows:

• Added paragraphs/segments on the revised submission are indicated by red color and attached with a capital letter (i.e., ADD-1, ADD-2, etc.).
• Modified paragraphs/segments on the revised submission are indicated by blue color and attached with a number (i.e., MOD-1, MOD-2, etc.).
• A revised character/word or grammatical correction is indicated by blue color.

We have now substantially deepened the paper. The revisions include adding new statement paragraphs, ambiguous terms, and subsection/paragraph in order to have a smooth flow of explanation. This has resulted in a 15-page version of the revised manuscript. We only added or revised the contents to make our schemes clearer as well as more understandable and solid. We absolutely did not arise any content that is inconsistent with the previous manuscript.

This part summarizes the revisions that have been made by following the suggestions and comments made by reviewers:

• Abstract has been revised to further clearly describe function type inference, challenges, and why we conduct this research.
• Introduction has been revised to explain in detail the significance of function type inference and its two challenges: the abstract level interface hinder the effectiveness of dynamic analysis, and the local semantic similarity cause false positive. Meanwhile, we have added some clear and concrete examples to show how our method works.
• Related work has been revised to add description to illustrate how function type inference improve the security of in-vehicles software. Meanwhile, we have added more new examples about the function type inference can improve the security of in-vehicles software.
• Propose RepFTI framework has been revised to describe the goal of RepFTI, and adjust the description to further highlight the advantages of our schemes.
• Evaluation has been revised to add new ablation experiment. Moreover, when evaluating the parsing scheme, we have added a comparison with the latest function type inference strategies with semantic representation-based and graph representation-based.
• Discussion The results of experiment need to discuss for demonstrating the effectiveness of RepFTI in-vehicles software environment. We use the data of experiment to prove the goal attainment of RepFTI.
• Conclusion The conclusion section lacks vision for the further of function type inference. We add the discussion about this.

 

Once again, we would like to express our gratitude for your valuable time to review our revised manuscript. The locations of the revisions in the manuscript are shown in the following table:

Label	ADD-1	ADD-2	ADD-3	ADD-4	ADD-5	ADD-6	ADD-7	ADD-8
Page	1	2	2	12	10	1,3	9,10	1,3
Label	MOD-1
Page	8

 

From now on, please kindly be noted that we will particularly response to each comment or suggestion of the editor and reviewers.

 

To Reviewer 1:

===Comment 1=======================================================

Title - "RepFTI: Representation-fused Function Type Inference for Vehicular Secure Software Systems" - The title suggests that something has been proposed that will improve the security of software in vehicles. This of course sounds very interesting, but unfortunately the manuscript does not directly support these expectations. First, the question of how the proposed method supports software security should be answered. Secondly, what is so specific about the proposed method that it can be applied specifically in the automotive industry?

===================================================================

Response to Comment 1

We would like to thank the reviewer for scrutinizing our work and giving insightful comments. Our parsing scheme is dataset-agnostic and engine-agnostic. Our manuscript lacks the detailed description about the relationship between the function type inference and the security of software on vehicles, which leads that RepFTI look likes can not support these expectations of title. We add the role and effect of the function type inference in the security of software in vehicles. And the target functions of current works are abstract interface, which reduce the expandability of function type inference methods. We provide the solution of bypassing the heterogeneous abstraction level for clearly observe real execution path and lower level function effect.

       For reviewer’s retrieval convenience, the added content that state the relationship between the function type inference and the security of software in vehicles is marked as “ADD-1” on Page 1, with the following content:

There are many phases of the life cycle of in-vehicle software that need function type inference to improve the security of the in-vehicle software. Two classic phases need the function type inference to improve the security of software in vehicles. In the test phase of software in vehicles, due to the rapid iteration of the software, the software engineers need a more accurate tool or strategy to find the target function that matches the semantic of vulnerability or the key function. Similarly, in the software operating, the vulnerability or the code defects of software make the vehicles lose connection or can not be under control, which needs a function type inference tool to early detect suspicious functions in the execution process. In the field of in-vehicle software, since the new modules of software and the new protocols of platform level are proposed, the SOA frame that has abstractions and standardized services interfaces decouple the application development and underlying platform support [1]. However, the different vehicle manufacturers have heterogeneous abstraction levels and standardized interfaces, which causes a new SOA map to a new function type inference strategy. More specifically, the service functions in the underlying platform are packaged, which leads to the detailed semantics of the function of the software in vehicles being lost.

===Comment 2=====================================================

Security - The very first sentence of the abstract states: "For enhancing the security of vehicular software systems, inversely identifying function types of binary files play a key role in threat discovering." Why is the identification of functional types crucial for threat detection? RepFTI is then claimed to be a "framework for secure vehicular software systems". This raises the expectation that RepFTI can analyze the software and provide some level of security as well as point out potential security issues. Sections 2.1-2.4 indicate that security should be considered in all phases of software development (design, testing, deployment and maintenance) and that dynamic analysis is required. However, section 4 states that "RepFTI can improve the observability of the target software." In conclusion it is claimed that "we can observe that different functions are distinguishable on the T-SNE plot, which is crucial to improving the security of complex vehicular software systems." Why is this so important? How can the T-SNE plot be used to improve security? In short, the theme of security runs throughout the manuscript, but it is not clearly presented how the proposed method improves security.

===================================================================

Response to Comment 2

We would like to thank the reviewer for scrupulously scrutinizing our work and giving insightful comments. In the whole manuscript, we lack the clear outline of improving the security of software in vehicles by the function type inference. We add the generalization of principle and the practical examples for explaining that the function type inference can improve the security of software in vehicles, and then we further introduce the relationship between the problems of improving the security of software in vehicles and the proposed method.

       For reviewer’s retrieval convenience, the added content that state why and how the proposed method improves security is marked as “ADD-2” on Page 2, with the following content:

Binary code is often available for the software in vehicles. At the same time, the compiler does not remain enough language-level semantic information, such as the function type, the original data structure, and so on, in the process of compilation. More specifically, recovering the semantics of function is important for applications, such as bug-searching [10–12], code clone detecting [13,14], patching and repairing [15–17], and patch analysis [18–20]. The function type inference task is a classic research direction in recovering the semantic of function. The function type inference methods often use the reliable disassembly of instructions to recovery the control flow, the data structure, or function semantics for recognizing function type. Additionally, this recovered information with higher-level semantic description can provide the more specialized analysis tasks.

In the software of vehicles, the function type inference has two sub-problems: recovering the semantics within the function and the control flow between functions. At first, the abstract level functions are often met when we try to trace the execution process of in-vehicle software, but the general dynamic binary analysis tools are hardly able to identify the abstract level interface. Then, the semantic feature can only find a similar execution process within the function, but how can we filter some similar local execution processes with different global dependency relationships? These are mainly problems in existing works. In this work, we extract the feature of function and recognize the function type by the semantic and the control flow of functions. Our start point is a list of assembly instructions at the function level, and the goal is to produce the observable function type without any compilation information for early and clearly finding the vulnerabilities and code defects.

===Comment 3=====================================================

Vehicular software - This term refers to a software system that is part of the vehicle or is used for its management and function. This includes software for engine management, brakes, safety functions such as stability control systems, and navigation and communication software. Why is the proposed solution suitable for vehicular software? I see a few points of contention here. Firstly, an assembler for the x86 architecture has been discussed. The registers and instructions used suggest this, if I am not mistaken. Frankly, the papers the current work is based on do the same, but nowhere is it mentioned that they are for the automotive industry. I am not saying that the x86 architecture is not used, but I do not think it is very typical for this application. Secondly, the code used to train and test the system does not belong to this software category either. ===================================================================

Response to Comment 3

We would like to thank the reviewer for scrupulously scrutinizing our work and giving insightful comments. We lack the description of adaptive work about vehicles software, when we infer function type. This work belongs to experiment preparation phase, which make us ignore to describe the adaptive work of RepFTI for running the experiment smoothly. We add specifical dynamic analysis method for bypassing the abstract level Interface of SOA.

For reviewer’s retrieval convenience, the added content that state the adaptive work for vehicles software is marked as “ADD-3” on Page 2, with the following content:

For the abstract level interface, if RepFTI uses these interface functions as samples for the function type inference, RepFTI loses the effectiveness of function type inference when another SOA structure replaces the old one. So we use the symbol export table and the symbol import table to bypass the abstract level interface for finding the underlying functions. When an ELF needs to make functions or variables available for use in other ELF files, this behavior is called Symbol Exporting. The most typical case is that a library exports symbols for use in an executable file, and ELF stores the exported symbols in a ".dynsym "section. For dynamic linker to find and use. In Windows PE, the concept of symbol Export is similar, and all exported symbols are centralized in a structure called an Export Table. In fact, the export table, in its simplest form, provides a mapping between symbol names and addresses so that the corresponding address can be looked up by a symbol. Basically, each of these symbols is an ASCII string, and we know that the symbol name may or may not be the same as the corresponding function or variable name because of the mechanism of symbol decoration. Due to these loaded memory addresses of libraries are same, these exported function addresses are written into the symbol import table. We test the availability of the abstract interface at first. If it fails, we rebuild the executable. After importing the function binding, we use the symbol import table of SOA libraries to find these underlying functions, and these abstract interfaces are supported by them.

===Comment 4=====================================================

Missing ablation study - The presented system has many components and two independent streams to create feature vectors. It would be good - and this is common in systems based on neural networks - to show how turning off certain parts of the system affects the quality of the final decision. This would also validate the entire concept.

===================================================================

Response to Comment 4

We would like to thank the reviewer for these insightful suggestions. How do these modules of RepFTI work together to improve the performance of function type inference respectively, which is unknown for us? We will try shut down one modules of RepFTI to check what is the performance penalty of this single module for a single query process of function type inference?

For reviewer’s retrieval convenience, the added paragraph that compares our scheme to recent advances is marked as “ADD-4” on Page 12, with the following content:

       Due to the limited computing resources of vehicles, the in-vehicle software requires high transparency for deployed security modules. We check the time consumption of two main parts of RepFTI, including the semantic representation recognizing module and the graph representation recognizing module. We record 1000 times the query of a single module of RepFTI and calculate their average values. And then, these time consume recordings of related current works are collected as baseline data for clearly comparing time efficiency, which includes GMNs [29], HGMN [46] and HGNN [47] are graph representation-based function type inference; InnerEye [31], Asm2Vec [22] and EKLAVYA[35] are semantic representation-based function type inference. The result of the evaluation meets our expected. Since the graph representation recognition takes on auxiliary work, the structure of the function calling graph does not include more complex data. The graph representation module of RepFTI does not consume much time, but the semantic representation module needs to learn and recognize most assembly instructions, which makes the result of the semantic representation module similar to other existing works.

===Comment 5=====================================================

Comparison with other solutions - In section 4.2.2. the authors compared RepFTI (in 4 variants) with three other methods. First, the current method names in the diagrams are DeepFTI instead of RetFTI. Secondly, both diagrams in Figure 6 are identical. Thirdly, variants of RepFTI are shown for different data sets, while the other methods are shown with one curve each. On which data were these methods tested or were the values simply taken from other work and for the completely different data sets?

===================================================================

Response to Comment 5

We would like to thank the reviewer for scrutinizing our work and giving insightful comments. There are two situations, include the low compilation optimization levels O0 and the high compilation optimization levels O2, as different experiment environments for displaying the performance of RepFTI. And the result of ROC is generated by the result of all epochs, which make the exterior of two sub-figures are similar. So, we add subfigure title to describe different situations for getting a better distinction. The experiment mistakenly used the old version of the name, we apologize for this. And the tested data source of other methods is unclear.

For reviewer’s retrieval convenience, the added paragraph that compares our scheme to recent advances is marked as “ADD-5” on Page 10, with the following content:

We refer to the experiment results of “Word2vec”, “Asm2vec”, “EKLAVYA” in their publication as comparison items, which shows the effectiveness and robustness of RepFTI.

===Comment 6=====================================================

FTI - The term "function type inference" is not standard and it should be explained what it means in the context of the current work (or provide a reference). This is also necessary because none of the methods/papers mentioned in Table 1 mention FTI and they are compared as FTI strategies. This needs to be defined right at the beginning of the work along with the motivation for the work itself. The motivation only becomes clear in sections 2.1 and 2.5.1. This should be pointed out right at the beginning.

===================================================================

Response to Comment 6

We would like to thank the reviewer for these insightful suggestions. We should be pointed out the motivation, and clearly explain what “function type inference” means in the current work by providing related reference. We add the motivation about “function type inference” and replace these old references with these work that relate to FTI.

For reviewer’s retrieval convenience, the added paragraph that compares our scheme to recent advances is marked as “ADD-6” on Page 1 and Page 3, with the following content:

Binary code is often available for the software in vehicles. At the same time, the compiler does not remain enough language-level semantic information, such as the function type, the original data structure, and so on, in the process of compilation. More specifically, recovering the semantics of function is important for applications, such as bug-searching [10–12], code clone detecting [13,14], patching and repairing [15–17], and patch analysis [18–20]. The function type inference task is a classic research direction in recovering the semantic of function. The function type inference methods often use the reliable disassembly of instructions to recovery the control flow, the data structure, or function semantics for recognizing function type. Additionally, this recovered information with higher-level semantic description can provide more specialized analysis tasks.

===Comment 7=====================================================

- It is necessary to additionally explain the general application of the cosine distance and then the context shown in Figure 4.

- The meaning of the t-SNE diagram is known, but the color coding in Figure 5 is not clear, nor is what can be specifically inferred from this image.

- The statement that the experiments were carried out on a server with a Snapdragon 8 Gen 2 processor is also not clear. Qualcomm's Snapdragon 8 series of mobile processors is not a stand-alone server processor.

===================================================================

Response to Comment 7

We would like to thank the reviewer for scrutinizing our work and giving insightful comments. Manuscript lacks the detail description about the experiment result and experimental setup. We add the application scenarios of the cosine distance, and introduce the meaning about the position distribution of different color dots and the distance between these different color dots in detail. And We re-describe the experimental configuration in detail.

For reviewer’s retrieval convenience, the added paragraph that compares our scheme to recent advances is marked as “ADD-7” on Page 9 and Page 10 and “MOD-1” on Page 8, with the following content:

The cosine distance measures the difference between two vectors in direction, not in real distance. When a pair of texts have a large length difference in similarity but similar content, if word frequency or word vector are used as features, their Euclidean distance in feature space is usually large. If cosine similarity is used, the angle between them may be small, so the similarity is identified as high. So the similarity of text tasks is often determined by cosine distance.

The different color dots of Fig. 5 are two-dimensional representations of high-dimensional vectors that are function node features in the function call graph. And the different colors of dots map to a kind of function calling type. The dots with the same color are clustered together, and the dots with different colors have distinct boundaries, which proves the function calling type can be recognized according to the function node feature clearly.

All the experiments of the train phase and inference phase are conducted on a Xeon(R) 8259CL CPU 2.50GHz X 16, one GTX 1060 GPU, 6GB memory, and the data collecting phase is conducted on an in-vehicle computer with a Snopdrogon 8Gen2, 512 GB SSD.

===Comment 8=====================================================

Other comments: Fig.1 - Avoid slanting the text boxes as the content is difficult to read. Also, there is no reference to Fig.1 in the text.

===================================================================

Response to Comment 8

We would like to thank the reviewer for scrutinizing our work and giving insightful comments. The figure 1 has the slanted text boxes that reduce the readability of figure, and we lack reference of figure 1 in the text. Wd fix this problem in figure and add the reference of figure 1.

For reviewer’s retrieval convenience, the added paragraph that compares our scheme to recent advances is marked as “ADD-8” on Page 1 and Page 3, with the following content:

In this section, we will introduce the proposed RepFTI framework in detail, as shown in Fig. 2, which mainly includes a novel dataset generation method, a semantic learning model, and a graph learning model.

References:

The number here corresponds to the reference number in the revised paper.

 

• Yu, D.; Xiao, A. The Digital Foundation Platform–A Multi-layered SOA Architecture for Intelligent Connected Vehicle Operating 462 System. arXiv preprint arXiv:2210.08818 2022.
• Cao, Y.; Xiao, C.; Cyr, B.; Zhou, Y.; Park, W.; Rampazzi, S.; Chen, Q.A.; Fu, K.; Mao, Z.M. Adversarial sensor attack on lidar-based 433 perception in autonomous driving. In Proceedings of the Proceedings of the 2019 ACM SIGSAC conference on computer and 434 communications security, 2019, pp. 2267–2281.
• Dibaei, M.; Zheng, X.; Jiang, K.; Abbas, R.; Liu, S.; Zhang, Y.; Xiang, Y.; Yu, S. Attacks and defences on intelligent connected vehicles: A survey. Digital Communications and Networks 2020, 6, 399–421.
• Hu, S.; Zhang, Q.; Weimerskirch, A.; Mao, Z.M. Gatekeeper: A gateway-based broadcast authentication protocol for the in-vehicle Ethernet. In Proceedings of the Proceedings of the 2022 ACM on Asia Conference on Computer and Communications Security, 2022, pp. 494–507.
• Ebrahimi, M.; Marksteiner, S.; Niˇckovi ́c, D.; Bloem, R.; Schögler, D.; Eisner, P.; Sprung, S.; Schober, T.; Chlup, S.; Schmittner, C.; et al. A Systematic Approach to Automotive Security. In Proceedings of the International Symposium on Formal Methods.Springer, 2023, pp. 598–609.
• Haney, J.M.; Lutters, W.G. " It’s { . . It’s}{Confusing. . . It’s} Dull": How Cybersecurity Advocates Overcome Negative Perceptions of Security. In Proceedings of the Fourteenth Symposium on Usable Privacy and Security (SOUPS 2018), 2018, pp. 411–425.
• Jing, P.; Tang, Q.; Du, Y.; Xue, L.; Luo, X.; Wang, T.; Nie, S.; Wu, S. Too good to be safe: Tricking lane detection in autonomous driving with crafted perturbations. In Proceedings of the 30th USENIX Security Symposium (USENIX Security 21), 2021, pp. 3237–3254.

 

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

- Please highlight your state of the art in the "INTRODUCTION" section and
compare your findings in the "EVALUATION AND DISCUSSION". The introduction should contextualize your study and give any
specialized information the general measurement or control reader may
require to understand what follows. It must describe the importance of
relevant earlier work and the challenges your work solves. It should also
list your work's comparators. The introduction should define the article's
contribution(s) and show how it's shown in the rest of the manuscript. A
typical introduction should be as brief as possible and would contain the
following:
1). An outline of the problem.
2). A review of the relevant literature, noting briefly the major
contributors and indicating:
   - What the main contributors did?
   - What the main contributors found?
3). A statement of unsolved problems and/or areas requiring improvement;
particularly the one(s) considered in your manuscript.
4). In regard to the above, describe what you will perform that has not been
done before (what are your new contributions?).
5). An outline of how the following sections show what you did and how its
relevance will be demonstrated .

- Expand the full term of an acronym when it is mentioned for the first time.
- Avoid writing incomplete paragraphs. Every paragraph should have at least
three sentences, one as the main or key sentence and at least two as
supporting sentences.

The columns in Fig.1 are not enough good!

Comments on the Quality of English Language

Some typos should be fixed!

Author Response

THE REPLY LETTER

----------------------------------------------------------------------------------------------------------------------

Paper Number: applsci-2992867

Title: RepFTI: Representation-fused Function Type Inference for Vehicular Secure Software Systems

Authors: Xiaoyu Yi, Gaolei Li, Jianhua Li and Ao ding

----------------------------------------------------------------------------------------------------------------------

Dear Editor and Reviewers,

We would like to thank the editor and the reviewers for the insightful comments and helpful suggestions on our paper. We have given careful consideration to each comment and suggestion made by the reviewers and editor for us. We also revised the previous manuscript according to these suggestions and comments.

In addition, we have revised several ambiguous words/terms, or text segments in order to make the manuscript clearer and more coherent. We now would like to introduce the way of MARKING the revision/addition that we have made in the revised submission, as follows:

• Added paragraphs/segments on the revised submission are indicated by red color and attached with a capital letter (i.e., ADD-1, ADD-2, etc.).
• Modified paragraphs/segments on the revised submission are indicated by blue color and attached with a number (i.e., MOD-1, MOD-2, etc.).
• A revised character/word or grammatical correction is indicated by blue color.

We have now substantially deepened the paper. The revisions include adding new statement paragraphs, ambiguous terms, and subsection/paragraph in order to have a smooth flow of explanation. This has resulted in a 15-page version of the revised manuscript. We only added or revised the contents to make our schemes clearer as well as more understandable and solid. We absolutely did not arise any content that is inconsistent with the previous manuscript.

This part summarizes the revisions that have been made by following the suggestions and comments made by reviewers:

• Abstract has been revised to further clearly describe function type inference, challenges, and why we conduct this research.
• Introduction has been revised to explain in detail the significance of function type inference and its two challenges: the abstract level interface hinder the effectiveness of dynamic analysis, and the local semantic similarity cause false positive. Meanwhile, we have added some clear and concrete examples to show how our method works.
• Related work has been revised to add description to illustrate how function type inference improve the security of in-vehicles software. Meanwhile, we have added more new examples about the function type inference can improve the security of in-vehicles software.
• Propose RepFTI framework has been revised to describe the goal of RepFTI, and adjust the description to further highlight the advantages of our schemes.
• Evaluation has been revised to add new ablation experiment. Moreover, when evaluating the parsing scheme, we have added a comparison with the latest function type inference strategies with semantic representation-based and graph representation-based.
• Discussion The results of experiment need to discuss for demonstrating the effectiveness of RepFTI in-vehicles software environment. We use the data of experiment to prove the goal attainment of RepFTI.
• Conclusion The conclusion section lacks vision for the further of function type inference. We add the discussion about this.

 

Once again, we would like to express our gratitude for your valuable time to review our revised manuscript. The locations of the revisions in the manuscript are shown in the following table:

Label	ADD-9
Page	1,2
Label	MOD-2	MOD-3
Page	3,8	4

 

From now on, please kindly be noted that we will particularly response to each comment or suggestion of the editor and reviewers.

 

 

To Reviewer 2:

===Comment 1=====================================================

Please highlight your state of the art in the "INTRODUCTION" section and compare your findings in the "EVALUATION AND DISCUSSION". The introduction should contextualize your study and give any specialized information the general measurement or control reader may require to understand what follows. It must describe the importance of relevant earlier work and the challenges your work solves. It should also list your work's comparators. The introduction should define the article's contribution(s) and show how it's shown in the rest of the manuscript. A typical introduction should be as brief as possible and would contain the following:

1). An outline of the problem.

2). A review of the relevant literature, noting briefly the major contributors and indicating:

   - What the main contributors did?

   - What the main contributors found?

3). A statement of unsolved problems and/or areas requiring improvement; particularly the one(s) considered in your manuscript.

4). In regard to the above, describe what you will perform that has not been done before (what are your new contributions?).

5). An outline of how the following sections show what you did and how its relevance will be demonstrated.

===================================================================

Response to Comment 1

We would like to thank the reviewer for scrutinizing our work and giving insightful suggestions. The section of INTRODUCTION lacks the detail background discussion of current work in function type inference and the description of key problems in the current work of function type inference. And then, the section of EVALUATION AND DISCUSSION need display the discovering in the experiment inference phase.

For reviewer’s retrieval convenience, the added paragraph that add new description in the section of INTRODUCTION and EVALUATION AND DISCUSSION is marked as “ADD-9” on Page 1, Page2.

(For sub-comment 1) Binary code is often available for the software in vehicles. At the same time, the compiler does not remain enough language-level semantic information, such as the function type, the original data structure, and so on, in the process of compilation. More specifically, recovering the semantics of function is important for applications, such as bug-searching [10–12], code clone detecting [13,14], patching and repairing [15–17], and patch analysis [18–20]. The function type inference task is a classic research direction in recovering the semantic of function. The function type inference methods often use the reliable disassembly of instructions to recovery the control flow, the data structure, or function semantics for recognizing function type. Additionally, this recovered information with higher-level semantic description can provide more specialized analysis tasks.

(For sub-comment 3) To ensure the availability and security of vehicular software systems, software security engineers need to make different decisions during the software life cycle. The two classic phases need function type inference to improve the security of software in vehicles. In the test phase of software in vehicles, due to the rapid iteration of the software, the software engineers need a more accurate tool or strategy to find the target function that matches the semantic of vulnerability or the key function. Similarly, in the software operating, the vulnerability or the code defects of software make the vehicles lose connection or can not be under control, which needs a function type inference tool to early detect suspicious functions in the execution process. In the field of in-vehicle software, since the new modules of software and the new protocols of platform level are proposed, the SOA frame that has abstractions and standardized services interfaces decouple the application development and underlying platform support [1]. However, the different vehicle manufacturers have heterogeneous abstraction levels and standardized interfaces, which causes a new SOA to map to a new function type inference strategy. More specifically, the service functions in the underlying platform are packaged, which leads to the detailed semantics of the function of the software in vehicles being lost.

(For sub-comment 2,4,5) In the software of vehicles, the function type inference has two sub-problems: recovering the semantics within the function and the control flow between functions. At first, the abstract level functions are often met when we try to trace the execution process of in-vehicle software, but the general dynamic binary analysis tools are hardly able to identify the abstract level interface. And then, the semantic feature can only find a similar execution process within a function, but how can we filter that some similar local execution process with different global dependency relationships? These are mainly problems in existing works. In this work, we extract the feature of function and recognize the function type by the semantic and the control flow of functions. Our start point is a list of assembly instructions at function level, and the goal is to produce the observable function type without any compilation information for early and clearly finding the vulnerabilities and code defects.

===Comment 2=====================================================

Expand the full term of an acronym when it is mentioned for the first time.

===================================================================

Response to Comment 2

We would like to thank the reviewer for scrutinizing our work and giving insightful comments. We will review all acronym and expand the full term of them.

For reviewer’s retrieval convenience, the modified paragraph that add the full term of acronym is marked as “MOD-2” on Page 3, Page8.

Service Oriented Architecture (SOA), Bidirectional Long Short Term Memory (Bi-LSTM), Long Short Term Memory (LSTM).

===Comment 3=====================================================

Avoid writing incomplete paragraphs. Every paragraph should have at least three sentences, one as the main or key sentence and at least two as supporting sentences.

The columns in Fig.1 are not enough good!

Comments on the Quality of English Language: Some typos should be fixed!

===================================================================

Response to Comment 3

We would like to thank the reviewer for scrutinizing our work and giving insightful comments and suggestions. We will review all paragraphs for finding incomplete paragraphs, and then we check the grammar of whole manuscripts.

For reviewer’s retrieval convenience, the added paragraph that is marked as “MOD-3” on Page 4.

Although the application of FTI has significantly improved security in software design and implementation, software testing, software deployment, and software maintenance, it also faces many challenges and unsolved problems. Specifically, in the environment of in-vehicle software, these problems and challenges can be amplified by some new features of in-vehicle software. Table 1 summarizes and compares existing approaches based on the representation types, the splitting granularity of target binary files, the encoding schemes, and the model architecture.

 

 

References:

The number here corresponds to the reference number in the revised paper.

 

• Yu, D.; Xiao, A. The Digital Foundation Platform–A Multi-layered SOA Architecture for Intelligent Connected Vehicle Operating 462 System. arXiv preprint arXiv:2210.08818 2022.
• Cao, Y.; Xiao, C.; Cyr, B.; Zhou, Y.; Park, W.; Rampazzi, S.; Chen, Q.A.; Fu, K.; Mao, Z.M. Adversarial sensor attack on lidar-based 433 perception in autonomous driving. In Proceedings of the Proceedings of the 2019 ACM SIGSAC conference on computer and 434 communications security, 2019, pp. 2267–2281.
• Dibaei, M.; Zheng, X.; Jiang, K.; Abbas, R.; Liu, S.; Zhang, Y.; Xiang, Y.; Yu, S. Attacks and defences on intelligent connected vehicles: A survey. Digital Communications and Networks 2020, 6, 399–421.
• Hu, S.; Zhang, Q.; Weimerskirch, A.; Mao, Z.M. Gatekeeper: A gateway-based broadcast authentication protocol for the in-vehicle Ethernet. In Proceedings of the Proceedings of the 2022 ACM on Asia Conference on Computer and Communications Security, 2022, pp. 494–507.
• Ebrahimi, M.; Marksteiner, S.; Niˇckovi ́c, D.; Bloem, R.; Schögler, D.; Eisner, P.; Sprung, S.; Schober, T.; Chlup, S.; Schmittner, C.; et al. A Systematic Approach to Automotive Security. In Proceedings of the International Symposium on Formal Methods.Springer, 2023, pp. 598–609.
• Haney, J.M.; Lutters, W.G. " It’s { . . It’s}{Confusing. . . It’s} Dull": How Cybersecurity Advocates Overcome Negative Perceptions of Security. In Proceedings of the Fourteenth Symposium on Usable Privacy and Security (SOUPS 2018), 2018, pp. 411–425.
• Jing, P.; Tang, Q.; Du, Y.; Xue, L.; Luo, X.; Wang, T.; Nie, S.; Wu, S. Too good to be safe: Tricking lane detection in autonomous driving with crafted perturbations. In Proceedings of the 30th USENIX Security Symposium (USENIX Security 21), 2021, pp. 3237–3254.

 

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

This paper is up-to-date and relevant. It contains new results in the field of vehicle software engineering that are useful for practice. Modern vehicle software systems are complex and multifunctional. They control the operation of all units and components of the vehicle and take into account the condition of the road, route, and many other factors. Modern vehicle software systems use wireless information and communication technologies and have many service interfaces, which greatly expands the possibilities of attacks. To ensure the availability and security of vehicle software, software engineers must make various decisions throughout the software lifecycle. However, vehicle software developers don't want to open source code. In order to increase security, software developers release software systems as packaged executable programs. To help software engineers make correct and efficient decisions, the development of intelligent function type inference (FTI) tools is imperative.

The authors of this paper have developed a new representation-based function type inference (RepFTI) framework for ensuring the safety of vehicle software systems. The proposed RepFTI uses the technology of merging multiple representations to reconstruct the entire dependence of target functions on data to infer their types. The authors also proposed a new dynamic tracking strategy. This strategy allows you to simultaneously capture command sequences and function call schedules, obtaining the corresponding function type.

The paper contains new and significant information adequate to justify publication. It demonstrates an adequate understanding of the relevant literature in the field and cites an appropriate range of literature sources. The paper's argument is built on an appropriate base of theory. All the results are presented clearly and analyzed appropriately.

There are some remarks to the paper:

1) The abstract does not demonstrate the results obtained in the paper.

2) The relevance of the work is confirmed by references to the list of old publications [2, 4-8]. Add references to publications of the last 5 years.

3) What is the aim of the paper? This paper has no aim. The aim should be outlined in the introduction, and whether the aim was reached should be mentioned in the conclusion. The aim statement should be more justified.

4) The Discussion is too short. No numerical results are displayed in the Discussion. The numerical results of this study should be added to the Discussion.

5) The future of this study should be added to the Conclusions.

 

Overall, the paper is very good, containing new and useful results. Its content and the results obtained align with the aims and scope of the Applied Sciences Journal. The paper underscores the necessity for intelligent function type inference (FTI) tools in the field of vehicle software engineering. It can be accepted after minor revision.

Author Response

THE REPLY LETTER

----------------------------------------------------------------------------------------------------------------------

Paper Number: applsci-2992867

Title: RepFTI: Representation-fused Function Type Inference for Vehicular Secure Software Systems

Authors: Xiaoyu Yi, Gaolei Li, Jianhua Li and Ao ding

----------------------------------------------------------------------------------------------------------------------

Dear Editor and Reviewers,

We would like to thank the editor and the reviewers for the insightful comments and helpful suggestions on our paper. We have given careful consideration to each comment and suggestion made by the reviewers and editor for us. We also revised the previous manuscript according to these suggestions and comments.

In addition, we have revised several ambiguous words/terms, or text segments in order to make the manuscript clearer and more coherent. We now would like to introduce the way of MARKING the revision/addition that we have made in the revised submission, as follows:

• Added paragraphs/segments on the revised submission are indicated by red color and attached with a capital letter (i.e., ADD-1, ADD-2, etc.).
• Modified paragraphs/segments on the revised submission are indicated by blue color and attached with a number (i.e., MOD-1, MOD-2, etc.).
• A revised character/word or grammatical correction is indicated by blue color.

We have now substantially deepened the paper. The revisions include adding new statement paragraphs, ambiguous terms, and subsection/paragraph in order to have a smooth flow of explanation. This has resulted in a 15-page version of the revised manuscript. We only added or revised the contents to make our schemes clearer as well as more understandable and solid. We absolutely did not arise any content that is inconsistent with the previous manuscript.

This part summarizes the revisions that have been made by following the suggestions and comments made by reviewers:

• Abstract has been revised to further clearly describe function type inference, challenges, and why we conduct this research.
• Introduction has been revised to explain in detail the significance of function type inference and its two challenges: the abstract level interface hinder the effectiveness of dynamic analysis, and the local semantic similarity cause false positive. Meanwhile, we have added some clear and concrete examples to show how our method works.
• Related work has been revised to add description to illustrate how function type inference improve the security of in-vehicles software. Meanwhile, we have added more new examples about the function type inference can improve the security of in-vehicles software.
• Propose RepFTI framework has been revised to describe the goal of RepFTI, and adjust the description to further highlight the advantages of our schemes.
• Evaluation has been revised to add new ablation experiment. Moreover, when evaluating the parsing scheme, we have added a comparison with the latest function type inference strategies with semantic representation-based and graph representation-based.
• Discussion The results of experiment need to discuss for demonstrating the effectiveness of RepFTI in-vehicles software environment. We use the data of experiment to prove the goal attainment of RepFTI.
• Conclusion The conclusion section lacks vision for the further of function type inference. We add the discussion about this.

 

Once again, we would like to express our gratitude for your valuable time to review our revised manuscript. The locations of the revisions in the manuscript are shown in the following table:

Label	ADD-11
Page	2
Label	MOD-5	MOD-6
Page	2	2

 

From now on, please kindly be noted that we will particularly response to each comment or suggestion of the editor and reviewers.

 

 

To Reviewer 3:

===Comment 1=====================================================

The abstract does not demonstrate the results obtained in the paper.

===================================================================

Response to Comment 1

We would like to thank the reviewer for scrutinizing our work and giving insightful comments and suggestions. The result of RepFTI dose not display in abstract. We add the description about the goal of this manuscript and the result of RepFTI can achieve.

The modified paragraph to outline the paradigm of crowdsourcing malware family annotation is marked as “MOD-4” on Page 1, with the following content:

With RepFTI, the abstract interface of in-vehicle software will be bypassed, which proposes a promising direction for other work that relies on reverse engineering to improve software security. Compared with other works, the efficiency of RepFTI is improved by 34.47%, and the accuracy of RepFTI is improved by 28.98%.

===Comment 2=====================================================

The relevance of the work is confirmed by references to the list of old publications [2, 4-8]. Add references to publications of the last 5 years.

===================================================================

Response to Comment 2

We would like to thank the reviewer for scrutinizing our work and giving insightful comments. These old references have unbeneficial effect for understanding the condition of existing works. We will find some new references about the similarity field of [2,4-8].

For reviewer’s retrieval convenience, the modified content that state our two observations is marked as “MOD-5” on Page 2, with the following content:

[2] Cao, Y.; Xiao, C.; Cyr, B.; Zhou, Y.; Park, W.; Rampazzi, S.; Chen, Q.A.; Fu, K.; Mao, Z.M. Adversarial sensor attack on lidar-based perception in autonomous driving. In Proceedings of the Proceedings of the 2019 ACM SIGSAC conference on computer and 434 communications security, 2019, pp. 2267–2281.

[4] Dibaei, M.; Zheng, X.; Jiang, K.; Abbas, R.; Liu, S.; Zhang, Y.; Xiang, Y.; Yu, S. Attacks and defences on intelligent connected vehicles: A survey. Digital Communications and Networks 2020, 6, 399–421.

[5] Hu, S.; Zhang, Q.; Weimerskirch, A.; Mao, Z.M. Gatekeeper: A gateway-based broadcast authentication protocol for the in-vehicle Ethernet. In Proceedings of the Proceedings of the 2022 ACM on Asia Conference on Computer and Communications Security, 2022, pp. 494–507.

[6] Ebrahimi, M.; Marksteiner, S.; Niˇckovi ́c, D.; Bloem, R.; Schögler, D.; Eisner, P.; Sprung, S.; Schober, T.; Chlup, S.; Schmittner, C.; et al. A Systematic Approach to Automotive Security. In Proceedings of the International Symposium on Formal Methods.Springer, 2023, pp. 598–609.

[7] Haney, J.M.; Lutters, W.G. " It’s {Scary. . . It’s}{Confusing. . . It’s} Dull": How Cybersecurity Advocates Overcome Negative Perceptions of Security. In Proceedings of the Fourteenth Symposium on Usable Privacy and Security (SOUPS 2018), 2018, pp. 411–425.

[8] Jing, P.; Tang, Q.; Du, Y.; Xue, L.; Luo, X.; Wang, T.; Nie, S.; Wu, S. Too good to be safe: Tricking lane detection in autonomous driving with crafted perturbations. In Proceedings of the 30th USENIX Security Symposium (USENIX Security 21), 2021, pp. 3237–3254.

===Comment 3=====================================================

What is the aim of the paper? This paper has no aim. The aim should be outlined in the introduction, and whether the aim was reached should be mentioned in the conclusion. The aim statement should be more justified.

===================================================================

Response to Comment 3

We would like to thank the reviewer for scrutinizing our work and giving insightful comments. The lack of specifical scenarios and these sub-goals of manuscript with random order make the goal become the unclearly description. We rebuild the outline of the goal and these sub-goals.

For reviewer’s retrieval convenience, the added paragraph that state how vendors generate labels is marked as “MOD-6” on Page 2, with the following content:

In the software of vehicles, The function type is important for bug-searching [10–12], code clone detecting [13,14], patching and repairing [15–17], and patch analysis [18–20]. The function type inference has two sub-problems: recovering the semantics within the function and the control flow between functions. At first, the abstract level functions are often met when we try to trace the execution process of in-vehicle software, but the general dynamic binary analysis tools are hardly able to identify the abstract level interface. Then, the semantic feature can only find a similarity execution process within a function, but how can we filter some similar local execution processes with different global dependency relationships? These are mainly problems in existing works. In this work, we extract the feature of function and recognize the function type by the semantic and the control flow of functions. Our start point is a list of assembly instructions at function level, and the goal is to produce the observable function type without any compilation information for early and clearly finding the vulnerabilities and code defects.

==Comment 4=====================================================

The Discussion is too short. No numerical results are displayed in the Discussion. The numerical results of this study should be added to the Discussion. ===================================================================

Response to Comment 4

We would like to thank the reviewer for scrutinizing our work and giving insightful comments. The results of experiment need to discuss for demonstrating the effectiveness of RepFTI in-vehicles software environment. We use the data of experiment to prove the goal attainment of RepFTI.

For reviewer’s retrieval convenience, the added paragraph that state the data of experiment for proving the goal attainment of RepFTI is marked as “ADD-10” on Page 2, with the following content:

For evaluating intra-functional semantic learning capacity, we establish a baseline by attempting to restore the features of the actual target function. We measure the feature vector of the filled execution path and the key reference feature vector. The results demonstrate effective control of the cosine distance within 0.6, thereby increasing the accuracy of downstream model evaluation for the pre-trained model. These results align with our expectations and indicate that the recovery of data dependence within the function meets the target requirements. For evaluating the recognition capability of the function calling relationship, we analyze the structural characteristics of function calls for the target node based on the call structures between the target function node and its adjacent nodes across all execution paths. Using all nodes of the execution paths related to the target function under the condition of achieving 90\% code coverage for the same software, we evaluate the data. We use a T-SNE graph to visualize the clustering of the same function, and the final results demonstrate a good clustering effect for the same function across different execution paths.

===Comment 5=======================================================

The future of this study should be added to the Conclusions.

===================================================================

Response to Comment 5

We would like to thank the reviewer for scrupulously scrutinizing our work and giving insightful comments and suggestions. The conclusion section lacks vision for the further of function type inference. We add the discussion about this.

Meanwhile, we have added concrete examples to illustrate “smallest index” when we first describe the two observations in introduction, the modified content that adds concrete examples is marked as “ADD-11” on Page 2, with the following content:

       However, there is still room for improvement. The graph representation and semantic representation can be adaptively adjusted based on the analysis objective. Furthermore, annotating a large volume of data is a complex task, and improvements can be made in the annotation process for a wide range of function types.

 

References:

The number here corresponds to the reference number in the revised paper.

 

• Yu, D.; Xiao, A. The Digital Foundation Platform–A Multi-layered SOA Architecture for Intelligent Connected Vehicle Operating 462 System. arXiv preprint arXiv:2210.08818 2022.
• Cao, Y.; Xiao, C.; Cyr, B.; Zhou, Y.; Park, W.; Rampazzi, S.; Chen, Q.A.; Fu, K.; Mao, Z.M. Adversarial sensor attack on lidar-based 433 perception in autonomous driving. In Proceedings of the Proceedings of the 2019 ACM SIGSAC conference on computer and 434 communications security, 2019, pp. 2267–2281.
• Dibaei, M.; Zheng, X.; Jiang, K.; Abbas, R.; Liu, S.; Zhang, Y.; Xiang, Y.; Yu, S. Attacks and defences on intelligent connected vehicles: A survey. Digital Communications and Networks 2020, 6, 399–421.
• Hu, S.; Zhang, Q.; Weimerskirch, A.; Mao, Z.M. Gatekeeper: A gateway-based broadcast authentication protocol for the in-vehicle Ethernet. In Proceedings of the Proceedings of the 2022 ACM on Asia Conference on Computer and Communications Security, 2022, pp. 494–507.
• Ebrahimi, M.; Marksteiner, S.; Niˇckovi ́c, D.; Bloem, R.; Schögler, D.; Eisner, P.; Sprung, S.; Schober, T.; Chlup, S.; Schmittner, C.; et al. A Systematic Approach to Automotive Security. In Proceedings of the International Symposium on Formal Methods.Springer, 2023, pp. 598–609.
• Haney, J.M.; Lutters, W.G. " It’s { . . It’s}{Confusing. . . It’s} Dull": How Cybersecurity Advocates Overcome Negative Perceptions of Security. In Proceedings of the Fourteenth Symposium on Usable Privacy and Security (SOUPS 2018), 2018, pp. 411–425.
• Jing, P.; Tang, Q.; Du, Y.; Xue, L.; Luo, X.; Wang, T.; Nie, S.; Wu, S. Too good to be safe: Tricking lane detection in autonomous driving with crafted perturbations. In Proceedings of the 30th USENIX Security Symposium (USENIX Security 21), 2021, pp. 3237–3254.

 

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The authors have responded to most of the comments and have thus certainly improved the comprehensibility of the paper. Remains unconfirmed:

- the importance of the x86 architecture for the automotive industry (the text mentions the registers and instructions characteristic of this architecture)

- the ablation study and

- the improvement of the readability of the text in Figure 1.

Maybe the comments mentioned are not that important, or maybe we did not understand each other, but I still believe that clarifying certain points would improve the quality of the work.

1. Why is assembly code used specifically for the x86 architecture? Give at least one example that justifies this. At what point in the development of the vehicle software are the libraries used for training and validating the model? In the test platform, the Snapdragon 8 rev. 2 was used, which is based on ARMv9 architecture (not x86 or A64). This part is still unclear to me.

2. We seem to have misunderstood each other regarding the ablation study. RepFTI uses graph and semantic representation together to produce better results than the competition. It would be interesting to see how efficient the classification of features is when only the semantic representation module and only the graph representation module are used, compared to the combined effect of both modules. Figure 8 shows that each of these modules on its own is more efficient than the other solutions in terms of execution time for a given code, which is certainly interesting. However, it remains unclear how effective the application of the "single stream off" model is in classifying features and how much more effective the application of both streams is than each individually. If I understand it correctly, the advantage lies in combining both streams. So this should be demonstrated versus applying each stream individually. Also, in Figure 8, the meaning of the terms "Normal Time" and "FTI Time" is not clear.

3. The slanted text in Figure 1 has not been corrected. Perhaps the authors think that it is esthetically nicer if the text (step 1 to step 4) and the boxes are slanted, but the readability of such text is worse.

Author Response

THE REPLY LETTER

----------------------------------------------------------------------------------------------------------------------

Paper Number: applsci-2992867

Title: RepFTI: Representation-fused Function Type Inference for Vehicular Secure Software Systems

Authors: Xiaoyu Yi, Gaolei Li, Jianhua Li and Ao ding

----------------------------------------------------------------------------------------------------------------------

Dear Editor and Reviewers,

We would like to thank the editor and the reviewers for the insightful comments and helpful suggestions on our paper. We have given careful consideration to each comment and suggestion made by the reviewers and editor for us. We also revised the previous manuscript according to these suggestions and comments.

In addition, we have revised several ambiguous words/terms, or text segments in order to make the manuscript clearer and more coherent. We now would like to introduce the way of MARKING the revision/addition that we have made in the revised submission, as follows:

• Added paragraphs/segments on the revised submission are indicated by red color and attached with a capital letter (i.e., ADD-1, ADD-2, etc.).
• Modified paragraphs/segments on the revised submission are indicated by blue color and attached with a number (i.e., MOD-1, MOD-2, etc.).
• A revised character/word or grammatical correction is indicated by blue color.

We have now substantially deepened the paper. The revisions include adding new statement paragraphs, ambiguous terms, and subsection/paragraph in order to have a smooth flow of explanation. This has resulted in a 15-page version of the revised manuscript. We only added or revised the contents to make our schemes clearer as well as more understandable and solid. We absolutely did not arise any content that is inconsistent with the previous manuscript.

This part summarizes the revisions that have been made by following the suggestions and comments made by reviewers:

• Abstract has been revised to further clearly describe function type inference, challenges, and why we conduct this research.
• Introduction has been revised to explain in detail the significance of function type inference and its two challenges: the abstract level interface hinder the effectiveness of dynamic analysis, and the local semantic similarity cause false positive. Meanwhile, we have added some clear and concrete examples to show how our method works.
• Related work has been revised to add description to illustrate how function type inference improve the security of in-vehicles software. Meanwhile, we have added more new examples about the function type inference can improve the security of in-vehicles software.
• Propose RepFTI framework has been revised to describe the goal of RepFTI, and adjust the description to further highlight the advantages of our schemes.
• Evaluation has been revised to add new ablation experiment. Moreover, when evaluating the parsing scheme, we have added a comparison with the latest function type inference strategies with semantic representation-based and graph representation-based.
• Discussion The results of experiment need to discuss for demonstrating the effectiveness of RepFTI in-vehicles software environment. We use the data of experiment to prove the goal attainment of RepFTI.
• Conclusion The conclusion section lacks vision for the further of function type inference. We add the discussion about this.

 

Once again, we would like to express our gratitude for your valuable time to review our revised manuscript. The locations of the revisions in the manuscript are shown in the following table:

Label	ADD-1	ADD-2
Page	2	6，7
Label	MOD-1
Page	1

 

From now on, please kindly be noted that we will particularly response to each comment or suggestion of the editor and reviewers.

 

 

To Reviewer 1:

===Comment 1=======================================================

Why is assembly code used specifically for the x86 architecture? Give at least one example that justifies this. At what point in the development of the vehicle software are the libraries used for training and validating the model? In the test platform, the Snapdragon 8 rev. 2 was used, which is based on ARMv9 architecture (not x86 or A64). This part is still unclear to me.

===================================================================

Response to Comment 1

We would like to thank the reviewer for scrutinizing our work and giving insightful comments. The reason that evaluation experiments focus on the assembly code of X86 architecture need a sample of the vehicle software development to demonstrate the tested libraries are used for this training and validating. The ARMv9 architecture is used for this experiment evaluation based on X86 architecture. First, due to the experiment environment being the virtualized E/E architecture of a vehicle based on the android car (X86 architecture), the assembly code X86 or A64 architecture is used to dynamically analyze a single executable file or software in the test phase. And, after we discussed with the team that built this experiment environment, the Snapdragon 8 rev. 2 is the wrong term in this test platform. We change the detailed description for introducing the experiment environment.

       For reviewer’s retrieval convenience, the added content that state the detail information about the experiment environment is marked as “MOD-1” on Page 1, with the following content:

Due to the lack various of vehicles, to test as many vehicle environments as possible, we built a virtualized host-based android for the CAN communication. All the experiments of the train phase and inference phase are conducted on the virtualized E/E architecture of a vehicle based on android car (X86 architecture) that is run on a Xeon(R) 8259CL CPU 2.50GHz X 16, one GTX 1060 GPU, 6GB memory.

===Comment 2=====================================================

We seem to have misunderstood each other regarding the ablation study. RepFTI uses graph and semantic representation together to produce better results than the competition. It would be interesting to see how efficient the classification of features is when only the semantic representation module and only the graph representation module are used, compared to the combined effect of both modules. Figure 8 shows that each of these modules on its own is more efficient than the other solutions in terms of execution time for a given code, which is certainly interesting. However, it remains unclear how effective the application of the "single stream off" model is in classifying features and how much more effective the application of both streams is than each individually. If I understand it correctly, the advantage lies in combining both streams. So this should be demonstrated versus applying each stream individually. Also, in Figure 8, the meaning of the terms "Normal Time" and "FTI Time" is not clear.

===================================================================

Response to Comment 2

We would like to thank the reviewer for scrupulously scrutinizing our work and giving insightful comments. The effectiveness of the “single stream off” of the RepFTI model should add control experiment in classifying features, and how much the advantage of both streams application than the individual stream application. The term “Normal time” and “FTI Time” is not clear description. We will try to close the single stream and recode the accuracy and standard deviation of function type identification. And, we change the term “Normal Time” to the term “The execution time of RepFTI without deployment”, and the term “FTI Time” is changed to the term “The execution time after deploying RepFTI”.

       For reviewer’s retrieval convenience, the added content that state why and how the proposed method improves security is marked as “ADD-1” on Page 2, with the following content:

To further test the effectiveness of the single representation of RepFTI, we close the semantic representation model and the graph representation model respectively for recording the corresponding accuracy of function type inference. For testing the effectiveness of function type inference based on the semantic representation model, we select current works as a baseline to clearly show the performance of the semantic representation model of RepFTI. Therein, these current works also use semantic representation to recognize the different parameters of the function, like the input type, the core logic, and so on. Therein, we select 1000 execution paths that can cover most code in each library, which are used as input parameters for this evaluation experiment. Then, in the same way, we also find similar current works about using a graph representation model and complete RepFTI (semantic+graph) to infer function type, which helps us to test the effectiveness based on graph representation and the effectiveness based on RepFTI. The experiment result is as expected. Using graph representation to recognize function type does not have excellent results, because the functions with similar call graph structure could be different functions in high probability. Although, using semantic representation to recognize function type has promising accuracy, but the standard deviation is higher than other current works. This status shows that the recognition of function types only by the semantic representation model has high instability.

===Comment 3=====================================================

The slanted text in Figure 1 has not been corrected. Perhaps the authors think that it is esthetically nicer if the text (step 1 to step 4) and the boxes are slanted, but the readability of such text is worse. ===================================================================

Response to Comment 3

We would like to thank the reviewer for scrupulously scrutinizing our work and giving insightful comments. This is another mistake, we will correct the slanted text and text box.

For reviewer’s retrieval convenience, the added content that state the adaptive work for vehicles software is marked as “ADD-2” on Page 6,7, with the following content:

Author Response File: Author Response.pdf