High-Performance Computing Storage Performance and Design Patterns—Btrfs and ZFS Performance for Different Use Cases

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The article evaluates advantages and drawbacks of btrfs and ZFS file systems. The paper is well written and organized.

 

The main contribution of the paper is a comparison between btrfs and ZFS file systems. 

 

The weak point of the article is that no figures were referenced in the text with enough explanations. Besides, the authors should have provided more details about the results presented in the charts (i.e. how many times ZFS was better or worse for a given metric). Also, I think the authors should have compared ext4 file system or, at least, give a very good reason to why not it was compaered.

 

 

 

Why the article did not evaluated ext4 e XFS?? They are older??? outdated ??? I think the authors should explain why they did not evaluated ext4 e XFS?? I think the authors should, at least, evaluate ext4 that continues to be widely used because of its well-132 rounded performance and dependability. 

 

Actually, I think the authors should mention if all these file systems are standard available during the installation or if it has to be installed after.

 

 

The authors did not referenced the figures in the text. All figures should be referenced in the text with an expalnation.

 

 

I think the authors should cite articles about the file systems: ext4, XFS, btrfs, and ZFS.

 

I think the authors should cite articles about FAT and NTFS.

 

Figure 3 should be referenced and detailed explained in the text, saying, for example, how many times ZFS was better for sequential read. What is the value in the y-axis????

 

The explanation for Figure 4 is also very short and do not reference the Figure.  What is the value in the y-axis???? The authors should explain in detail each test that were carried out and in what situations it is better ZFS or btrfs.

 

 

What is the value in the y-axis for figure 5, 6, 7, 8 and 9  ????

 

Typos 

 

line 31: "considered [1]There" -> "considered [1]. There"

 

 

lines 47-48: the following sentence should be corrected -> "for a test High-Performance Computing (HPC) application 47 that we selected for the test."

 

In line 323 describe the acronym FIO

Comments on the Quality of English Language

The article is well written.

Author Response

Dear reviewer,

Thank you for your input. I will take all of your comments into account when I send the paper revision. I need to do a couple of rounds of tests for that, which will require a bit of time, but I'll submit the paper revision in the next 48 hours or so. Thank you for your time and suggestions.

Regarding the first reviewer’s comments:

• We added references to all the figures.
• EXT4 and XFS were added to the text and results.
• We added text about FAT and NTFS and references about those two filesystems.
• We re-did all our charts so that they’re much easier to read with values for the X/Y axis.
• We added EXT4 and XFS to our charts for a more extensive data set.
• We corrected the cited typo in the text.
• We corrected the sentence as per the reviewer’s request.
• We added the explanation for the FIO acronym.

Reviewer 2 Report

Comments and Suggestions for Authors

The proposed work  for review has a contemporary theme, because storage systems are used by more and more people, and probably almost everyone has their own, remote or home storage. Therefore, choosing an appropriate file system is important. The work is written comprehensibly, without complex expressions. The work has the following strengths:
1. The authors have made a brief but very concise overview of the file systems currently in use;
2. They have looked in detail at the two file systems that they will compare in their work, mentioning all their capabilities;
3. A comparison was made between the two file systems related to the most important functionalities they support

The work also has some flaws that, if removed, will make their study even more in-depth and interesting:
1. The literature is quite old. Part of it should be replaced with a new one, from the last 4-5 years;
2. It's a good idea to add a new section titled "Problem Status" or some other, more suitable title. In this section, the authors can explain what prompted them to do this research, why such research should be done, and more;
3. To add a new section entitled "Experimental setup and study methodology" or another, according to the authors, more appropriate title. In this section, the authors should describe the experimental setup - what they used to do the research: workstation or server; what operating system was used; what kind of storage system (local on USB or NAS) was used; what kind of disks were used (sata, SAS, SSD, NVME, Ramdisk, if SAS disks were used, at what speeds 10k or 15k, etc..............). Describe what tool/program was used to simulate the different situations they investigated (Sequential read, Random access, mixed load,............. etc.);
4. To section 6, if possible, add one more study that is related to the applicability of the considered file systems when accessing/using them remotely (via the Internet). Since almost everyone now has remote disk space, it would be interesting to study how and if file sharing over the Internet has any effect on the productivity/performance of the two file systems. Such research would enrich the work even more.

Author Response

Dear reviewer,

Thank you for your input. I will consider all of your comments when I send the paper revision. The only thing that I'd prefer not to do is ramdisk, for two reasons. First, for the intended use cases, especially HPC, ramdisk sizes on servers are too small, making it unfeasible to use ramdisk from a technology standpoint. The second, albeit a bit less critical, reason. Ramdisk on Linux is a technology that almost everyone avoids using, as it requires loading while booting the OS and storing when powering the server off. I need to do a couple of rounds of tests and add the information that you asked for, which will require some time. Still, I'll submit the paper revision in the next 48 hours or so. Thank you for your time and suggestions; I sincerely appreciate that. 

Regarding the  reviewer’s comments:

• We added many more references, especially more contemporary ones.
• We added the “Problem status” section.
• We added the “Experimental setup and methodology" section, explaining the tools and hardware used.
• Although requested, we didn’t include any file-sharing data in our results as this use case does not apply to our paper – we can’t imagine someone having a running database or an HPC app running via SMB or NFS over the Internet – it would be unbearably slow, with high latency and problematic from a security perspective. However, we note that this would be an excellent topic to cover in a different paper related to a different topic – the suggestion merits further research. We’re seriously considering doing a paper on this topic in the future, but more from the home user/office user perspective.

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

Everything is OK. I do not have any other remarks or comments.