#704 closed Feature Request (wontfix)

ZFS and heavy resource utilization

Reported by: friedg Owned by:
Priority: major Milestone:
Component: System Configuration Version: 9.1-RC3
Keywords: zfs, ufs Cc: trac-bugs@…


PCBSD with zfs has high resource (processing and memory) utilization and has been the target of multiple user complaints on web forums. The canned answer is usually with regard to kernel memory allocation (mem_wire, mem_active, mem_inactive, and mem_cache) for optimal performance and that unused memory is wasted memory.

The complaint is not due to kernel maximizing memory allocation, it's with respect to mem_active memory being considerably higher than if a ufs or a Linux ext3 or 4 file system were being used. ZFS has an inherently higher mem_active memory utilization due to the arc algorithm and sha256 checksum operations on hard drive reads. Increasing the zfs cache puts a higher load on mem_active and decreasing zfs cache puts a higher load on the processor, even at idle. This may lead to a certain level of dissatisfaction with regard to system performance and give PCBSD a poor reputation. I've noticed 100mb of mem_active being used on a headless PCBSD/zfs installation, which is far higher than a normal 20mb with ufs. I've also witnessed 50% processor utilization when the system is idle with a core2quad processor.

The proponents of zfs stipulate the data integrity advantage it has with respect to ufs, ext4, and fat32. However, the end user may not recognize this, especially with poor performance. A compromise could be if the default pcbsd installation utilized ufs for system partitions and zfs for user data (/usr and /home).

Change History (2)

comment:1 Changed 14 months ago by mjollnirpaul

1st: The default checksum is fletcher4 and should need _much_ less CPU than sha256.

1b: sha256 is the default when deduplication is enabled for the zpool. Thus, I guess it's clear why some curious users complain about zfs beeing slow and causing a high load on their system. They are just happy to try out new things w/o looking for information first... ;)

1c: It could be an idea too write a short explanation in the manual/handbook, release notes, maybe even patch the zfs utility to print out a big fat warning about the consequences of enabling deduplication and that the common understanding is that it is nonsense for >99.999% of desktop usage scenarios and needs about 5G RAM per 1T disk space.

1d: This also could explain 50% CPU usage.

1e: Another reason for high CPU load is clearly compression, necessarily if you start scratching the plates on a gzip'ed filesystem the CPU load will go up. gzip-compression is only a good choice for filesystem that are rarely used!

1f: default compression (lzjb) is ok for a rotational HD, questionable on an SDD (i.e. you should think about it, test it, use only for less often used FS, etc) and bogus for SSD that do compression themself.

2nd: The arc size automagically adjusts itself to the systems utilisation, i.e. it shrinks when the system needs more memory for applications or the kernel. This works fine for >99.999% of typical desktop usage.

3rd: Given that "unused memory is wasted memory", 100M active mem on ZFS is 5 times better than 20M on UFS, right? It should be clear that the cacheing scheme of UFS is much simpler and inferiour to an ARC in most usage scenarios.

comment:2 Changed 14 months ago by kris

  • Resolution set to wontfix
  • Status changed from new to closed

ZFS is an enterprise grade file-system, and as such will rely more heavily on system resources. The user is welcome to do additional "tuning" to fit their specific needs, but that always has a cost, possibly speed in the case of shrinking the ARC size, etc. I'm not sure I like the idea of doing a hybrid ZFS / ufs install though, there are HUGE advantages to having dedicated ZFS disks, boot-environments, reliability, check-summing, scrubs, adding disks to the pool, etc.

Note: See TracTickets for help on using tickets.