Real world benchmarking of filesystems

havoc

Busy exploring ZFS with all it's dials and knobs available for tuning. Using fio as first go for testing, but starting to realize it's poorly suited to the task at hand.

Simple example - the fio disk tests are random data. So can't see the effect of compression vs no compression. Same for all the caching levels. Same for deduplication impact. fio is a disk test not a filesystem test

So trying to figure out what else can I use? I can whip up a script doing some stuff like deploying a VM/LXC etc and time it. Copy files around etc.

Wondering whether there is something prebuilt like that already though before I whip out my DIY bash skills...cough LLM cough

Neoon

Use a YABS to show off

havoc

@Neoon said:
Use a YABS to show off

Alas I don't think I'm impressing you lot with my janky homeserver.

But yeah basically want something that feels snappy for real world homelab proxmox use. So if I'm gonna spend a bunch of time tuning may as well try to measure the right metrics

legendary

@havoc said: I whip out my DIY bash skills

Are you self-proclaimed bash maister? Can you rewrite autoconf scripts?

P.S. As mild training, you may study x86 assembler written in bash: https://lists.gnu.org/archive/html/bug-bash/2001-02/msg00054.html

havoc

@legendary said:

@havoc said: I whip out my DIY bash skills

Well sometimes it takes me two tries to figure out whether the script needs to start with #! or !# but yes absolute bash master right here

skhron

@havoc said: Simple example - the fio disk tests are random data.

https://linux.die.net/man/1/fio

Have you tried this knob?

zero_buffers
    Initialise buffers with all zeros. Default: fill buffers with random data. 

yoursunny

We use postmark for I/O speed benchmarks.

havoc

@yoursunny said:
We use postmark for I/O speed benchmarks.

Oh that looks promising. Got an error on trying to compile it.

/* Counters */
int files_created;  /* number of files created */
int files_deleted;  /* number of files deleted */
int files_read;     /* number of files read */
int files_appended; /* number of files appended */
float bytes_written; /* number of bytes written to files */
float bytes_read;    /* number of bytes read from files */

@skhron said:

@havoc said: Simple example - the fio disk tests are random data.

https://linux.die.net/man/1/fio

Have you tried this knob?

zero_buffers
    Initialise buffers with all zeros. Default: fill buffers with random data. 

Wasn't aware of it, but ultimately same issue - zeros compress too well. Also not anything near realistic

havoc

Decided to build custom scripts that simulate real proxmox tasks. Want to set up tuned ZFS volumes for each so don't think i'll get around real world tasks. So far LXC testing looks like this:

Created container in seconds:
4.04
Started container in seconds
3.50
Updated container in seconds
3.35
File search in seconds
11.00
Ran histogram in seconds
1.63
Stopped container in seconds
2.20
Destroyed container in  seconds
3.51
All except update
25.88

No just need to figure out a way to wrap this in a loop to test various variables

yoursunny

@havoc said:

@yoursunny said:
We use postmark for I/O speed benchmarks.

Oh that looks promising. Got an error on trying to compile it.

postmark is available in Debian repository:

https://packages.debian.org/bookworm/postmark
https://packages.ubuntu.com/noble/postmark