Sysbench fileio read tests and the --validate option

Last updated on

Late in March 2022 a provider reported to VPSBenchmarks a flaw in the sysbench fileio read metrics: when Sysbench writes the files used for testing in a prepare command with the default parameters, the test files are filled with zeros. That's problematic because some storage hardware optimizes for this situation and when it's asked to read zero filled files, it only reads the metadata and skips reading the actual files. Obviously this is much faster than it would be with normal files. This gives an unfair advantage to providers using this type of hardware.

Fortunately, Sysbench has a command-line option called "validate". The documentation for the validate option says: "Perform validation of test results where possible (default: off)".
Further down in the man page in the fileio section, they add: "When the global --validate option is used with the fileio test mode, Sysbench performs checksums validation on all data read from the disk. On each write operation the block is filled with random values, then the checksum is calculated and stored in the block along with the offset of this block within a file. On each read operation the block is validated by comparing the stored offset with the real offset, and the stored checksum with the real calculated checksum."

This is what we want: the fileio prepare command should fill the test files with random bytes and we think this should be the default behavior of the fileio prepare command.

After the bug was verified end of March 2022, VPSBenchmarks started running fileio read tests with both the "validate" option and without it and collecting the new metrics. Review of the data for the 27 trials that have run since reveals that about 25% of them returned read metrics that were substantially higher (and thus wrong) without the validate option.

Switching to the new metrics took some time because VPSBenchmarks grades represent the relative performance of a server compared to others on every single metric. Calculating the grade on a metric requires having enough trials with the new metric to have meaningful min, max, median and several percentiles values. But as of mid-May 2022, we have enough data, all Sysbench fileio read tests are run using the validate option and all grades are based on those new metrics for new trials.




Screeners

Find the VPS that fits your requirements in seconds


Screener

Best VPS 2022

Provider Finder


Share this page

Latest Tweets by @vpsbenchmarks

Blog

Sysbench fileio read tests and the --validate option Late in March 2022 a provider reported to VPSBenchmarks a flaw in the sysbench fileio read metrics: ...

More...

Be Featured on VPSBenchmarks

Cloud Server Providers, would you like your products to be tested at vpsbenchmarks.com?

Check out our pricing page and sign up

Disclaimer

We are proud to provide objective and impartial benchmark data on this website. VPSBenchmarks receives support from some of the providers featured here but all tests are conducted the same way regardless of our relationships with them.

If you find our benchmarks valuable, you can help by making your cloud purchases using the Provider Affiliate Buttons displayed throughout the site.

Be the first to learn about new Best VPS rankings. Subscribe to our newsletter.