Lossless audio codec comparison revision 5 - hi-res part

Introduction

This document compares the performance of lossless audio codecs on high resolution material, which are PCM audio files with more than 16 bit per sample or a samplerate higher than 48kHz.

The test corpus consists of four 96kHz, 24 bit per sample sources, three 192kHz, 24 bit per sample sources, one 192kHz, 16 bit per sample source and two 352.8kHz, 24 bit per sample sources.

Method

To compare the performance of each codec, the following steps are followed for each combination of corpus file, codec and codec setting:

• A WAV file is placed on a large enough ramdisk
• An MD5sum is calculated for the WAV file excluding its header
• The WAV file is encoded by the chosen codec provided with the required settings. The amount of CPU time required to do this conversion is measured and the resulting filesize is recorded
• The encoded file is decoded by the chosen codec. The amount of CPU time required to do this conversion is measured
• An MD5sum is calculated for the resulting decoded file excluding its header
• The MD5sum of the provided WAV file and the decoded file are compared

The following codecs and settings are used:

Codec	Settings used
FLAC	-0, -1, -2, -3, -4, -5, -6, -7, -8
ALAC	--fast, [-]
WavPack	-f, [-], -h, -hh, -x4f, -x4, -x4h, -x4hh
TAK	-p0, -p0e, -p0m, -p1, -p1e, -p1m, -p2, -p2e, -p2m, -p3, -p3e, -p3m, -p4, -p4e, -p4m
OptimFROG	presets 0 through 10 and max
Monkeys Audio	-c1000, -c2000, -c3000, -c4000, -c5000
La	[-], -high
MP4ALS	-a -o5, -a, -a -o20, -a -o40, -a -b, -a -b -o20, -a -b -o40, -a -b -o1023, -7
WMA	[-]
Shorten	[-]
TTA	[-]

For each codec, the latest Windows binary provided by the author of the codec is used, no specially tuned compiles are used. In case of ALAC, encoding is done with refalac64 as provided by QAAC. In case of WMA, encoding is done with WMAEncode.exe (which uses the encoder provided by Windows 10) and decoding with FFmpeg 5.0.

Not all codecs appear in all results, for example Shorten and La only support 16 bit per sample sources and WMA does not support samplerates above 96kHz.

Measurements are made on a Windows 10 machine with a AMD A4-5000 CPU with 4GB of RAM. This CPU has all x86 instruction set extensions up to and including AVX (i.e. lacks AVX2). Measuring the CPU-time used is done with timer64.exe, part of the 7-max/7-benchmark suite.

Timing is done per track. This measured time is divided by the content length, i.e. the execution time of the encoding or decoding process in seconds divided by the playback length of the track or chapter in seconds. The result of this division is called CPU-usage. The filesize of the encoded file is divided by the filesize of the original WAV file to calculate a compression.

Results per source are obtained by averaging the compression and CPU-usage so each track or chapter contributes the same amount to the average, i.e. length of the track or chapter is not incorporated.

The average of all sources is obtained by averaging the results per source, again without any weighing. The total results are therefore not influenced by the length of the corpus content, each source contributes an equal amount to the average.

Results (PDF)

• Average of all hi-res sources
• Average of hi-res sources up to 192kHz
• Results per source, PDF, CSV

The best results will be obtained in the bottom left corner of the graphs: this represents the best compression (smallest file size) and the lowest CPU-usage (fastest compression and/or decompression).

In the graph each codec is represented by a group of markers connected by a line. Each combination of settings as mentioned in the previous table corresponds to one marker, in the order listed in the table. The first combination of settings is (usually) the fastest and the last the slowest. Therefore, the marker closest to the upper left corner of the graph corresponds to the first listed combination of settings, and the marker closest to the lower right corner corresponds to the last listed combination of settings.

Discussion

Looking at the average of hi-res sources up and until 192kHz, TAK seems to be the clear winner. There is not a single combination of settings for TAK where it is beat in both file size and either encoding or decoding speed by any other codec. Its decoding speed is second only to FLAC, but the difference is small while producing much smaller files.

When looking at all hi-res sources, TAK is no longer present (as it does not support sample rates above 192kHz). If encoding speed is deemed of lesser importance than decoding speed, WavPack performs very well when used with the -x4 setting.

TTA holds a interesting spot, with reasonably fast encoding and decoding and a reasonable compression. WavPack -x4 compressed better and decodes faster but encodes slower. ALS compresses better but is slighly slower for both encoding and decoding.

OptimFROG with --preset max produces the smallest files in this test, albeit at the cost of very slow encoding and decoding. Except for the 96kHz files, the test CPU was not able to decode the OptimFROG --preset max files in realtime, thus making playback impossible.

Monkey's Audio shows erratic behaviour, with its two of its faster presets (-c2000 and -c4000) outperforming the next slower preset (-c3000 and -c5000 respectively) by a non-negligeable margin.

When looking at the individual results, the chiptune source (Junichi Masuda, Go Ichinose - Pokemon Gold) stands out as very different.