ktf's lossless codec comparison

The graphs

These graphs show how much a codec compresses as a function of time. The computer I used was my own notebook with a Turion ML-34, running 64-bit Linux. Encoding and decoding were done on a ramdisk, so the harddisk isn't a bottleneck for the fastest codecs. As you can see, there are 4 main graphs: encoding with a linear scale, decoding with a linear scale, encoding with a logarithmic scale and decoding with a logarithmic scale. As a linear scale looks probably better understandable, the logarithmic scale is a better representation of speed.

For example: the difference between 1x realtime and 2x realtime is much bigger then the difference between 200x realtime and 201x realtime. If you look at the difference between la without parameters and la high on a logarithmic scale, you can see it is comparable to the difference between FLAC -7 and FLAC -8. You can't see that on a linear scale, but it is easier to read exact values.

I wasn't able to put the codec parameters in the graph without making it unreadable, so I'll try to make it clear here. The points are interconnected based on which one was tested first, second, third, in order, so if you tracks the line in between from roughly the upper right to down left, you'll go past the following settings:

FLAC: -0; -1; -2; -3; -4; -5; -6; -7; -8
TAK: -p0; -p0e; -p0m; -p1; -p1e; -p1m; -p2; -p2e; -p2m; -p3; -p3e; -p3m; -p4; -p4e; -p4m
TTA: [none]
Shorten: [none]
alac@FFmpeg: [none]
WavPack: -f; [none]; -h; -hh
APE: -c1000; -c2000; -c3000; -c4000; -c5000
OFR: --mode fast; --mode normal; --mode high; --mode highnew; --mode best; --mode bestnew
ALS: [none]; -7
LA: [none]; -high; -high -noseek

And logarithmic

Please note, these graphs are not balanced yet, see next chapter

The music

The music used so far in this test

The featured codecs

Of course, it is important to know which versions of the codecs are used.

• FLAC 1.2.1
• TAK 1.1.1 with wine 1.1.18
• TTA 3.4.1
• Shorten 3.6.1
• alac@FFmpeg: Fresh SVN-r18820
• Wavpack: 4.50.1
• APE: 3.99
• OFR: 4.600ex, Linux/AMD64
• ALS: RM22rev1
• LA: 0.4

What I'm going to do next...

• Adding more (classical) music
• Investigate whether it is interesting to make separate graphs for different genre's or not
• Doing an in-depth analysis to make other interesting graphs (for example, the spread of compression over all test samples, to determine which codec is most allround and which one is least, and things like that)
• Testing which (low-level) benchmark is representative for this test, so you can up- or downscale the results to you CPU
• Adding weighted graphs with a questionaire, for example:
  1. What are you going to do with the codec? archival (1:1), transcoding (1:5), listening (1:20) - (weights, encoding:decoding)
  2. What kind of music do you listen? (add weights)
  3. Can you/do you want to read a logaritmic scale?
  4. Are you going to code surround/24-bit/>48khz recordings? If so, please add weights
  Of course, I can only create such a comprehensive test if I have enough test material, so I'll have to search for it or buy some :)
• Add some eye-candy to this page
• Thinking about what I'm going to put here...