github: https://github.com/dsnet/compress

godoc: https://godoc.org/github.com/dsnet/compress/bzip2

I finished writing a bzip2 encoder. I haven't heavily tested it, and

there's still alot to do in the area of optimizations. However, it still

performs relatively well relative to the C implementation and is actually

faster in encoding than the standard Go library is at in decoding. The

speed is about half that of the C version, but I think I can close them gap

to be about 1.5x or even 1x. The compression ratio is about 10% worse since

I haven't implemented a K-means based huffman encoder yet. I have plans to

add a decoder eventually that will outperform the standard library version.

BENCHMARK: bz2:encRate

benchmark ds MB/s delta cgo MB/s delta

digits.txt:1:1e4 5.44 1.00x 7.55 1.39x

digits.txt:1:1e5 5.57 1.00x 12.51 2.25x

digits.txt:1:1e6 5.76 1.00x 12.88 2.24x

digits.txt:6:1e4 4.86 1.00x 7.47 1.54x

digits.txt:6:1e5 5.54 1.00x 12.46 2.25x

digits.txt:6:1e6 5.39 1.00x 12.46 2.31x

digits.txt:9:1e4 4.66 1.00x 7.54 1.62x

digits.txt:9:1e5 5.48 1.00x 12.60 2.30x

digits.txt:9:1e6 5.02 1.00x 12.11 2.41x

twain.txt:1:1e4 5.26 1.00x 7.89 1.50x

twain.txt:1:1e5 6.02 1.00x 12.89 2.14x

twain.txt:1:1e6 6.19 1.00x 12.83 2.07x

twain.txt:6:1e4 4.75 1.00x 8.07 1.70x

twain.txt:6:1e5 6.02 1.00x 12.93 2.15x

twain.txt:6:1e6 6.09 1.00x 12.51 2.06x

twain.txt:9:1e4 4.54 1.00x 7.93 1.75x

twain.txt:9:1e5 6.00 1.00x 12.50 2.08x

twain.txt:9:1e6 5.73 1.00x 12.13 2.12x

BENCHMARK: bz2:ratio

benchmark ds ratio delta cgo ratio delta

digits.txt:1:1e4 2.20x 1.00x 2.28x 1.04x

digits.txt:1:1e5 2.23x 1.00x 2.32x 1.04x

digits.txt:1:1e6 2.22x 1.00x 2.31x 1.04x

digits.txt:6:1e4 2.20x 1.00x 2.28x 1.04x

digits.txt:6:1e5 2.23x 1.00x 2.32x 1.04x

digits.txt:6:1e6 2.12x 1.00x 2.23x 1.05x

digits.txt:9:1e4 2.20x 1.00x 2.28x 1.04x

digits.txt:9:1e5 2.23x 1.00x 2.32x 1.04x

digits.txt:9:1e6 2.05x 1.00x 2.16x 1.05x

twain.txt:1:1e4 2.07x 1.00x 2.25x 1.09x

twain.txt:1:1e5 2.68x 1.00x 2.90x 1.08x

twain.txt:1:1e6 2.68x 1.00x 2.89x 1.08x

twain.txt:6:1e4 2.07x 1.00x 2.25x 1.09x

twain.txt:6:1e5 2.68x 1.00x 2.90x 1.08x

twain.txt:6:1e6 2.87x 1.00x 3.12x 1.09x

twain.txt:9:1e4 2.07x 1.00x 2.25x 1.09x

twain.txt:9:1e5 2.68x 1.00x 2.90x 1.08x

twain.txt:9:1e6 2.87x 1.00x 3.12x 1.09x

digits.txt:1:1e4 5.44 1.00x 7.55 1.39x

digits.txt:1:1e5 5.57 1.00x 12.51 2.25x

digits.txt:1:1e6 5.76 1.00x 12.88 2.24x

digits.txt:6:1e4 4.86 1.00x 7.47 1.54x

digits.txt:6:1e5 5.54 1.00x 12.46 2.25x

digits.txt:6:1e6 5.39 1.00x 12.46 2.31x

digits.txt:9:1e4 4.66 1.00x 7.54 1.62x

digits.txt:9:1e5 5.48 1.00x 12.60 2.30x

digits.txt:9:1e6 5.02 1.00x 12.11 2.41x

twain.txt:1:1e4 5.26 1.00x 7.89 1.50x

twain.txt:1:1e5 6.02 1.00x 12.89 2.14x

twain.txt:1:1e6 6.19 1.00x 12.83 2.07x

twain.txt:6:1e4 4.75 1.00x 8.07 1.70x

twain.txt:6:1e5 6.02 1.00x 12.93 2.15x

twain.txt:6:1e6 6.09 1.00x 12.51 2.06x

twain.txt:9:1e4 4.54 1.00x 7.93 1.75x

twain.txt:9:1e5 6.00 1.00x 12.50 2.08x

twain.txt:9:1e6 5.73 1.00x 12.13 2.12x

BENCHMARK: bz2:ratio

benchmark ds ratio delta cgo ratio delta

digits.txt:1:1e4 2.20x 1.00x 2.28x 1.04x

digits.txt:1:1e5 2.23x 1.00x 2.32x 1.04x

digits.txt:1:1e6 2.22x 1.00x 2.31x 1.04x

digits.txt:6:1e4 2.20x 1.00x 2.28x 1.04x

digits.txt:6:1e5 2.23x 1.00x 2.32x 1.04x

digits.txt:6:1e6 2.12x 1.00x 2.23x 1.05x

digits.txt:9:1e4 2.20x 1.00x 2.28x 1.04x

digits.txt:9:1e5 2.23x 1.00x 2.32x 1.04x

digits.txt:9:1e6 2.05x 1.00x 2.16x 1.05x

twain.txt:1:1e4 2.07x 1.00x 2.25x 1.09x

twain.txt:1:1e5 2.68x 1.00x 2.90x 1.08x

twain.txt:1:1e6 2.68x 1.00x 2.89x 1.08x

twain.txt:6:1e4 2.07x 1.00x 2.25x 1.09x

twain.txt:6:1e5 2.68x 1.00x 2.90x 1.08x

twain.txt:6:1e6 2.87x 1.00x 3.12x 1.09x

twain.txt:9:1e4 2.07x 1.00x 2.25x 1.09x

twain.txt:9:1e5 2.68x 1.00x 2.90x 1.08x

twain.txt:9:1e6 2.87x 1.00x 3.12x 1.09x

This package is still in development, but I would appreciate if people can

try using it and tell me about any bugs. I'll make a bigger announcement

about it when I feel that the package is stable enough. I eventually plan

on merging it into the standard library, but that definitely won't happen

until go1.7 or later.

JT

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.