JT

On Tuesday, December 8, 2015 at 5:48:53 PM UTC-8, thebroke...@gmail.com

wrote:

Just an update, I have completed a working decoder. It decompresses at

~20MB/s, which about 25% faster than the C library (~16MB/s), and 200%

faster than the Go standard library (~5MB/s).

https://godoc.org/github.com/dsnet/compress/bzip2

Enjoy.

JT

On Monday, December 7, 2015 at 11:55:41 PM UTC-8, thebroke...@gmail.com

wrote:

--~20MB/s, which about 25% faster than the C library (~16MB/s), and 200%

faster than the Go standard library (~5MB/s).

https://godoc.org/github.com/dsnet/compress/bzip2

Enjoy.

JT

On Monday, December 7, 2015 at 11:55:41 PM UTC-8, thebroke...@gmail.com

wrote:

Hey all,

github: https://github.com/dsnet/compress

godoc: https://godoc.org/github.com/dsnet/compress/bzip2

I finished writing a bzip2 encoder. I haven't heavily tested it, and

there's still alot to do in the area of optimizations. However, it still

performs relatively well relative to the C implementation and is actually

faster in encoding than the standard Go library is at in decoding. The

speed is about half that of the C version, but I think I can close them gap

to be about 1.5x or even 1x. The compression ratio is about 10% worse since

I haven't implemented a K-means based huffman encoder yet. I have plans to

add a decoder eventually that will outperform the standard library version.

BENCHMARK: bz2:encRate

This package is still in development, but I would appreciate if people

can try using it and tell me about any bugs. I'll make a bigger

announcement about it when I feel that the package is stable enough. I

eventually plan on merging it into the standard library, but that

definitely won't happen until go1.7 or later.

JT

github: https://github.com/dsnet/compress

godoc: https://godoc.org/github.com/dsnet/compress/bzip2

I finished writing a bzip2 encoder. I haven't heavily tested it, and

there's still alot to do in the area of optimizations. However, it still

performs relatively well relative to the C implementation and is actually

faster in encoding than the standard Go library is at in decoding. The

speed is about half that of the C version, but I think I can close them gap

to be about 1.5x or even 1x. The compression ratio is about 10% worse since

I haven't implemented a K-means based huffman encoder yet. I have plans to

add a decoder eventually that will outperform the standard library version.

BENCHMARK: bz2:encRate

benchmark ds MB/s delta cgo MB/s delta

digits.txt:1:1e4 5.44 1.00x 7.55 1.39x

digits.txt:1:1e5 5.57 1.00x 12.51 2.25x

digits.txt:1:1e6 5.76 1.00x 12.88 2.24x

digits.txt:6:1e4 4.86 1.00x 7.47 1.54x

digits.txt:6:1e5 5.54 1.00x 12.46 2.25x

digits.txt:6:1e6 5.39 1.00x 12.46 2.31x

digits.txt:9:1e4 4.66 1.00x 7.54 1.62x

digits.txt:9:1e5 5.48 1.00x 12.60 2.30x

digits.txt:9:1e6 5.02 1.00x 12.11 2.41x

twain.txt:1:1e4 5.26 1.00x 7.89 1.50x

twain.txt:1:1e5 6.02 1.00x 12.89 2.14x

twain.txt:1:1e6 6.19 1.00x 12.83 2.07x

twain.txt:6:1e4 4.75 1.00x 8.07 1.70x

twain.txt:6:1e5 6.02 1.00x 12.93 2.15x

twain.txt:6:1e6 6.09 1.00x 12.51 2.06x

twain.txt:9:1e4 4.54 1.00x 7.93 1.75x

twain.txt:9:1e5 6.00 1.00x 12.50 2.08x

twain.txt:9:1e6 5.73 1.00x 12.13 2.12x

BENCHMARK: bz2:ratio

benchmark ds ratio delta cgo ratio delta

digits.txt:1:1e4 2.20x 1.00x 2.28x 1.04x

digits.txt:1:1e5 2.23x 1.00x 2.32x 1.04x

digits.txt:1:1e6 2.22x 1.00x 2.31x 1.04x

digits.txt:6:1e4 2.20x 1.00x 2.28x 1.04x

digits.txt:6:1e5 2.23x 1.00x 2.32x 1.04x

digits.txt:6:1e6 2.12x 1.00x 2.23x 1.05x

digits.txt:9:1e4 2.20x 1.00x 2.28x 1.04x

digits.txt:9:1e5 2.23x 1.00x 2.32x 1.04x

digits.txt:9:1e6 2.05x 1.00x 2.16x 1.05x

twain.txt:1:1e4 2.07x 1.00x 2.25x 1.09x

twain.txt:1:1e5 2.68x 1.00x 2.90x 1.08x

twain.txt:1:1e6 2.68x 1.00x 2.89x 1.08x

twain.txt:6:1e4 2.07x 1.00x 2.25x 1.09x

twain.txt:6:1e5 2.68x 1.00x 2.90x 1.08x

twain.txt:6:1e6 2.87x 1.00x 3.12x 1.09x

twain.txt:9:1e4 2.07x 1.00x 2.25x 1.09x

twain.txt:9:1e5 2.68x 1.00x 2.90x 1.08x

twain.txt:9:1e6 2.87x 1.00x 3.12x 1.09x

digits.txt:1:1e4 5.44 1.00x 7.55 1.39x

digits.txt:1:1e5 5.57 1.00x 12.51 2.25x

digits.txt:1:1e6 5.76 1.00x 12.88 2.24x

digits.txt:6:1e4 4.86 1.00x 7.47 1.54x

digits.txt:6:1e5 5.54 1.00x 12.46 2.25x

digits.txt:6:1e6 5.39 1.00x 12.46 2.31x

digits.txt:9:1e4 4.66 1.00x 7.54 1.62x

digits.txt:9:1e5 5.48 1.00x 12.60 2.30x

digits.txt:9:1e6 5.02 1.00x 12.11 2.41x

twain.txt:1:1e4 5.26 1.00x 7.89 1.50x

twain.txt:1:1e5 6.02 1.00x 12.89 2.14x

twain.txt:1:1e6 6.19 1.00x 12.83 2.07x

twain.txt:6:1e4 4.75 1.00x 8.07 1.70x

twain.txt:6:1e5 6.02 1.00x 12.93 2.15x

twain.txt:6:1e6 6.09 1.00x 12.51 2.06x

twain.txt:9:1e4 4.54 1.00x 7.93 1.75x

twain.txt:9:1e5 6.00 1.00x 12.50 2.08x

twain.txt:9:1e6 5.73 1.00x 12.13 2.12x

BENCHMARK: bz2:ratio

benchmark ds ratio delta cgo ratio delta

digits.txt:1:1e4 2.20x 1.00x 2.28x 1.04x

digits.txt:1:1e5 2.23x 1.00x 2.32x 1.04x

digits.txt:1:1e6 2.22x 1.00x 2.31x 1.04x

digits.txt:6:1e4 2.20x 1.00x 2.28x 1.04x

digits.txt:6:1e5 2.23x 1.00x 2.32x 1.04x

digits.txt:6:1e6 2.12x 1.00x 2.23x 1.05x

digits.txt:9:1e4 2.20x 1.00x 2.28x 1.04x

digits.txt:9:1e5 2.23x 1.00x 2.32x 1.04x

digits.txt:9:1e6 2.05x 1.00x 2.16x 1.05x

twain.txt:1:1e4 2.07x 1.00x 2.25x 1.09x

twain.txt:1:1e5 2.68x 1.00x 2.90x 1.08x

twain.txt:1:1e6 2.68x 1.00x 2.89x 1.08x

twain.txt:6:1e4 2.07x 1.00x 2.25x 1.09x

twain.txt:6:1e5 2.68x 1.00x 2.90x 1.08x

twain.txt:6:1e6 2.87x 1.00x 3.12x 1.09x

twain.txt:9:1e4 2.07x 1.00x 2.25x 1.09x

twain.txt:9:1e5 2.68x 1.00x 2.90x 1.08x

twain.txt:9:1e6 2.87x 1.00x 3.12x 1.09x

This package is still in development, but I would appreciate if people

can try using it and tell me about any bugs. I'll make a bigger

announcement about it when I feel that the package is stable enough. I

eventually plan on merging it into the standard library, but that

definitely won't happen until go1.7 or later.

JT

You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.