Just kidding, it's 400% faster than the standard library ;)


On Tuesday, December 8, 2015 at 5:48:53 PM UTC-8, thebroke...@gmail.com
Just an update, I have completed a working decoder. It decompresses at
~20MB/s, which about 25% faster than the C library (~16MB/s), and 200%
faster than the Go standard library (~5MB/s).




On Monday, December 7, 2015 at 11:55:41 PM UTC-8, thebroke...@gmail.com
Hey all,

github: https://github.com/dsnet/compress
godoc: https://godoc.org/github.com/dsnet/compress/bzip2

I finished writing a bzip2 encoder. I haven't heavily tested it, and
there's still alot to do in the area of optimizations. However, it still
performs relatively well relative to the C implementation and is actually
faster in encoding than the standard Go library is at in decoding. The
speed is about half that of the C version, but I think I can close them gap
to be about 1.5x or even 1x. The compression ratio is about 10% worse since
I haven't implemented a K-means based huffman encoder yet. I have plans to
add a decoder eventually that will outperform the standard library version.

BENCHMARK: bz2:encRate
benchmark ds MB/s delta cgo MB/s delta
digits.txt:1:1e4 5.44 1.00x 7.55 1.39x
digits.txt:1:1e5 5.57 1.00x 12.51 2.25x
digits.txt:1:1e6 5.76 1.00x 12.88 2.24x
digits.txt:6:1e4 4.86 1.00x 7.47 1.54x
digits.txt:6:1e5 5.54 1.00x 12.46 2.25x
digits.txt:6:1e6 5.39 1.00x 12.46 2.31x
digits.txt:9:1e4 4.66 1.00x 7.54 1.62x
digits.txt:9:1e5 5.48 1.00x 12.60 2.30x
digits.txt:9:1e6 5.02 1.00x 12.11 2.41x
twain.txt:1:1e4 5.26 1.00x 7.89 1.50x
twain.txt:1:1e5 6.02 1.00x 12.89 2.14x
twain.txt:1:1e6 6.19 1.00x 12.83 2.07x
twain.txt:6:1e4 4.75 1.00x 8.07 1.70x
twain.txt:6:1e5 6.02 1.00x 12.93 2.15x
twain.txt:6:1e6 6.09 1.00x 12.51 2.06x
twain.txt:9:1e4 4.54 1.00x 7.93 1.75x
twain.txt:9:1e5 6.00 1.00x 12.50 2.08x
twain.txt:9:1e6 5.73 1.00x 12.13 2.12x
BENCHMARK: bz2:ratio
benchmark ds ratio delta cgo ratio delta
digits.txt:1:1e4 2.20x 1.00x 2.28x 1.04x
digits.txt:1:1e5 2.23x 1.00x 2.32x 1.04x
digits.txt:1:1e6 2.22x 1.00x 2.31x 1.04x
digits.txt:6:1e4 2.20x 1.00x 2.28x 1.04x
digits.txt:6:1e5 2.23x 1.00x 2.32x 1.04x
digits.txt:6:1e6 2.12x 1.00x 2.23x 1.05x
digits.txt:9:1e4 2.20x 1.00x 2.28x 1.04x
digits.txt:9:1e5 2.23x 1.00x 2.32x 1.04x
digits.txt:9:1e6 2.05x 1.00x 2.16x 1.05x
twain.txt:1:1e4 2.07x 1.00x 2.25x 1.09x
twain.txt:1:1e5 2.68x 1.00x 2.90x 1.08x
twain.txt:1:1e6 2.68x 1.00x 2.89x 1.08x
twain.txt:6:1e4 2.07x 1.00x 2.25x 1.09x
twain.txt:6:1e5 2.68x 1.00x 2.90x 1.08x
twain.txt:6:1e6 2.87x 1.00x 3.12x 1.09x
twain.txt:9:1e4 2.07x 1.00x 2.25x 1.09x
twain.txt:9:1e5 2.68x 1.00x 2.90x 1.08x
twain.txt:9:1e6 2.87x 1.00x 3.12x 1.09x

This package is still in development, but I would appreciate if people
can try using it and tell me about any bugs. I'll make a bigger
announcement about it when I feel that the package is stable enough. I
eventually plan on merging it into the standard library, but that
definitely won't happen until go1.7 or later.

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

Discussion Posts


Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 3 of 11 | next ›
Discussion Overview
groupgolang-nuts @
postedDec 8, '15 at 7:55a
activeDec 13, '15 at 12:53p



site design / logo © 2021 Grokbase