On Tuesday, October 20, 2015 at 12:19:00 PM UTC+2, Klaus Post wrote:
On Monday, 19 October 2015 17:57:07 UTC+2, Giovanni Bajo wrote:


I've released a native LZO1X implementation in Go:

Very nice - lzo1x-1 seems very competitive with the Snappy both in terms
of speed and compression.

Matt Mahoneys 10GB corpus: lzo1-1 is 5% faster and 2% better compression
than Snappy with 64KB blocks.
enwik9: Snappy is 30% faster, but has 4.5% worse compression. Again 64KB
random bytes: Incredibly fast. Beats everything by an order of magnitude.
Of course there is no compression gain.

It seems that lzo-1 get slower as compression ration increases (to a
certain point), which is an interesting characteristic. Especially its
ability to skip uncompressible content is very interesting, and gives it a
huge edge as a general compressor.

"1x-999" seems VERY slow. 5-10MB/s. I don't suspect this to be competitive
with 64KB blocks. You should consider adding some options in-between.
I've now exposed the original compression levels for 999. I guess they
might be interesting for someone.

I've also published my benchmarks for compressors.

It would help if you added an io.ReadCloser interface. I assume it would
be possible for you to design the re-implementation, so it doesn't require
all content to be in memory.
Yes, it'll probably require a circular buffer with the max look-behind size.

I have not tested decompression, but I would expect it to be good as well.
It should be pretty fast. I've implemented a couple of tricks to be able to
implement io.Reader without too much overhead, though I'm sure it's
possible to do even better.

Giovanni Bajo

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

Discussion Posts


Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 5 of 5 | next ›
Discussion Overview
groupgolang-nuts @
postedOct 19, '15 at 3:56p
activeOct 22, '15 at 9:04p



site design / logo © 2022 Grokbase