FAQ

On Tuesday, October 20, 2015 at 12:19:00 PM UTC+2, Klaus Post wrote:
Hi!
On Monday, 19 October 2015 17:57:07 UTC+2, Giovanni Bajo wrote:

Hello,

I've released a native LZO1X implementation in Go:

https://github.com/rasky/go-lzo
Very nice - lzo1x-1 seems very competitive with the Snappy both in terms
of speed and compression.

Matt Mahoneys 10GB corpus: lzo1-1 is 5% faster and 2% better compression
than Snappy with 64KB blocks.
enwik9: Snappy is 30% faster, but has 4.5% worse compression. Again 64KB
blocks.
random bytes: Incredibly fast. Beats everything by an order of magnitude.
Of course there is no compression gain.

It seems that lzo-1 get slower as compression ration increases (to a
certain point), which is an interesting characteristic. Especially its
ability to skip uncompressible content is very interesting, and gives it a
huge edge as a general compressor.

"1x-999" seems VERY slow. 5-10MB/s. I don't suspect this to be competitive
with 64KB blocks. You should consider adding some options in-between.
I've now exposed the original compression levels for 999. I guess they
might be interesting for someone.

I've also published my benchmarks for compressors.

It would help if you added an io.ReadCloser interface. I assume it would
be possible for you to design the re-implementation, so it doesn't require
all content to be in memory.
Yes, it'll probably require a circular buffer with the max look-behind size.

I have not tested decompression, but I would expect it to be good as well.
It should be pretty fast. I've implemented a couple of tricks to be able to
implement io.Reader without too much overhead, though I'm sure it's
possible to do even better.

Giovanni Bajo

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

Discussion Posts

Previous

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 5 of 5 | next ›
Discussion Overview
groupgolang-nuts @
categoriesgo
postedOct 19, '15 at 3:56p
activeOct 22, '15 at 9:04p
posts5
users3
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase