I have worked on optimizing the standard library deflate function, and I am
happy to announce revised gzip/zip packages, that on x64 is about *30-50%
faster with slightly improved compression*. It contains no cgo.
All packages are drop-in replacements for the standard libraries, so you
can use them by simply changing imports.
The biggest gains are on machines with SSE4.2 instructions available on
Intel Nehalem (2009) and AMD Bulldozer (2012). The optimized functions are:
* Minimum matches are 4 bytes, this leads to fewer searches and better
* Stronger hash (iSCSI CRC32) for matches on x64 with SSE 4.2 support. This
leads to fewer hash collisions.
* Literal byte matching using SSE 4.2 for faster long-match comparisons.
* Bulk hashing on matches.
* Much faster dictionary indexing with NewWriterDict()/Reset().
* Make Bit Coder faster by assuming we are on a 64 bit CPU.
* CRC32 optimized for 10x speedup on SSE 4.2. Available
For benchmarks see the project page.
In short, there will be better compression at levels 1 to 4 and about 1.5
times the throughput at higher compression levels.
Furthermore "pgzip" (multi-cpu gzip for longer streams) has also been
updated to the new deflate/crc32, so it you update the repo you will also
get a "free" speed boost there. See https://github.com/klauspost/pgzip
Comments, questions and other feedback is very welcome!
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to email@example.com.
For more options, visit https://groups.google.com/d/optout.