FAQ
Thanks again Evan,

Your four changes, particularly the first two ones have had a dramatical
impact in
performance. Now, go is the fastest implementation:

0.03u 4.19s 41.34r 2192kB c
0.04u 8.58s 40.03r 2192kB go

I still have to do some more tests but I find this particularly remarkable
specially
considering how the c version was implemented.

I used your code and fixed a bug (not introduced by your changes).

So to sum up what we did:

1. operate in bytes[]
2. when reading from the reader use ReadSlice() to get your bytes
3. To reuse slices reset their lengths ([:0]) so we reuse the memory we
have already allocated and
we avoid GC operations.
4. Implement the iterations by creating objects (setting objects to the
main struct) instead of
using closures.
5. Use .SplitN() instead of Split() since we only care about the first part
of the string, we about also GC ops.

Latest code here: https://gist.github.com/3886423 (I am still using my
files package to
access Xopen).

Way to go!
-drd


On Saturday, October 13, 2012 3:15:12 PM UTC-5, Evan Shaw wrote:

On Sun, Oct 14, 2012 at 6:47 AM, drio <driod...@gmail.com <javascript:>>
wrote:
Please take a look to the code (https://gist.github.com/3882029) and let me
know if you have more comments.
One big improvement would be, instead of setting seq and qual to nil
on line 55, resetting their lengths. In other words:

seq = seq[:0]
qual = qual[:0]

This way you're reusing space you've already allocated instead of
allocating new space. This cuts about 20% of the processing time for
some sample file I found from a Google search.

I get another measurable improvement using bytes.SplitN(last, space,
1) since you're only using the first result from bytes.Split. You
might be able to do even better with bytes.IndexByte, but I had
trouble getting it to work with my sample file.

As for the closures, I think it would be more idiomatic (and
potentially more efficient) to create objects instead of closures.
Closures and single-method objects are basically equivalent, but when
in doubt I'd usually choose an object. I get another measurable speed
improvement by using an object in place of the closures.

My version of the code, with the above changes and a few smaller
refactorings, is here: https://gist.github.com/3885963

(I pasted your Xopen function in my version too since the go tool was
having trouble getting your package.)

I don't know how to verify that it actually still works correctly, but
my transformations were fairly mechanical so hopefully I haven't
screwed anything up.

- Evan
--

Search Discussions

Discussion Posts

Previous

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 10 of 10 | next ›
Discussion Overview
groupgolang-nuts @
categoriesgo
postedOct 13, '12 at 5:02a
activeOct 13, '12 at 10:47p
posts10
users4
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase