FAQ
Wouldn't it be nice to have

type ChReader interface {
ChRead ((p chan byte) (n int, err error)
}
type ChWriter interface {
  ChWrite(p chan byte) (n int, err error)
}

and os.File and net.Conn to implement it? Or is it bad idea in general, and am I missing something critical?

My usecase - I want to receive large text (say 10Mb) from the Conn and parse it for unique terms. Seems I can do it on the fly. So i don't want to allocate []byte of such a size, especially while serving 1000 Conn.

About possible lowlevel implementation - stdlib os.File.Read use syscall read

 ssize_t read(int fd, void *buf, size_t count);

which want *buf
Looks like Hchan in /src/runtime/chan.h has such pointer. Alike with net.Conn

Where am I generally wrong?

--
Kostarev Ilya

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Volker Dobler at Dec 10, 2014 at 1:17 pm

    Am Mittwoch, 10. Dezember 2014 13:50:13 UTC+1 schrieb Uvelichitel:
    [...]
    type ChReader interface {
    ChRead ((p chan byte) (n int, err error)
    }
    type ChWriter interface {
    ChWrite(p chan byte) (n int, err error)
    }

    and os.File and net.Conn to implement it? Or is it bad idea in general,
    and am I missing something critical?

    My usecase - I want to receive large text (say 10Mb) from the Conn and
    parse it for unique terms. Seems I can do it on the fly. So i don't want to
    allocate []byte of such a size, especially while serving 1000 Conn.
    [...]
    How would you use ChRead? You call it and it starts feeding bytes into p
    until
    p is closed? Or will it close p once "done" with reading? What are the
    return
    values? Error reporting happens only at the call site of ChRead and the
    consumer of the bytes sent on p just reads until p is closed? How do you
    intend to signal EOF or an error to the consumer of p?

    Your usecase does not require to allocate 1000 x 10Mb, but maybe
    just 1000 x 1Kb and you reuse that 1k until you are done reading the
    full 10Mb. Did you profile, that this is the bottleneck of your code?

    V.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at Dec 10, 2014 at 1:40 pm
    Try to stay with the reader concept as long as possible, stack readers on top of one another and write programs that consume io.Readers not []byte.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Hariharan Srinath at Dec 10, 2014 at 2:40 pm
    Shouldn't wrapping whatever reader you're using in a bufio.Reader give you buffered reads? The default size is 4096 bytes but there's a function that creates a buffered reader with specifIan let size (min 16 bytes I think)

    Regards
    Srinath

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedDec 10, '14 at 12:48p
activeDec 10, '14 at 2:40p
posts4
users4
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase