FAQ
Hi,

I'm trying to implement scgi protocol support to an existing xmlrpc
library, using a custom rpc.ClientCodec.
My problem is, that the ClientCodec in the ReadResponseHeader expecting to
set at least the "Seq" field of the Response struct.
As I understand from the source, it must match with the original "Seq" from
the Request, otherwise may the wrong "Call" will return to the client.
The problem is, that the xmlrpc response does not provide the "Seq" number,
neither the original "ServiceMethod" or arguments.
So if 2 rpc method invoked through the same Client from multiple goroutine,
I can't identify in the response which one returned.
The obvious solution to do the calls sequentially, communicating between
the "WriteRequest" and "ReadResponseHeader" calls with a channel for
example, but it's ugly.

I thinking an other way, where creating a struct, saving the "Seq" and
"ServiceMethod" and fire a separate goroutine, which do the call and read
the response, save it to the same struct, then send it to the
"ReadResponseHeader" method, through a channel.
This way I could be able to match the responses and the "Seq" id-s, I'm
unsure only one thing.
The documentation of the net.Conn says, "Multiple goroutines may invoke
methods on a Conn simultaneously", which is good, but what if:
Calling a long running rpc method from the A goroutine and a short running
one from the B goroutine.
It means the "WriteRequest" will be invoked 2 times, so I creating 2
struct, e.g.: with "Seq" 0(A) and 1(B) and trying to read the responses
from 2 separate goroutine.
I assume(depend from the server) it would be possible that the goroutine
with "Seq" 0 read the response of the "Seq" 1 request, because the server
finish earlier with the shorter rpc method.
In this case, 2 goroutine waits to read from the connection and when it's
ready to receive, the go scheduler will pick one randomly, so may the
responses will reverse.
Am I right?
If so, is there a way to make the rpc calls concurrent safely, without
reversing the responses?

Thanks,
Norbert

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Kevin Malachowski at Oct 25, 2014 at 4:41 pm
    The race-detector <https://blog.golang.org/race-detector> is a pretty
    awesome tool that allows you to easily find out if you're making the wrong
    assumptions about concurrent code and also even better for catching
    outright mistakes. Try whipping up a short example that attempts to do what
    you're doing and run it under the race detector. If it passes, it doesn't
    have any races. If it fails, you will need to do things a different way
    because races are by definition non-deterministic and that's a bad thing.

    When I deal with network connections I usually end up with a goroutine
    reading and a separate goroutine writing, unless the protocol I'm
    implementing is a simple call-response like HTTP (in which case it's one
    goroutine per connection that alternates listening and writing). I use the
    concurrency primitives (channels, waitgroups, mutexes, etc) to allow the
    reader and writer to communicate between themselves as well as with other
    goroutines that may exist. It allows me to immediately abstract away the
    annoyance that is dealing with a primitive byte stream and therefore
    integrates into other code better.
    On Saturday, October 25, 2014 2:33:43 AM UTC-7, Norbert Csibra wrote:

    Hi,

    I'm trying to implement scgi protocol support to an existing xmlrpc
    library, using a custom rpc.ClientCodec.
    My problem is, that the ClientCodec in the ReadResponseHeader expecting to
    set at least the "Seq" field of the Response struct.
    As I understand from the source, it must match with the original "Seq"
    from the Request, otherwise may the wrong "Call" will return to the client.
    The problem is, that the xmlrpc response does not provide the "Seq"
    number, neither the original "ServiceMethod" or arguments.
    So if 2 rpc method invoked through the same Client from multiple
    goroutine, I can't identify in the response which one returned.
    The obvious solution to do the calls sequentially, communicating between
    the "WriteRequest" and "ReadResponseHeader" calls with a channel for
    example, but it's ugly.

    I thinking an other way, where creating a struct, saving the "Seq" and
    "ServiceMethod" and fire a separate goroutine, which do the call and read
    the response, save it to the same struct, then send it to the
    "ReadResponseHeader" method, through a channel.
    This way I could be able to match the responses and the "Seq" id-s, I'm
    unsure only one thing.
    The documentation of the net.Conn says, "Multiple goroutines may invoke
    methods on a Conn simultaneously", which is good, but what if:
    Calling a long running rpc method from the A goroutine and a short running
    one from the B goroutine.
    It means the "WriteRequest" will be invoked 2 times, so I creating 2
    struct, e.g.: with "Seq" 0(A) and 1(B) and trying to read the responses
    from 2 separate goroutine.
    I assume(depend from the server) it would be possible that the goroutine
    with "Seq" 0 read the response of the "Seq" 1 request, because the server
    finish earlier with the shorter rpc method.
    In this case, 2 goroutine waits to read from the connection and when it's
    ready to receive, the go scheduler will pick one randomly, so may the
    responses will reverse.
    Am I right?
    If so, is there a way to make the rpc calls concurrent safely, without
    reversing the responses?

    Thanks,
    Norbert
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Norbert Csibra at Oct 25, 2014 at 5:37 pm
    I'm not sure we are on the same page, because I think in this case there is
    no race, at least the question is not about a possible race condition.
    I know about race detector and I'm agree that's a pretty awesome tool.
    In a very simple form, the question is that the net.Conn is work in FIFO or
    not, but in this form it maybe misleading, that's why I tried to explain
    the whole story. :)
    On Saturday, October 25, 2014 6:41:46 PM UTC+2, Kevin Malachowski wrote:

    The race-detector <https://blog.golang.org/race-detector> is a pretty
    awesome tool that allows you to easily find out if you're making the wrong
    assumptions about concurrent code and also even better for catching
    outright mistakes. Try whipping up a short example that attempts to do what
    you're doing and run it under the race detector. If it passes, it doesn't
    have any races. If it fails, you will need to do things a different way
    because races are by definition non-deterministic and that's a bad thing.

    When I deal with network connections I usually end up with a goroutine
    reading and a separate goroutine writing, unless the protocol I'm
    implementing is a simple call-response like HTTP (in which case it's one
    goroutine per connection that alternates listening and writing). I use the
    concurrency primitives (channels, waitgroups, mutexes, etc) to allow the
    reader and writer to communicate between themselves as well as with other
    goroutines that may exist. It allows me to immediately abstract away the
    annoyance that is dealing with a primitive byte stream and therefore
    integrates into other code better.
    On Saturday, October 25, 2014 2:33:43 AM UTC-7, Norbert Csibra wrote:

    Hi,

    I'm trying to implement scgi protocol support to an existing xmlrpc
    library, using a custom rpc.ClientCodec.
    My problem is, that the ClientCodec in the ReadResponseHeader expecting
    to set at least the "Seq" field of the Response struct.
    As I understand from the source, it must match with the original "Seq"
    from the Request, otherwise may the wrong "Call" will return to the client.
    The problem is, that the xmlrpc response does not provide the "Seq"
    number, neither the original "ServiceMethod" or arguments.
    So if 2 rpc method invoked through the same Client from multiple
    goroutine, I can't identify in the response which one returned.
    The obvious solution to do the calls sequentially, communicating between
    the "WriteRequest" and "ReadResponseHeader" calls with a channel for
    example, but it's ugly.

    I thinking an other way, where creating a struct, saving the "Seq" and
    "ServiceMethod" and fire a separate goroutine, which do the call and read
    the response, save it to the same struct, then send it to the
    "ReadResponseHeader" method, through a channel.
    This way I could be able to match the responses and the "Seq" id-s, I'm
    unsure only one thing.
    The documentation of the net.Conn says, "Multiple goroutines may invoke
    methods on a Conn simultaneously", which is good, but what if:
    Calling a long running rpc method from the A goroutine and a short
    running one from the B goroutine.
    It means the "WriteRequest" will be invoked 2 times, so I creating 2
    struct, e.g.: with "Seq" 0(A) and 1(B) and trying to read the responses
    from 2 separate goroutine.
    I assume(depend from the server) it would be possible that the goroutine
    with "Seq" 0 read the response of the "Seq" 1 request, because the server
    finish earlier with the shorter rpc method.
    In this case, 2 goroutine waits to read from the connection and when it's
    ready to receive, the go scheduler will pick one randomly, so may the
    responses will reverse.
    Am I right?
    If so, is there a way to make the rpc calls concurrent safely, without
    reversing the responses?

    Thanks,
    Norbert
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Kevin Malachowski at Oct 25, 2014 at 5:47 pm
    Sorry, I didn't include some internal narrative which caused me to mention races.

    To me I assumed your question about two goroutines reading from the same connection. At least in the case of TCP this doesn't really make sense: since TCP is stream oriented how are you supposed to divvy up a stream of bytes to more than one reader? It just seems inherently racey to me.

    I don't know much about RPC and forgot for a bit that net.Conn is an interface rather than an implementation so it could make sense in the context of that protocol.

    Still though, I suggest just doing a small example and trying it. I don't think it'd be a long program. If it always works then I'd imagine you did it correctly. If it never works or spuriously fails then clearly you'll have to try another solution.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Norbert Csibra at Oct 25, 2014 at 8:01 pm
    Sorry, I introduced some confusion with the goroutines, those purpose in
    this case, is not to read parallel from the connection, just to not block
    the "WriteRequest" method.
    But your point is true of course, wouldn't make sense to try reading
    parallel from the same stream.
    I will try to write a test for it, just as you said, not easy to catch this
    kind of problems, that's why I asked.
    On Saturday, October 25, 2014 7:47:30 PM UTC+2, Kevin Malachowski wrote:

    Sorry, I didn't include some internal narrative which caused me to mention
    races.

    To me I assumed your question about two goroutines reading from the same
    connection. At least in the case of TCP this doesn't really make sense:
    since TCP is stream oriented how are you supposed to divvy up a stream of
    bytes to more than one reader? It just seems inherently racey to me.

    I don't know much about RPC and forgot for a bit that net.Conn is an
    interface rather than an implementation so it could make sense in the
    context of that protocol.

    Still though, I suggest just doing a small example and trying it. I don't
    think it'd be a long program. If it always works then I'd imagine you did
    it correctly. If it never works or spuriously fails then clearly you'll
    have to try another solution.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedOct 25, '14 at 9:33a
activeOct 25, '14 at 8:01p
posts5
users2
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase