FAQ
Hi,

I tried the new race detector for the first time on all of production
Go code I got my hands dirty with - and it was nice to get "only" 1
warning.

However, that one warning seems spurious to me.

The repository import path is "github.com/cznic/dns/msg". The warning part for

$ go test -v -race github.com/cznic/dns/msg

reads:

=== RUN TestExchange0
==================
WARNING: DATA RACE
Read by goroutine 7:
github.com/cznic/dns/msg.func·003()
/home/jnml/src/github.com/cznic/dns/msg/message.go:542 +0x13f

Previous write by goroutine 6:
github.com/cznic/dns/msg.TestExchange0()
/home/jnml/src/github.com/cznic/dns/msg/all_test.go:126 +0x389
testing.tRunner()
/home/jnml/go/src/pkg/testing/testing.go:301 +0xe8

Goroutine 7 (running) created at:
github.com/cznic/dns/msg.(*Message).GoExchangeBuf()
/home/jnml/src/github.com/cznic/dns/msg/message.go:548 +0x129
github.com/cznic/dns/msg.(*Message).GoExchange()
/home/jnml/src/github.com/cznic/dns/msg/message.go:554 +0x82
github.com/cznic/dns/msg.TestExchange0()
/home/jnml/src/github.com/cznic/dns/msg/all_test.go:125 +0x37a
testing.tRunner()
/home/jnml/go/src/pkg/testing/testing.go:301 +0xe8

Goroutine 6 (running) created at:
testing.RunTests()
/home/jnml/go/src/pkg/testing/testing.go:377 +0xaec
testing.Main()
/home/jnml/go/src/pkg/testing/testing.go:313 +0xcd
main.main()
github.com/cznic/dns/msg/_test/_testmain.go:51 +0xda
runtime.main()
/home/jnml/go/src/pkg/runtime/proc.c:248 +0x91

==================

The TestExhange0 function is verifying that `GoExchangeBuf` is immune
to its reply channel being already closed when the answer is ready to
be sent to it. This is attempted by closing the channel in the test
immediately when returned from `GoExchangeBuf` and setting up some
timeouts (hackishly, not really deterministic in the general case). I
think that channel operations are safe, `close` included, so I'm not
sure how to interpret the warning. Or, I am maybe on the wrong track,
not seeing the real place where the race _does_ occur.

Can anybody please hint me if the problem is mine or the race detector
is giving me a false positive? In the later case, is that a bug or an
artifact of the used method and the warnings have to be finally judged
by human inspection (no problem with that)? Thanks in advance.

Running `659f7a1810dd` "syscall: fix data races in LazyDLL/LazyProc"
on x86_64 Ubuntu 12.04.

-j

--

Search Discussions

  • Dmitry Vyukov at Nov 17, 2012 at 8:08 am

    On Friday, November 16, 2012 2:21:00 PM UTC+4, Jan Mercl wrote:

    Hi,

    I tried the new race detector for the first time on all of production
    Go code I got my hands dirty with - and it was nice to get "only" 1
    warning.

    However, that one warning seems spurious to me.

    The repository import path is "github.com/cznic/dns/msg". The warning
    part for

    $ go test -v -race github.com/cznic/dns/msg

    reads:

    === RUN TestExchange0
    ==================
    WARNING: DATA RACE
    Read by goroutine 7:
    github.com/cznic/dns/msg.func·003()<http://github.com/cznic/dns/msg.func%C2%B7003()>
    /home/jnml/src/github.com/cznic/dns/msg/message.go:542 +0x13f

    Previous write by goroutine 6:
    github.com/cznic/dns/msg.TestExchange0()
    /home/jnml/src/github.com/cznic/dns/msg/all_test.go:126 +0x389
    testing.tRunner()
    /home/jnml/go/src/pkg/testing/testing.go:301 +0xe8

    Goroutine 7 (running) created at:
    github.com/cznic/dns/msg.(*Message).GoExchangeBuf()
    /home/jnml/src/github.com/cznic/dns/msg/message.go:548 +0x129
    github.com/cznic/dns/msg.(*Message).GoExchange()
    /home/jnml/src/github.com/cznic/dns/msg/message.go:554 +0x82
    github.com/cznic/dns/msg.TestExchange0()
    /home/jnml/src/github.com/cznic/dns/msg/all_test.go:125 +0x37a
    testing.tRunner()
    /home/jnml/go/src/pkg/testing/testing.go:301 +0xe8

    Goroutine 6 (running) created at:
    testing.RunTests()
    /home/jnml/go/src/pkg/testing/testing.go:377 +0xaec
    testing.Main()
    /home/jnml/go/src/pkg/testing/testing.go:313 +0xcd
    main.main()
    github.com/cznic/dns/msg/_test/_testmain.go:51 +0xda
    runtime.main()
    /home/jnml/go/src/pkg/runtime/proc.c:248 +0x91

    ==================

    The TestExhange0 function is verifying that `GoExchangeBuf` is immune
    to its reply channel being already closed when the answer is ready to
    be sent to it. This is attempted by closing the channel in the test
    immediately when returned from `GoExchangeBuf` and setting up some
    timeouts (hackishly, not really deterministic in the general case). I
    think that channel operations are safe, `close` included, so I'm not
    sure how to interpret the warning. Or, I am maybe on the wrong track,
    not seeing the real place where the race _does_ occur.

    Can anybody please hint me if the problem is mine or the race detector
    is giving me a false positive? In the later case, is that a bug or an
    artifact of the used method and the warnings have to be finally judged
    by human inspection (no problem with that)? Thanks in advance.
    Hi,

    This warning is intentional. It says that chan send is concurrent with
    close. In 99% of cases it is a bug. It's especially unpleasant if usually
    it happens send->close, but then one day in production due to some delays
    it happens as close->send and the program crashes. The race detector allows
    to catch it even in send->close executions (i.e. early during testing).

    The method of the race detector does not produce "false warnings" (like
    static analysis tools), and generally one does not need "final human
    judgement". This case is somewhat special, because the spec clearly says
    that in this case panic is raised, so one can recover from it. And I see
    that's what you are doing. But I do not want to turn the warning off for
    all. So this is kinda "false warning" in some sense.

    But can you describe what you want to achieve allowing receivers to close
    the chan? This is kind of considered "bad practice". When one closes a
    chan, he may assume that he unblocks something or cancels some outstanding
    work. But this is not the case here. The sole result is panic raise and
    panic suppressions. What I am missing?

    Another take away is that the race detector must provide better stack
    traces that clearly point to chan close and chan send.

    Thanks!

    --
  • Dmitry Vyukov at Nov 17, 2012 at 11:56 am

    On Saturday, November 17, 2012 12:08:49 PM UTC+4, Dmitry Vyukov wrote:
    On Friday, November 16, 2012 2:21:00 PM UTC+4, Jan Mercl wrote:

    Hi,

    I tried the new race detector for the first time on all of production
    Go code I got my hands dirty with - and it was nice to get "only" 1
    warning.

    However, that one warning seems spurious to me.

    The repository import path is "github.com/cznic/dns/msg". The warning
    part for

    $ go test -v -race github.com/cznic/dns/msg

    reads:

    === RUN TestExchange0
    ==================
    WARNING: DATA RACE
    Read by goroutine 7:
    github.com/cznic/dns/msg.func·003()<http://github.com/cznic/dns/msg.func%C2%B7003()>
    /home/jnml/src/github.com/cznic/dns/msg/message.go:542 +0x13f

    Previous write by goroutine 6:
    github.com/cznic/dns/msg.TestExchange0()
    /home/jnml/src/github.com/cznic/dns/msg/all_test.go:126 +0x389
    testing.tRunner()
    /home/jnml/go/src/pkg/testing/testing.go:301 +0xe8

    Goroutine 7 (running) created at:
    github.com/cznic/dns/msg.(*Message).GoExchangeBuf()
    /home/jnml/src/github.com/cznic/dns/msg/message.go:548 +0x129
    github.com/cznic/dns/msg.(*Message).GoExchange()
    /home/jnml/src/github.com/cznic/dns/msg/message.go:554 +0x82
    github.com/cznic/dns/msg.TestExchange0()
    /home/jnml/src/github.com/cznic/dns/msg/all_test.go:125 +0x37a
    testing.tRunner()
    /home/jnml/go/src/pkg/testing/testing.go:301 +0xe8

    Goroutine 6 (running) created at:
    testing.RunTests()
    /home/jnml/go/src/pkg/testing/testing.go:377 +0xaec
    testing.Main()
    /home/jnml/go/src/pkg/testing/testing.go:313 +0xcd
    main.main()
    github.com/cznic/dns/msg/_test/_testmain.go:51 +0xda
    runtime.main()
    /home/jnml/go/src/pkg/runtime/proc.c:248 +0x91

    ==================

    The TestExhange0 function is verifying that `GoExchangeBuf` is immune
    to its reply channel being already closed when the answer is ready to
    be sent to it. This is attempted by closing the channel in the test
    immediately when returned from `GoExchangeBuf` and setting up some
    timeouts (hackishly, not really deterministic in the general case). I
    think that channel operations are safe, `close` included, so I'm not
    sure how to interpret the warning. Or, I am maybe on the wrong track,
    not seeing the real place where the race _does_ occur.

    Can anybody please hint me if the problem is mine or the race detector
    is giving me a false positive? In the later case, is that a bug or an
    artifact of the used method and the warnings have to be finally judged
    by human inspection (no problem with that)? Thanks in advance.
    Hi,

    This warning is intentional. It says that chan send is concurrent with
    close. In 99% of cases it is a bug. It's especially unpleasant if usually
    it happens send->close, but then one day in production due to some delays
    it happens as close->send and the program crashes. The race detector allows
    to catch it even in send->close executions (i.e. early during testing).

    The method of the race detector does not produce "false warnings" (like
    static analysis tools), and generally one does not need "final human
    judgement". This case is somewhat special, because the spec clearly says
    that in this case panic is raised, so one can recover from it. And I see
    that's what you are doing. But I do not want to turn the warning off for
    all. So this is kinda "false warning" in some sense.

    But can you describe what you want to achieve allowing receivers to close
    the chan? This is kind of considered "bad practice". When one closes a
    chan, he may assume that he unblocks something or cancels some outstanding
    work. But this is not the case here. The sole result is panic raise and
    panic suppressions. What I am missing?

    Another take away is that the race detector must provide better stack
    traces that clearly point to chan close and chan send.

    Thanks!

    Forgot to mention that if you want to continue using the detector and keep
    the current design, you can put this test into a separate file with "//
    +build !race" build tag. This will just exclude the test under race
    detector. It looks like the simplest solution to the problem.

    --
  • Jan Mercl at Nov 17, 2012 at 3:58 pm
    On Sat, Nov 17, 2012 at 12:56 PM, Dmitry Vyukov wrote:

    Thank you very much for all of the detailed information, it is very
    helpful! I decided to change the API semantics in a minor way and to
    rather rewrite the client code in this case (to not carelessly close
    the returned channel if the answer was received from elsewhere
    sooner). All tests pass w/o race warnings now ;-)

    -j

    --
  • Si guy at Nov 29, 2012 at 1:02 am
    Hi,

    A question:
    Is there a difference between OSs for the same program?
    i.e. I have some production code but my production machine(linux/amsd64)
    can't handle the race detector overhead, so I build it for my windows/amd64
    machine and test it on there. Will this matter? How much?

    Thanks
    -Simon Watt

    P.S.: Thank you to the heavens for this tool.

    --
  • Brainman at Nov 29, 2012 at 1:52 am

    On Thursday, 29 November 2012 12:02:25 UTC+11, si guy wrote:
    ... Will this matter? How much? ...
    I am pretty sure it will. Every program has some bits of "platform specific
    code". These bits could make huge difference. Do not forget, that "standard
    packages" are full of such "platform specific" code too.

    Alex

    --
  • Bryanturley at Nov 29, 2012 at 4:53 am

    On Wednesday, November 28, 2012 7:02:25 PM UTC-6, si guy wrote:
    Hi,

    A question:
    Is there a difference between OSs for the same program?
    i.e. I have some production code but my production machine(linux/amsd64)
    can't handle the race detector overhead, so I build it for my windows/amd64
    machine and test it on there. Will this matter? How much?
    How much different is the actual hardware?

    --
  • Si guy at Nov 29, 2012 at 8:23 am
    Desktop: Wintel 2600k 4core, 8gb, ssd
    Server: rackspace linux amd64 4core, 512mb, ??disk

    Is that different enough to make a difference? Probably I think. But I also test on a Linux boot and a restricted memory Linux VM both on the desktop.

    The issue is that under load the race detector version uses almost 3x memory, (Up to ~700m from ~200m) so it's hard to test for races in a proper environment. I'm using an appropriately scaled rackspace vm for testing as well.

    Thanks,

    -Simon Watt

    --
  • Si guy at Nov 29, 2012 at 8:57 am
    The only bug I've seen is mangling of some function names. I'll post output tomorrow.

    -Simon Watt

    --
  • Dmitry Vyukov at Nov 29, 2012 at 7:38 am

    On Thu, Nov 29, 2012 at 5:02 AM, si guy wrote:
    Hi,

    A question:
    Is there a difference between OSs for the same program?
    i.e. I have some production code but my production machine(linux/amsd64)
    can't handle the race detector overhead, so I build it for my windows/amd64
    machine and test it on there. Will this matter? How much?
    I would say not much. The difference can be caused by
    platform-specific code in std lib (but it should provide the same
    interface to user code, so it should not affect user code a lot) and
    platform-specific code in your project (usually it's nothing or very
    little).
    The race detector operation itself is basically the same.

    Btw, what is the problem for the prod machine? Memory or speed? Both
    can be improved, but for now I am mostly concentrated on getting it
    work.

    --
  • Dmitry Vyukov at Nov 29, 2012 at 7:39 am

    On Thu, Nov 29, 2012 at 5:02 AM, si guy wrote:
    Hi,

    A question:
    Is there a difference between OSs for the same program?
    i.e. I have some production code but my production machine(linux/amsd64)
    can't handle the race detector overhead, so I build it for my windows/amd64
    machine and test it on there. Will this matter? How much?

    Thanks
    -Simon Watt

    P.S.: Thank you to the heavens for this tool.
    Have you found any bugs with it? It's very motivating to hear success
    stories when you are working on tool like this.

    --

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedNov 16, '12 at 10:20a
activeNov 29, '12 at 8:57a
posts11
users5
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase