FAQ
FYI, a recent new failure in tools/go/gccgoimporter may (or may not) be
related to this. I just disabled a test to get the build going again (
https://go-review.googlesource.com/#/c/4750/ ).

The failure showed up on the dashboard at the same time.

- gri
On Thu, Feb 12, 2015 at 1:10 AM, 'Dmitry Vyukov' via golang-dev wrote:

Is there any chance you have a data race on maps? I can imagine that
failure modes change after this change. Try to run the program with -race
flag.

On Thu, Feb 12, 2015 at 11:01 AM, Dmitry Vyukov wrote:
On Thu, Feb 12, 2015 at 1:50 AM, wrote:
Recently Dyukov kindly submitted 85e7bee; it updates the GC to ignore maps
which don't contain pointers.

At about the same time I started have weird corruption issues with maps. I
finally tracked them down to a fairly simple case in my code. I can't
replicate it yet in a sandbox or highly simplified format, but I wanted to
get this out soon, so that people know they should probably revert 85e7 if
they're having weird troubles. And of course, hopefully someone smarter than
me will see the problem immediately.

In brief, I create maps as follows:

fixedmap[keystruct][5]valstruct
varmap[keystruct][]valstruct

And proceed to dump between 1 and 50 million entries in them, using a method
like:

a := [5]struct{}
a[0] = somedata
...

fixedmap[key] = a


After each forced GC, or call to runtime.GC(), at least the fixed map is
corrupted. It may return data from another entry, or just what looks like
random bytes.

I spent this morning using git bisect and it it turned up this
particular
commit.

I also checked out tip and reverted only this commit; that seems to work
fine.

I am running with two small patches. Tip with these patches works fine, so I
believe it's something in Dyukov's original patch.

encoding/gob/decoder.go:15
-const tooBig = 1 << 30
+const tooBig = 1 << 34

and
--- a/src/runtime/malloc2.go
+++ b/src/runtime/malloc2.go
@@ -136,7 +136,7 @@ const (
// See http://golang.org/issue/5402 and
http://golang.org/issue/5236.
// On other 64-bit platforms, we limit the arena to 128GB, or 37
bits.
// On 32-bit, we don't bother limiting anything, so we use the full
32-bit address.
- _MHeapMap_TotalBits = (_64bit*goos_windows)*35 +
(_64bit*(1-goos_windows))*37 + (1-_64bit)*32
+ _MHeapMap_TotalBits = (_64bit*goos_windows)*35 +
(_64bit*(1-goos_windows))*38 + (1-_64bit)*32

@@ -145,7 +145,7 @@ const (
// 2, 3, and 4 are all plausible maximums depending
// on the hardware details of the machine. The garbage
// collector scales well to 32 cpus.
- _MaxGcproc = 32
+ _MaxGcproc = 64

Can you please provide the exact reproducer program?
I am running the following test and it does not fail:

package main

import (
"runtime"
"fmt"
"os"
)

func main() {
m := make(map[int][2]int)
for i := 0; i < 1e6; i++ {
m[i] = [2]int{i, i+1}
if (i%111) == 0 {
runtime.GC()
if len(m) != i+1 {
fmt.Printf("bad len: %v/%v\n", len(m), i+1)
os.Exit(1)
}
for j := 0; j <= i; j++ {
v := m[j]
if v[0] != j || v[1] != j+1 {
fmt.Printf("bad value %v: %v/%v\n", j, v[0], v[1])
os.Exit(1)
}
}
}
}
}
--
You received this message because you are subscribed to the Google Groups
"golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to golang-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 6 of 18 | next ›
Discussion Overview
groupgolang-dev @
categoriesgo
postedFeb 11, '15 at 11:27p
activeFeb 20, '15 at 6:47a
posts18
users5
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase