Grokbase › Groups › Go › golang-dev › November 2012
FAQ
Change 97b8514db41b broke the windows-386-ec2 build:
http://build.golang.org/log/146a2a7d9b24441dc14602a1293918191d4e75f1

runtime, runtime/cgo: track memory allocated by non-Go code

Otherwise a poorly timed GC can collect the memory before it
is returned to the Go program.

R=golang-dev, dave, dvyukov, minux.ma
CC=golang-dev
http://codereview.appspot.com/6819119

http://code.google.com/p/go/source/detail?r=97b8514db41b

$ tail -200 < log
crypto/sha512
database/sql/driver
database/sql
debug/gosym
encoding/ascii85
encoding/base32
encoding/csv
exp/ebnf
exp/ebnflint
exp/types
exp/gotype
exp/html/atom
exp/html
exp/norm
exp/locale/collate
hash/fnv
exp/locale/collate/build
exp/locale/collate/tools/colcmp
exp/utf8string
exp/winfsnotify
hash/crc64
image/color
image
image/draw
image/gif
image/jpeg
image/png
log/syslog
math/cmplx
net/http/cgi
net/http/fcgi
net/http/httptest
net/http/httputil
net/mail
net/rpc
net/rpc/jsonrpc
net/smtp
old/netchan
os/signal
os/user
runtime/cgo
testing
testing/iotest
testing/quick


# Testing packages.
ok cmd/api 0.235s
? cmd/cgo [no test files]
ok cmd/fix 3.654s
ok cmd/go 0.236s
? cmd/godoc [no test files]
ok cmd/gofmt 0.262s
? cmd/vet [no test files]
? cmd/yacc [no test files]
ok archive/tar 0.125s
ok archive/zip 0.256s
ok bufio 0.154s
ok bytes 0.350s
ok compress/bzip2 0.325s
ok compress/flate 1.860s
ok compress/gzip 0.091s
ok compress/lzw 0.369s
ok compress/zlib 3.678s
ok container/heap 0.138s
ok container/list 0.075s
ok container/ring 0.129s
? crypto [no test files]
ok crypto/aes 0.110s
ok crypto/cipher 0.131s
ok crypto/des 0.242s
ok crypto/dsa 0.159s
ok crypto/ecdsa 0.251s
ok crypto/elliptic 0.149s
ok crypto/hmac 0.128s
ok crypto/md5 0.078s
ok crypto/rand 0.215s
ok crypto/rc4 0.028s
ok crypto/rsa 0.594s
ok crypto/sha1 0.147s
ok crypto/sha256 0.080s
ok crypto/sha512 0.135s
ok crypto/subtle 0.098s
ok crypto/tls 0.750s
ok crypto/x509 2.924s
? crypto/x509/pkix [no test files]
ok database/sql 0.110s
ok database/sql/driver 0.164s
ok debug/dwarf 0.033s
ok debug/elf 0.149s
ok debug/gosym 0.127s
ok debug/macho 0.114s
ok debug/pe 0.101s
ok encoding/ascii85 0.128s
ok encoding/asn1 0.104s
ok encoding/base32 0.195s
ok encoding/base64 0.150s
ok encoding/binary 0.097s
ok encoding/csv 0.175s
ok encoding/gob 0.171s
ok encoding/hex 0.131s
ok encoding/json 0.207s
ok encoding/pem 0.083s
ok encoding/xml 0.158s
ok errors 0.135s
ok exp/ebnf 0.077s
ok exp/ebnflint 0.138s
ok exp/gotype 2.215s
ok exp/html 0.440s
ok exp/html/atom 0.085s
ok exp/locale/collate 0.213s
ok exp/locale/collate/build 0.115s
? exp/locale/collate/tools/colcmp [no test files]
ok exp/norm 2.282s
ok exp/types 0.282s
ok exp/utf8string 0.120s
ok exp/winfsnotify 0.218s
ok expvar 0.057s
ok flag 0.098s
ok fmt 0.239s
ok go/ast 0.167s
ok go/build 0.659s
ok go/doc 0.225s
ok go/parser 0.176s
ok go/printer 0.953s
ok go/scanner 0.114s
ok go/token 0.113s
? hash [no test files]
ok hash/adler32 0.144s
ok hash/crc32 0.131s
ok hash/crc64 0.171s
ok hash/fnv 0.100s
ok html 0.194s
ok html/template 0.198s
ok image 0.407s
ok image/color 0.145s
ok image/draw 0.206s
? image/gif [no test files]
ok image/jpeg 0.543s
ok image/png 0.240s
ok index/suffixarray 0.149s
ok io 0.096s
ok io/ioutil 0.125s
ok log 0.079s
? log/syslog [no test files]
ok math 0.129s
ok math/big 0.879s
ok math/cmplx 0.087s
ok math/rand 0.165s
ok mime 0.184s
ok mime/multipart 0.442s
ok net 1.397s
--- FAIL: TestRequestLimit (1.02 seconds)
serve_test.go:1043: Do: Get http://127.0.0.1:55581: WSARecv tcp
127.0.0.1:55582: An established connection was aborted by the software in
your host machine.
FAIL
FAIL net/http 7.649s
ok net/http/cgi 0.263s
ok net/http/fcgi 0.195s
ok net/http/httptest 0.206s
ok net/http/httputil 0.199s
? net/http/pprof [no test files]
ok net/mail 0.199s
ok net/rpc 0.208s
ok net/rpc/jsonrpc 0.226s
ok net/smtp 0.159s
ok net/textproto 0.183s
ok net/url 0.129s
ok old/netchan 0.106s
ok os 0.298s
ok os/exec 0.542s
ok os/signal 2.237s
ok os/user 11.440s
ok path 0.145s
ok path/filepath 0.656s
ok reflect 0.119s
ok regexp 0.308s
ok regexp/syntax 3.827s
ok runtime 7.054s
? runtime/cgo [no test files]
ok runtime/debug 0.132s
ok runtime/pprof 1.290s
ok sort 0.299s
ok strconv 2.093s
ok strings 0.331s
ok sync 0.191s
ok sync/atomic 0.182s
ok syscall 0.129s
? testing [no test files]
? testing/iotest [no test files]
ok testing/quick 0.099s
ok text/scanner 0.128s
ok text/tabwriter 0.131s
ok text/template 0.189s
ok text/template/parse 0.085s
ok time 53.780s
ok unicode 0.138s
ok unicode/utf16 0.126s
ok unicode/utf8 0.124s
? unsafe [no test files]

Search Discussions

  • Brainman at Nov 11, 2012 at 10:33 pm ⇧
    I think this failure in TestRequestLimitis is to be expected - connection
    is reset by the server once it has determined that incoming headers are too
    large.

    Alex
  • Brad Fitzpatrick at Nov 11, 2012 at 11:55 pm ⇧
    Maybe. The test's comment is very clear that it wants to succeed, though.

    The server does:

    fmt.Fprintf(c.rwc, "HTTP/1.1 %s\r\n\r\n", msg)

    the "413 Request Entity Too Large" msg in server.go before we close the
    connection, and that's on the unbuffered net.Conn.

    So why does every other platform read the final data before the close, but
    Windows sometimes reads the close first?

    The server.go could be changed to write that string, close just the write
    side of the TCP connection, sleep for 250 milliseconds or so, and then
    close the whole connection, but that feels unnecessary. I'd like to
    understand why Windows is different before we resort to that.

    On Sun, Nov 11, 2012 at 2:33 PM, brainman wrote:

    I think this failure in TestRequestLimitis is to be expected - connection
    is reset by the server once it has determined that incoming headers are too
    large.

    Alex
  • Brainman at Nov 12, 2012 at 5:23 am ⇧

    On Monday, 12 November 2012 10:55:31 UTC+11, Brad Fitzpatrick wrote:

    The server does:
    fmt.Fprintf(c.rwc, "HTTP/1.1 %s\r\n\r\n", msg)
    the "413 Request Entity Too Large" msg in server.go before we close the
    connection, ...

    The problem, as I see it, is unread by the server data. I do not think you
    will be 100% until you read those. I understand why you do not want to read
    those, but then you will have connection resets sometimes. I think it is
    one or the other.
    and that's on the unbuffered net.Conn.
    I think it is irrelevant here. Any network IO is buffered - inside kernel,
    in the drivers, inside routers forwarding your message.
    So why does every other platform read the final data before the close,
    but Windows sometimes reads the close first?

    I do not know. But here is a similar example. A diff to 422c00e8e022:

    diff --git a/src/pkg/net/net_test.go b/src/pkg/net/net_test.go
    --- a/src/pkg/net/net_test.go
    +++ b/src/pkg/net/net_test.go
    @@ -213,3 +213,78 @@
    t.Fatal(err)
    }
    }
    +
    +func TestALEX(t *testing.T) {
    + l, err := Listen("tcp", "127.0.0.1:0")
    + if err != nil {
    + t.Fatal(err)
    + }
    + defer l.Close()
    +
    + read := func(r io.Reader) error {
    + var m [1]byte
    + _, err := r.Read(m[:])
    + return err
    + }
    +
    + write := func(w io.Writer) error {
    + var m [1]byte
    + _, err := w.Write(m[:])
    + return err
    + }
    +
    + ec := make(chan error)
    +
    + go func() (err error) {
    + defer func() {
    + ec <- err
    + }()
    +
    + c, err := l.Accept()
    + if err != nil {
    + t.Fatal(err)
    + }
    + defer c.Close()
    +
    + err = read(c)
    + if err != nil {
    + return err
    + }
    + err = write(c)
    + if err != nil {
    + return err
    + }
    +
    + if err != nil {
    + return err
    + }
    + return nil
    + }()
    +
    + c, err := Dial("tcp", l.Addr().String())
    + if err != nil {
    + t.Fatal(err)
    + }
    + defer c.Close()
    +
    + err = write(c)
    + if err != nil {
    + t.Fatal(err)
    + }
    +
    + var m [10000000]byte
    + _, err = c.Write(m[:])
    + if err != nil {
    + t.Fatal(err)
    + }
    +
    + err = read(c)
    + if err != nil {
    + t.Fatal(err)
    + }
    +
    + err = <-ec
    + if err != nil {
    + t.Fatal(err)
    + }
    +}

    This runs fine on windows-386, but fails on linux-386 with:

    # go test -v -run=ALEX
    === RUN TestALEX
    --- FAIL: TestALEX (0.00 seconds)
    net_test.go:278: write tcp 127.0.0.1:37936: connection reset by peer
    FAIL
    exit status 1
    FAIL net 0.009s
    The server.go could be changed to write that string, close just the write
    side of the TCP connection, sleep for 250 milliseconds or so, and then
    close the whole connection, ...

    I do not think it will fix the problem, because you are still not prepared
    to read those bytes in fly. I think that will be the only way.

    Alex
  • Brad Fitzpatrick at Nov 12, 2012 at 5:29 am ⇧

    On Sun, Nov 11, 2012 at 9:23 PM, brainman wrote:
    On Monday, 12 November 2012 10:55:31 UTC+11, Brad Fitzpatrick wrote:

    The server does:
    fmt.Fprintf(c.rwc, "HTTP/1.1 %s\r\n\r\n", msg)
    the "413 Request Entity Too Large" msg in server.go before we close the
    connection, ...

    The problem, as I see it, is unread by the server data. I do not think you
    will be 100% until you read those. I understand why you do not want to read
    those, but then you will have connection resets sometimes. I think it is
    one or the other.

    I don't follow.

    read? server data?
  • Brainman at Nov 12, 2012 at 5:32 am ⇧

    On Monday, 12 November 2012 16:29:17 UTC+11, Brad Fitzpatrick wrote:

    I don't follow.

    read? server data?
    You read only first int64(c.server.maxHeaderBytes()) + 4096 bytes from the
    client. Don't you? What happens to the other bytes sent by the client?

    Alex
  • Brad Fitzpatrick at Nov 12, 2012 at 5:34 am ⇧

    On Sun, Nov 11, 2012 at 9:32 PM, brainman wrote:
    On Monday, 12 November 2012 16:29:17 UTC+11, Brad Fitzpatrick wrote:


    I don't follow.

    read? server data?
    You read only first int64(c.server.maxHeaderBytes()) + 4096 bytes from the
    client. Don't you? What happens to the other bytes sent by the client?
    I don't read those.
  • Brainman at Nov 12, 2012 at 5:37 am ⇧

    On Monday, 12 November 2012 16:34:49 UTC+11, Brad Fitzpatrick wrote:
    ..
    I don't read those.
    I think that is the problem. Look at the example I posted, "it does not
    read some bytes" too. See what the result of "not reading" is. On linux.

    Alex
  • Brad Fitzpatrick at Nov 12, 2012 at 5:41 am ⇧

    On Sun, Nov 11, 2012 at 9:37 PM, brainman wrote:
    On Monday, 12 November 2012 16:34:49 UTC+11, Brad Fitzpatrick wrote:

    ..

    I don't read those.
    I think that is the problem. Look at the example I posted, "it does not
    read some bytes" too. See what the result of "not reading" is. On linux.
    I thought we were talking about windows. It's the one flaking on that test.

    Can we make it act like all the other platforms?
  • Brainman at Nov 12, 2012 at 5:46 am ⇧

    On Monday, 12 November 2012 16:41:04 UTC+11, Brad Fitzpatrick wrote:

    I thought we were talking about windows. ...
    You are talking about windows. :-) I think your assumptions are wrong.
    (That you could leave some unread data on TCP connection and still be able
    to close it without reset).

    Can we make it act like all the other platforms?
    I really do not know how.

    Alex
  • Brad Fitzpatrick at Nov 12, 2012 at 6:28 am ⇧
    Sent http://codereview.appspot.com/6826084/
    On Nov 11, 2012 9:46 PM, "brainman" wrote:
    On Monday, 12 November 2012 16:41:04 UTC+11, Brad Fitzpatrick wrote:


    I thought we were talking about windows. ...
    You are talking about windows. :-) I think your assumptions are wrong.
    (That you could leave some unread data on TCP connection and still be able
    to close it without reset).

    Can we make it act like all the other platforms?
    I really do not know how.

    Alex

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-dev @
categoriesgo
postedNov 10, '12 at 7:32p
activeNov 12, '12 at 6:28a
posts11
users3
websitegolang.org

3 users in discussion

Brainman: 5 posts Brad Fitzpatrick: 5 posts Builder: 1 post

Content

People

Support

Translate

site design / logo © 2022 Grokbase