FAQ
Hi all,

I'm facing a strange issue with http.Get

I wrote something like this in my code:

func (f *myFetcher) Fetch(url string) (string, []string, error) {

// get html contents

resp, err := http.Get(url)
if err != nil {
return "", nil, fmt.Errorf("not found: %s", url);
}

// read to body bytes
defer resp.Body.Close();

bodyBytes, err := ioutil.ReadAll(resp.Body);
if err != nil {
return "", nil, fmt.Errorf("Failed to read content: %s", err)
}

// read to buffer

buf := bytes.NewBuffer(bodyBytes)
bodyStr := buf.String()
return bodyStr, nil, err

}

But then when I run this in main():

_, _, err := fetcher.Fetch("http://www.pixelactionstudio.com/");

if err != nil {
fmt.Println(fmt.Errorf("Error: %v", err));
} else {
fmt.Println("success");
}

It failed and show this:

Error: Failed to read content: use of closed network connection

I tried to change the URL to another websites, some of them works fine and
some are not.
Yet again, I have no idea why this happen. Seems some servers doesn't like
http.Get or something.

What could be my problem?

--

Search Discussions

  • Minux at Jan 9, 2013 at 11:43 am

    On Wed, Jan 9, 2013 at 7:06 PM, Koala Yeung wrote:

    Hi all,

    I'm facing a strange issue with http.Get

    I wrote something like this in my code:

    func (f *myFetcher) Fetch(url string) (string, []string, error) {

    // get html contents

    resp, err := http.Get(url)
    if err != nil {
    return "", nil, fmt.Errorf("not found: %s", url);
    }

    // read to body bytes
    defer resp.Body.Close();

    bodyBytes, err := ioutil.ReadAll(resp.Body);
    if err != nil {
    return "", nil, fmt.Errorf("Failed to read content: %s", err)
    }

    // read to buffer
    why not just return string(bodyBytes), nil, err? I don't understand why you
    use a bytes.Buffer for this
    purpose.
    buf := bytes.NewBuffer(bodyBytes)
    bodyStr := buf.String()
    return bodyStr, nil, err

    }

    But then when I run this in main():

    _, _, err := fetcher.Fetch("http://www.**pixelactionstudio.com/<http://www.pixelactionstudio.com/>
    ");

    if err != nil {
    fmt.Println(fmt.Errorf("Error: %v", err));
    } else {
    fmt.Println("success");
    }

    It failed and show this:

    Error: Failed to read content: use of closed network connection

    I tried to change the URL to another websites, some of them works fine and
    some are not.
    Yet again, I have no idea why this happen. Seems some servers doesn't like
    http.Get or something.

    What could be my problem?
    Seems like the net/http is reusing closed keep-alive idle connections.

    you can try using &http.Transport{DisableKeepAlives: true} as
    DefaultClient.Transport and
    see if the problem goes away.

    --
  • Chen Yufei at Jan 9, 2013 at 12:05 pm

    On 2013年1月9日Wednesday at 下午7:42, minux wrote:
    On Wed, Jan 9, 2013 at 7:06 PM, Koala Yeung (mailto:koalay@gmail.com)> wrote:
    Hi all,

    I'm facing a strange issue with http.Get

    I wrote something like this in my code:

    func (f *myFetcher) Fetch(url string) (string, []string, error) {

    // get html contents

    resp, err := http.Get(url)
    if err != nil {
    return "", nil, fmt.Errorf("not found: %s", url);
    }

    // read to body bytes
    defer resp.Body.Close();

    bodyBytes, err := ioutil.ReadAll(resp.Body);
    if err != nil {
    return "", nil, fmt.Errorf("Failed to read content: %s", err)
    }

    // read to buffer
    why not just return string(bodyBytes), nil, err? I don't understand why you use a bytes.Buffer for this
    purpose.
    buf := bytes.NewBuffer(bodyBytes)
    bodyStr := buf.String()
    return bodyStr, nil, err

    }

    But then when I run this in main():

    _, _, err := fetcher.Fetch("http://www.pixelactionstudio.com/");

    if err != nil {
    fmt.Println(fmt.Errorf("Error: %v", err));
    } else {
    fmt.Println("success");
    }

    It failed and show this:

    Error: Failed to read content: use of closed network connection

    I tried to change the URL to another websites, some of them works fine and some are not.
    Yet again, I have no idea why this happen. Seems some servers doesn't like http.Get or something.

    What could be my problem?
    Seems like the net/http is reusing closed keep-alive idle connections.

    you can try using &http.Transport{DisableKeepAlives: true} as DefaultClient.Transport and
    see if the problem goes away.
    Is it intentional that the net/http package does not retry the request when the server connection is closed?

    I guess not retrying the request would cause some trouble for users not familiar with HTTP persistent connection, as DisableKeepAlive is false by default.

    Regarding the "use of closed network connection" error, I didn't find an easy way to detect this particular error as this error is not exported in the net package. But detecting this error is useful in some cases. (Detecting closed server connection is one such case.)
    --


    --

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedJan 9, '13 at 11:13a
activeJan 9, '13 at 12:05p
posts3
users3
websitegolang.org

3 users in discussion

Chen Yufei: 1 post Minux: 1 post Koala Yeung: 1 post

People

Translate

site design / logo © 2022 Grokbase