FAQ
I am downloading an image and every once in a while I don't get all the
bytes for the image

resp, err := http..Get(this.RadarStation.ImageUrl)

image, err = ioutil.ReadAll(resp.Body)


Just recently this call downloaded 8349 bytes of the image


When I download the same image using Chrome I get the full image which is 8417 bytes.


Am I missing a flush or something to guarantee I get the entire file?

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Konstantin Khomoutov at Sep 17, 2013 at 4:03 pm

    On Tue, 17 Sep 2013 08:19:55 -0700 (PDT) William Kennedy wrote:

    I am downloading an image and every once in a while I don't get all
    the bytes for the image

    resp, err := http..Get(this.RadarStation.ImageUrl)

    image, err = ioutil.ReadAll(resp.Body)


    Just recently this call downloaded 8349 bytes of the image


    When I download the same image using Chrome I get the full image
    which is 8417 bytes.


    Am I missing a flush or something to guarantee I get the entire file?
    resp.Body is a socket reader; flushing is about files you write to, not
    read from.

    A dumb question: do you check for err != nil after
    ioutil.ReadAll(resp.Body) finishes?

    Aren't you doing this download in a goroutine, and your main() exits
    before actually waiting for all the goroutines to finish (thus killing
    them hardly)?

    Another question: are you calling resp.Body.Close() checking the error
    it returns?

    By the way, if you just want to save the image to a file, a better way
    would be to

    func closeOrPanic(c io.Closer) {
       err := c.Close()
       if err != nil {
         panic(err)
       }
    }

    ...

    f, err := os.OpenFile(...)
    // check for error here
    defer closeOrPanic(f)
    b := resp.Body
    defer closeOrPanic(b)
    _, err := io.Copy(f, b)
    // check for error here

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Islandberry23 at Sep 17, 2013 at 4:03 pm

    On Tuesday, September 17, 2013 8:19:55 AM UTC-7, William Kennedy wrote:

    Am I missing a flush or something to guarantee I get the entire file?
    Check resp.StatusCode == 200 to confirm that you are getting the image and
    not an error message from the server.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • William Kennedy at Sep 17, 2013 at 4:08 pm
    I am getting the image, just seems not the whole image sometimes. The
    browser can display it but ImageMagic says the image is corrupt. Really
    just need to keep this in memory. No question is stupid and it was not
    necessary to say that. I am pulling at straws and was hoping that somehow,
    maybe, I was not getting everything from the socket stream.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • William Kennedy at Sep 17, 2013 at 5:50 pm
    Looks like I had a bug closing the Body. It was not happening every time
    and this was causing all my problems. Sorry for wasting everyone's time on
    this. The new code is cleaner and no problems ever since.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • William Kennedy at Sep 17, 2013 at 6:19 pm
    I didn't fix it with the closing of the Body.

    There are times when I download an image and I don't get the whole thing.
    Missing around 60+ bytes.

    Here is a url for an
    image: http://radar.weather.gov/ridge/RadarImg/N0R/DYX_N0R_0.gif

    However this image changes ever 120 seconds. Once it changes I won't have a
    problem with that particular url. If I use the browser to download the same
    image, when I get a notification that the image is corrupt, I get a larger
    image back.

    resp, err := http.Get(this.RadarStation.ImageUrl)


    defer resp.Body.Close()


    if err != nil {

         // There is no error

    }


    image, err = ioutil.ReadAll(resp.Body)


    I don't know what else to check or try?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • William Kennedy at Sep 18, 2013 at 3:42 am
    I got lucky and saw the browser pull down the same corrupted gif image as the http Go code was doing. I think there may be a bad server in the farm and I finally hit it with the browser. Having the browser finally pull down the image the same way the go code did was a relief. I am downloading many images at a time. The browser rendering a corrupted gif image didn't help the situation either.

    Thanks for all the help and suggestions.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedSep 17, '13 at 3:20p
activeSep 18, '13 at 3:42a
posts7
users3
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase