FAQ
I couldn't find a good example of how to do multipart file upload with some
extra params, so I wrote one:
http://matt.aimonetti.net/posts/2013/07/01/golang-multipart-file-upload-example/

Feel free to let me know if I missed something or did something wrong.

- Matt

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Jonathan at Jul 3, 2013 at 2:13 pm
    Instead of buffering the file into memory twice, you can leverage io.Pipe
    and a goroutine to stream the part from disk:
    http://play.golang.org/p/xfNSyyTd15

    Jonathan
    On Tuesday, July 2, 2013 11:32:28 AM UTC-4, Matt Aimonetti wrote:

    I couldn't find a good example of how to do multipart file upload with
    some extra params, so I wrote one:

    http://matt.aimonetti.net/posts/2013/07/01/golang-multipart-file-upload-example/

    Feel free to let me know if I missed something or did something wrong.

    - Matt
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jonathan at Jul 3, 2013 at 2:16 pm
    Fixed two mistakes that I made (this is still completely untested):
    http://play.golang.org/p/eEFBMGMNTW

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Matt Aimonetti at Jul 4, 2013 at 7:05 pm
    Having to use a goroutine makes the example quite more complex, and withou
    the goroutine the code just blocks. However, I agree that I should have use
    `copy` instead of reading the content of the file in memory.
    Good point also about using "file/path". I updated the blog post with your
    suggestions, thanks.

    On Wed, Jul 3, 2013 at 7:15 AM, wrote:

    Fixed two mistakes that I made (this is still completely untested):
    http://play.golang.org/p/eEFBMGMNTW

    --
    You received this message because you are subscribed to a topic in the
    Google Groups "golang-nuts" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/golang-nuts/WIFky_-5qpU/unsubscribe.
    To unsubscribe from this group and all its topics, send an email to
    golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jonathan Rudenberg at Jul 4, 2013 at 7:15 pm

    On Jul 4, 2013, at 3:04 PM, Matt Aimonetti wrote:

    Having to use a goroutine makes the example quite more complex, and withou the goroutine the code just blocks.
    Sure, it's a bit more complex, but it demonstrates the correct way to do this with streaming IO vs reading the entire file into memory for no good reason.

    Also, I noticed another bug:

         var bodyContent []byte
         // …
         resp.Body.Read(bodyContent)

    bodyContent is a nil slice, so the Read call will read zero bytes. Instead, you can do this:

         buf := &bytes.Buffer{}
         _, err := buf.ReadFrom(resp.Body)

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Matt Aimonetti at Jul 4, 2013 at 7:49 pm
    oh man, how did I miss this huge bug o_O
    Thanks, I updated the blog post ince again.
    I added a link to your example with the goroutine, and while it's probably
    the best approach it does add more complexity.

    Can you help me better understand the difference between using a pipe and
    copying the memory?
    It's probably a stupid question but I'm not quite sure I understand the
    problem. Are you saying that allocation wise, using copy vs piping is the
    same thing, but the difference is that using copy requires that we read the
    entire source in memory instead of reading by small chunks? (which could be
    a problem in case of big files)
    If that's the case, wouldn't something like bufio helps?
    Again, I realize it might sound like a stupid question but I would like to
    make sure I fully understand the problem.

    Thanks



    On Thu, Jul 4, 2013 at 12:15 PM, Jonathan Rudenberg
    wrote:
    On Jul 4, 2013, at 3:04 PM, Matt Aimonetti wrote:

    Having to use a goroutine makes the example quite more complex, and
    withou the goroutine the code just blocks.

    Sure, it's a bit more complex, but it demonstrates the correct way to do
    this with streaming IO vs reading the entire file into memory for no good
    reason.

    Also, I noticed another bug:

    var bodyContent []byte
    // …
    resp.Body.Read(bodyContent)

    bodyContent is a nil slice, so the Read call will read zero bytes.
    Instead, you can do this:

    buf := &bytes.Buffer{}
    _, err := buf.ReadFrom(resp.Body)
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jonathan Rudenberg at Jul 4, 2013 at 8:13 pm

    On Jul 4, 2013, at 3:48 PM, Matt Aimonetti wrote:

    Are you saying that allocation wise, using copy vs piping is the same thing, but the difference is that using copy requires that we read the entire source in memory instead of reading by small chunks? (which could be a problem in case of big files)
    You are copying the entire file from disk into an in-memory buffer (the bytes.Buffer that you are encoding the multipart request body into). This will fail when the file is too large to fit into memory and is also slower, because you have to wait to read the entire file into memory.

    The pipe that my implementation uses never stores more than the chunk size used in io.Copy in memory, streaming it from the disk through to the socket one chunk at a time.
    If that's the case, wouldn't something like bufio helps?
    bufio exists to solve a different problem by combining lots of tiny read/write syscalls and providing helpers for some standard operations like reading a line at a time.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedJul 2, '13 at 3:32p
activeJul 4, '13 at 8:13p
posts7
users2
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase