FAQ
I need to write a nested MIME multipart body — i.e. a multipart that contains parts that are themselves multipart. I can’t find a way to do this without creating a secondary data buffer, which increases the memory usage.

Let’s say I already have a multipart.Writer for the outer body, and I now want to write a new Part that’s itself multipart. The two things I need to call are
* writer.CreatePart, to start a new part in the outer body;
* multipart.NewWriter, to create the nested multipart body.
But there’s a chicken-and-egg problem with these: I need to give CreatePart the headers to include, including the all-important Content-Type, which contains a “separator” parameter derived from the new Writer. But to create the Writer I need an io.Writer for it to write into, which is what CreatePart returns. So each one of them needs a value provided by the other, and I can’t call either one first!

The workaround I’m using is to create a bytes.Buffer and tell NewWriter to write into that. Then after I’m done writing the nested body I get the byte-array from the Buffer, create the Part, and write the bytes into it.

This works, but ends up doing a lot of in-memory copying. I’m worried about that since this code is going to typically be running on file attachments that could be arbitrarily huge. Ideally I’d stream them straight from their origin to the HTTP response without creating in-memory copies. Is there any way around this?

—Jens

Search Discussions

  • Tamás Gulácsi at Jul 20, 2013 at 9:25 am
    You need to work leaf-first, and wrap the parts as you walk upwards in the tree.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Kyle Lemons at Jul 20, 2013 at 10:19 pm
    I suspect you can create a proxy writer that has the behavior you want by
    just buffering at the beginning.

    (yes, I know I didn't check errors, you should:)
    http://play.golang.org/p/E9RFUqmRFl

    On Fri, Jul 19, 2013 at 4:58 PM, Jens Alfke wrote:

    I need to write a nested MIME multipart body — i.e. a multipart that
    contains parts that are themselves multipart. I can’t find a way to do this
    without creating a secondary data buffer, which increases the memory usage.

    Let’s say I already have a multipart.Writer for the outer body, and I now
    want to write a new Part that’s itself multipart. The two things I need to
    call are
    * writer.CreatePart, to start a new part in the outer body;
    * multipart.NewWriter, to create the nested multipart body.
    But there’s a chicken-and-egg problem with these: I need to give
    CreatePart the headers to include, including the all-important
    Content-Type, which contains a “separator” parameter derived from the new
    Writer. But to create the Writer I need an io.Writer for it to write into,
    which is what CreatePart returns. So each one of them needs a value
    provided by the other, and I can’t call either one first!

    The workaround I’m using is to create a bytes.Buffer and tell NewWriter to
    write into that. Then after I’m done writing the nested body I get the
    byte-array from the Buffer, create the Part, and write the bytes into it.

    This works, but ends up doing a lot of in-memory copying. I’m worried
    about that since this code is going to typically be running on file
    attachments that could be arbitrarily huge. Ideally I’d stream them
    straight from their origin to the HTTP response without creating in-memory
    copies. Is there any way around this?

    —Jens
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • David DENG at Jul 20, 2013 at 11:41 pm
    You can generate the boundary by your own, then you get the header for
    CreatePart, call NewWriter, and set the boundary by calling
    multipart.Writer.SetBoundary.

    This is the boundary generation code in the built-in lib:

    func randomBoundary() string {
             var buf [30]byte
             _, err := io.ReadFull(rand.Reader, buf[:])
             if err != nil {
                     panic(err)
             }
             return fmt.Sprintf("%x", buf[:])
    }


    David
    On Saturday, July 20, 2013 7:58:06 AM UTC+8, Jens Alfke wrote:

    I need to write a nested MIME multipart body — i.e. a multipart that
    contains parts that are themselves multipart. I can’t find a way to do this
    without creating a secondary data buffer, which increases the memory usage.

    Let’s say I already have a multipart.Writer for the outer body, and I now
    want to write a new Part that’s itself multipart. The two things I need to
    call are
    * writer.CreatePart, to start a new part in the outer body;
    * multipart.NewWriter, to create the nested multipart body.
    But there’s a chicken-and-egg problem with these: I need to give
    CreatePart the headers to include, including the all-important
    Content-Type, which contains a “separator” parameter derived from the new
    Writer. But to create the Writer I need an io.Writer for it to write into,
    which is what CreatePart returns. So each one of them needs a value
    provided by the other, and I can’t call either one first!

    The workaround I’m using is to create a bytes.Buffer and tell NewWriter to
    write into that. Then after I’m done writing the nested body I get the
    byte-array from the Buffer, create the Part, and write the bytes into it.

    This works, but ends up doing a lot of in-memory copying. I’m worried
    about that since this code is going to typically be running on file
    attachments that could be arbitrarily huge. Ideally I’d stream them
    straight from their origin to the HTTP response without creating in-memory
    copies. Is there any way around this?

    —Jens
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedJul 19, '13 at 11:58p
activeJul 20, '13 at 11:41p
posts4
users4
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase