FAQ
Hello,

The current implementation of cookiejar.Cookies(url.URL) returns cookies
with only attributes Name and Value being set. It would be more beneficial
if they contain most attributes (with the exception of max-age) set in a
response by the server as this enables code extracting cookies to extend
further processing such as adding persistence storage. I'd prepare a patch
if the request makes sense.

Thanks,
tan

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Nigel Tao at Jul 30, 2015 at 1:49 am

    On Thu, Jul 30, 2015 at 9:50 AM, tandalat wrote:
    The current implementation of cookiejar.Cookies(url.URL) returns cookies
    with only attributes Name and Value being set. It would be more beneficial
    if they contain most attributes (with the exception of max-age) set in a
    response by the server as this enables code extracting cookies to extend
    further processing such as adding persistence storage. I'd prepare a patch
    if the request makes sense.
    Re persistent storage, there was a lot of discussion with Dr. Volker
    Dobler (CC'ed) and I a couple of years ago about how to do a cookie
    jar with pluggable storage, but we never landed on a satisfactory
    design, let alone a perfect one, so what's in the standard library is
    not persistent.

    If you come to us with an existence proof of a good design for a
    persistent cookie jar, I'm happy to look at it, but in the meantime,
    if you want persistence, it''s probably best to fork
    net/http/cookiejar for now. Once a change lands in the standard
    library and is released, we can no longer change its API, and I don't
    know what the right API is yet.

    https://groups.google.com/forum/#!topicsearchin/golang-dev/cookiejar$20-%22broken$20by%22
    and
    https://groups.google.com/d/topic/golang-dev/7Jcbgrbah2s/discussion
    might be places to start looking for previous discussions.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Volker Dobler at Jul 30, 2015 at 8:03 am

    On Thu, Jul 30, 2015 at 3:49 AM, Nigel Tao wrote:
    On Thu, Jul 30, 2015 at 9:50 AM, tandalat wrote:
    The current implementation of cookiejar.Cookies(url.URL) returns cookies
    with only attributes Name and Value being set. It would be more beneficial
    if they contain most attributes (with the exception of max-age) set in a
    response by the server as this enables code extracting cookies to extend
    further processing such as adding persistence storage. I'd prepare a patch
    if the request makes sense.
    Re persistent storage, there was a lot of discussion with Dr. Volker
    Dobler (CC'ed) and I a couple of years ago about how to do a cookie
    jar with pluggable storage, but we never landed on a satisfactory
    design, let alone a perfect one, so what's in the standard library is
    not persistent.

    If you come to us with an existence proof of a good design for a
    persistent cookie jar, I'm happy to look at it, but in the meantime,
    if you want persistence, it''s probably best to fork
    net/http/cookiejar for now. Once a change lands in the standard
    library and is released, we can no longer change its API, and I don't
    know what the right API is yet.
    Yes, a persistent cookiejar with proper error handling will be very hard.

    But If I understand Tan correctly he is suggesting to just add some
    API to retrieve all cookie entries from the jar. Maybe something like

         // UsedETLDs returns all effective top level domains for which
         // the Jar contains cookies.
         func (j *Jar) UsedETLDs() []string

         // CookiesFor returns all stored (and not expired) cookies for
         // the given effective top level domain etld.
         func (j *Jar) CookiesFor(etld string) []Entry

    with Entry some exported form of cookiejar.entry.
    (While the two stage retrieval might look strange it has some
    benefits.)

    Based on this API a user of package cookiejar could extract all
    stored cookies and persist them herself. Probably loading them
    back into a Jar would require some help to safeguard against subtle
    errors and PSL changes.

    Coming up with the "right" API here will be difficult, but might be
    simpler than a real persistent cookiejar.

    V.
    https://groups.google.com/forum/#!topicsearchin/golang-dev/cookiejar$20-%22broken$20by%22
    and
    https://groups.google.com/d/topic/golang-dev/7Jcbgrbah2s/discussion
    might be places to start looking for previous discussions.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Tandalat at Jul 31, 2015 at 8:36 am

    On Thursday, July 30, 2015 at 1:04:24 AM UTC-7, Volker Dobler wrote:
    But If I understand Tan correctly he is suggesting to just add some
    API to retrieve all cookie entries from the jar. Maybe something like

    // UsedETLDs returns all effective top level domains for which
    // the Jar contains cookies.
    func (j *Jar) UsedETLDs() []string

    // CookiesFor returns all stored (and not expired) cookies for
    // the given effective top level domain etld.
    func (j *Jar) CookiesFor(etld string) []Entry

    with Entry some exported form of cookiejar.entry.
    (While the two stage retrieval might look strange it has some
    benefits.)
    Yes, indeed. Is there a reason why cookie objects returned by
    cookiejar.Cookies() do not have other fields set other than Name and Value?
    In the source code for cookiejar.cookies(...):

    225 for _, e := range selected {226 cookies = append(cookies, &http.Cookie{Name: e.Name, Value: e.Value})227 }


    My proposal is simply filling up other fields like so,

      for _, e := range selected {
             c := &http.Cookie{
                      Name: e.Name,
                      Value: e.Value,
                      Path: e.Path,
                      Domain: e.Domain,
                      Expires: e.Expires,
                      Secure: e.Secure,
                      HttpOnly: e.HttpOnly,
                   }
            cookies = append(cookies, c)
    }


    What's the catch here?


    Based on this API a user of package cookiejar could extract all
    stored cookies and persist them herself. Probably loading them
    back into a Jar would require some help to safeguard against subtle
    errors and PSL changes.
    Won't keying the urls (used to retrieve cookies) for serialized cookies on
    disk or database and later using them to load the cookies back into the jar
    keep things consistent? For example,

             store[u.String()] = jar.Cookies(u)
             ... serialize to some storage ...
             ... load from storage
             for k, v := range store {
                      jar.SetCookies(url.MustParse(k), v)
             }
             ....

    Thanks
    tan


    Coming up with the "right" API here will be difficult, but might be
    simpler than a real persistent cookiejar.
    V.

    https://groups.google.com/forum/#!topicsearchin/golang-dev/cookiejar$20-%22broken$20by%22
    and
    https://groups.google.com/d/topic/golang-dev/7Jcbgrbah2s/discussion
    might be places to start looking for previous discussions.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Volker Dobler at Jul 31, 2015 at 9:28 am
    On Fri, Jul 31, 2015 at 10:36 AM, tandalat wrote:
    [...]
    Yes, indeed. Is there a reason why cookie objects returned by
    cookiejar.Cookies() do not have other fields set other than Name and Value?
    In the source code for cookiejar.cookies(...):

    225 for _, e := range selected {
    226 cookies = append(cookies, &http.Cookie{Name: e.Name, Value: e.Value})
    227 }


    My proposal is simply filling up other fields like so,

    for _, e := range selected {
    c := &http.Cookie{
    Name: e.Name,
    Value: e.Value,
    Path: e.Path,
    Domain: e.Domain,
    Expires: e.Expires,
    Secure: e.Secure,
    HttpOnly: e.HttpOnly,
    }
    cookies = append(cookies, c)
    }


    What's the catch here?
    1. This would break the Go API compatibility guaranty.

    2. Cookies() returns http.Cookies suitable for generating
    a Cookie header. If all fields would be filled up you would
    get a cookie which would serialize to a Set-Cookie header.
    (package http could clear the other fields).

    3. How would you differentiate between a domain and a
    host cookie?

    4. You lose the creation and last access time, so you
    cannot use this for properly persisting and re-loading
    the cookies.
    Based on this API a user of package cookiejar could extract all
    stored cookies and persist them herself. Probably loading them
    back into a Jar would require some help to safeguard against subtle
    errors and PSL changes.

    Won't keying the urls (used to retrieve cookies) for serialized cookies on
    disk or database and later using them to load the cookies back into the jar
    keep things consistent? For example,

    store[u.String()] = jar.Cookies(u)
    ... serialize to some storage ...
    ... load from storage
    for k, v := range store {
    jar.SetCookies(url.MustParse(k), v)
    }
    ....
    Basically yes, but there might be some ugly corner cases like
    domain cookies which are no longer allowed because the
    domain is now considered a ETLD (because the PSL used
    to re-load the cookies differs from the one used during storing).
    And possible more if I thought about it longer.

    The main problem with keying by _URL_ is the Path field:
    You cannot know which URLs (i.e. which paths) you have
    to request as Cookies() returns only cookies which match
    the path of the URL. How would you know that the jar
    contains a cookie with Path="/some/obscure/deep/path"?

    V.

    [...]

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Roger peppe at Jul 31, 2015 at 7:28 pm
    FWIW I would love to see a decent disk-persistent cookiejar implementation.
    I forked the stdlib cookiejar package to make
    github.com/juju/persistentcookiejar as a very quick hack but it's
    demonstrably wrong in the face of concurrent users, and I'm sure for other
    reasons too.
    On 31 Jul 2015 10:28, "Volker Dobler" wrote:

    On Fri, Jul 31, 2015 at 10:36 AM, tandalat wrote:
    [...]
    Yes, indeed. Is there a reason why cookie objects returned by
    cookiejar.Cookies() do not have other fields set other than Name and Value?
    In the source code for cookiejar.cookies(...):

    225 for _, e := range selected {
    226 cookies = append(cookies, &http.Cookie{Name: e.Name, Value: e.Value})
    227 }


    My proposal is simply filling up other fields like so,

    for _, e := range selected {
    c := &http.Cookie{
    Name: e.Name,
    Value: e.Value,
    Path: e.Path,
    Domain: e.Domain,
    Expires: e.Expires,
    Secure: e.Secure,
    HttpOnly: e.HttpOnly,
    }
    cookies = append(cookies, c)
    }


    What's the catch here?
    1. This would break the Go API compatibility guaranty.

    2. Cookies() returns http.Cookies suitable for generating
    a Cookie header. If all fields would be filled up you would
    get a cookie which would serialize to a Set-Cookie header.
    (package http could clear the other fields).

    3. How would you differentiate between a domain and a
    host cookie?

    4. You lose the creation and last access time, so you
    cannot use this for properly persisting and re-loading
    the cookies.
    Based on this API a user of package cookiejar could extract all
    stored cookies and persist them herself. Probably loading them
    back into a Jar would require some help to safeguard against subtle
    errors and PSL changes.

    Won't keying the urls (used to retrieve cookies) for serialized cookies on
    disk or database and later using them to load the cookies back into the jar
    keep things consistent? For example,

    store[u.String()] = jar.Cookies(u)
    ... serialize to some storage ...
    ... load from storage
    for k, v := range store {
    jar.SetCookies(url.MustParse(k), v)
    }
    ....
    Basically yes, but there might be some ugly corner cases like
    domain cookies which are no longer allowed because the
    domain is now considered a ETLD (because the PSL used
    to re-load the cookies differs from the one used during storing).
    And possible more if I thought about it longer.

    The main problem with keying by _URL_ is the Path field:
    You cannot know which URLs (i.e. which paths) you have
    to request as Cookies() returns only cookies which match
    the path of the URL. How would you know that the jar
    contains a cookie with Path="/some/obscure/deep/path"?

    V.

    [...]

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Tandalat at Jul 31, 2015 at 8:04 am

    On Wednesday, July 29, 2015 at 6:49:30 PM UTC-7, Nigel Tao wrote:

    Re persistent storage, there was a lot of discussion with Dr. Volker
    Dobler (CC'ed) and I a couple of years ago about how to do a cookie
    jar with pluggable storage, but we never landed on a satisfactory
    design, let alone a perfect one, so what's in the standard library is
    not persistent.

    If you come to us with an existence proof of a good design for a
    persistent cookie jar, I'm happy to look at it, but in the meantime,
    if you want persistence, it''s probably best to fork
    net/http/cookiejar for now. Once a change lands in the standard
    library and is released, we can no longer change its API, and I don't
    know what the right API is yet.


    https://groups.google.com/forum/#!topicsearchin/golang-dev/cookiejar$20-%22broken$20by%22
    and
    https://groups.google.com/d/topic/golang-dev/7Jcbgrbah2s/discussion
    might be places to start looking for previous discussions.
    I'd read these threads before posting the change request for http.cookiejar
    and was aware of the difficulty to come up with a satisfactory design.
    Hence, my proposal isn't about a persistent cookie jar per se but an
    ability for other cookie processors to implement any extra handling
    required by their respective applications, and a consistency with original
    cookie information set in the response's header. A reasonable expectation
    is that cookie attributes (domain, path, expires, httponly etc.) sent by an
    application from the server side should be readily obtained via a cookie
    object returned by http.cookiejar.

    I was caught by surprise when all I could get out of a cookie object
    returned by cookiejar.Cookies() ([] http.Cookie) was just the name and
    value of the cookie while http.Cookie implements all fields defined for a
    Set-Cookie in RFC 6265. This pretty much limits the use of http.cookiejar
    to http.Client and provides little benefits for other cookie processors to
    perform any extra handling such as using domain, path, expires etc. for
    automated services (single sign-on, for instance) or persisting any
    non-session cookies etc.


    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedJul 30, '15 at 12:55a
activeJul 31, '15 at 7:28p
posts7
users4
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase