FAQ
I need to create a type which implements the etcdserverpb.KVServer
interface[0]. Note that package context is imported by its canonical
import path[1]. Also note that etcd has a vendor folder in its root
which includes the context package[2].

[0] https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L1724-L1743
[1] https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L18
[2] https://github.com/coreos/etcd/tree/de801b500b1f2ab237757052edd87901a5906153/vendor/golang.org/x/net/context

The etcd repo contains packages that are intended to be imported by
other packages, such as this one. But it also contains binaries, so I
think the presence of the vendor folder is correct. Here is my
code[3]. It fails to build because etcdserverpb.KVServer.Compact
expects "github.com/coreos/etcd/vendor/golang.org/x/net/context".Context
rather than "golang.org/x/net/context".Context[4], which as far as I
know I cannot possibly provide.

[3] https://gist.github.com/peterbourgon/79f44f89d5c334b5f05a291f25fe6bd8
[4] https://gist.github.com/peterbourgon/304c21802b3531cd8d0b50247ca41522

What went wrong, and how can I fix it?

Cheers,
Peter.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Jonathan Gaillard at Mar 31, 2016 at 3:12 pm
    A bit of shell in your build routine to move up any nested vendor packages if your putting any packages in vendor at build time. Otherwise vendoring tools like govendor do this already.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Sam Boyer at Mar 31, 2016 at 3:20 pm
    AFAIK, you can't fix it without hacking etcd's vendor - assuming that etcd
    is under your project's vendor, you'd need to move x/net/context from under
    etcd/vendor to your own vendor/ directory. We've been referring to that
    process as 'flattening'.

    While vendor does allow you to use the canonical import path in the source
    code (yaaaaay), the compiler still rewrites those paths to what you see
    there ("github.com/coreos/etcd/vendor/golang.org/x/net/context"). Thus, any
    shared deps have to be pushed up to the topmost vendor directory.

    Most of the package management tools are working on solutions to this. I'm
    working on a comprehensive solver (https://github.com/sdboyer/vsolver) -
    with glide as the first integration target - that'll be able to figure out
    if it's safe to flatten in such a way, and explain why if it's not.
    On Thursday, March 31, 2016 at 11:01:22 AM UTC-4, Peter Bourgon wrote:

    I need to create a type which implements the etcdserverpb.KVServer
    interface[0]. Note that package context is imported by its canonical
    import path[1]. Also note that etcd has a vendor folder in its root
    which includes the context package[2].

    [0]
    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L1724-L1743
    [1]
    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L18
    [2]
    https://github.com/coreos/etcd/tree/de801b500b1f2ab237757052edd87901a5906153/vendor/golang.org/x/net/context

    The etcd repo contains packages that are intended to be imported by
    other packages, such as this one. But it also contains binaries, so I
    think the presence of the vendor folder is correct. Here is my
    code[3]. It fails to build because etcdserverpb.KVServer.Compact
    expects "github.com/coreos/etcd/vendor/golang.org/x/net/context".Context
    rather than "golang.org/x/net/context".Context[4], which as far as I
    know I cannot possibly provide.

    [3] https://gist.github.com/peterbourgon/79f44f89d5c334b5f05a291f25fe6bd8
    [4] https://gist.github.com/peterbourgon/304c21802b3531cd8d0b50247ca41522

    What went wrong, and how can I fix it?

    Cheers,
    Peter.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Peter Bourgon at Mar 31, 2016 at 3:24 pm
    Thanks for the replies. My sample code was misleading, the type that
    implements etcdserverpb.KVServer is itself in a library, so its repo
    has no vendor folder by design. My users should be able to `go get` my
    code, or use it in their own projects, without requiring any tricks in
    their build process. Does that make the problem intractable?
    On Thu, Mar 31, 2016 at 5:20 PM, Sam Boyer wrote:
    AFAIK, you can't fix it without hacking etcd's vendor - assuming that etcd
    is under your project's vendor, you'd need to move x/net/context from under
    etcd/vendor to your own vendor/ directory. We've been referring to that
    process as 'flattening'.

    While vendor does allow you to use the canonical import path in the source
    code (yaaaaay), the compiler still rewrites those paths to what you see
    there ("github.com/coreos/etcd/vendor/golang.org/x/net/context"). Thus, any
    shared deps have to be pushed up to the topmost vendor directory.

    Most of the package management tools are working on solutions to this. I'm
    working on a comprehensive solver (https://github.com/sdboyer/vsolver) -
    with glide as the first integration target - that'll be able to figure out
    if it's safe to flatten in such a way, and explain why if it's not.
    On Thursday, March 31, 2016 at 11:01:22 AM UTC-4, Peter Bourgon wrote:

    I need to create a type which implements the etcdserverpb.KVServer
    interface[0]. Note that package context is imported by its canonical
    import path[1]. Also note that etcd has a vendor folder in its root
    which includes the context package[2].

    [0]
    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L1724-L1743
    [1]
    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L18
    [2]
    https://github.com/coreos/etcd/tree/de801b500b1f2ab237757052edd87901a5906153/vendor/golang.org/x/net/context

    The etcd repo contains packages that are intended to be imported by
    other packages, such as this one. But it also contains binaries, so I
    think the presence of the vendor folder is correct. Here is my
    code[3]. It fails to build because etcdserverpb.KVServer.Compact
    expects "github.com/coreos/etcd/vendor/golang.org/x/net/context".Context
    rather than "golang.org/x/net/context".Context[4], which as far as I
    know I cannot possibly provide.

    [3] https://gist.github.com/peterbourgon/79f44f89d5c334b5f05a291f25fe6bd8
    [4] https://gist.github.com/peterbourgon/304c21802b3531cd8d0b50247ca41522

    What went wrong, and how can I fix it?

    Cheers,
    Peter.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Sam Boyer at Mar 31, 2016 at 3:32 pm
    If you can't assume a package manager, then yup, I think it's intractable.

    This exemplifies why "just committing vendor" is kind of a shortsighted
    solution. And, because this can easily happen recursively (e.g. etcd could
    import a project with main+lib packages that itself chooses to vendor),
    it's one that increases in harmfulness to the ecosystem as adoption of it
    increases.
    On Thursday, March 31, 2016 at 11:24:24 AM UTC-4, Peter Bourgon wrote:

    Thanks for the replies. My sample code was misleading, the type that
    implements etcdserverpb.KVServer is itself in a library, so its repo
    has no vendor folder by design. My users should be able to `go get` my
    code, or use it in their own projects, without requiring any tricks in
    their build process. Does that make the problem intractable?

    On Thu, Mar 31, 2016 at 5:20 PM, Sam Boyer <samuel....@gmail.com
    <javascript:>> wrote:
    AFAIK, you can't fix it without hacking etcd's vendor - assuming that etcd
    is under your project's vendor, you'd need to move x/net/context from under
    etcd/vendor to your own vendor/ directory. We've been referring to that
    process as 'flattening'.

    While vendor does allow you to use the canonical import path in the source
    code (yaaaaay), the compiler still rewrites those paths to what you see
    there ("github.com/coreos/etcd/vendor/golang.org/x/net/context"). Thus, any
    shared deps have to be pushed up to the topmost vendor directory.

    Most of the package management tools are working on solutions to this. I'm
    working on a comprehensive solver (https://github.com/sdboyer/vsolver) -
    with glide as the first integration target - that'll be able to figure out
    if it's safe to flatten in such a way, and explain why if it's not.
    On Thursday, March 31, 2016 at 11:01:22 AM UTC-4, Peter Bourgon wrote:

    I need to create a type which implements the etcdserverpb.KVServer
    interface[0]. Note that package context is imported by its canonical
    import path[1]. Also note that etcd has a vendor folder in its root
    which includes the context package[2].

    [0]
    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L1724-L1743
    [1]
    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L18
    [2]
    https://github.com/coreos/etcd/tree/de801b500b1f2ab237757052edd87901a5906153/vendor/golang.org/x/net/context
    The etcd repo contains packages that are intended to be imported by
    other packages, such as this one. But it also contains binaries, so I
    think the presence of the vendor folder is correct. Here is my
    code[3]. It fails to build because etcdserverpb.KVServer.Compact
    expects "github.com/coreos/etcd/vendor/golang.org/x/net/context".Context
    rather than "golang.org/x/net/context".Context[4], which as far as I
    know I cannot possibly provide.

    [3]
    https://gist.github.com/peterbourgon/79f44f89d5c334b5f05a291f25fe6bd8
    [4]
    https://gist.github.com/peterbourgon/304c21802b3531cd8d0b50247ca41522
    What went wrong, and how can I fix it?

    Cheers,
    Peter.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Peter Bourgon at Mar 31, 2016 at 3:39 pm
    Thanks. Committing vendor is fine, I think, as long as nobody imports
    your repo. That is, as long as you're exclusively a package main. As
    soon as you have something that others may want to import, all bets
    are off.

    This is very disappointing. It makes my project significantly more
    difficult to use as intended. Maybe so difficult that it will prevent
    widespread use.

    On Thu, Mar 31, 2016 at 5:32 PM, Sam Boyer wrote:
    If you can't assume a package manager, then yup, I think it's intractable.

    This exemplifies why "just committing vendor" is kind of a shortsighted
    solution. And, because this can easily happen recursively (e.g. etcd could
    import a project with main+lib packages that itself chooses to vendor), it's
    one that increases in harmfulness to the ecosystem as adoption of it
    increases.
    On Thursday, March 31, 2016 at 11:24:24 AM UTC-4, Peter Bourgon wrote:

    Thanks for the replies. My sample code was misleading, the type that
    implements etcdserverpb.KVServer is itself in a library, so its repo
    has no vendor folder by design. My users should be able to `go get` my
    code, or use it in their own projects, without requiring any tricks in
    their build process. Does that make the problem intractable?
    On Thu, Mar 31, 2016 at 5:20 PM, Sam Boyer wrote:
    AFAIK, you can't fix it without hacking etcd's vendor - assuming that
    etcd
    is under your project's vendor, you'd need to move x/net/context from
    under
    etcd/vendor to your own vendor/ directory. We've been referring to that
    process as 'flattening'.

    While vendor does allow you to use the canonical import path in the
    source
    code (yaaaaay), the compiler still rewrites those paths to what you see
    there ("github.com/coreos/etcd/vendor/golang.org/x/net/context"). Thus,
    any
    shared deps have to be pushed up to the topmost vendor directory.

    Most of the package management tools are working on solutions to this.
    I'm
    working on a comprehensive solver (https://github.com/sdboyer/vsolver) -
    with glide as the first integration target - that'll be able to figure
    out
    if it's safe to flatten in such a way, and explain why if it's not.
    On Thursday, March 31, 2016 at 11:01:22 AM UTC-4, Peter Bourgon wrote:

    I need to create a type which implements the etcdserverpb.KVServer
    interface[0]. Note that package context is imported by its canonical
    import path[1]. Also note that etcd has a vendor folder in its root
    which includes the context package[2].

    [0]

    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L1724-L1743
    [1]

    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L18
    [2]

    https://github.com/coreos/etcd/tree/de801b500b1f2ab237757052edd87901a5906153/vendor/golang.org/x/net/context

    The etcd repo contains packages that are intended to be imported by
    other packages, such as this one. But it also contains binaries, so I
    think the presence of the vendor folder is correct. Here is my
    code[3]. It fails to build because etcdserverpb.KVServer.Compact
    expects
    "github.com/coreos/etcd/vendor/golang.org/x/net/context".Context
    rather than "golang.org/x/net/context".Context[4], which as far as I
    know I cannot possibly provide.

    [3]
    https://gist.github.com/peterbourgon/79f44f89d5c334b5f05a291f25fe6bd8
    [4]
    https://gist.github.com/peterbourgon/304c21802b3531cd8d0b50247ca41522

    What went wrong, and how can I fix it?

    Cheers,
    Peter.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Sam Boyer at Mar 31, 2016 at 4:08 pm
    Sure. And, right, sorry, I should've been clearer - if your repo only has
    main package(s), there's no problem in the case where you've committed your
    vendor. When I say '"just committing vendor" is kind of shortsighted,' it's
    a bigger-picture guess about how having that easy approach available might
    drive some systemic things:

    - plenty of folks just won't be aware of the harm, and will go ahead and
    publish split main/lib repos with committed vendor dirs
    - folks who are aware of it will be discouraged from splitting useful parts
    out of main packages, because the sudden steep increase in maintenance
    difficulty that presents
    - wary of these problems, folks may become even less inclined to rely on
    other deps than they are now. (ironic, given that vendor dirs were supposed
    to do the opposite)

    and people end up in the spot you're in now - a bit despondent because
    sharing code just got a little more infeasible.

    that said, it's not at all intractable in general. we just need the right
    tooling. i, for one, am working feverishly to that end :)
    On Thursday, March 31, 2016 at 11:39:59 AM UTC-4, Peter Bourgon wrote:

    Thanks. Committing vendor is fine, I think, as long as nobody imports
    your repo. That is, as long as you're exclusively a package main. As
    soon as you have something that others may want to import, all bets
    are off.
    This is very disappointing. It makes my project significantly more
    difficult to use as intended. Maybe so difficult that it will prevent
    widespread use.


    On Thu, Mar 31, 2016 at 5:32 PM, Sam Boyer <samuel....@gmail.com
    <javascript:>> wrote:
    If you can't assume a package manager, then yup, I think it's
    intractable.
    This exemplifies why "just committing vendor" is kind of a shortsighted
    solution. And, because this can easily happen recursively (e.g. etcd could
    import a project with main+lib packages that itself chooses to vendor), it's
    one that increases in harmfulness to the ecosystem as adoption of it
    increases.
    On Thursday, March 31, 2016 at 11:24:24 AM UTC-4, Peter Bourgon wrote:

    Thanks for the replies. My sample code was misleading, the type that
    implements etcdserverpb.KVServer is itself in a library, so its repo
    has no vendor folder by design. My users should be able to `go get` my
    code, or use it in their own projects, without requiring any tricks in
    their build process. Does that make the problem intractable?
    On Thu, Mar 31, 2016 at 5:20 PM, Sam Boyer wrote:
    AFAIK, you can't fix it without hacking etcd's vendor - assuming that
    etcd
    is under your project's vendor, you'd need to move x/net/context from
    under
    etcd/vendor to your own vendor/ directory. We've been referring to
    that
    process as 'flattening'.

    While vendor does allow you to use the canonical import path in the
    source
    code (yaaaaay), the compiler still rewrites those paths to what you
    see
    there ("github.com/coreos/etcd/vendor/golang.org/x/net/context").
    Thus,
    any
    shared deps have to be pushed up to the topmost vendor directory.

    Most of the package management tools are working on solutions to
    this.
    I'm
    working on a comprehensive solver (https://github.com/sdboyer/vsolver)
    -
    with glide as the first integration target - that'll be able to
    figure
    out
    if it's safe to flatten in such a way, and explain why if it's not.

    On Thursday, March 31, 2016 at 11:01:22 AM UTC-4, Peter Bourgon
    wrote:
    I need to create a type which implements the etcdserverpb.KVServer
    interface[0]. Note that package context is imported by its canonical
    import path[1]. Also note that etcd has a vendor folder in its root
    which includes the context package[2].

    [0]
    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L1724-L1743
    [1]
    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L18
    [2]
    https://github.com/coreos/etcd/tree/de801b500b1f2ab237757052edd87901a5906153/vendor/golang.org/x/net/context
    The etcd repo contains packages that are intended to be imported by
    other packages, such as this one. But it also contains binaries, so
    I
    think the presence of the vendor folder is correct. Here is my
    code[3]. It fails to build because etcdserverpb.KVServer.Compact
    expects
    "github.com/coreos/etcd/vendor/golang.org/x/net/context".Context
    rather than "golang.org/x/net/context".Context[4], which as far as
    I
    know I cannot possibly provide.

    [3]
    https://gist.github.com/peterbourgon/79f44f89d5c334b5f05a291f25fe6bd8
    [4]
    https://gist.github.com/peterbourgon/304c21802b3531cd8d0b50247ca41522
    What went wrong, and how can I fix it?

    Cheers,
    Peter.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniel Theophanes at Mar 31, 2016 at 5:13 pm
    Everything is working as intended.

    You need to copy the vendor packages to your own vendor folder to "flatten"
    them. Don't use git sub-modules: see
    https://www.reddit.com/r/golang/comments/4cptba/best_practice_for_vendoring_in_libraries/

    In this case because etcd copies packages locally, you need to use a vendor
    tool when consuming those packages.
    I would personally suggest github.com/kardianos/govendor, but any tool that
    "flattens" the vendor repo will functionally work.
    On Thursday, March 31, 2016 at 8:01:22 AM UTC-7, Peter Bourgon wrote:

    I need to create a type which implements the etcdserverpb.KVServer
    interface[0]. Note that package context is imported by its canonical
    import path[1]. Also note that etcd has a vendor folder in its root
    which includes the context package[2].

    [0]
    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L1724-L1743
    [1]
    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L18
    [2]
    https://github.com/coreos/etcd/tree/de801b500b1f2ab237757052edd87901a5906153/vendor/golang.org/x/net/context

    The etcd repo contains packages that are intended to be imported by
    other packages, such as this one. But it also contains binaries, so I
    think the presence of the vendor folder is correct. Here is my
    code[3]. It fails to build because etcdserverpb.KVServer.Compact
    expects "github.com/coreos/etcd/vendor/golang.org/x/net/context".Context
    rather than "golang.org/x/net/context".Context[4], which as far as I
    know I cannot possibly provide.

    [3] https://gist.github.com/peterbourgon/79f44f89d5c334b5f05a291f25fe6bd8
    [4] https://gist.github.com/peterbourgon/304c21802b3531cd8d0b50247ca41522

    What went wrong, and how can I fix it?

    Cheers,
    Peter.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Peter Bourgon at Mar 31, 2016 at 8:41 pm
    Upon further thought, I think this is actually a very serious problem.
    The vendoring in coreos/etcd makes it impossible for other packages to
    implement its interfaces. This generalizes to: vendoring in •any repo•
    makes it •impossible• to interact with packages in that repo in ways
    that require type identity with vendored deps.

    I've created https://github.com/peterbourgon/wtf as a minimal example
    to demonstrate this problem.

    I'd appreciate it if someone involved with the design of
    GO15VENDOREXPERIMENT could comment. Russ?

    On Thu, Mar 31, 2016 at 7:13 PM, Daniel Theophanes wrote:
    Everything is working as intended.

    You need to copy the vendor packages to your own vendor folder to "flatten"
    them. Don't use git sub-modules: see
    https://www.reddit.com/r/golang/comments/4cptba/best_practice_for_vendoring_in_libraries/

    In this case because etcd copies packages locally, you need to use a vendor
    tool when consuming those packages.
    I would personally suggest github.com/kardianos/govendor, but any tool that
    "flattens" the vendor repo will functionally work.

    On Thursday, March 31, 2016 at 8:01:22 AM UTC-7, Peter Bourgon wrote:

    I need to create a type which implements the etcdserverpb.KVServer
    interface[0]. Note that package context is imported by its canonical
    import path[1]. Also note that etcd has a vendor folder in its root
    which includes the context package[2].

    [0]
    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L1724-L1743
    [1]
    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L18
    [2]
    https://github.com/coreos/etcd/tree/de801b500b1f2ab237757052edd87901a5906153/vendor/golang.org/x/net/context

    The etcd repo contains packages that are intended to be imported by
    other packages, such as this one. But it also contains binaries, so I
    think the presence of the vendor folder is correct. Here is my
    code[3]. It fails to build because etcdserverpb.KVServer.Compact
    expects "github.com/coreos/etcd/vendor/golang.org/x/net/context".Context
    rather than "golang.org/x/net/context".Context[4], which as far as I
    know I cannot possibly provide.

    [3] https://gist.github.com/peterbourgon/79f44f89d5c334b5f05a291f25fe6bd8
    [4] https://gist.github.com/peterbourgon/304c21802b3531cd8d0b50247ca41522

    What went wrong, and how can I fix it?

    Cheers,
    Peter.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Konstantin Shaposhnikov at Apr 1, 2016 at 12:10 am
    I asked the same question some time ago on
    golang-dev: https://groups.google.com/d/msg/golang-dev/WebP4dLV1b0/Lhk4hpwJEgAJ.
    Read the thread to see Russ's replies.
    On Friday, 1 April 2016 04:41:25 UTC+8, Peter Bourgon wrote:

    Upon further thought, I think this is actually a very serious problem.
    The vendoring in coreos/etcd makes it impossible for other packages to
    implement its interfaces. This generalizes to: vendoring in •any repo•
    makes it •impossible• to interact with packages in that repo in ways
    that require type identity with vendored deps.

    I've created https://github.com/peterbourgon/wtf as a minimal example
    to demonstrate this problem.

    I'd appreciate it if someone involved with the design of
    GO15VENDOREXPERIMENT could comment. Russ?


    On Thu, Mar 31, 2016 at 7:13 PM, Daniel Theophanes <kard...@gmail.com
    <javascript:>> wrote:
    Everything is working as intended.

    You need to copy the vendor packages to your own vendor folder to "flatten"
    them. Don't use git sub-modules: see
    https://www.reddit.com/r/golang/comments/4cptba/best_practice_for_vendoring_in_libraries/
    In this case because etcd copies packages locally, you need to use a vendor
    tool when consuming those packages.
    I would personally suggest github.com/kardianos/govendor, but any tool that
    "flattens" the vendor repo will functionally work.

    On Thursday, March 31, 2016 at 8:01:22 AM UTC-7, Peter Bourgon wrote:

    I need to create a type which implements the etcdserverpb.KVServer
    interface[0]. Note that package context is imported by its canonical
    import path[1]. Also note that etcd has a vendor folder in its root
    which includes the context package[2].

    [0]
    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L1724-L1743
    [1]
    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L18
    [2]
    https://github.com/coreos/etcd/tree/de801b500b1f2ab237757052edd87901a5906153/vendor/golang.org/x/net/context
    The etcd repo contains packages that are intended to be imported by
    other packages, such as this one. But it also contains binaries, so I
    think the presence of the vendor folder is correct. Here is my
    code[3]. It fails to build because etcdserverpb.KVServer.Compact
    expects "github.com/coreos/etcd/vendor/golang.org/x/net/context".Context
    rather than "golang.org/x/net/context".Context[4], which as far as I
    know I cannot possibly provide.

    [3]
    https://gist.github.com/peterbourgon/79f44f89d5c334b5f05a291f25fe6bd8
    [4]
    https://gist.github.com/peterbourgon/304c21802b3531cd8d0b50247ca41522
    What went wrong, and how can I fix it?

    Cheers,
    Peter.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Peter Bourgon at Apr 1, 2016 at 8:13 am
    Thanks, Konstantin. Money quote there from Russ seems to be
    Vendored code is a private copy of something.
    That is, "vendor" is explicitly "internal" as well.
    I am trying to boil this down to a rule. If

    - Your repo contains any package that isn't package main, and
    - Any of those packages expose types from any of their dependencies, and
    - It's •possible• that a third party would want to import any of those packages,

    Then you •must not• use vendoring in that repo.

    Is that sufficiently expressed?


    On Fri, Apr 1, 2016 at 2:10 AM, Konstantin Shaposhnikov
    wrote:
    I asked the same question some time ago on golang-dev:
    https://groups.google.com/d/msg/golang-dev/WebP4dLV1b0/Lhk4hpwJEgAJ. Read
    the thread to see Russ's replies.
    On Friday, 1 April 2016 04:41:25 UTC+8, Peter Bourgon wrote:

    Upon further thought, I think this is actually a very serious problem.
    The vendoring in coreos/etcd makes it impossible for other packages to
    implement its interfaces. This generalizes to: vendoring in •any repo•
    makes it •impossible• to interact with packages in that repo in ways
    that require type identity with vendored deps.

    I've created https://github.com/peterbourgon/wtf as a minimal example
    to demonstrate this problem.

    I'd appreciate it if someone involved with the design of
    GO15VENDOREXPERIMENT could comment. Russ?


    On Thu, Mar 31, 2016 at 7:13 PM, Daniel Theophanes <kard...@gmail.com>
    wrote:
    Everything is working as intended.

    You need to copy the vendor packages to your own vendor folder to
    "flatten"
    them. Don't use git sub-modules: see

    https://www.reddit.com/r/golang/comments/4cptba/best_practice_for_vendoring_in_libraries/

    In this case because etcd copies packages locally, you need to use a
    vendor
    tool when consuming those packages.
    I would personally suggest github.com/kardianos/govendor, but any tool
    that
    "flattens" the vendor repo will functionally work.

    On Thursday, March 31, 2016 at 8:01:22 AM UTC-7, Peter Bourgon wrote:

    I need to create a type which implements the etcdserverpb.KVServer
    interface[0]. Note that package context is imported by its canonical
    import path[1]. Also note that etcd has a vendor folder in its root
    which includes the context package[2].

    [0]

    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L1724-L1743
    [1]

    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L18
    [2]

    https://github.com/coreos/etcd/tree/de801b500b1f2ab237757052edd87901a5906153/vendor/golang.org/x/net/context

    The etcd repo contains packages that are intended to be imported by
    other packages, such as this one. But it also contains binaries, so I
    think the presence of the vendor folder is correct. Here is my
    code[3]. It fails to build because etcdserverpb.KVServer.Compact
    expects
    "github.com/coreos/etcd/vendor/golang.org/x/net/context".Context
    rather than "golang.org/x/net/context".Context[4], which as far as I
    know I cannot possibly provide.

    [3]
    https://gist.github.com/peterbourgon/79f44f89d5c334b5f05a291f25fe6bd8
    [4]
    https://gist.github.com/peterbourgon/304c21802b3531cd8d0b50247ca41522

    What went wrong, and how can I fix it?

    Cheers,
    Peter.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Pierre Durand at Apr 1, 2016 at 8:48 am
    Or don't rewrite import path.

    Le vendredi 1 avril 2016 10:13:47 UTC+2, Peter Bourgon a écrit :
    Thanks, Konstantin. Money quote there from Russ seems to be
    Vendored code is a private copy of something.
    That is, "vendor" is explicitly "internal" as well.
    I am trying to boil this down to a rule. If

    - Your repo contains any package that isn't package main, and
    - Any of those packages expose types from any of their dependencies, and
    - It's •possible• that a third party would want to import any of those
    packages,

    Then you •must not• use vendoring in that repo.

    Is that sufficiently expressed?


    On Fri, Apr 1, 2016 at 2:10 AM, Konstantin Shaposhnikov
    <k.shapo...@gmail.com <javascript:>> wrote:
    I asked the same question some time ago on golang-dev:
    https://groups.google.com/d/msg/golang-dev/WebP4dLV1b0/Lhk4hpwJEgAJ. Read
    the thread to see Russ's replies.
    On Friday, 1 April 2016 04:41:25 UTC+8, Peter Bourgon wrote:

    Upon further thought, I think this is actually a very serious problem.
    The vendoring in coreos/etcd makes it impossible for other packages to
    implement its interfaces. This generalizes to: vendoring in •any repo•
    makes it •impossible• to interact with packages in that repo in ways
    that require type identity with vendored deps.

    I've created https://github.com/peterbourgon/wtf as a minimal example
    to demonstrate this problem.

    I'd appreciate it if someone involved with the design of
    GO15VENDOREXPERIMENT could comment. Russ?


    On Thu, Mar 31, 2016 at 7:13 PM, Daniel Theophanes <kard...@gmail.com>
    wrote:
    Everything is working as intended.

    You need to copy the vendor packages to your own vendor folder to
    "flatten"
    them. Don't use git sub-modules: see
    https://www.reddit.com/r/golang/comments/4cptba/best_practice_for_vendoring_in_libraries/
    In this case because etcd copies packages locally, you need to use a
    vendor
    tool when consuming those packages.
    I would personally suggest github.com/kardianos/govendor, but any
    tool
    that
    "flattens" the vendor repo will functionally work.

    On Thursday, March 31, 2016 at 8:01:22 AM UTC-7, Peter Bourgon wrote:

    I need to create a type which implements the etcdserverpb.KVServer
    interface[0]. Note that package context is imported by its canonical
    import path[1]. Also note that etcd has a vendor folder in its root
    which includes the context package[2].

    [0]
    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L1724-L1743
    [1]
    https://github.com/coreos/etcd/blob/de801b5/etcdserver/etcdserverpb/rpc.pb.go#L18
    [2]
    https://github.com/coreos/etcd/tree/de801b500b1f2ab237757052edd87901a5906153/vendor/golang.org/x/net/context
    The etcd repo contains packages that are intended to be imported by
    other packages, such as this one. But it also contains binaries, so
    I
    think the presence of the vendor folder is correct. Here is my
    code[3]. It fails to build because etcdserverpb.KVServer.Compact
    expects
    "github.com/coreos/etcd/vendor/golang.org/x/net/context".Context
    rather than "golang.org/x/net/context".Context[4], which as far as
    I
    know I cannot possibly provide.

    [3]
    https://gist.github.com/peterbourgon/79f44f89d5c334b5f05a291f25fe6bd8
    [4]
    https://gist.github.com/peterbourgon/304c21802b3531cd8d0b50247ca41522
    What went wrong, and how can I fix it?

    Cheers,
    Peter.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Peter Waller at Apr 1, 2016 at 8:51 am
    I have hit this exact situation and spent some time thinking about it.

    I wanted to vendor some code from elsewhere without copying the source,
    modifying the source, or using external vendoring tools. `git submodule`
    works just fine for everything we use so far, a fairly substantial number
    of dependencies. It has a lot of nice properties. But then I hit this case
    where there was some perfectly good code that I wanted to use, but
    couldn't, without violating the rule "just submodule it".

    The structure looked something like:

    /cmd/foo
    /pkg/foo
    /vendor/bar

    Upstream was unsympathetic and didn't want to change anything. Which is
    fair enough. Thinking about it further though, they couldn't have fixed it
    for me, within their repository. They would have had to introduce a
    separate repository for main to live in.

    My first thought was that they should move /vendor/bar to /cmd/vendor/bar.
    But only when I tried to do it in another circumstance did I realise I had
    missed the obvious: When you do that, the vendoring doesn't apply to
    /pkg/bar anymore. So your only choice if you want /pkg/foo to be externally
    importable is to have another repository for /cmd/, which contains both
    /vendor/pkg/foo and /vendor/bar.

    One style of solution I thought of is some sort of vendor package directory
    masking. If I use vendoring, I may want to "hide" a vendor directory
    (nested within a vendor directory) from the build system, so that I may put
    my own vendor'd packages in its place. I even started prototyping this
    behaviour with a fuse go program.

    But in the end, this was all massively overkill for my problem and I found
    another solution which didn't require importing the module at all.

    I do fear though that the more vendoring is used by packages I may want to
    import, the less I can do so.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Sam Boyer at Apr 1, 2016 at 11:12 am

    On Friday, April 1, 2016 at 4:51:30 AM UTC-4, Peter Waller wrote:
    I have hit this exact situation and spent some time thinking about it.

    I wanted to vendor some code from elsewhere without copying the source,
    modifying the source, or using external vendoring tools. `git submodule`
    works just fine for everything we use so far, a fairly substantial number
    of dependencies. It has a lot of nice properties. But then I hit this case
    where there was some perfectly good code that I wanted to use, but
    couldn't, without violating the rule "just submodule it".

    The structure looked something like:

    /cmd/foo
    /pkg/foo
    /vendor/bar

    Upstream was unsympathetic and didn't want to change anything. Which is
    fair enough. Thinking about it further though, they couldn't have fixed it
    for me, within their repository. They would have had to introduce a
    separate repository for main to live in.

    My first thought was that they should move /vendor/bar to /cmd/vendor/bar.
    But only when I tried to do it in another circumstance did I realise I had
    missed the obvious: When you do that, the vendoring doesn't apply to
    /pkg/bar anymore. So your only choice if you want /pkg/foo to be externally
    importable is to have another repository for /cmd/, which contains both
    /vendor/pkg/foo and /vendor/bar.

    One style of solution I thought of is some sort of vendor package
    directory masking. If I use vendoring, I may want to "hide" a vendor
    directory (nested within a vendor directory) from the build system, so that
    I may put my own vendor'd packages in its place. I even started prototyping
    this behaviour with a fuse go program.

    But in the end, this was all massively overkill for my problem and I found
    another solution which didn't require importing the module at all.

    I do fear though that the more vendoring is used by packages I may want to
    import, the less I can do so.
    yep, this is the recursive nastiness i was referencing earlier.

    i really doubt there's any kind of static filesystem magic we could work
    out that would solve this problem for folks. it's tooling, or pain.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at Apr 1, 2016 at 9:01 am
    I think it could be more succinctly expressed as:

    Libraries must not vendor code.

    What's a library? You're code is, if some other project imports it.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Peter Waller at Apr 1, 2016 at 9:17 am

    On 1 April 2016 at 10:01, Dave Cheney wrote:

    Libraries must not vendor code.

    What's a library? You're code is, if some other project imports it.

    Some open questions:

    (Let's say I have something intended to be used as both a library and a
    program :)

    Is it OK to have a library /pkg/foo and a program /cmd/foo in the same
    repository? (My intuitive guess: a reasonable thing to want?)

    Is /cmd/foo allowed to vendor /pkg/foo's dependencies? (yes)

    Must that program also vendor that /pkg/foo? (seems like it might be
    required if you want to use vendoring in /cmd/foo)

    Can you suggest a directory structure? (seems like you need a copy of
    /pkg/foo in the repository, which lives under /cmd/vendor or
    /cmd/foo/vendor)

    Something like:

    /cmd/foo
    /cmd/vendor/pkg/foo
    /cmd/vendor/foo-dependency
    /pkg/foo

    The copy at /cmd/vendor/pkg/foo (or /pkg/foo depending on your perspective)
    is unfortunate, but I don't currently see another way. I'd love for there
    to be one :)

    What this achieves is it puts a package at /pkg/foo which is importable
    from the outside world which doesn't pull in any vendor'd packages.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Peter Waller at Apr 1, 2016 at 9:35 am

    On 1 April 2016 at 10:17, Peter Waller wrote:
    What this achieves is it puts a package at /pkg/foo which is importable
    from the outside world which doesn't pull in any vendor'd packages.
    What is especially unfortunate about this state of affairs is that
    essentially everyone who ever makes repositories which are both cmd/ and
    pkg/ (and use vendoring) has to get this "right"*. Otherwise `go get` (and,
    e.g, pinning upstream with git submodules) won't work without further
    intervention. As an importer of code there is nothing I can do except
    something more complicated.


    * Not to claim that what I suggested in the previous post is right.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at Apr 1, 2016 at 10:15 am

    On Friday, 1 April 2016 20:18:11 UTC+11, Peter Waller wrote:
    On 1 April 2016 at 10:01, Dave Cheney <da...@cheney.net <javascript:>>
    wrote:
    Libraries must not vendor code.

    What's a library? You're code is, if some other project imports it.

    Some open questions:

    (Let's say I have something intended to be used as both a library and a
    program :)
    Don't do that.

    Is it OK to have a library /pkg/foo and a program /cmd/foo in the same
    repository? (My intuitive guess: a reasonable thing to want?)
    Yes, the advice for using /vendor is it should go at the top of your
    repository so it's scope covers all the code in the repository, ie

    github.com/you/foo/vendor

    Is /cmd/foo allowed to vendor /pkg/foo's dependencies? (yes)
    Sounds like a bad idea.

    Must that program also vendor that /pkg/foo? (seems like it might be
    required if you want to use vendoring in /cmd/foo)
    Not if they are in the same repository, and the should be, because they are
    tightly coupled.

    Can you suggest a directory structure? (seems like you need a copy of
    /pkg/foo in the repository, which lives under /cmd/vendor or
    /cmd/foo/vendor)

    Something like:

    /cmd/foo
    /cmd/vendor/pkg/foo
    /cmd/vendor/foo-dependency
    /pkg/foo

    The copy at /cmd/vendor/pkg/foo (or /pkg/foo depending on your
    perspective) is unfortunate, but I don't currently see another way. I'd
    love for there to be one :)
    That's too complicated IMO, vendoring is per repo (per project), so

    github.com/you/foo/vendor

    What this achieves is it puts a package at /pkg/foo which is importable
    from the outside world which doesn't pull in any vendor'd packages.
    Don't do that, you're making it to hard on yourself. If it's a library;
    designed to be imported by _other_ repositories, put it in it's own repo.
    If it's a library that is part of the final application (what gb calls a
    project), then don't split it into it's own repo, that's just making it
    harder than it needs to be.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Peter Waller at Apr 1, 2016 at 10:20 am

    On 1 April 2016 at 11:15, Dave Cheney wrote:
    Don't do that, you're making it to hard on yourself. If it's a library;
    designed to be imported by _other_ repositories, put it in it's own repo.
    If it's a library that is part of the final application (what gb calls a
    project), then don't split it into it's own repo, that's just making it
    harder than it needs to be.
    The problem isn't that I do that - in itself - but that there are packages
    in the wild that do this that I may want to import.

    I'm willing to change my behaviour where I become aware that is it wrong.

    On the other hand, I don't see what's wrong with having both a library and
    a main function in one package. I mean, etcd does it, right? Are they doing
    it wrong? If so, please tell them for me :)

    I write much of my code so that main() is really a tiny program which just
    calls a library. Are you saying this has to live in its own repository, or
    am I misunderstanding?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at Apr 1, 2016 at 10:26 am

    On Friday, 1 April 2016 21:20:45 UTC+11, Peter Waller wrote:
    On 1 April 2016 at 11:15, Dave Cheney <da...@cheney.net <javascript:>>
    wrote:
    Don't do that, you're making it to hard on yourself. If it's a library;
    designed to be imported by _other_ repositories, put it in it's own repo.
    If it's a library that is part of the final application (what gb calls a
    project), then don't split it into it's own repo, that's just making it
    harder than it needs to be.
    The problem isn't that I do that - in itself - but that there are packages
    in the wild that do this that I may want to import.

    I'm willing to change my behaviour where I become aware that is it wrong.

    On the other hand, I don't see what's wrong with having both a library and
    a main function in one package. I mean, etcd does it, right? Are they doing
    it wrong? If so, please tell them for me :)
    This is just my opinion, feel free to return it for the complete purchase
    amount, or simply discard if you don't agree with it.

    I write much of my code so that main() is really a tiny program which just
    calls a library. Are you saying this has to live in its own repository, or
    am I misunderstanding?
    If the library and the code that calls it are related, then they should be
    in the same repository.

    If the library has many consumers then you should not also place vendored
    code in that repository -- otherwise consumers of your library will find
    themselves in the same position as Peter Bourgon has found himself.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Peter Waller at Apr 1, 2016 at 10:31 am

    On 1 April 2016 at 11:26, Dave Cheney wrote:
    If the library has many consumers then you should not also place vendored
    code in that repository -- otherwise consumers of your library will find
    themselves in the same position as Peter Bourgon has found himself.
    An interesting thing about this is that it places the choice in the hands
    of the producer, and as a consumer my life is harder.

    They have to knowingly make the "correct" choice, and also not work against
    my interests as a consumer.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Sam Boyer at Apr 1, 2016 at 11:15 am
    FWIW, IMO this is the danger I referenced earlier - vendoring troubles
    discouraging people from having anything other than a main package. It's
    harmful to the ecosystem.
    On Friday, April 1, 2016 at 6:26:10 AM UTC-4, Dave Cheney wrote:

    On Friday, 1 April 2016 21:20:45 UTC+11, Peter Waller wrote:
    On 1 April 2016 at 11:15, Dave Cheney wrote:

    Don't do that, you're making it to hard on yourself. If it's a library;
    designed to be imported by _other_ repositories, put it in it's own repo.
    If it's a library that is part of the final application (what gb calls a
    project), then don't split it into it's own repo, that's just making it
    harder than it needs to be.
    The problem isn't that I do that - in itself - but that there are
    packages in the wild that do this that I may want to import.

    I'm willing to change my behaviour where I become aware that is it wrong.

    On the other hand, I don't see what's wrong with having both a library
    and a main function in one package. I mean, etcd does it, right? Are they
    doing it wrong? If so, please tell them for me :)
    This is just my opinion, feel free to return it for the complete purchase
    amount, or simply discard if you don't agree with it.
    I write much of my code so that main() is really a tiny program which
    just calls a library. Are you saying this has to live in its own
    repository, or am I misunderstanding?
    If the library and the code that calls it are related, then they should be
    in the same repository.

    If the library has many consumers then you should not also place vendored
    code in that repository -- otherwise consumers of your library will find
    themselves in the same position as Peter Bourgon has found himself.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Konstantin Shaposhnikov at Apr 1, 2016 at 11:20 am
    I wonder if extending "go test" to use dependencies from vendor_test
    directory in some cases would help? This is just an idea though, I
    haven't thought through all the details.
    On 1 April 2016 at 19:15, Sam Boyer wrote:
    FWIW, IMO this is the danger I referenced earlier - vendoring troubles
    discouraging people from having anything other than a main package. It's
    harmful to the ecosystem.

    On Friday, April 1, 2016 at 6:26:10 AM UTC-4, Dave Cheney wrote:

    On Friday, 1 April 2016 21:20:45 UTC+11, Peter Waller wrote:
    On 1 April 2016 at 11:15, Dave Cheney wrote:

    Don't do that, you're making it to hard on yourself. If it's a library;
    designed to be imported by _other_ repositories, put it in it's own repo. If
    it's a library that is part of the final application (what gb calls a
    project), then don't split it into it's own repo, that's just making it
    harder than it needs to be.

    The problem isn't that I do that - in itself - but that there are
    packages in the wild that do this that I may want to import.

    I'm willing to change my behaviour where I become aware that is it wrong.

    On the other hand, I don't see what's wrong with having both a library
    and a main function in one package. I mean, etcd does it, right? Are they
    doing it wrong? If so, please tell them for me :)
    This is just my opinion, feel free to return it for the complete purchase
    amount, or simply discard if you don't agree with it.

    I write much of my code so that main() is really a tiny program which
    just calls a library. Are you saying this has to live in its own repository,
    or am I misunderstanding?

    If the library and the code that calls it are related, then they should be
    in the same repository.

    If the library has many consumers then you should not also place vendored
    code in that repository -- otherwise consumers of your library will find
    themselves in the same position as Peter Bourgon has found himself.
    --
    You received this message because you are subscribed to a topic in the
    Google Groups "golang-nuts" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/golang-nuts/AnMr9NL6dtc/unsubscribe.
    To unsubscribe from this group and all its topics, send an email to
    golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Peter Waller at Apr 1, 2016 at 10:23 am

    On 1 April 2016 at 11:15, Dave Cheney wrote:
    On Friday, 1 April 2016 20:18:11 UTC+11, Peter Waller wrote:

    Something like:

    /cmd/foo
    /cmd/vendor/pkg/foo
    /cmd/vendor/foo-dependency
    /pkg/foo

    The copy at /cmd/vendor/pkg/foo (or /pkg/foo depending on your
    perspective) is unfortunate, but I don't currently see another way. I'd
    love for there to be one :)
    That's too complicated IMO, vendoring is per repo (per project)
    FWIW, this is not something I was seriously suggesting, but intended to be
    a straw man.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Sam Boyer at Apr 1, 2016 at 11:16 am

    On Friday, April 1, 2016 at 5:18:11 AM UTC-4, Peter Waller wrote:
    On 1 April 2016 at 10:01, Dave Cheney <da...@cheney.net <javascript:>>
    wrote:
    Libraries must not vendor code.

    What's a library? You're code is, if some other project imports it.

    Some open questions:

    (Let's say I have something intended to be used as both a library and a
    program :)
    I disagree with Dave - It's perfectly fine to do this. glide supports it -
    you can have carefully-controlled dependencies, but needn't commit your
    vendor directory. (If you can rely on downstream folks to use glide, then
    it's safe for you to commit vendor, if you want).

    Is it OK to have a library /pkg/foo and a program /cmd/foo in the same
    repository? (My intuitive guess: a reasonable thing to want?)

    Is /cmd/foo allowed to vendor /pkg/foo's dependencies? (yes)

    Must that program also vendor that /pkg/foo? (seems like it might be
    required if you want to use vendoring in /cmd/foo)

    Can you suggest a directory structure? (seems like you need a copy of
    /pkg/foo in the repository, which lives under /cmd/vendor or
    /cmd/foo/vendor)

    Something like:

    /cmd/foo
    /cmd/vendor/pkg/foo
    /cmd/vendor/foo-dependency
    /pkg/foo

    The copy at /cmd/vendor/pkg/foo (or /pkg/foo depending on your
    perspective) is unfortunate, but I don't currently see another way. I'd
    love for there to be one :)

    What this achieves is it puts a package at /pkg/foo which is importable
    from the outside world which doesn't pull in any vendor'd packages.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Ian Davis at Apr 1, 2016 at 9:33 am

    On Fri, Apr 1, 2016, at 10:01 AM, Dave Cheney wrote:
    I think it could be more succinctly expressed as:

    Libraries must not vendor code.

    What's a library? You're code is, if some other project imports it.

    The corollary to your rule is:

    Repositories that contain vendored code should not be imported.

    (since if it were a library intended for importing then the author would
    not be using vendoring)

    I'm being facetious here, but my growing feeling is that the current
    vendoring approach for Go is harmful for code reuse.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Alex Bligh at Apr 1, 2016 at 10:51 am

    On 1 Apr 2016, at 10:01, Dave Cheney wrote:

    I think it could be more succinctly expressed as:

    Libraries must not vendor code.

    What's a library? You're code is, if some other project imports it.
    Does that mean if your code is a library (you want others to import it)
    you cannot have within it a test suite which requires vendored code?
    If so, that's a bit unfortunate for interoperability tests.

    --
    Alex Bligh




    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at Apr 1, 2016 at 10:53 am

    On Fri, Apr 1, 2016 at 9:51 PM, Alex Bligh wrote:
    On 1 Apr 2016, at 10:01, Dave Cheney wrote:

    I think it could be more succinctly expressed as:

    Libraries must not vendor code.

    What's a library? You're code is, if some other project imports it.
    Does that mean if your code is a library (you want others to import it)
    you cannot have within it a test suite which requires vendored code?
    If so, that's a bit unfortunate for interoperability tests.
    Based on Peter's findings, yes, that appears to be the case.
    --
    Alex Bligh


    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Jakob Borg at Apr 1, 2016 at 11:02 am
    2016-04-01 12:51 GMT+02:00 Alex Bligh <alex@alex.org.uk>:
    On 1 Apr 2016, at 10:01, Dave Cheney wrote:

    I think it could be more succinctly expressed as:

    Libraries must not vendor code.

    What's a library? You're code is, if some other project imports it.
    Does that mean if your code is a library (you want others to import it)
    you cannot have within it a test suite which requires vendored code?
    If so, that's a bit unfortunate for interoperability tests.
    As I understand it, vendored dependencies themselves are not the
    problem here, but *exposing* them such as returning or taking as
    arguments types from them. So vendoring x/net/context and having
    methods accept a context.Ctx from the outside is a no-no. But a
    vendored dependency for tests, or internal use only, or as a
    dependency to some cmd/* package in the same repo ought to be
    acceptable.

    I don't think this changes anything for the base rule of "libraries
    should not vendor" - that still holds.

    //jb

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Jakob Borg at Apr 1, 2016 at 11:07 am

    2016-04-01 13:02 GMT+02:00 Jakob Borg <jakob@nym.se>:
    So vendoring x/net/context and having
    methods accept a context.Ctx from the outside is a no-no.
    Adding another thought here, this specific case only sounds even
    vaguely acceptable because it's - at this point - such a common type
    that it "feels" like the standard library. I.e. we commonly accept
    io.Writers so why not context.Contexts? But in most other cases you
    would not expose a method that takes some interface from an unrelated
    package - you'd declare a matching interface of your own to avoid the
    dependency to begin with. That's rather the point of having implicitly
    satisfied interfaces after all?

    //jb

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Ian Davis at Apr 1, 2016 at 11:13 am

    On Fri, Apr 1, 2016, at 12:07 PM, Jakob Borg wrote:
    2016-04-01 13:02 GMT+02:00 Jakob Borg <jakob@nym.se>:
    So vendoring x/net/context and having
    methods accept a context.Ctx from the outside is a no-no.
    Adding another thought here, this specific case only sounds even
    vaguely acceptable because it's - at this point - such a common type
    that it "feels" like the standard library. I.e. we commonly accept
    io.Writers so why not context.Contexts? But in most other cases you
    would not expose a method that takes some interface from an unrelated
    package - you'd declare a matching interface of your own to avoid the
    dependency to begin with. That's rather the point of having implicitly
    satisfied interfaces after all?
    But this implies that libraries should wrap all their dependencies which
    is not always practical. For example, imagine a game library that relies
    on OpenGL. Because it can't possibly cater for every use case it's
    likely the library will want to expose hooks to directly access some of
    the raw OpenGL functionality, e.g. handles to buffers. If the game
    library has to alias those types then Go's own lexical matching suggests
    it would also need to provide compatible aliases for the entire OpenGL
    API.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at Apr 1, 2016 at 11:20 am
    That sounds utterly miserable. Both you, the library author, and the other
    you, the library consumer are walking on eggshells; one is petrified of
    accidentally leaking -- I don't think that concept is well understood --
    it's never been a think that Go developers have had to worry about because
    there was only every one type for a given import path before -- and the
    other is petrified of being passed a type which will could fail an equality
    check when code is refactored, or a dependency updated.

    My suggestion above may not be absolute, i'm sure with scrupulous attention
    to detail by a very dedicate development team, backed by tooling which
    hasn't been developed yet, it could be made to work. But it seems simpler,
    and far more explainable, to adopt a "only main packages vendor" maxim.
    On Friday, 1 April 2016 22:14:09 UTC+11, Ian Davis wrote:
    On Fri, Apr 1, 2016, at 12:07 PM, Jakob Borg wrote:
    2016-04-01 13:02 GMT+02:00 Jakob Borg <ja...@nym.se <javascript:>>:
    So vendoring x/net/context and having
    methods accept a context.Ctx from the outside is a no-no.
    Adding another thought here, this specific case only sounds even
    vaguely acceptable because it's - at this point - such a common type
    that it "feels" like the standard library. I.e. we commonly accept
    io.Writers so why not context.Contexts? But in most other cases you
    would not expose a method that takes some interface from an unrelated
    package - you'd declare a matching interface of your own to avoid the
    dependency to begin with. That's rather the point of having implicitly
    satisfied interfaces after all?
    But this implies that libraries should wrap all their dependencies which
    is not always practical. For example, imagine a game library that relies
    on OpenGL. Because it can't possibly cater for every use case it's
    likely the library will want to expose hooks to directly access some of
    the raw OpenGL functionality, e.g. handles to buffers. If the game
    library has to alias those types then Go's own lexical matching suggests
    it would also need to provide compatible aliases for the entire OpenGL
    API.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Ian Davis at Apr 1, 2016 at 11:30 am

    On Fri, Apr 1, 2016, at 12:20 PM, Dave Cheney wrote:
    My suggestion above may not be absolute, i'm sure with scrupulous
    attention to detail by a very dedicate development team, backed by
    tooling which hasn't been developed yet, it could be made to work. But
    it seems simpler, and far more explainable, to adopt a "only main
    packages vendor" maxim.
    I agree that this is a natural consequence of the vendor experiment.

    It means vendoring doesn't help me, as a library author, when I want to
    provide tests and example code that interact with my library and its
    dependencies. I will have to provide those in a separate repository with
    a vendor directory. And ironically I should actually vendor my library
    into that repo too.

    Possibly an alternative is to copy the packages I need and convert them
    to internal packages in my library. I'm not sure that's a good idea.



    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Sam Boyer at Apr 1, 2016 at 11:32 am
    It does help you, as a library author, if you're willing to use tooling -
    which does not *require* that your users rely on that tooling (though they
    could get additional benefit if they do). the only restriction is that you
    can't commit the resulting vendor dir.
    On Friday, April 1, 2016 at 7:30:36 AM UTC-4, Ian Davis wrote:


    On Fri, Apr 1, 2016, at 12:20 PM, Dave Cheney wrote:

    My suggestion above may not be absolute, i'm sure with scrupulous
    attention to detail by a very dedicate development team, backed by tooling
    which hasn't been developed yet, it could be made to work. But it seems
    simpler, and far more explainable, to adopt a "only main packages vendor"
    maxim.


    I agree that this is a natural consequence of the vendor experiment.

    It means vendoring doesn't help me, as a library author, when I want to
    provide tests and example code that interact with my library and its
    dependencies. I will have to provide those in a separate repository with a
    vendor directory. And ironically I should actually vendor my library into
    that repo too.

    Possibly an alternative is to copy the packages I need and convert them to
    internal packages in my library. I'm not sure that's a good idea.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Ian Davis at Apr 1, 2016 at 11:49 am

    On Fri, Apr 1, 2016, at 12:32 PM, Sam Boyer wrote:
    It does help you, as a library author, if you're willing to use
    tooling - which does not *require* that your users rely on that
    tooling (though they could get additional benefit if they do). the
    only restriction is that you can't commit the resulting vendor dir.
    I'm wondering what the best approach is for this scenario: as a library
    author I want to commit my dependencies so I can reliably reconstruct a
    workable version of the library at any commit for bug triage. I want
    that to work even if my dependencies rewrite history in their own
    repositories.  A secondary convenience of committing dependencies is
    that there is zero effort for me (or others) to start working on a
    library even if my local development environment has changed.



    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Peter Bourgon at Apr 1, 2016 at 11:52 am

    On Fri, Apr 1, 2016 at 1:49 PM, Ian Davis wrote:
    On Fri, Apr 1, 2016, at 12:32 PM, Sam Boyer wrote:
    It does help you, as a library author, if you're willing to use tooling -
    which does not *require* that your users rely on that tooling (though they
    could get additional benefit if they do). the only restriction is that you
    can't commit the resulting vendor dir.
    I'm wondering what the best approach is for this scenario: as a library
    author I want to commit my dependencies so I can reliably reconstruct a
    workable version of the library at any commit for bug triage. I want that to
    work even if my dependencies rewrite history in their own repositories. A
    secondary convenience of committing dependencies is that there is zero
    effort for me (or others) to start working on a library even if my local
    development environment has changed.
    A nonstandard vendor folder e.g. _vendor + a blessed build+test
    procedure as encoded in e.g. a Makefile.

    This line of thought is far from the original problem.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Sam Boyer at Apr 1, 2016 at 12:16 pm
    IMO any distance between the two is superficial, because attempting to
    solve just the one problem will make another one pop out somewhere else.
    The original problem is really just a symptom of a deeper issue, and as
    long as we keep slapping band-aids, we're gonna have a bad time.
    On Friday, April 1, 2016 at 7:52:26 AM UTC-4, Peter Bourgon wrote:

    On Fri, Apr 1, 2016 at 1:49 PM, Ian Davis <m...@iandavis.com <javascript:>>
    wrote:
    On Fri, Apr 1, 2016, at 12:32 PM, Sam Boyer wrote:
    It does help you, as a library author, if you're willing to use tooling
    -
    which does not *require* that your users rely on that tooling (though
    they
    could get additional benefit if they do). the only restriction is that
    you
    can't commit the resulting vendor dir.
    I'm wondering what the best approach is for this scenario: as a library
    author I want to commit my dependencies so I can reliably reconstruct a
    workable version of the library at any commit for bug triage. I want that to
    work even if my dependencies rewrite history in their own repositories. A
    secondary convenience of committing dependencies is that there is zero
    effort for me (or others) to start working on a library even if my local
    development environment has changed.
    A nonstandard vendor folder e.g. _vendor + a blessed build+test
    procedure as encoded in e.g. a Makefile.

    This line of thought is far from the original problem.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Sam Boyer at Apr 1, 2016 at 12:14 pm

    On Friday, April 1, 2016 at 7:50:05 AM UTC-4, Ian Davis wrote:

    On Fri, Apr 1, 2016, at 12:32 PM, Sam Boyer wrote:

    It does help you, as a library author, if you're willing to use tooling -
    which does not *require* that your users rely on that tooling (though they
    could get additional benefit if they do). the only restriction is that you
    can't commit the resulting vendor dir.


    I'm wondering what the best approach is for this scenario: as a library
    author I want to commit my dependencies so I can reliably reconstruct a
    workable version of the library at any commit for bug triage.
    Yes, that's what a lock file does - it's a precise list of immutable
    revisions that will be placed into the vendor/ directory when the
    appropriate command is run.

    I want that to work even if my dependencies rewrite history in their own
    repositories.
    The only real defense against this detaching yourself from upstream somehow
    - one version is by committing dependencies. Another is forking. (glide
    allows both now, though support for the former is more recent and
    experimental.)

    A secondary convenience of committing dependencies is that there is zero
    effort for me (or others) to start working on a library even if my local
    development environment has changed.
    Zero effort, yes - for the use case of contributing to your library
    directly.

    The key observation here, IMO, is that *only one* actor in the dependency
    graph can have final, authoritative control over which versions of
    dependencies are used. When you commit vendor, you're making that
    authoritative choice - and thus, not a good choice for libraries. At least,
    not without something that other folks who're relying on your library can
    use to *un*make that choice, by excising your library's vendor directory,
    and moving its contents somewhere else. That's what tooling, like glide,
    does.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Konstantin Shaposhnikov at Apr 1, 2016 at 11:38 am

    It means vendoring doesn't help me, as a library author, when I want to
    provide tests and example code that interact with my library and its
    dependencies. I will have to provide those in a separate repository with a
    vendor directory. And ironically I should actually vendor my library into
    that repo too.
    With a separate repository it is not possible to use/test unexported
    functionality in the tests.

    Possibly an alternative is to copy the packages I need and convert them to
    internal packages in my library. I'm not sure that's a good idea.
    If you can convert dependencies to internal packages this means that
    your library doesn't use these dependencies in the exported API. So
    the problem that is discussed in this thread doesn't apply.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Sam Boyer at Apr 1, 2016 at 11:30 am

    On Friday, April 1, 2016 at 7:20:09 AM UTC-4, Dave Cheney wrote:
    That sounds utterly miserable. Both you, the library author, and the other
    you, the library consumer are walking on eggshells; one is petrified of
    accidentally leaking -- I don't think that concept is well understood --
    it's never been a think that Go developers have had to worry about because
    there was only every one type for a given import path before -- and the
    other is petrified of being passed a type which will could fail an equality
    check when code is refactored, or a dependency updated.

    My suggestion above may not be absolute, i'm sure with scrupulous
    attention to detail by a very dedicate development team, backed by tooling
    which hasn't been developed yet, it could be made to work. But it seems
    simpler, and far more explainable, to adopt a "only main packages vendor"
    maxim.
    It's a performable analysis, and not one that has to be that difficult.

    You've expressed to me previously that you chose to focus specifically on
    main packages with gb because it seemed like a reasonable first step - not
    because it was all there was. Have you changed your view? This language
    seems more absolute than what you've used in the past.

    On Friday, 1 April 2016 22:14:09 UTC+11, Ian Davis wrote:
    On Fri, Apr 1, 2016, at 12:07 PM, Jakob Borg wrote:
    2016-04-01 13:02 GMT+02:00 Jakob Borg <ja...@nym.se>:
    So vendoring x/net/context and having
    methods accept a context.Ctx from the outside is a no-no.
    Adding another thought here, this specific case only sounds even
    vaguely acceptable because it's - at this point - such a common type
    that it "feels" like the standard library. I.e. we commonly accept
    io.Writers so why not context.Contexts? But in most other cases you
    would not expose a method that takes some interface from an unrelated
    package - you'd declare a matching interface of your own to avoid the
    dependency to begin with. That's rather the point of having implicitly
    satisfied interfaces after all?
    But this implies that libraries should wrap all their dependencies which
    is not always practical. For example, imagine a game library that relies
    on OpenGL. Because it can't possibly cater for every use case it's
    likely the library will want to expose hooks to directly access some of
    the raw OpenGL functionality, e.g. handles to buffers. If the game
    library has to alias those types then Go's own lexical matching suggests
    it would also need to provide compatible aliases for the entire OpenGL
    API.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at Apr 1, 2016 at 11:39 am
    Nope, I don't think so. I'm pretty sure that the only way the vendor experiment can be used _without_ extra tooling is by library writers forgoing it.

    Wrt to what gb does, if you follow the ideas of "rolling up" dependencies, you eventually end up at something that looks very similar to what gb does, dependencies are specified by the project, the final code that links a binary.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Sam Boyer at Apr 1, 2016 at 12:42 pm

    On Friday, April 1, 2016 at 7:39:32 AM UTC-4, Dave Cheney wrote:
    Nope, I don't think so.

    Maybe I've missed something in my considerations. I'll see, once I set down
    to write the type escape analysis code. I think there's a simple, easy,
    local-only analysis which has holes, and a whole-reachable-codebase
    analysis that's harder.

    I'm pretty sure that the only way the vendor experiment can be used
    _without_ extra tooling is by library writers forgoing it.

    Yes, that I definitely agree with. I may have missed that caveat in what
    you were saying earlier - sorry.

    Wrt to what gb does, if you follow the ideas of "rolling up" dependencies,
    you eventually end up at something that looks very similar to what gb does,
    dependencies are specified by the project, the final code that links a
    binary.
    Right. At this point, I think that's what all of the (major) tools do.

    I know I've made this argument to you before, but I'll repeat it for the
    mailing list - where we differ, IIRC, is dealing with the intermediaries.
    Because gb is only focused on projects with compilable binaries, it assigns
    total control (a good thing) but also total responsibility (not so good) to
    the project for deciding the versions of all deps, including transitive
    ones. Total *responsibility* for deciding the versions of transitive deps
    is less than great, because it more or less means I, as a project author,
    have to go in and research the relationships between my deps, and the deps
    of my deps.

    I'm not saying you shouldn't do that - just that a) it's often not
    feasible, and b) if I depend on A, and A depends on B, then it'd be great
    if A at least provided some suggestions about where to start my research
    about what versions of B it works with.

    Which is the approach glide takes: both libs and projects with binaries can
    use it to specify the versions they'd *like* to use, which then produces a
    list of immutable revisions of what it *will* use (when it's the top-level
    decider), and that then translates into a vendor directory that may (for a
    project with only `main` pkgs) or may not (for project with lib pkgs) be
    committed.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Jakob Borg at Apr 1, 2016 at 11:22 am

    2016-04-01 13:13 GMT+02:00 Ian Davis <me@iandavis.com>:
    On Fri, Apr 1, 2016, at 12:07 PM, Jakob Borg wrote:
    2016-04-01 13:02 GMT+02:00 Jakob Borg <jakob@nym.se>:
    So vendoring x/net/context and having
    methods accept a context.Ctx from the outside is a no-no.
    Adding another thought here, this specific case only sounds even
    vaguely acceptable because it's - at this point - such a common type
    that it "feels" like the standard library. I.e. we commonly accept
    io.Writers so why not context.Contexts? But in most other cases you
    would not expose a method that takes some interface from an unrelated
    package - you'd declare a matching interface of your own to avoid the
    dependency to begin with. That's rather the point of having implicitly
    satisfied interfaces after all?
    But this implies that libraries should wrap all their dependencies which
    is not always practical. For example, imagine a game library that relies
    on OpenGL.
    No, I'm just saying a small interface is not necessarily the best
    thing to have an external dependency for. In the context case this has
    become practice anyway for reasons of tooling etc. In most other cases
    I don't think we'd import, never mind *vendor*, a package just for a
    small interface. See also left-pad... :)

    For something like OpenGL, by all means do expose the necessary types
    but then don't vendor the packages in your library.

    //jb

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Konstantin Shaposhnikov at Apr 1, 2016 at 11:12 am

    I don't think this changes anything for the base rule of "libraries
    should not vendor" - that still holds.
    I wouldn't say that this rule is absolute though. There are some cases
    when it makes sense for a library to vendor some or all of its
    dependencies. For example Go standard library (when Go 1.7 is out)
    vendors golang.org/x/net/http2/hpack

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Ian Davis at Apr 1, 2016 at 11:20 am

    On Fri, Apr 1, 2016, at 12:02 PM, Jakob Borg wrote:
    2016-04-01 12:51 GMT+02:00 Alex Bligh <alex@alex.org.uk>:
    On 1 Apr 2016, at 10:01, Dave Cheney wrote:

    I think it could be more succinctly expressed as:

    Libraries must not vendor code.

    What's a library? You're code is, if some other project imports it.
    Does that mean if your code is a library (you want others to import it)
    you cannot have within it a test suite which requires vendored code?
    If so, that's a bit unfortunate for interoperability tests.
    As I understand it, vendored dependencies themselves are not the
    problem here, but *exposing* them such as returning or taking as
    arguments types from them. So vendoring x/net/context and having
    methods accept a context.Ctx from the outside is a no-no. But a
    vendored dependency for tests, or internal use only, or as a
    dependency to some cmd/* package in the same repo ought to be
    acceptable.
    Vendoring for internal use could lead to duplicate instances of packages
    ending up in the final binary. If one of those packages initializes some
    external resource via an init function and expects exclusive access to
    it then that would be a problem.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Sam Boyer at Apr 1, 2016 at 11:27 am

    On Friday, April 1, 2016 at 7:02:55 AM UTC-4, Jakob Borg wrote:
    2016-04-01 12:51 GMT+02:00 Alex Bligh <al...@alex.org.uk <javascript:>>:
    On 1 Apr 2016, at 10:01, Dave Cheney <da...@cheney.net <javascript:>>
    wrote:
    I think it could be more succinctly expressed as:

    Libraries must not vendor code.

    What's a library? You're code is, if some other project imports it.
    Does that mean if your code is a library (you want others to import it)
    you cannot have within it a test suite which requires vendored code?
    If so, that's a bit unfortunate for interoperability tests.
    As I understand it, vendored dependencies themselves are not the
    problem here, but *exposing* them such as returning or taking as
    arguments types from them. So vendoring x/net/context and having
    methods accept a context.Ctx from the outside is a no-no. But a
    vendored dependency for tests, or internal use only, or as a
    dependency to some cmd/* package in the same repo ought to be
    acceptable.
    Yep. There's an analysis I intend to put in the solver engine I'm working
    on (and, thus, into glide) that determines whether or not a types from a
    dependent package escape from a package. More or less like pointer escape
    analysis. Hard to do completely because of interfaces, but still gets us a
    lot of the way there.

    I don't think this changes anything for the base rule of "libraries
    should not vendor" - that still holds.
    I do wish we could stop using the word "vendor" to refer to just
    "committing your vendor directory." That is *one* approach, but it is not
    the only approach (though I know Dave would prefer it to be). Conflating
    the word with just the one approach distorts the discussion.
    //jb
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Konstantin Shaposhnikov at Apr 1, 2016 at 11:06 am


    Does that mean if your code is a library (you want others to import it)
    you cannot have within it a test suite which requires vendored code?
    If so, that's a bit unfortunate for interoperability tests.

    Only if public (exported) API of your library uses types from its dependant
    packages (like in Peter's example).

    I agree that this situation is unfortunate.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Wojciech S. Czarnecki at Apr 1, 2016 at 1:15 pm
    Dnia 2016-03-31, o godz. 17:01:02
    Peter Bourgon <peter@bourgon.org> napisał(a):
    The etcd repo contains packages that are intended to be imported by
    other packages, such as this one. But it also contains binaries, so I
    think the presence of the vendor folder is correct. Here is my
    code[3]. It fails to build because etcdserverpb.KVServer.Compact
    expects "github.com/coreos/etcd/vendor/golang.org/x/net/context".Context
    rather than "golang.org/x/net/context".Context[4], which as far as I
    know I cannot possibly provide.
    What went wrong, and how can I fix it?
    My strong gut feeling is that whole vendorexperiment failed miserably.

    I think this way of vendoring should be withdrawn ASAP, ie starting at
    1.7 as deprecated. Anything this 'vendor' path allows for is achievable by
    'internal' path. Its beter to back off now than later. Just before it will
    contaminate then destroy whole packages ecosystem.

    A 'feature' that need to be fenced by so much warnings starting with
    'do not use it with/at/for' is not a feature but a loud fail.

    So, with great respect to go dev team I dare to say that the Emperor has no
    clothes. Errare humanum est, sed in errare perseverare diabolicum.
    Cheers,
    Peter.
    --
    Wojciech S. Czarnecki
            ^oo^ OHIR-RIPE

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniel Theophanes at Apr 1, 2016 at 1:45 pm
    The conclusion of this thread is both a sad one and inaccurate. Checking in
    the vendor folder is viral, but not bad. If a dep you use checks in souce
    in the vendor folder then you need to as well. I recommend a good tool to
    help such as govendor.

    This consequence is the same as if you used import path rewriting under the
    internal folder.

    It is not viral to include a vendor metadata file specifying versions and
    revisions.
    On Fri, Apr 1, 2016, 06:15 Wojciech S. Czarnecki wrote:

    Dnia 2016-03-31, o godz. 17:01:02
    Peter Bourgon <peter@bourgon.org> napisał(a):
    The etcd repo contains packages that are intended to be imported by
    other packages, such as this one. But it also contains binaries, so I
    think the presence of the vendor folder is correct. Here is my
    code[3]. It fails to build because etcdserverpb.KVServer.Compact
    expects "github.com/coreos/etcd/vendor/golang.org/x/net/context".Context
    rather than "golang.org/x/net/context".Context[4], which as far as I
    know I cannot possibly provide.
    What went wrong, and how can I fix it?
    My strong gut feeling is that whole vendorexperiment failed miserably.

    I think this way of vendoring should be withdrawn ASAP, ie starting at
    1.7 as deprecated. Anything this 'vendor' path allows for is achievable by
    'internal' path. Its beter to back off now than later. Just before it will
    contaminate then destroy whole packages ecosystem.

    A 'feature' that need to be fenced by so much warnings starting with
    'do not use it with/at/for' is not a feature but a loud fail.

    So, with great respect to go dev team I dare to say that the Emperor has no
    clothes. Errare humanum est, sed in errare perseverare diabolicum.
    Cheers,
    Peter.
    --
    Wojciech S. Czarnecki
    ^oo^ OHIR-RIPE

    --
    You received this message because you are subscribed to a topic in the
    Google Groups "golang-nuts" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/golang-nuts/AnMr9NL6dtc/unsubscribe.
    To unsubscribe from this group and all its topics, send an email to
    golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Zellyn at Apr 1, 2016 at 2:08 pm
    "main packages should vendor" does *not* work if your repository contains
    both library code and binaries. Demonstration:
    https://github.com/zellyn/wtf2

    Zellyn
    On Friday, April 1, 2016 at 9:46:05 AM UTC-4, Daniel Theophanes wrote:

    The conclusion of this thread is both a sad one and inaccurate. Checking
    in the vendor folder is viral, but not bad. If a dep you use checks in
    souce in the vendor folder then you need to as well. I recommend a good
    tool to help such as govendor.

    This consequence is the same as if you used import path rewriting under
    the internal folder.

    It is not viral to include a vendor metadata file specifying versions and
    revisions.

    On Fri, Apr 1, 2016, 06:15 Wojciech S. Czarnecki <oh...@fairbe.org
    <javascript:>> wrote:
    Dnia 2016-03-31, o godz. 17:01:02
    Peter Bourgon <pe...@bourgon.org <javascript:>> napisał(a):
    The etcd repo contains packages that are intended to be imported by
    other packages, such as this one. But it also contains binaries, so I
    think the presence of the vendor folder is correct. Here is my
    code[3]. It fails to build because etcdserverpb.KVServer.Compact
    expects "github.com/coreos/etcd/vendor/golang.org/x/net/context ".Context
    rather than "golang.org/x/net/context".Context[4], which as far as I
    know I cannot possibly provide.
    What went wrong, and how can I fix it?
    My strong gut feeling is that whole vendorexperiment failed miserably.

    I think this way of vendoring should be withdrawn ASAP, ie starting at
    1.7 as deprecated. Anything this 'vendor' path allows for is achievable by
    'internal' path. Its beter to back off now than later. Just before it will
    contaminate then destroy whole packages ecosystem.

    A 'feature' that need to be fenced by so much warnings starting with
    'do not use it with/at/for' is not a feature but a loud fail.

    So, with great respect to go dev team I dare to say that the Emperor has
    no
    clothes. Errare humanum est, sed in errare perseverare diabolicum.
    Cheers,
    Peter.
    --
    Wojciech S. Czarnecki
    ^oo^ OHIR-RIPE

    --
    You received this message because you are subscribed to a topic in the
    Google Groups "golang-nuts" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/golang-nuts/AnMr9NL6dtc/unsubscribe.
    To unsubscribe from this group and all its topics, send an email to
    golang-nuts...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniel Theophanes at Apr 1, 2016 at 2:44 pm
    Hi Zellyn,

    If you didn't understand wtf2 is not how you use the vendor folder when it
    was introduced months ago, you never took the time to understand it in the
    first place.

    A package won't build if you have two different dep packages that looks the
    same but you try to use them as the same package. If this is how you want
    to use the vendor folder, why didn't you try it back when go1.5 was
    released?

    I support the vendor folder because it does indeed work for projects and it
    is standard, not because you can't break it in some way.
    On Fri, Apr 1, 2016, 07:08 Zellyn wrote:

    "main packages should vendor" does *not* work if your repository contains
    both library code and binaries. Demonstration:
    https://github.com/zellyn/wtf2

    Zellyn

    On Friday, April 1, 2016 at 9:46:05 AM UTC-4, Daniel Theophanes wrote:

    The conclusion of this thread is both a sad one and inaccurate. Checking
    in the vendor folder is viral, but not bad. If a dep you use checks in
    souce in the vendor folder then you need to as well. I recommend a good
    tool to help such as govendor.

    This consequence is the same as if you used import path rewriting under
    the internal folder.

    It is not viral to include a vendor metadata file specifying versions and
    revisions.

    On Fri, Apr 1, 2016, 06:15 Wojciech S. Czarnecki <oh...@fairbe.org>
    wrote:
    Dnia 2016-03-31, o godz. 17:01:02
    Peter Bourgon <pe...@bourgon.org> napisał(a):

    The etcd repo contains packages that are intended to be imported by
    other packages, such as this one. But it also contains binaries, so I
    think the presence of the vendor folder is correct. Here is my
    code[3]. It fails to build because etcdserverpb.KVServer.Compact
    expects "github.com/coreos/etcd/vendor/golang.org/x/net/context ".Context
    rather than "golang.org/x/net/context".Context[4], which as far as I
    know I cannot possibly provide.
    What went wrong, and how can I fix it?
    My strong gut feeling is that whole vendorexperiment failed miserably.

    I think this way of vendoring should be withdrawn ASAP, ie starting at
    1.7 as deprecated. Anything this 'vendor' path allows for is achievable
    by
    'internal' path. Its beter to back off now than later. Just before it
    will
    contaminate then destroy whole packages ecosystem.

    A 'feature' that need to be fenced by so much warnings starting with
    'do not use it with/at/for' is not a feature but a loud fail.

    So, with great respect to go dev team I dare to say that the Emperor has
    no
    clothes. Errare humanum est, sed in errare perseverare diabolicum.
    Cheers,
    Peter.
    --
    Wojciech S. Czarnecki
    ^oo^ OHIR-RIPE

    --
    You received this message because you are subscribed to a topic in the
    Google Groups "golang-nuts" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/golang-nuts/AnMr9NL6dtc/unsubscribe.
    To unsubscribe from this group and all its topics, send an email to
    golang-nuts...@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to a topic in the
    Google Groups "golang-nuts" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/golang-nuts/AnMr9NL6dtc/unsubscribe.
    To unsubscribe from this group and all its topics, send an email to
    golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedMar 31, '16 at 3:01p
activeApr 1, '16 at 3:34p
posts54
users13
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase