FAQ
If you use the vendor experiment to put external packages into the vendor/
folder and then others pull your package into theirs it's easy to run into
a problem. For example see
https://github.com/mattfarina/golang-broken-vendor.

I've not seen instruction in how to avoid that situation. And, it becomes
complex for some projects such as k8s. k8s is an application in its own
right and others can include it (or parts of it) for their SDK to k8s.
Applications that have a dual purpose can cause problems.

I'm curious what thought has been put into this situation.

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Russ Cox at Nov 3, 2015 at 7:25 pm

    On Tue, Nov 3, 2015 at 12:28 PM, Matt Farina wrote:

    If you use the vendor experiment to put external packages into the vendor/
    folder and then others pull your package into theirs it's easy to run into
    a problem. For example see
    https://github.com/mattfarina/golang-broken-vendor.

    I've not seen instruction in how to avoid that situation. And, it becomes
    complex for some projects such as k8s. k8s is an application in its own
    right and others can include it (or parts of it) for their SDK to k8s.
    Applications that have a dual purpose can cause problems.

    I'm curious what thought has been put into this situation.
    If you want to share code with others and can't be sure they all set
    GO15VENDOREXPERIMENT, then don't use vendor directories yet. Wait until
    vendor support is on by default in Go 1.6 (and you can be sure those you
    want to share with are all up to 1.6).

    Russ

    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matt Farina at Nov 3, 2015 at 7:48 pm
    Unfortunately, waiting for 1.6 isn't going to solve this problem. It may
    only get worse. Let me explain.

    Quite often you don't know who you're going to share with. The code goes up
    on GitHub and none or many people may use it. If package authors don't know
    about this behavior they may store the packages they use in their vendor/
    folder. Then, when someone else uses that package and has their own
    dependencies in their vendor/ folder this problem can arise. In that case,
    what's someone to do?

    I found this problem in practice by using packages where everyone expected
    the GO15VENDOREXPERIMENT to be on. People were not aware of this behavior.

    Once the vendor experiment is on by default a larger audience will be open
    to this problem.

    On Tuesday, November 3, 2015 at 2:25:32 PM UTC-5, rsc wrote:

    On Tue, Nov 3, 2015 at 12:28 PM, Matt Farina <matt....@gmail.com
    <javascript:>> wrote:
    If you use the vendor experiment to put external packages into the
    vendor/ folder and then others pull your package into theirs it's easy to
    run into a problem. For example see
    https://github.com/mattfarina/golang-broken-vendor.

    I've not seen instruction in how to avoid that situation. And, it becomes
    complex for some projects such as k8s. k8s is an application in its own
    right and others can include it (or parts of it) for their SDK to k8s.
    Applications that have a dual purpose can cause problems.

    I'm curious what thought has been put into this situation.
    If you want to share code with others and can't be sure they all set
    GO15VENDOREXPERIMENT, then don't use vendor directories yet. Wait until
    vendor support is on by default in Go 1.6 (and you can be sure those you
    want to share with are all up to 1.6).

    Russ

    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Russ Cox at Nov 3, 2015 at 8:44 pm

    On Tue, Nov 3, 2015 at 2:48 PM, Matt Farina wrote:

    Unfortunately, waiting for 1.6 isn't going to solve this problem. It may
    only get worse. Let me explain.

    Quite often you don't know who you're going to share with. The code goes
    up on GitHub and none or many people may use it. If package authors don't
    know about this behavior they may store the packages they use in their
    vendor/ folder. Then, when someone else uses that package and has their own
    dependencies in their vendor/ folder this problem can arise. In that case,
    what's someone to do?

    I found this problem in practice by using packages where everyone expected
    the GO15VENDOREXPERIMENT to be on. People were not aware of this behavior.

    Once the vendor experiment is on by default a larger audience will be open
    to this problem.
    I am not sure I understand the problem. I thought the problem was that some
    people had vendor semantics enabled and some did not. That's a short term
    problem: eventually close enough to everyone will have them enabled
    (because they're using Go 1.6 or later).

    But it sounds like you are describing a problem that arises even when
    everyone involved agrees that vendor semantics are enabled. What is the
    problem in that case?

    Thanks.
    Russ

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matt Farina at Nov 3, 2015 at 8:47 pm
    @rsc did you look at the example I linked off to? When the same package is
    in 2 vendor/ locations. It could be more but the linked example only has 2.
    In case you missed it, there example is at
    https://github.com/mattfarina/golang-broken-vendor

    It has to do with how, in practice, people use the vendor/ directories.
    This issue has been seen in the wild.
    On Tuesday, November 3, 2015 at 3:44:29 PM UTC-5, rsc wrote:

    On Tue, Nov 3, 2015 at 2:48 PM, Matt Farina <matt....@gmail.com
    <javascript:>> wrote:
    Unfortunately, waiting for 1.6 isn't going to solve this problem. It may
    only get worse. Let me explain.

    Quite often you don't know who you're going to share with. The code goes
    up on GitHub and none or many people may use it. If package authors don't
    know about this behavior they may store the packages they use in their
    vendor/ folder. Then, when someone else uses that package and has their own
    dependencies in their vendor/ folder this problem can arise. In that case,
    what's someone to do?

    I found this problem in practice by using packages where everyone
    expected the GO15VENDOREXPERIMENT to be on. People were not aware of this
    behavior.

    Once the vendor experiment is on by default a larger audience will be
    open to this problem.
    I am not sure I understand the problem. I thought the problem was that
    some people had vendor semantics enabled and some did not. That's a short
    term problem: eventually close enough to everyone will have them enabled
    (because they're using Go 1.6 or later).

    But it sounds like you are describing a problem that arises even when
    everyone involved agrees that vendor semantics are enabled. What is the
    problem in that case?

    Thanks.
    Russ

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Minux at Nov 3, 2015 at 9:59 pm

    On Tue, Nov 3, 2015 at 3:47 PM, Matt Farina wrote:

    @rsc did you look at the example I linked off to? When the same package is
    in 2 vendor/ locations. It could be more but the linked example only has 2.
    In case you missed it, there example is at
    https://github.com/mattfarina/golang-broken-vendor

    It has to do with how, in practice, people use the vendor/ directories.
    This issue has been seen in the wild.
    It's working as intended. Two vendored copies of the same package are not
    the same package (their import paths are different).

    The vendor experiment merely provides a mechanism, the vendor policy is
    not and shouldn't be enforced by the go tool.

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at Nov 3, 2015 at 10:09 pm
    So here's the problem,

    The way the vendor rule works at the moment is vendor/x is imported as "x",
    but linked as " vendor/x"

    This introduces all the problems of putting version numbers in imports, aka
    gopkg.in. specifically

    * duplicate calls to init routines to register drivers and loggers
    * types that fail type assertion checks.

    However, to do the opposite, vendor/x linking as x would conflict with
    other x's, possibly from GOPATH or other nested, or sibling vendor
    structures.

    This would cause linking failures as symbols wouldn't match, or worse, no
    linking failures and structures being the wrong size at run time.

    Either way is problematic, and I don't see a way to resolve Matt's
    concerns.

    Thanks

    Dave
    On Wed, 4 Nov 2015, 08:59 minux wrote:
    On Tue, Nov 3, 2015 at 3:47 PM, Matt Farina wrote:

    @rsc did you look at the example I linked off to? When the same package
    is in 2 vendor/ locations. It could be more but the linked example only has
    2. In case you missed it, there example is at
    https://github.com/mattfarina/golang-broken-vendor

    It has to do with how, in practice, people use the vendor/ directories.
    This issue has been seen in the wild.
    It's working as intended. Two vendored copies of the same package are not
    the same package (their import paths are different).

    The vendor experiment merely provides a mechanism, the vendor policy is
    not and shouldn't be enforced by the go tool.

    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matt Farina at Nov 3, 2015 at 11:28 pm
    @dave If this is what we have I see two possibilities of ways to use it in
    practice. Something we'll need to teach people.

        1. No one stores their dependences in the vendor/ in their VCS (unless
        you really know what you're doing... an exception). Then you use tooling to
        populate the vendor/ folder when needed (dev, test, and build
        environments). The tooling uses config files (and nested ones in
        dependencies) to populate it. This allows you to use it and have different
        versions of packages in different applications. This is basically the model
        used by PHP, node.js, and many other languages. Let the tooling manage your
        vendor/ folder based on config so you can avoid bad situations.
        2. Tooling walks through all the dependencies and moves vendor
        dependencies to the top level. This may be opt-in and would need to rely on
        some config (to handle knowledge about versions so you can deal with the
        same package multiple times but different versions).

    Really, the first case it tooling to manage you vendor/ folder to avoid the
    problems. The second it tooling to fix the problems.

    I personally prefer the former because it's an easier problem to solve and
    is generally known because so many other languages tooling operate this
    way. It's familiar and we can learn from their struggles.

    Without tooling to "manage" this it quickly becomes painful.

    Unless you have a better idea (and I'm looking for one).

    - Matt
    On Tuesday, November 3, 2015 at 5:09:53 PM UTC-5, Dave Cheney wrote:

    So here's the problem,

    The way the vendor rule works at the moment is vendor/x is imported as
    "x", but linked as " vendor/x"

    This introduces all the problems of putting version numbers in imports,
    aka gopkg.in. specifically

    * duplicate calls to init routines to register drivers and loggers
    * types that fail type assertion checks.

    However, to do the opposite, vendor/x linking as x would conflict with
    other x's, possibly from GOPATH or other nested, or sibling vendor
    structures.

    This would cause linking failures as symbols wouldn't match, or worse, no
    linking failures and structures being the wrong size at run time.

    Either way is problematic, and I don't see a way to resolve Matt's
    concerns.

    Thanks

    Dave
    On Wed, 4 Nov 2015, 08:59 minux <mi...@golang.org <javascript:>> wrote:

    On Tue, Nov 3, 2015 at 3:47 PM, Matt Farina <matt....@gmail.com
    <javascript:>> wrote:
    @rsc did you look at the example I linked off to? When the same package
    is in 2 vendor/ locations. It could be more but the linked example only has
    2. In case you missed it, there example is at
    https://github.com/mattfarina/golang-broken-vendor

    It has to do with how, in practice, people use the vendor/ directories.
    This issue has been seen in the wild.
    It's working as intended. Two vendored copies of the same package are not
    the same package (their import paths are different).

    The vendor experiment merely provides a mechanism, the vendor policy is
    not and shouldn't be enforced by the go tool.

    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at Nov 3, 2015 at 11:33 pm

    On Wed, Nov 4, 2015 at 10:28 AM, Matt Farina wrote:

    @dave If this is what we have I see two possibilities of ways to use it in
    practice. Something we'll need to teach people.

    1. No one stores their dependences in the vendor/ in their VCS (unless
    you really know what you're doing... an exception). Then you use tooling to
    populate the vendor/ folder when needed (dev, test, and build
    environments). The tooling uses config files (and nested ones in
    dependencies) to populate it. This allows you to use it and have different
    versions of packages in different applications. This is basically the model
    used by PHP, node.js, and many other languages. Let the tooling manage your
    vendor/ folder based on config so you can avoid bad situations.
    This wound nullify what I see as the main value of the vendor directory;
    vendoring code into your repository. If this cannot be done, then what is
    the point of vendor ? Why not just have tools that manage GOPATH directly,
    like https://launchpad.net/godeps ?

    1. Tooling walks through all the dependencies and moves vendor
    dependencies to the top level. This may be opt-in and would need to rely on
    some config (to handle knowledge about versions so you can deal with the
    same package multiple times but different versions).

    This sounds like a better option, but how do you handle conflicts between
    a deps that live in a (flattened) vendor/ folder, and raw, or transitively
    (see below) in GOPATH ?

    github.com/
             /foo/bar/
                   /vendor/
                         /logger
             /quxx/frob
                  /vendor/
                         /logger

    If quxx/frob imports "logger" and "foo/bar", which copy of logger is
    "foo/bar" going to link against ? I don't think flattening can solve this.


    Really, the first case it tooling to manage you vendor/ folder to avoid
    the problems. The second it tooling to fix the problems.

    I personally prefer the former because it's an easier problem to solve and
    is generally known because so many other languages tooling operate this
    way. It's familiar and we can learn from their struggles.

    Without tooling to "manage" this it quickly becomes painful.

    Unless you have a better idea (and I'm looking for one).

    - Matt
    On Tuesday, November 3, 2015 at 5:09:53 PM UTC-5, Dave Cheney wrote:

    So here's the problem,

    The way the vendor rule works at the moment is vendor/x is imported as
    "x", but linked as " vendor/x"

    This introduces all the problems of putting version numbers in imports,
    aka gopkg.in. specifically

    * duplicate calls to init routines to register drivers and loggers
    * types that fail type assertion checks.

    However, to do the opposite, vendor/x linking as x would conflict with
    other x's, possibly from GOPATH or other nested, or sibling vendor
    structures.

    This would cause linking failures as symbols wouldn't match, or worse, no
    linking failures and structures being the wrong size at run time.

    Either way is problematic, and I don't see a way to resolve Matt's
    concerns.

    Thanks

    Dave
    On Wed, 4 Nov 2015, 08:59 minux wrote:
    On Tue, Nov 3, 2015 at 3:47 PM, Matt Farina wrote:

    @rsc did you look at the example I linked off to? When the same package
    is in 2 vendor/ locations. It could be more but the linked example only has
    2. In case you missed it, there example is at
    https://github.com/mattfarina/golang-broken-vendor

    It has to do with how, in practice, people use the vendor/ directories.
    This issue has been seen in the wild.
    It's working as intended. Two vendored copies of the same package are not
    the same package (their import paths are different).

    The vendor experiment merely provides a mechanism, the vendor policy is
    not and shouldn't be enforced by the go tool.

    --
    You received this message because you are subscribed to the Google
    Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-dev+...@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at Nov 3, 2015 at 11:36 pm
    Replying to myself on the second point. It would appear that if you vendor/
    anything, you must therefore vendor everything as a partial solution
    introduces ambiguity. This is probably something tooling can help with.
    On Wed, Nov 4, 2015 at 10:33 AM, Dave Cheney wrote:


    On Wed, Nov 4, 2015 at 10:28 AM, Matt Farina wrote:

    @dave If this is what we have I see two possibilities of ways to use it
    in practice. Something we'll need to teach people.

    1. No one stores their dependences in the vendor/ in their VCS
    (unless you really know what you're doing... an exception). Then you use
    tooling to populate the vendor/ folder when needed (dev, test, and build
    environments). The tooling uses config files (and nested ones in
    dependencies) to populate it. This allows you to use it and have different
    versions of packages in different applications. This is basically the model
    used by PHP, node.js, and many other languages. Let the tooling manage your
    vendor/ folder based on config so you can avoid bad situations.
    This wound nullify what I see as the main value of the vendor directory;
    vendoring code into your repository. If this cannot be done, then what is
    the point of vendor ? Why not just have tools that manage GOPATH directly,
    like https://launchpad.net/godeps ?

    1. Tooling walks through all the dependencies and moves vendor
    dependencies to the top level. This may be opt-in and would need to rely on
    some config (to handle knowledge about versions so you can deal with the
    same package multiple times but different versions).

    This sounds like a better option, but how do you handle conflicts between
    a deps that live in a (flattened) vendor/ folder, and raw, or transitively
    (see below) in GOPATH ?

    github.com/
    /foo/bar/
    /vendor/
    /logger
    /quxx/frob
    /vendor/
    /logger

    If quxx/frob imports "logger" and "foo/bar", which copy of logger is
    "foo/bar" going to link against ? I don't think flattening can solve this.


    Really, the first case it tooling to manage you vendor/ folder to avoid
    the problems. The second it tooling to fix the problems.

    I personally prefer the former because it's an easier problem to solve
    and is generally known because so many other languages tooling operate this
    way. It's familiar and we can learn from their struggles.

    Without tooling to "manage" this it quickly becomes painful.

    Unless you have a better idea (and I'm looking for one).

    - Matt
    On Tuesday, November 3, 2015 at 5:09:53 PM UTC-5, Dave Cheney wrote:

    So here's the problem,

    The way the vendor rule works at the moment is vendor/x is imported as
    "x", but linked as " vendor/x"

    This introduces all the problems of putting version numbers in imports,
    aka gopkg.in. specifically

    * duplicate calls to init routines to register drivers and loggers
    * types that fail type assertion checks.

    However, to do the opposite, vendor/x linking as x would conflict with
    other x's, possibly from GOPATH or other nested, or sibling vendor
    structures.

    This would cause linking failures as symbols wouldn't match, or worse,
    no linking failures and structures being the wrong size at run time.

    Either way is problematic, and I don't see a way to resolve Matt's
    concerns.

    Thanks

    Dave
    On Wed, 4 Nov 2015, 08:59 minux wrote:
    On Tue, Nov 3, 2015 at 3:47 PM, Matt Farina wrote:

    @rsc did you look at the example I linked off to? When the same
    package is in 2 vendor/ locations. It could be more but the linked example
    only has 2. In case you missed it, there example is at
    https://github.com/mattfarina/golang-broken-vendor

    It has to do with how, in practice, people use the vendor/
    directories. This issue has been seen in the wild.
    It's working as intended. Two vendored copies of the same package are
    not
    the same package (their import paths are different).

    The vendor experiment merely provides a mechanism, the vendor policy is
    not and shouldn't be enforced by the go tool.

    --
    You received this message because you are subscribed to the Google
    Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-dev+...@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matt Farina at Nov 3, 2015 at 11:58 pm
    Dave,

    In ref to #1, what if you're working on more than one application and they
    use different versions of the same dependency? How do you handle that with
    one gopath?

    Oh, and I understand how a project specific workspace solves that. I'm
    attempting to look at this through the go tool.

    To flatten multiple projects to the top level and handle versions you'd
    need a record of versions. Ideally, to handle mild differences they'd be
    versions as semver constraints and the tooling picks the best version to
    meet those (if one is available).

    If you need that record of versions somewhere and to have to restructure a
    codebase what benefit is there to vendoring?

    - Matt
    On Tuesday, November 3, 2015, Dave Cheney wrote:

    Replying to myself on the second point. It would appear that if you
    vendor/ anything, you must therefore vendor everything as a partial
    solution introduces ambiguity. This is probably something tooling can help
    with.

    On Wed, Nov 4, 2015 at 10:33 AM, Dave Cheney <dave@cheney.net
    wrote:

    On Wed, Nov 4, 2015 at 10:28 AM, Matt Farina <matt.farina@gmail.com
    wrote:
    @dave If this is what we have I see two possibilities of ways to use it
    in practice. Something we'll need to teach people.

    1. No one stores their dependences in the vendor/ in their VCS
    (unless you really know what you're doing... an exception). Then you use
    tooling to populate the vendor/ folder when needed (dev, test, and build
    environments). The tooling uses config files (and nested ones in
    dependencies) to populate it. This allows you to use it and have different
    versions of packages in different applications. This is basically the model
    used by PHP, node.js, and many other languages. Let the tooling manage your
    vendor/ folder based on config so you can avoid bad situations.
    This wound nullify what I see as the main value of the vendor directory;
    vendoring code into your repository. If this cannot be done, then what is
    the point of vendor ? Why not just have tools that manage GOPATH directly,
    like https://launchpad.net/godeps ?

    1. Tooling walks through all the dependencies and moves vendor
    dependencies to the top level. This may be opt-in and would need to rely on
    some config (to handle knowledge about versions so you can deal with the
    same package multiple times but different versions).

    This sounds like a better option, but how do you handle conflicts
    between a deps that live in a (flattened) vendor/ folder, and raw, or
    transitively (see below) in GOPATH ?

    github.com/
    /foo/bar/
    /vendor/
    /logger
    /quxx/frob
    /vendor/
    /logger

    If quxx/frob imports "logger" and "foo/bar", which copy of logger is
    "foo/bar" going to link against ? I don't think flattening can solve this.


    Really, the first case it tooling to manage you vendor/ folder to avoid
    the problems. The second it tooling to fix the problems.

    I personally prefer the former because it's an easier problem to solve
    and is generally known because so many other languages tooling operate this
    way. It's familiar and we can learn from their struggles.

    Without tooling to "manage" this it quickly becomes painful.

    Unless you have a better idea (and I'm looking for one).

    - Matt
    On Tuesday, November 3, 2015 at 5:09:53 PM UTC-5, Dave Cheney wrote:

    So here's the problem,

    The way the vendor rule works at the moment is vendor/x is imported as
    "x", but linked as " vendor/x"

    This introduces all the problems of putting version numbers in imports,
    aka gopkg.in. specifically

    * duplicate calls to init routines to register drivers and loggers
    * types that fail type assertion checks.

    However, to do the opposite, vendor/x linking as x would conflict with
    other x's, possibly from GOPATH or other nested, or sibling vendor
    structures.

    This would cause linking failures as symbols wouldn't match, or worse,
    no linking failures and structures being the wrong size at run time.

    Either way is problematic, and I don't see a way to resolve Matt's
    concerns.

    Thanks

    Dave
    On Wed, 4 Nov 2015, 08:59 minux wrote:

    On Tue, Nov 3, 2015 at 3:47 PM, Matt Farina <matt....@gmail.com>
    wrote:
    @rsc did you look at the example I linked off to? When the same
    package is in 2 vendor/ locations. It could be more but the linked example
    only has 2. In case you missed it, there example is at
    https://github.com/mattfarina/golang-broken-vendor

    It has to do with how, in practice, people use the vendor/
    directories. This issue has been seen in the wild.
    It's working as intended. Two vendored copies of the same package are
    not
    the same package (their import paths are different).

    The vendor experiment merely provides a mechanism, the vendor policy is
    not and shouldn't be enforced by the go tool.

    --
    You received this message because you are subscribed to the Google
    Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-dev+...@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google
    Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-dev+unsubscribe@googlegroups.com
    <javascript:_e(%7B%7D,'cvml','golang-dev%2bunsubscribe@googlegroups.com');>
    .
    For more options, visit https://groups.google.com/d/optout.
    --
    Matt Farina
    mattfarina.com

    Go in Practice <http://goinpracticebook.com/> - A book of Recipes for the
    Go programming language.

    Engineered Web <http://engineeredweb.com/> - A blog on cloud computing and
    web technologies.

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at Nov 4, 2015 at 12:01 am

    On Wed, Nov 4, 2015 at 10:58 AM, Matt Farina wrote:

    Dave,

    In ref to #1, what if you're working on more than one application and they
    use different versions of the same dependency?
    This has never been possible, the linker prevents it.

    How do you handle that with one gopath?
    If you have one entry in your gopath, then there can only be one revision
    of the code for that package there. If you have multiple, then the first
    found wins, for the same reason you cannot have a package called "net" in
    GOPATH, GOROOT always preceeds it.

    Oh, and I understand how a project specific workspace solves that. I'm
    attempting to look at this through the go tool.

    To flatten multiple projects to the top level and handle versions you'd
    need a record of versions. Ideally, to handle mild differences they'd be
    versions as semver constraints and the tooling picks the best version to
    meet those (if one is available).

    If you need that record of versions somewhere and to have to restructure a
    codebase what benefit is there to vendoring?
    Indeed.

    - Matt

    On Tuesday, November 3, 2015, Dave Cheney wrote:

    Replying to myself on the second point. It would appear that if you
    vendor/ anything, you must therefore vendor everything as a partial
    solution introduces ambiguity. This is probably something tooling can help
    with.
    On Wed, Nov 4, 2015 at 10:33 AM, Dave Cheney wrote:



    On Wed, Nov 4, 2015 at 10:28 AM, Matt Farina <matt.farina@gmail.com>
    wrote:
    @dave If this is what we have I see two possibilities of ways to use it
    in practice. Something we'll need to teach people.

    1. No one stores their dependences in the vendor/ in their VCS
    (unless you really know what you're doing... an exception). Then you use
    tooling to populate the vendor/ folder when needed (dev, test, and build
    environments). The tooling uses config files (and nested ones in
    dependencies) to populate it. This allows you to use it and have different
    versions of packages in different applications. This is basically the model
    used by PHP, node.js, and many other languages. Let the tooling manage your
    vendor/ folder based on config so you can avoid bad situations.
    This wound nullify what I see as the main value of the vendor directory;
    vendoring code into your repository. If this cannot be done, then what is
    the point of vendor ? Why not just have tools that manage GOPATH directly,
    like https://launchpad.net/godeps ?

    1. Tooling walks through all the dependencies and moves vendor
    dependencies to the top level. This may be opt-in and would need to rely on
    some config (to handle knowledge about versions so you can deal with the
    same package multiple times but different versions).

    This sounds like a better option, but how do you handle conflicts
    between a deps that live in a (flattened) vendor/ folder, and raw, or
    transitively (see below) in GOPATH ?

    github.com/
    /foo/bar/
    /vendor/
    /logger
    /quxx/frob
    /vendor/
    /logger

    If quxx/frob imports "logger" and "foo/bar", which copy of logger is
    "foo/bar" going to link against ? I don't think flattening can solve this.


    Really, the first case it tooling to manage you vendor/ folder to avoid
    the problems. The second it tooling to fix the problems.

    I personally prefer the former because it's an easier problem to solve
    and is generally known because so many other languages tooling operate this
    way. It's familiar and we can learn from their struggles.

    Without tooling to "manage" this it quickly becomes painful.

    Unless you have a better idea (and I'm looking for one).

    - Matt
    On Tuesday, November 3, 2015 at 5:09:53 PM UTC-5, Dave Cheney wrote:

    So here's the problem,

    The way the vendor rule works at the moment is vendor/x is imported as
    "x", but linked as " vendor/x"

    This introduces all the problems of putting version numbers in
    imports, aka gopkg.in. specifically

    * duplicate calls to init routines to register drivers and loggers
    * types that fail type assertion checks.

    However, to do the opposite, vendor/x linking as x would conflict with
    other x's, possibly from GOPATH or other nested, or sibling vendor
    structures.

    This would cause linking failures as symbols wouldn't match, or worse,
    no linking failures and structures being the wrong size at run time.

    Either way is problematic, and I don't see a way to resolve Matt's
    concerns.

    Thanks

    Dave
    On Wed, 4 Nov 2015, 08:59 minux wrote:

    On Tue, Nov 3, 2015 at 3:47 PM, Matt Farina <matt....@gmail.com>
    wrote:
    @rsc did you look at the example I linked off to? When the same
    package is in 2 vendor/ locations. It could be more but the linked example
    only has 2. In case you missed it, there example is at
    https://github.com/mattfarina/golang-broken-vendor

    It has to do with how, in practice, people use the vendor/
    directories. This issue has been seen in the wild.
    It's working as intended. Two vendored copies of the same package are
    not
    the same package (their import paths are different).

    The vendor experiment merely provides a mechanism, the vendor policy
    is
    not and shouldn't be enforced by the go tool.

    --
    You received this message because you are subscribed to the Google
    Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it,
    send an email to golang-dev+...@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google
    Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    Matt Farina
    mattfarina.com

    Go in Practice <http://goinpracticebook.com/> - A book of Recipes for the
    Go programming language.

    Engineered Web <http://engineeredweb.com/> - A blog on cloud computing
    and web technologies.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at Nov 4, 2015 at 12:02 am
    On Wed, Nov 4, 2015 at 11:01 AM, Dave Cheney wrote:
    On Wed, Nov 4, 2015 at 10:58 AM, Matt Farina wrote:

    Dave,

    In ref to #1, what if you're working on more than one application and
    they use different versions of the same dependency?
    This has never been possible, the linker prevents it.
    Sorry that response was incomplete. In the example I work on daily in Juju,
    every time we git checkout a branch, we must remember to run godeps -u
    dependencies.tsv to adjust the rest of the GOPATH to match the expectations
    of this branch. This approach is error prone, and a key driver behind the
    project based approach that gb uses.

    How do you handle that with one gopath?
    If you have one entry in your gopath, then there can only be one revision
    of the code for that package there. If you have multiple, then the first
    found wins, for the same reason you cannot have a package called "net" in
    GOPATH, GOROOT always preceeds it.

    Oh, and I understand how a project specific workspace solves that. I'm
    attempting to look at this through the go tool.

    To flatten multiple projects to the top level and handle versions you'd
    need a record of versions. Ideally, to handle mild differences they'd be
    versions as semver constraints and the tooling picks the best version to
    meet those (if one is available).

    If you need that record of versions somewhere and to have to restructure
    a codebase what benefit is there to vendoring?
    Indeed.

    - Matt

    On Tuesday, November 3, 2015, Dave Cheney wrote:

    Replying to myself on the second point. It would appear that if you
    vendor/ anything, you must therefore vendor everything as a partial
    solution introduces ambiguity. This is probably something tooling can help
    with.
    On Wed, Nov 4, 2015 at 10:33 AM, Dave Cheney wrote:



    On Wed, Nov 4, 2015 at 10:28 AM, Matt Farina <matt.farina@gmail.com>
    wrote:
    @dave If this is what we have I see two possibilities of ways to use
    it in practice. Something we'll need to teach people.

    1. No one stores their dependences in the vendor/ in their VCS
    (unless you really know what you're doing... an exception). Then you use
    tooling to populate the vendor/ folder when needed (dev, test, and build
    environments). The tooling uses config files (and nested ones in
    dependencies) to populate it. This allows you to use it and have different
    versions of packages in different applications. This is basically the model
    used by PHP, node.js, and many other languages. Let the tooling manage your
    vendor/ folder based on config so you can avoid bad situations.
    This wound nullify what I see as the main value of the vendor
    directory; vendoring code into your repository. If this cannot be done,
    then what is the point of vendor ? Why not just have tools that manage
    GOPATH directly, like https://launchpad.net/godeps ?

    1. Tooling walks through all the dependencies and moves vendor
    dependencies to the top level. This may be opt-in and would need to rely on
    some config (to handle knowledge about versions so you can deal with the
    same package multiple times but different versions).

    This sounds like a better option, but how do you handle conflicts
    between a deps that live in a (flattened) vendor/ folder, and raw, or
    transitively (see below) in GOPATH ?

    github.com/
    /foo/bar/
    /vendor/
    /logger
    /quxx/frob
    /vendor/
    /logger

    If quxx/frob imports "logger" and "foo/bar", which copy of logger is
    "foo/bar" going to link against ? I don't think flattening can solve this.


    Really, the first case it tooling to manage you vendor/ folder to
    avoid the problems. The second it tooling to fix the problems.

    I personally prefer the former because it's an easier problem to solve
    and is generally known because so many other languages tooling operate this
    way. It's familiar and we can learn from their struggles.

    Without tooling to "manage" this it quickly becomes painful.

    Unless you have a better idea (and I'm looking for one).

    - Matt
    On Tuesday, November 3, 2015 at 5:09:53 PM UTC-5, Dave Cheney wrote:

    So here's the problem,

    The way the vendor rule works at the moment is vendor/x is imported
    as "x", but linked as " vendor/x"

    This introduces all the problems of putting version numbers in
    imports, aka gopkg.in. specifically

    * duplicate calls to init routines to register drivers and loggers
    * types that fail type assertion checks.

    However, to do the opposite, vendor/x linking as x would conflict
    with other x's, possibly from GOPATH or other nested, or sibling vendor
    structures.

    This would cause linking failures as symbols wouldn't match, or
    worse, no linking failures and structures being the wrong size at run time.

    Either way is problematic, and I don't see a way to resolve Matt's
    concerns.

    Thanks

    Dave
    On Wed, 4 Nov 2015, 08:59 minux wrote:

    On Tue, Nov 3, 2015 at 3:47 PM, Matt Farina <matt....@gmail.com>
    wrote:
    @rsc did you look at the example I linked off to? When the same
    package is in 2 vendor/ locations. It could be more but the linked example
    only has 2. In case you missed it, there example is at
    https://github.com/mattfarina/golang-broken-vendor

    It has to do with how, in practice, people use the vendor/
    directories. This issue has been seen in the wild.
    It's working as intended. Two vendored copies of the same package
    are not
    the same package (their import paths are different).

    The vendor experiment merely provides a mechanism, the vendor policy
    is
    not and shouldn't be enforced by the go tool.

    --
    You received this message because you are subscribed to the Google
    Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it,
    send an email to golang-dev+...@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google
    Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    Matt Farina
    mattfarina.com

    Go in Practice <http://goinpracticebook.com/> - A book of Recipes for
    the Go programming language.

    Engineered Web <http://engineeredweb.com/> - A blog on cloud computing
    and web technologies.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matt Farina at Nov 4, 2015 at 12:44 am
    Dave,

    With the vendor/ directory each application can have their own versions of
    a packages. They just need to be in the top level vendor/ directory and the
    linker works. That's what makes the GO15VENDOREXPERIMENT interesting. Not
    because of the ability to store outside pacakges in your vcs repo. It's
    interesting because it provides the first sane way to handle multiple
    versions of packages. At least that's why it's interesting to me.

    - Matt
    On Tuesday, November 3, 2015 at 7:02:52 PM UTC-5, Dave Cheney wrote:



    On Wed, Nov 4, 2015 at 11:01 AM, Dave Cheney <da...@cheney.net
    <javascript:>> wrote:

    On Wed, Nov 4, 2015 at 10:58 AM, Matt Farina <matt....@gmail.com
    <javascript:>> wrote:
    Dave,

    In ref to #1, what if you're working on more than one application and
    they use different versions of the same dependency?
    This has never been possible, the linker prevents it.
    Sorry that response was incomplete. In the example I work on daily in
    Juju, every time we git checkout a branch, we must remember to run godeps
    -u dependencies.tsv to adjust the rest of the GOPATH to match the
    expectations of this branch. This approach is error prone, and a key driver
    behind the project based approach that gb uses.

    How do you handle that with one gopath?
    If you have one entry in your gopath, then there can only be one revision
    of the code for that package there. If you have multiple, then the first
    found wins, for the same reason you cannot have a package called "net" in
    GOPATH, GOROOT always preceeds it.

    Oh, and I understand how a project specific workspace solves that. I'm
    attempting to look at this through the go tool.

    To flatten multiple projects to the top level and handle versions you'd
    need a record of versions. Ideally, to handle mild differences they'd be
    versions as semver constraints and the tooling picks the best version
    to meet those (if one is available).

    If you need that record of versions somewhere and to have to restructure
    a codebase what benefit is there to vendoring?
    Indeed.

    - Matt


    On Tuesday, November 3, 2015, Dave Cheney <da...@cheney.net
    <javascript:>> wrote:
    Replying to myself on the second point. It would appear that if you
    vendor/ anything, you must therefore vendor everything as a partial
    solution introduces ambiguity. This is probably something tooling can help
    with.
    On Wed, Nov 4, 2015 at 10:33 AM, Dave Cheney wrote:



    On Wed, Nov 4, 2015 at 10:28 AM, Matt Farina <matt.farina@gmail.com>
    wrote:
    @dave If this is what we have I see two possibilities of ways to use
    it in practice. Something we'll need to teach people.

    1. No one stores their dependences in the vendor/ in their VCS
    (unless you really know what you're doing... an exception). Then you use
    tooling to populate the vendor/ folder when needed (dev, test, and build
    environments). The tooling uses config files (and nested ones in
    dependencies) to populate it. This allows you to use it and have different
    versions of packages in different applications. This is basically the model
    used by PHP, node.js, and many other languages. Let the tooling manage your
    vendor/ folder based on config so you can avoid bad situations.
    This wound nullify what I see as the main value of the vendor
    directory; vendoring code into your repository. If this cannot be done,
    then what is the point of vendor ? Why not just have tools that manage
    GOPATH directly, like https://launchpad.net/godeps ?

    1. Tooling walks through all the dependencies and moves vendor
    dependencies to the top level. This may be opt-in and would need to rely on
    some config (to handle knowledge about versions so you can deal with the
    same package multiple times but different versions).

    This sounds like a better option, but how do you handle conflicts
    between a deps that live in a (flattened) vendor/ folder, and raw, or
    transitively (see below) in GOPATH ?

    github.com/
    /foo/bar/
    /vendor/
    /logger
    /quxx/frob
    /vendor/
    /logger

    If quxx/frob imports "logger" and "foo/bar", which copy of logger is
    "foo/bar" going to link against ? I don't think flattening can solve this.


    Really, the first case it tooling to manage you vendor/ folder to
    avoid the problems. The second it tooling to fix the problems.

    I personally prefer the former because it's an easier problem to
    solve and is generally known because so many other languages tooling
    operate this way. It's familiar and we can learn from their struggles.

    Without tooling to "manage" this it quickly becomes painful.

    Unless you have a better idea (and I'm looking for one).

    - Matt
    On Tuesday, November 3, 2015 at 5:09:53 PM UTC-5, Dave Cheney wrote:

    So here's the problem,

    The way the vendor rule works at the moment is vendor/x is imported
    as "x", but linked as " vendor/x"

    This introduces all the problems of putting version numbers in
    imports, aka gopkg.in. specifically

    * duplicate calls to init routines to register drivers and loggers
    * types that fail type assertion checks.

    However, to do the opposite, vendor/x linking as x would conflict
    with other x's, possibly from GOPATH or other nested, or sibling vendor
    structures.

    This would cause linking failures as symbols wouldn't match, or
    worse, no linking failures and structures being the wrong size at run time.

    Either way is problematic, and I don't see a way to resolve Matt's
    concerns.

    Thanks

    Dave
    On Wed, 4 Nov 2015, 08:59 minux wrote:

    On Tue, Nov 3, 2015 at 3:47 PM, Matt Farina <matt....@gmail.com>
    wrote:
    @rsc did you look at the example I linked off to? When the same
    package is in 2 vendor/ locations. It could be more but the linked example
    only has 2. In case you missed it, there example is at
    https://github.com/mattfarina/golang-broken-vendor

    It has to do with how, in practice, people use the vendor/
    directories. This issue has been seen in the wild.
    It's working as intended. Two vendored copies of the same package
    are not
    the same package (their import paths are different).

    The vendor experiment merely provides a mechanism, the vendor
    policy is
    not and shouldn't be enforced by the go tool.

    --
    You received this message because you are subscribed to the Google
    Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it,
    send an email to golang-dev+...@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google
    Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it,
    send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    Matt Farina
    mattfarina.com

    Go in Practice <http://goinpracticebook.com/> - A book of Recipes for
    the Go programming language.

    Engineered Web <http://engineeredweb.com/> - A blog on cloud computing
    and web technologies.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Minux at Nov 4, 2015 at 12:39 am

    On Tue, Nov 3, 2015 at 6:28 PM, Matt Farina wrote:

    @dave If this is what we have I see two possibilities of ways to use it in
    practice. Something we'll need to teach people.

    1. No one stores their dependences in the vendor/ in their VCS (unless
    you really know what you're doing... an exception). Then you use tooling to
    populate the vendor/ folder when needed (dev, test, and build
    environments). The tooling uses config files (and nested ones in
    dependencies) to populate it. This allows you to use it and have different
    versions of packages in different applications. This is basically the model
    used by PHP, node.js, and many other languages. Let the tooling manage your
    vendor/ folder based on config so you can avoid bad situations.
    2. Tooling walks through all the dependencies and moves vendor
    dependencies to the top level. This may be opt-in and would need to rely on
    some config (to handle knowledge about versions so you can deal with the
    same package multiple times but different versions).

    Really, the first case it tooling to manage you vendor/ folder to avoid
    the problems. The second it tooling to fix the problems.
    The way I see the problem is that, library packages should not vendor
    packages unless
    absolutely necessary. And if they do, they must make sure that the API
    interfaces from
    the vendored packages are not leaked to the outside of the library package.
    (I.e. if you
    make a private copy of net/http in your package, it's fine if its use is
    purely internal to
    the package.)

    Only applications should vendor packages.

    To this end, I actually like the original recommendation of vendoring by
    rewriting import
    paths. At least people know that vendored packages are different from the
    original one.

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matt Farina at Nov 4, 2015 at 12:48 am
    @minux what's an application? For example, Kubernetes is both an
    application and internally there's an SDK that others should import. Docker
    is an application and many people import it to leverage it. They are both.
    It's not always cut and dry.

    I would not expect most people to read the design document for vendor/.
    Instead, I would expect them to think it operates the same way this
    directory does for PHP, node.js, and virtually every other language that
    has something comparable. Most people don't read the docs... especially the
    design docs. They first go with their intuition and experience. Experience
    with other languages has trained a lot of devs how something like this
    should work.
    On Tuesday, November 3, 2015 at 7:39:37 PM UTC-5, minux wrote:


    On Tue, Nov 3, 2015 at 6:28 PM, Matt Farina <matt....@gmail.com
    <javascript:>> wrote:
    @dave If this is what we have I see two possibilities of ways to use it
    in practice. Something we'll need to teach people.

    1. No one stores their dependences in the vendor/ in their VCS
    (unless you really know what you're doing... an exception). Then you use
    tooling to populate the vendor/ folder when needed (dev, test, and build
    environments). The tooling uses config files (and nested ones in
    dependencies) to populate it. This allows you to use it and have different
    versions of packages in different applications. This is basically the model
    used by PHP, node.js, and many other languages. Let the tooling manage your
    vendor/ folder based on config so you can avoid bad situations.
    2. Tooling walks through all the dependencies and moves vendor
    dependencies to the top level. This may be opt-in and would need to rely on
    some config (to handle knowledge about versions so you can deal with the
    same package multiple times but different versions).

    Really, the first case it tooling to manage you vendor/ folder to avoid
    the problems. The second it tooling to fix the problems.
    The way I see the problem is that, library packages should not vendor
    packages unless
    absolutely necessary. And if they do, they must make sure that the API
    interfaces from
    the vendored packages are not leaked to the outside of the library
    package. (I.e. if you
    make a private copy of net/http in your package, it's fine if its use is
    purely internal to
    the package.)

    Only applications should vendor packages.

    To this end, I actually like the original recommendation of vendoring by
    rewriting import
    paths. At least people know that vendored packages are different from the
    original one.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Yongjian Xu at Nov 4, 2015 at 5:27 pm
    I think what minux@ meant by "application" is that it is a binary
    containing a main package. For a library (e.g. as you said it could be
    something common abstracted out as a library from an application), the
    question of how to resolve multiple versions of the same vendored package
    seems unavoidable... (which is a hard problem I think and the original spec
    also chosen to leave it out)
    On Tue, Nov 3, 2015 at 4:48 PM Matt Farina wrote:

    @minux what's an application? For example, Kubernetes is both an
    application and internally there's an SDK that others should import. Docker
    is an application and many people import it to leverage it. They are both.
    It's not always cut and dry.

    I would not expect most people to read the design document for vendor/.
    Instead, I would expect them to think it operates the same way this
    directory does for PHP, node.js, and virtually every other language that
    has something comparable. Most people don't read the docs... especially the
    design docs. They first go with their intuition and experience. Experience
    with other languages has trained a lot of devs how something like this
    should work.
    On Tuesday, November 3, 2015 at 7:39:37 PM UTC-5, minux wrote:

    On Tue, Nov 3, 2015 at 6:28 PM, Matt Farina wrote:

    @dave If this is what we have I see two possibilities of ways to use it
    in practice. Something we'll need to teach people.

    1. No one stores their dependences in the vendor/ in their VCS
    (unless you really know what you're doing... an exception). Then you use
    tooling to populate the vendor/ folder when needed (dev, test, and build
    environments). The tooling uses config files (and nested ones in
    dependencies) to populate it. This allows you to use it and have different
    versions of packages in different applications. This is basically the model
    used by PHP, node.js, and many other languages. Let the tooling manage your
    vendor/ folder based on config so you can avoid bad situations.
    2. Tooling walks through all the dependencies and moves vendor
    dependencies to the top level. This may be opt-in and would need to rely on
    some config (to handle knowledge about versions so you can deal with the
    same package multiple times but different versions).

    Really, the first case it tooling to manage you vendor/ folder to avoid
    the problems. The second it tooling to fix the problems.
    The way I see the problem is that, library packages should not vendor
    packages unless
    absolutely necessary. And if they do, they must make sure that the API
    interfaces from
    the vendored packages are not leaked to the outside of the library
    package. (I.e. if you
    make a private copy of net/http in your package, it's fine if its use is
    purely internal to
    the package.)

    Only applications should vendor packages.

    To this end, I actually like the original recommendation of vendoring by
    rewriting import
    paths. At least people know that vendored packages are different from the
    original one.
    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matt Farina at Nov 4, 2015 at 9:05 pm
    @Jimxu I'm not sure version handling and packages is too hard to solve.
    Every other modern language ecosystem has it solved. That includes Rust,
    PHP, JavaScript, Java, Ruby, Python, Objective-C, Swift, and others. Not
    only do they have this problem solved, they don't run into issues like the
    one I originally outlined here.
    On Wednesday, November 4, 2015 at 12:27:05 PM UTC-5, Jimxu wrote:

    I think what minux@ meant by "application" is that it is a binary
    containing a main package. For a library (e.g. as you said it could be
    something common abstracted out as a library from an application), the
    question of how to resolve multiple versions of the same vendored package
    seems unavoidable... (which is a hard problem I think and the original spec
    also chosen to leave it out)
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Yongjian Xu at Nov 4, 2015 at 9:46 pm

    On Wed, Nov 4, 2015 at 1:05 PM Matt Farina wrote:

    @Jimxu I'm not sure version handling and packages is too hard to solve.
    Every other modern language ecosystem has it solved. That includes Rust,
    PHP, JavaScript, Java, Ruby, Python, Objective-C, Swift, and others. Not
    only do they have this problem solved, they don't run into issues like the
    one I originally outlined here.
    Sure. I was just suspecting that this would be a real issue under the
    current implementation of the Go toolchain. I am not familiar with how
    other languages had solved this issue. Are they just pick one of the copies
    and compile that or their toolchains natively support compile/link multiple
    versions into one binary? Or they rely on some kind of config system input?
    On Wednesday, November 4, 2015 at 12:27:05 PM UTC-5, Jimxu wrote:

    I think what minux@ meant by "application" is that it is a binary
    containing a main package. For a library (e.g. as you said it could be
    something common abstracted out as a library from an application), the
    question of how to resolve multiple versions of the same vendored package
    seems unavoidable... (which is a hard problem I think and the original spec
    also chosen to leave it out)
    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matt Farina at Nov 4, 2015 at 11:12 pm
    @jimxu There are two ways other languages handle this. In some cases they
    can cleanly handle multiple versions. In other cases they flatten the
    dependency tree. In both cases they have tooling that basically does this
    for them. It's mostly automated.

    The timing of your question is great because I just wrote
    http://engineeredweb.com/blog/2015/pkg-mgr-overview/ for an entirely
    different purpose that talks about how the package managers for other
    languages handle packages. Maybe it will be useful here as well.
    On Wednesday, November 4, 2015 at 4:46:48 PM UTC-5, Jimxu wrote:



    On Wed, Nov 4, 2015 at 1:05 PM Matt Farina <matt....@gmail.com
    <javascript:>> wrote:
    @Jimxu I'm not sure version handling and packages is too hard to solve.
    Every other modern language ecosystem has it solved. That includes Rust,
    PHP, JavaScript, Java, Ruby, Python, Objective-C, Swift, and others. Not
    only do they have this problem solved, they don't run into issues like the
    one I originally outlined here.
    Sure. I was just suspecting that this would be a real issue under the
    current implementation of the Go toolchain. I am not familiar with how
    other languages had solved this issue. Are they just pick one of the copies
    and compile that or their toolchains natively support compile/link multiple
    versions into one binary? Or they rely on some kind of config system input?
    On Wednesday, November 4, 2015 at 12:27:05 PM UTC-5, Jimxu wrote:

    I think what minux@ meant by "application" is that it is a binary
    containing a main package. For a library (e.g. as you said it could be
    something common abstracted out as a library from an application), the
    question of how to resolve multiple versions of the same vendored package
    seems unavoidable... (which is a hard problem I think and the original spec
    also chosen to leave it out)
    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Snafex at Nov 5, 2015 at 10:32 am
    This sort of issue around the vendor experiment is one I've been concerned
    with for some time. I'm glad it's finally being discussed. I have a fairly
    specific opinion:
    On Tuesday, November 3, 2015 at 4:39:37 PM UTC-8, minux wrote:

    Only applications should vendor packages.
      ^ This. And I believe a way to do this is to only have the vendor/ pull
    from a location relative to the "main" package of an application (rather
    than relative to the current package). This will prevent impossible diamond
    dependencies (package a vendors package c, package b vendors packages c,
    c.type returned by a.func is incompatible with c.type accepted by b.func),
    I illustrate this issue in a simple application here:
    https://github.com/conslo/vendor_problem. However by leaving the dynamic
    nature of the vendor folder location, one can still share a vendor folder
    among multiple projects (github.com/vendor as a silly example for anything
    from github)

    I don't personally agree with rewriting import paths, and I think the
    community has shown it doesn't either (hence this experiment). And as for
    letting external tooling solve these problems, I feel that goes against the
    idea that anything should be "go get-able", requiring tooling outside
    what's provided by the language to interact with projects seems... silly.
    Other languages solve these problems by having package managers: a
    canonical source with canonical names, and version numbers for packages.
    Neither of these are being provided or enforced, and expecting a tool to do
    some sort of versioning resolution without any sort of in-built version
    identification (semver, anyone?) seems... also silly.

    In addition, the general theme I've seen, including with this vendor
    experiment, seems to assume "a future version of package x will always be
    compatible with a previous version of package x". When you work in a
    totally vendored environment where there's only one version of any package
    at any time (sound familiar?) this works fine; breaking changes are made
    atomically, everyone is compatible with each other, and bad versions are
    fixed and never seen again. However, this is not what open source is like;
    when a package needs to make breaking changes, it has to maintain the old
    version for a while so people have time to transition, people try to pin to
    versions, but we're not all pinned to the same one. There are multiple
    versions (some/many incompatible with each other) of any one dependency in
    use. This is why communities have, mostly, learned to use semantic
    versioning: I need a feature in 1.5, and you need 1.6, so above 1.6 and
    below 2.0 should work for both of us, if we find a version that doesn't
    work there should be a patch version after it that fixes the break. As of
    right now there is not widely accepted mechanism for declaring versions of
    dependencies, just a canonical URL and an expectation that that URL will
    never have changes that break me, there's gopkg.in (which is awesome!), but
    if a package has internal packages it doesn't work unless the developer
    specifically includes it (because, you know, no relative imports ;)). Some
    things enable pinning, but they generally pin to a commit hash, not a
    "version", so any dependency resolution isn't really enabled. Perhaps a
    discussion of versioning as a whole is in order? An endorsed mechanism for
    declaring versions (tags/branches? I know git/mercurial but not bazaar or
    others), and/or an endorsed method of declaring versions of dependencies
    would be incredible (for the later, I actually think a language addition
    for suffixing the version in the import would be awesome: import name
    "path" "version").

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matt Farina at Nov 5, 2015 at 11:46 am
    I think what you're saying is we need:

        1. A place to store dependencies other than the GOPATH for a main package
        2. A way to record what versions of a dependency a package relies on.
        Then the tooling can try to resolve the right one to put in the store for
        #1.

    For anyone wonder why a per application store, rather than the GOPATH,
    that's because different applications will resolve to different versions
    for different applications.

    For versioning there is already a proposal to use SemVer
    at https://github.com/golang/go/issues/12302.

    While I like gopkg.in it's limited to public projects on GitHub. There are
    just so many cases where that won't work that Go needs to support.

    I'm wary of putting the version in the GOPATH because, like you said, it's
    a version range. How would you put ">= 1.2.3, < 2.0.0, != 1.4.5" in the
    import path? If it's just a single version you have a problem of resolution
    in a complex project.

    And, this is why I feel this is stuck.

    Given the way Google works with code I wouldn't ask them to handle #2.
    People who use the tooling should be the ones to write it. As for a #1
    store, is a vendor/ directory in main the best option or should there be
    something better?
    On Thursday, November 5, 2015 at 3:32:54 AM UTC-7, sna...@gmail.com wrote:

    This sort of issue around the vendor experiment is one I've been concerned
    with for some time. I'm glad it's finally being discussed. I have a fairly
    specific opinion:
    On Tuesday, November 3, 2015 at 4:39:37 PM UTC-8, minux wrote:

    Only applications should vendor packages.
    ^ This. And I believe a way to do this is to only have the vendor/ pull
    from a location relative to the "main" package of an application (rather
    than relative to the current package). This will prevent impossible diamond
    dependencies (package a vendors package c, package b vendors packages c,
    c.type returned by a.func is incompatible with c.type accepted by b.func),
    I illustrate this issue in a simple application here:
    https://github.com/conslo/vendor_problem. However by leaving the dynamic
    nature of the vendor folder location, one can still share a vendor folder
    among multiple projects (github.com/vendor as a silly example for
    anything from github)

    I don't personally agree with rewriting import paths, and I think the
    community has shown it doesn't either (hence this experiment). And as for
    letting external tooling solve these problems, I feel that goes against the
    idea that anything should be "go get-able", requiring tooling outside
    what's provided by the language to interact with projects seems... silly.
    Other languages solve these problems by having package managers: a
    canonical source with canonical names, and version numbers for packages.
    Neither of these are being provided or enforced, and expecting a tool to do
    some sort of versioning resolution without any sort of in-built version
    identification (semver, anyone?) seems... also silly.

    In addition, the general theme I've seen, including with this vendor
    experiment, seems to assume "a future version of package x will always be
    compatible with a previous version of package x". When you work in a
    totally vendored environment where there's only one version of any package
    at any time (sound familiar?) this works fine; breaking changes are made
    atomically, everyone is compatible with each other, and bad versions are
    fixed and never seen again. However, this is not what open source is like;
    when a package needs to make breaking changes, it has to maintain the old
    version for a while so people have time to transition, people try to pin to
    versions, but we're not all pinned to the same one. There are multiple
    versions (some/many incompatible with each other) of any one dependency in
    use. This is why communities have, mostly, learned to use semantic
    versioning: I need a feature in 1.5, and you need 1.6, so above 1.6 and
    below 2.0 should work for both of us, if we find a version that doesn't
    work there should be a patch version after it that fixes the break. As of
    right now there is not widely accepted mechanism for declaring versions of
    dependencies, just a canonical URL and an expectation that that URL will
    never have changes that break me, there's gopkg.in (which is awesome!),
    but if a package has internal packages it doesn't work unless the developer
    specifically includes it (because, you know, no relative imports ;)). Some
    things enable pinning, but they generally pin to a commit hash, not a
    "version", so any dependency resolution isn't really enabled. Perhaps a
    discussion of versioning as a whole is in order? An endorsed mechanism for
    declaring versions (tags/branches? I know git/mercurial but not bazaar or
    others), and/or an endorsed method of declaring versions of dependencies
    would be incredible (for the later, I actually think a language addition
    for suffixing the version in the import would be awesome: import name
    "path" "version").
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Snafex at Nov 5, 2015 at 12:18 pm

    On Thursday, November 5, 2015 at 3:46:52 AM UTC-8, Matt Farina wrote:
    I'm wary of putting the version in the GOPATH because, like you said, it's
    a version range. How would you put ">= 1.2.3, < 2.0.0, != 1.4.5" in the
    import path? If it's just a single version you have a problem of resolution
    in a complex project.
    This isn't precisely what I suggested. It wouldn't be in the import path,
    but rather an optional piece of metadata:

    import (
         name "github.com/some/package" ">=1.2.3,<2.0.0,!=1.4.5"
    )

    It could also be a comment on the same line, rather than part of the syntax
    (so it's just for external tooling), though on second thought requiring a
    version specification in every file that uses a package seems excessive.

    I agree with your two pointed list. And personally I feel a vendor/
    director in main is an excellent option (motivated by the acceptance so
    far) for #1. I'll have to disagree with you on #2 a bit though, assuming
    12302 <https://github.com/golang/go/issues/12302> gets accepted in some
    form, I feel it *is* google's responsibility to give us a canonical way to
    deal with these things, solving problems like this early on (rather than
    letting someone make a most-of-the-way solution get widely accepted after
    it's become a large enough problem) I feel are intensely important to the
    health of the ecosystem. I don't expect them to make the tool itself
    (though accepting one into the core tools later would be nice, ie: pip),
    but they should give us something to build on top of, a standardized method
    for declaration for version and dependencies (ie: dist_utils). It can be
    simple, I rather like the Go vendor file spec
    <https://github.com/kardianos/vendor-spec>, and combined with a canonical
    acceptance of semver it would give us everything we need to make our own
    tools.

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matt Farina at Nov 5, 2015 at 12:41 pm
    The problem with having the version in the import path is three fold:

        - It's impractical to update on large projects
        - It's annoying to get right every place you import a package
        - There are times you need to pin or lock to a specific version rather
        than keep to a SemVer range

    These problems have been solved by every other modern programming language
    community in just about the same way. I recently wrote about how they do it
    and why we should care
    <http://engineeredweb.com/blog/2015/pkg-mgr-overview/>.

    At first glance the vendor-spec appears to work. But, in the past year of
    working on Glide <https://github.com/Masterminds/glide> I've encountered a
    number of ways to get packages it doesn't account for. I crafted the pkg
    <https://github.com/mattfarina/pkg> spec which is an alternative in the
    same vein as all the other modern package managers that handles the
    straight forward cases and the more complicated ones we've run into. It can
    be just as simple or move up in complexity as required.

    Package management has been solved by every other modern programming
    language/platform and they do it in similar ways. There are clear patterns.
    On Thursday, November 5, 2015 at 5:18:32 AM UTC-7, sna...@gmail.com wrote:


    On Thursday, November 5, 2015 at 3:46:52 AM UTC-8, Matt Farina wrote:

    I'm wary of putting the version in the GOPATH because, like you said,
    it's a version range. How would you put ">= 1.2.3, < 2.0.0, != 1.4.5" in
    the import path? If it's just a single version you have a problem of
    resolution in a complex project.
    This isn't precisely what I suggested. It wouldn't be in the import path,
    but rather an optional piece of metadata:

    import (
    name "github.com/some/package" ">=1.2.3,<2.0.0,!=1.4.5"
    )

    It could also be a comment on the same line, rather than part of the
    syntax (so it's just for external tooling), though on second thought
    requiring a version specification in every file that uses a package seems
    excessive.

    I agree with your two pointed list. And personally I feel a vendor/
    director in main is an excellent option (motivated by the acceptance so
    far) for #1. I'll have to disagree with you on #2 a bit though, assuming
    12302 <https://github.com/golang/go/issues/12302> gets accepted in some
    form, I feel it *is* google's responsibility to give us a canonical way to
    deal with these things, solving problems like this early on (rather than
    letting someone make a most-of-the-way solution get widely accepted after
    it's become a large enough problem) I feel are intensely important to the
    health of the ecosystem. I don't expect them to make the tool itself
    (though accepting one into the core tools later would be nice, ie: pip),
    but they should give us something to build on top of, a standardized method
    for declaration for version and dependencies (ie: dist_utils). It can be
    simple, I rather like the Go vendor file spec
    <https://github.com/kardianos/vendor-spec>, and combined with a canonical
    acceptance of semver it would give us everything we need to make our own
    tools.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matt Farina at Nov 5, 2015 at 12:45 pm
    Also, I've come up with a hole in the idea of only having a vendor/
    directory on main projects.

    It's a practical issue. If you're working on a library and require external
    dependencies where do you put them? What if they are a different version of
    what's in your GOPATH? Or, there is a fork you need to use in place of the
    canonical package? How do you automate handling that?

    What we really need is the concept of a project in addition to a package.
    The project can have a vendor/ space and there is only one in any given
    project. That means if you depend on another library/project only your top
    level one is included when looking for packages.
    On Thursday, November 5, 2015 at 5:18:32 AM UTC-7, sna...@gmail.com wrote:


    On Thursday, November 5, 2015 at 3:46:52 AM UTC-8, Matt Farina wrote:

    I'm wary of putting the version in the GOPATH because, like you said,
    it's a version range. How would you put ">= 1.2.3, < 2.0.0, != 1.4.5" in
    the import path? If it's just a single version you have a problem of
    resolution in a complex project.
    This isn't precisely what I suggested. It wouldn't be in the import path,
    but rather an optional piece of metadata:

    import (
    name "github.com/some/package" ">=1.2.3,<2.0.0,!=1.4.5"
    )

    It could also be a comment on the same line, rather than part of the
    syntax (so it's just for external tooling), though on second thought
    requiring a version specification in every file that uses a package seems
    excessive.

    I agree with your two pointed list. And personally I feel a vendor/
    director in main is an excellent option (motivated by the acceptance so
    far) for #1. I'll have to disagree with you on #2 a bit though, assuming
    12302 <https://github.com/golang/go/issues/12302> gets accepted in some
    form, I feel it *is* google's responsibility to give us a canonical way to
    deal with these things, solving problems like this early on (rather than
    letting someone make a most-of-the-way solution get widely accepted after
    it's become a large enough problem) I feel are intensely important to the
    health of the ecosystem. I don't expect them to make the tool itself
    (though accepting one into the core tools later would be nice, ie: pip),
    but they should give us something to build on top of, a standardized method
    for declaration for version and dependencies (ie: dist_utils). It can be
    simple, I rather like the Go vendor file spec
    <https://github.com/kardianos/vendor-spec>, and combined with a canonical
    acceptance of semver it would give us everything we need to make our own
    tools.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Travis Johnson at Nov 5, 2015 at 1:10 pm
    Your issue was addressed earlier, but I didn't quote it: libraries
    shouldn't vendor dependencies. If you're working on the library as part of
    a project, that projects vendor/ directory will suffice, if you're just
    working on the library itself, then your testing setup can be utilized, or
    the vendor/ directory could also be allowed for running tests (though this
    ambiguity between tests and use would concern me), or this could bring up a
    point that we should work in virtualenv-style environments instead of
    single global one.

    But I feel those are problems we can rely on external tooling to solve,
    it'd be nice if we all used the same ones, but it doesn't actually harm
    much if different developers use different ones, so experimentation and a
    variety of tools is ok.

    Whereas if everyone starts using differing methods of version and
    dependency declaration, it could make things quite difficult: my package
    manager uses the vendor-spec file, but I want to use a library that expects
    your pkg file format. So using different tools is ok, so long as the method
    by which they communicate is common, this is why I feel the
    decision/proposal on how this is done needs to come from "on high" as it
    were, because during the time spent deciding communally what is best will
    be slow and frustrating as we have many packages using (or worse, not
    using) many different methods of versioning/dependency declaration, and
    several tools develop that are incompatible.

    I agree with your synopsis of my import suggestion, it was an off-the-top
    suggestion to get things going. I pointed to the vendor-spec as a launching
    point, agree with a lot of the decisions in your package spec, both could
    be utilized as good examples.
    On Thu, Nov 5, 2015 at 4:45 AM Matt Farina wrote:

    Also, I've come up with a hole in the idea of only having a vendor/
    directory on main projects.

    It's a practical issue. If you're working on a library and require
    external dependencies where do you put them? What if they are a different
    version of what's in your GOPATH? Or, there is a fork you need to use in
    place of the canonical package? How do you automate handling that?

    What we really need is the concept of a project in addition to a package.
    The project can have a vendor/ space and there is only one in any given
    project. That means if you depend on another library/project only your top
    level one is included when looking for packages.

    On Thursday, November 5, 2015 at 5:18:32 AM UTC-7, sna...@gmail.com wrote:


    On Thursday, November 5, 2015 at 3:46:52 AM UTC-8, Matt Farina wrote:

    I'm wary of putting the version in the GOPATH because, like you said,
    it's a version range. How would you put ">= 1.2.3, < 2.0.0, != 1.4.5" in
    the import path? If it's just a single version you have a problem of
    resolution in a complex project.
    This isn't precisely what I suggested. It wouldn't be in the import path,
    but rather an optional piece of metadata:

    import (
    name "github.com/some/package" ">=1.2.3,<2.0.0,!=1.4.5"
    )

    It could also be a comment on the same line, rather than part of the
    syntax (so it's just for external tooling), though on second thought
    requiring a version specification in every file that uses a package seems
    excessive.

    I agree with your two pointed list. And personally I feel a vendor/
    director in main is an excellent option (motivated by the acceptance so
    far) for #1. I'll have to disagree with you on #2 a bit though, assuming
    12302 <https://github.com/golang/go/issues/12302> gets accepted in some
    form, I feel it *is* google's responsibility to give us a canonical way to
    deal with these things, solving problems like this early on (rather than
    letting someone make a most-of-the-way solution get widely accepted after
    it's become a large enough problem) I feel are intensely important to the
    health of the ecosystem. I don't expect them to make the tool itself
    (though accepting one into the core tools later would be nice, ie: pip),
    but they should give us something to build on top of, a standardized method
    for declaration for version and dependencies (ie: dist_utils). It can be
    simple, I rather like the Go vendor file spec
    <https://github.com/kardianos/vendor-spec>, and combined with a
    canonical acceptance of semver it would give us everything we need to make
    our own tools.
    --
    -Travis
    617-549-3856

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matt Farina at Nov 5, 2015 at 1:34 pm
    @Travis That solution was my original thought and that's how Glide operates
    in practice. IIRC, the other tools do as well. But, that's convention laid
    on top of a feature. Many developers don't read the docs and don't follow
    conventions. Let me illustrate with two examples,

        1. I've run into cases where nested vendor/ directories exist and people
        blame the tools rather than understand the process
        2. Where people don't understand why, even when given examples, you
        should only have one vendor directory. So, they don't follow it and share
        that.

    Have you read the book "JavaScript The Good Parts"? If you haven't, it
    describes using a subset of JavaScript and avoiding others. It appears this
    is building that same kind of thing in Go. Shouldn't we solve this before
    the experiment is turned on for everyone? So it's not just a convention.

    Working on Glide we now support 3 spec formats. I see more coming in the
    not too distant future. This is complicated. I believe a spec needs to be
    use case driven rather than following the opinions of a handful of
    developers. We have package management use cases and we need a spec that
    contains enough details for tools to implement them. I started that at
    https://github.com/mattfarina/pkg/tree/master/use_cases and I think I need
    to revisit it with what I've learned.

    Before we have any kind of spec we need to get the SemVer proposal in. It's
    going to need to be a multi-step process to get there.
    On Thursday, November 5, 2015 at 6:10:57 AM UTC-7, Travis Johnson wrote:

    Your issue was addressed earlier, but I didn't quote it: libraries
    shouldn't vendor dependencies. If you're working on the library as part of
    a project, that projects vendor/ directory will suffice, if you're just
    working on the library itself, then your testing setup can be utilized, or
    the vendor/ directory could also be allowed for running tests (though this
    ambiguity between tests and use would concern me), or this could bring up a
    point that we should work in virtualenv-style environments instead of
    single global one.

    But I feel those are problems we can rely on external tooling to solve,
    it'd be nice if we all used the same ones, but it doesn't actually harm
    much if different developers use different ones, so experimentation and a
    variety of tools is ok.

    Whereas if everyone starts using differing methods of version and
    dependency declaration, it could make things quite difficult: my package
    manager uses the vendor-spec file, but I want to use a library that expects
    your pkg file format. So using different tools is ok, so long as the method
    by which they communicate is common, this is why I feel the
    decision/proposal on how this is done needs to come from "on high" as it
    were, because during the time spent deciding communally what is best will
    be slow and frustrating as we have many packages using (or worse, not
    using) many different methods of versioning/dependency declaration, and
    several tools develop that are incompatible.

    I agree with your synopsis of my import suggestion, it was an off-the-top
    suggestion to get things going. I pointed to the vendor-spec as a launching
    point, agree with a lot of the decisions in your package spec, both could
    be utilized as good examples.

    On Thu, Nov 5, 2015 at 4:45 AM Matt Farina <matt....@gmail.com
    <javascript:>> wrote:
    Also, I've come up with a hole in the idea of only having a vendor/
    directory on main projects.

    It's a practical issue. If you're working on a library and require
    external dependencies where do you put them? What if they are a different
    version of what's in your GOPATH? Or, there is a fork you need to use in
    place of the canonical package? How do you automate handling that?

    What we really need is the concept of a project in addition to a package.
    The project can have a vendor/ space and there is only one in any given
    project. That means if you depend on another library/project only your top
    level one is included when looking for packages.


    On Thursday, November 5, 2015 at 5:18:32 AM UTC-7, sna...@gmail.com
    wrote:
    On Thursday, November 5, 2015 at 3:46:52 AM UTC-8, Matt Farina wrote:

    I'm wary of putting the version in the GOPATH because, like you said,
    it's a version range. How would you put ">= 1.2.3, < 2.0.0, != 1.4.5" in
    the import path? If it's just a single version you have a problem of
    resolution in a complex project.
    This isn't precisely what I suggested. It wouldn't be in the import
    path, but rather an optional piece of metadata:

    import (
    name "github.com/some/package" ">=1.2.3,<2.0.0,!=1.4.5"
    )

    It could also be a comment on the same line, rather than part of the
    syntax (so it's just for external tooling), though on second thought
    requiring a version specification in every file that uses a package seems
    excessive.

    I agree with your two pointed list. And personally I feel a vendor/
    director in main is an excellent option (motivated by the acceptance so
    far) for #1. I'll have to disagree with you on #2 a bit though, assuming
    12302 <https://github.com/golang/go/issues/12302> gets accepted in some
    form, I feel it *is* google's responsibility to give us a canonical way to
    deal with these things, solving problems like this early on (rather than
    letting someone make a most-of-the-way solution get widely accepted after
    it's become a large enough problem) I feel are intensely important to the
    health of the ecosystem. I don't expect them to make the tool itself
    (though accepting one into the core tools later would be nice, ie: pip),
    but they should give us something to build on top of, a standardized method
    for declaration for version and dependencies (ie: dist_utils). It can be
    simple, I rather like the Go vendor file spec
    <https://github.com/kardianos/vendor-spec>, and combined with a
    canonical acceptance of semver it would give us everything we need to make
    our own tools.
    --
    -Travis
    617-549-3856
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Snafex at Nov 5, 2015 at 2:15 pm

    On Thursday, November 5, 2015 at 5:34:55 AM UTC-8, Matt Farina wrote:
    Shouldn't we solve this before the experiment is turned on for everyone?
    So it's not just a convention.
    Yes yes yes. This is one of the points I'm trying to make; restricting the
    vendor/ directory to only "main" packages was my attempt at a fix, and I
    meant by changing out the vendor experiment works, not just by convention.

    Before we have any kind of spec we need to get the SemVer proposal in. It's
    going to need to be a multi-step process to get there.
    Yep. But I believe the plan was for the vendor experiment to be on by
    default in 1.6, and I think this thread has done a decent job at
    highlighting that this should not be the case.

    I believe a spec needs to be use case driven rather than following the
    opinions of a handful of developers.
    Agreed. The proposal for a spec should come from someone who's dealt with
    these sorts of problems, and should draw heavily from existing solutions
    for other languages. But until some central body with enough attention and
    pull comes out and says "this is what you should use
    <http://europa.eu/rapid/press-release_IP-10-1776_en.htm?locale=en>" then
    we'll have to keep dealing with a slew of competing standards
    <https://xkcd.com/927/>. Once we have an endorsed standard we can iterate
    on it for improvement, and it could be designed with extensions in mind so
    use cases not 100% covered could be supported by tools until if/when it
    gets moved into the spec.

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Russ Cox at Nov 5, 2015 at 8:04 pm
    This is a good discussion to have, but please keep in mind that it's all
    about how to use the vendor/ behavior. This is going to be decided by
    convention and 3rd-party vendor-management tools, not by restrictions in
    the go command itself. The way you find it natural to use vendor/ and the
    way another person finds it natural to use vendor/ may be different, both
    can be valid, and we don't want to force one or the other.

    Thanks.
    Russ



    On Thu, Nov 5, 2015 at 9:15 AM, wrote:

    On Thursday, November 5, 2015 at 5:34:55 AM UTC-8, Matt Farina wrote:

    Shouldn't we solve this before the experiment is turned on for everyone?
    So it's not just a convention.
    Yes yes yes. This is one of the points I'm trying to make; restricting the
    vendor/ directory to only "main" packages was my attempt at a fix, and I
    meant by changing out the vendor experiment works, not just by convention.

    Before we have any kind of spec we need to get the SemVer proposal in.
    It's going to need to be a multi-step process to get there.
    Yep. But I believe the plan was for the vendor experiment to be on by
    default in 1.6, and I think this thread has done a decent job at
    highlighting that this should not be the case.

    I believe a spec needs to be use case driven rather than following the
    opinions of a handful of developers.
    Agreed. The proposal for a spec should come from someone who's dealt with
    these sorts of problems, and should draw heavily from existing solutions
    for other languages. But until some central body with enough attention and
    pull comes out and says "this is what you should use
    <http://europa.eu/rapid/press-release_IP-10-1776_en.htm?locale=en>" then
    we'll have to keep dealing with a slew of competing standards
    <https://xkcd.com/927/>. Once we have an endorsed standard we can iterate
    on it for improvement, and it could be designed with extensions in mind so
    use cases not 100% covered could be supported by tools until if/when it
    gets moved into the spec.

    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Snafex at Nov 11, 2015 at 8:25 pm
    I feel like this discussion has pointed out a problem with using the
    GO15VENDOREXPERIMENT. While I agree that 3rd-party vendor-management tools
    could solve this, if the experiment becomes enabled by default (as is
    planned for 1.6) then some interactions with packages that currently work
    (because the vendor is off by default, so everything uses my global
    versions instead of the vendored one), won't anymore.

    Packages I could (and presently do) previously use with only the core tools
    will become only usable if I use a 3rd-party tool.

    I was also trying to make the point that the lack of agreement in how to
    declare dependency versions impedes the creation of such a tool. As Matt
    said, he's had to implement reading 3 different types of requirements files.

    Is this something we're ok with? As previously stated, people don't read
    design docs, or follow directions, so simply deciding how to use vendor by
    convention won't solve the problem. Are we ok with giving the ecosystem and
    on-by-default mechanism that will force people downstream to use a
    3rd-party tool?
    On Thursday, November 5, 2015 at 12:04:24 PM UTC-8, rsc wrote:

    This is a good discussion to have, but please keep in mind that it's all
    about how to use the vendor/ behavior. This is going to be decided by
    convention and 3rd-party vendor-management tools, not by restrictions in
    the go command itself. The way you find it natural to use vendor/ and the
    way another person finds it natural to use vendor/ may be different, both
    can be valid, and we don't want to force one or the other.

    Thanks.
    Russ



    On Thu, Nov 5, 2015 at 9:15 AM, <sna...@gmail.com <javascript:>> wrote:

    On Thursday, November 5, 2015 at 5:34:55 AM UTC-8, Matt Farina wrote:

    Shouldn't we solve this before the experiment is turned on for everyone?
    So it's not just a convention.
    Yes yes yes. This is one of the points I'm trying to make; restricting
    the vendor/ directory to only "main" packages was my attempt at a fix, and
    I meant by changing out the vendor experiment works, not just by convention.

    Before we have any kind of spec we need to get the SemVer proposal in.
    It's going to need to be a multi-step process to get there.
    Yep. But I believe the plan was for the vendor experiment to be on by
    default in 1.6, and I think this thread has done a decent job at
    highlighting that this should not be the case.

    I believe a spec needs to be use case driven rather than following the
    opinions of a handful of developers.
    Agreed. The proposal for a spec should come from someone who's dealt with
    these sorts of problems, and should draw heavily from existing solutions
    for other languages. But until some central body with enough attention and
    pull comes out and says "this is what you should use
    <http://europa.eu/rapid/press-release_IP-10-1776_en.htm?locale=en>" then
    we'll have to keep dealing with a slew of competing standards
    <https://xkcd.com/927/>. Once we have an endorsed standard we can
    iterate on it for improvement, and it could be designed with extensions in
    mind so use cases not 100% covered could be supported by tools until
    if/when it gets moved into the spec.

    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Kyle Wolfe at Nov 11, 2015 at 9:20 pm

    On Wednesday, November 11, 2015 at 3:25:31 PM UTC-5, sna...@gmail.com wrote:
    I feel like this discussion has pointed out a problem with using the
    GO15VENDOREXPERIMENT. While I agree that 3rd-party vendor-management tools
    could solve this, if the experiment becomes enabled by default (as is
    planned for 1.6) then some interactions with packages that currently work
    (because the vendor is off by default, so everything uses my global
    versions instead of the vendored one), won't anymore.

    Packages I could (and presently do) previously use with only the core
    tools will become only usable if I use a 3rd-party tool.

    I was also trying to make the point that the lack of agreement in how to
    declare dependency versions impedes the creation of such a tool. As Matt
    said, he's had to implement reading 3 different types of requirements files.

    Is this something we're ok with? As previously stated, people don't read
    design docs, or follow directions, so simply deciding how to use vendor by
    convention won't solve the problem. Are we ok with giving the ecosystem and
    on-by-default mechanism that will force people downstream to use a
    3rd-party tool?

    I would say adding in a warning during build -v to show that a single
    import path is being resolved to two different real paths in the same
    project may be a helpful start to this issue. What if a new rule were added
    in to use the lowest level vendor path for the project? In the case of your
    test project, vendor/a.go would be used in place of vendor/b/vendor/a.go.
    This is definitely not something I'd suggest doing without the user knowing
    about it during build, though.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at Nov 11, 2015 at 9:21 pm
    If the argument is that people don't read design docs, or follow
    directions, then it seems unlikely they will heed a warning.
    On Thu, Nov 12, 2015 at 8:20 AM, Kyle Wolfe wrote:
    On Wednesday, November 11, 2015 at 3:25:31 PM UTC-5, sna...@gmail.com wrote:

    I feel like this discussion has pointed out a problem with using the
    GO15VENDOREXPERIMENT. While I agree that 3rd-party vendor-management tools
    could solve this, if the experiment becomes enabled by default (as is
    planned for 1.6) then some interactions with packages that currently work
    (because the vendor is off by default, so everything uses my global versions
    instead of the vendored one), won't anymore.

    Packages I could (and presently do) previously use with only the core
    tools will become only usable if I use a 3rd-party tool.

    I was also trying to make the point that the lack of agreement in how to
    declare dependency versions impedes the creation of such a tool. As Matt
    said, he's had to implement reading 3 different types of requirements files.

    Is this something we're ok with? As previously stated, people don't read
    design docs, or follow directions, so simply deciding how to use vendor by
    convention won't solve the problem. Are we ok with giving the ecosystem and
    on-by-default mechanism that will force people downstream to use a 3rd-party
    tool?
    I would say adding in a warning during build -v to show that a single import
    path is being resolved to two different real paths in the same project may
    be a helpful start to this issue. What if a new rule were added in to use
    the lowest level vendor path for the project? In the case of your test
    project, vendor/a.go would be used in place of vendor/b/vendor/a.go. This is
    definitely not something I'd suggest doing without the user knowing about it
    during build, though.
    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Kyle Wolfe at Nov 11, 2015 at 9:26 pm

    On Wednesday, November 11, 2015 at 4:21:43 PM UTC-5, Dave Cheney wrote:
    If the argument is that people don't read design docs, or follow
    directions, then it seems unlikely they will heed a warning.
      So keep things hidden behind the scenes, then? While I agree, people
    should do their homework, there's no reason that things like this should
    not be presented to them.

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniel Theophanes at Nov 11, 2015 at 10:36 pm
    Kyle, snafex,

    Now that go1.5 has been out for a while, what projects have you used it on
    and what tools did you use? Was your experience confusing or troubling and
    if so, what aspects were?

    Thanks, -Daniel

    On Wed, Nov 11, 2015 at 1:26 PM Kyle Wolfe wrote:

    On Wednesday, November 11, 2015 at 4:21:43 PM UTC-5, Dave Cheney wrote:

    If the argument is that people don't read design docs, or follow
    directions, then it seems unlikely they will heed a warning.
    So keep things hidden behind the scenes, then? While I agree, people
    should do their homework, there's no reason that things like this should
    not be presented to them.

    --
    You received this message because you are subscribed to a topic in the
    Google Groups "golang-dev" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/golang-dev/WebP4dLV1b0/unsubscribe.
    To unsubscribe from this group and all its topics, send an email to
    golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Travis Johnson at Nov 11, 2015 at 11:24 pm
    I use vendor/ for my "main" packages, but never with libraries, I do this
    by modifying my GOPATH to include the vendor/ directory of whatever project
    I'm presently working on.

    If I try to enable the vendor experiment instead of modifying my GOPATH, my
    projects break, because I use other people's libraries who also vendor with
    the vendor/ directory, and diamond dependencies immediately break the build.
    On Wed, Nov 11, 2015 at 2:36 PM Daniel Theophanes wrote:

    Kyle, snafex,

    Now that go1.5 has been out for a while, what projects have you used it on
    and what tools did you use? Was your experience confusing or troubling and
    if so, what aspects were?

    Thanks, -Daniel

    On Wed, Nov 11, 2015 at 1:26 PM Kyle Wolfe wrote:

    On Wednesday, November 11, 2015 at 4:21:43 PM UTC-5, Dave Cheney wrote:

    If the argument is that people don't read design docs, or follow
    directions, then it seems unlikely they will heed a warning.
    So keep things hidden behind the scenes, then? While I agree, people
    should do their homework, there's no reason that things like this should
    not be presented to them.

    --
    You received this message because you are subscribed to a topic in the
    Google Groups "golang-dev" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/golang-dev/WebP4dLV1b0/unsubscribe.
    To unsubscribe from this group and all its topics, send an email to
    golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    -Travis
    617-549-3856

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniel Theophanes at Nov 11, 2015 at 11:56 pm
    Hi Travis,

    May I recommend using a tool to resolve that? If you used a tool like
    github.com/kardianos/govendor, it would take care of those diamond
    dependencies. I think other tools would do that as well.

    -Daniel

    On Wed, Nov 11, 2015 at 3:24 PM Travis Johnson wrote:

    I use vendor/ for my "main" packages, but never with libraries, I do this
    by modifying my GOPATH to include the vendor/ directory of whatever project
    I'm presently working on.

    If I try to enable the vendor experiment instead of modifying my GOPATH,
    my projects break, because I use other people's libraries who also vendor
    with the vendor/ directory, and diamond dependencies immediately break the
    build.
    On Wed, Nov 11, 2015 at 2:36 PM Daniel Theophanes wrote:

    Kyle, snafex,

    Now that go1.5 has been out for a while, what projects have you used it
    on and what tools did you use? Was your experience confusing or troubling
    and if so, what aspects were?

    Thanks, -Daniel


    On Wed, Nov 11, 2015 at 1:26 PM Kyle Wolfe <kyle.a.wolfe@gmail.com>
    wrote:
    On Wednesday, November 11, 2015 at 4:21:43 PM UTC-5, Dave Cheney wrote:

    If the argument is that people don't read design docs, or follow
    directions, then it seems unlikely they will heed a warning.
    So keep things hidden behind the scenes, then? While I agree, people
    should do their homework, there's no reason that things like this should
    not be presented to them.

    --
    You received this message because you are subscribed to a topic in the
    Google Groups "golang-dev" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/golang-dev/WebP4dLV1b0/unsubscribe.
    To unsubscribe from this group and all its topics, send an email to
    golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    -Travis
    617-549-3856
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Travis Johnson at Nov 12, 2015 at 2:51 am
    Lovely of you to advertise your tool, and I'm sure it works great, but it
    doesn't solve the problem. So I'll restate something I said earlier:

    Are we ok with *requiring* users to use a 3rd party tool for things made
    only using the standard tools?

    As it stands, if I am only using the standard toolchains (single global
    GOPATH, no manipulation etc), and things work, and I "go get" a library
    that uses the vendor/ directory (but no custom build process), I will be
    *required* to use a 3rd party tool (such as govendor) as of Go 1.6.
    On Wed, Nov 11, 2015 at 3:56 PM Daniel Theophanes wrote:

    Hi Travis,

    May I recommend using a tool to resolve that? If you used a tool like
    github.com/kardianos/govendor, it would take care of those diamond
    dependencies. I think other tools would do that as well.

    -Daniel

    On Wed, Nov 11, 2015 at 3:24 PM Travis Johnson wrote:

    I use vendor/ for my "main" packages, but never with libraries, I do this
    by modifying my GOPATH to include the vendor/ directory of whatever project
    I'm presently working on.

    If I try to enable the vendor experiment instead of modifying my GOPATH,
    my projects break, because I use other people's libraries who also vendor
    with the vendor/ directory, and diamond dependencies immediately break the
    build.

    On Wed, Nov 11, 2015 at 2:36 PM Daniel Theophanes <kardianos@gmail.com>
    wrote:
    Kyle, snafex,

    Now that go1.5 has been out for a while, what projects have you used it
    on and what tools did you use? Was your experience confusing or troubling
    and if so, what aspects were?

    Thanks, -Daniel


    On Wed, Nov 11, 2015 at 1:26 PM Kyle Wolfe <kyle.a.wolfe@gmail.com>
    wrote:
    On Wednesday, November 11, 2015 at 4:21:43 PM UTC-5, Dave Cheney wrote:

    If the argument is that people don't read design docs, or follow
    directions, then it seems unlikely they will heed a warning.
    So keep things hidden behind the scenes, then? While I agree, people
    should do their homework, there's no reason that things like this should
    not be presented to them.

    --
    You received this message because you are subscribed to a topic in the
    Google Groups "golang-dev" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/golang-dev/WebP4dLV1b0/unsubscribe.
    To unsubscribe from this group and all its topics, send an email to
    golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    -Travis
    617-549-3856
    --
    -Travis
    617-549-3856

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniel Theophanes at Nov 12, 2015 at 2:55 am
    Unless go get fetches into the vendor folder, which I don't think it does,
    nothing really changes unless you copy it in, probably with a tool.
    On Wed, Nov 11, 2015, 18:51 Travis Johnson wrote:

    Lovely of you to advertise your tool, and I'm sure it works great, but it
    doesn't solve the problem. So I'll restate something I said earlier:

    Are we ok with *requiring* users to use a 3rd party tool for things made
    only using the standard tools?

    As it stands, if I am only using the standard toolchains (single global
    GOPATH, no manipulation etc), and things work, and I "go get" a library
    that uses the vendor/ directory (but no custom build process), I will be
    *required* to use a 3rd party tool (such as govendor) as of Go 1.6.
    On Wed, Nov 11, 2015 at 3:56 PM Daniel Theophanes wrote:

    Hi Travis,

    May I recommend using a tool to resolve that? If you used a tool like
    github.com/kardianos/govendor, it would take care of those diamond
    dependencies. I think other tools would do that as well.

    -Daniel

    On Wed, Nov 11, 2015 at 3:24 PM Travis Johnson wrote:

    I use vendor/ for my "main" packages, but never with libraries, I do
    this by modifying my GOPATH to include the vendor/ directory of whatever
    project I'm presently working on.

    If I try to enable the vendor experiment instead of modifying my GOPATH,
    my projects break, because I use other people's libraries who also vendor
    with the vendor/ directory, and diamond dependencies immediately break the
    build.

    On Wed, Nov 11, 2015 at 2:36 PM Daniel Theophanes <kardianos@gmail.com>
    wrote:
    Kyle, snafex,

    Now that go1.5 has been out for a while, what projects have you used it
    on and what tools did you use? Was your experience confusing or troubling
    and if so, what aspects were?

    Thanks, -Daniel


    On Wed, Nov 11, 2015 at 1:26 PM Kyle Wolfe <kyle.a.wolfe@gmail.com>
    wrote:
    On Wednesday, November 11, 2015 at 4:21:43 PM UTC-5, Dave Cheney wrote:

    If the argument is that people don't read design docs, or follow
    directions, then it seems unlikely they will heed a warning.
    So keep things hidden behind the scenes, then? While I agree, people
    should do their homework, there's no reason that things like this should
    not be presented to them.

    --
    You received this message because you are subscribed to a topic in the
    Google Groups "golang-dev" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/golang-dev/WebP4dLV1b0/unsubscribe.
    To unsubscribe from this group and all its topics, send an email to
    golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    -Travis
    617-549-3856
    --
    -Travis
    617-549-3856
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Travis Johnson at Nov 12, 2015 at 2:58 am
    It doesn't, but it does download the vendor/ folder as well, and since the
    experiment is scheduled to be on by default in 1.6, "go build" will use
    that libraries vendor/ directory for imports.
    On Wed, Nov 11, 2015 at 6:55 PM Daniel Theophanes wrote:

    Unless go get fetches into the vendor folder, which I don't think it does,
    nothing really changes unless you copy it in, probably with a tool.
    On Wed, Nov 11, 2015, 18:51 Travis Johnson wrote:

    Lovely of you to advertise your tool, and I'm sure it works great, but it
    doesn't solve the problem. So I'll restate something I said earlier:

    Are we ok with *requiring* users to use a 3rd party tool for things made
    only using the standard tools?

    As it stands, if I am only using the standard toolchains (single global
    GOPATH, no manipulation etc), and things work, and I "go get" a library
    that uses the vendor/ directory (but no custom build process), I will be
    *required* to use a 3rd party tool (such as govendor) as of Go 1.6.

    On Wed, Nov 11, 2015 at 3:56 PM Daniel Theophanes <kardianos@gmail.com>
    wrote:
    Hi Travis,

    May I recommend using a tool to resolve that? If you used a tool like
    github.com/kardianos/govendor, it would take care of those diamond
    dependencies. I think other tools would do that as well.

    -Daniel

    On Wed, Nov 11, 2015 at 3:24 PM Travis Johnson wrote:

    I use vendor/ for my "main" packages, but never with libraries, I do
    this by modifying my GOPATH to include the vendor/ directory of whatever
    project I'm presently working on.

    If I try to enable the vendor experiment instead of modifying my
    GOPATH, my projects break, because I use other people's libraries who also
    vendor with the vendor/ directory, and diamond dependencies immediately
    break the build.

    On Wed, Nov 11, 2015 at 2:36 PM Daniel Theophanes <kardianos@gmail.com>
    wrote:
    Kyle, snafex,

    Now that go1.5 has been out for a while, what projects have you used
    it on and what tools did you use? Was your experience confusing or
    troubling and if so, what aspects were?

    Thanks, -Daniel


    On Wed, Nov 11, 2015 at 1:26 PM Kyle Wolfe <kyle.a.wolfe@gmail.com>
    wrote:
    On Wednesday, November 11, 2015 at 4:21:43 PM UTC-5, Dave Cheney
    wrote:
    If the argument is that people don't read design docs, or follow
    directions, then it seems unlikely they will heed a warning.
    So keep things hidden behind the scenes, then? While I agree, people
    should do their homework, there's no reason that things like this should
    not be presented to them.

    --
    You received this message because you are subscribed to a topic in
    the Google Groups "golang-dev" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/golang-dev/WebP4dLV1b0/unsubscribe.
    To unsubscribe from this group and all its topics, send an email to
    golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    -Travis
    617-549-3856
    --
    -Travis
    617-549-3856
    --
    -Travis
    617-549-3856

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matt Farina at Nov 12, 2015 at 3:06 am
    Daniel,

    The issue is with the default behavior. Defaults are important because a
    lot of people are only going to do that. In this case that's what we're
    getting at. The default behavior for a feature that's going to be on by
    default in 1.6 provides ample opportunity to create build breaking
    situations.

    This isn't good for developer experience. It's going to cause a lot of
    headaches.

    The fact that we talk about 3rd party tools to correct a common problem
    brought on by a default behavior isn't good. I'm not naming tools because
    it's not about using a tool to correct the behavior. We need to shine a
    light on the problems from the default behavior.

    Matt
    On Wednesday, November 11, 2015 at 9:58:02 PM UTC-5, Travis Johnson wrote:

    It doesn't, but it does download the vendor/ folder as well, and since the
    experiment is scheduled to be on by default in 1.6, "go build" will use
    that libraries vendor/ directory for imports.

    On Wed, Nov 11, 2015 at 6:55 PM Daniel Theophanes <kard...@gmail.com
    <javascript:>> wrote:
    Unless go get fetches into the vendor folder, which I don't think it
    does, nothing really changes unless you copy it in, probably with a tool.

    On Wed, Nov 11, 2015, 18:51 Travis Johnson <sna...@gmail.com
    <javascript:>> wrote:
    Lovely of you to advertise your tool, and I'm sure it works great, but
    it doesn't solve the problem. So I'll restate something I said earlier:

    Are we ok with *requiring* users to use a 3rd party tool for things made
    only using the standard tools?

    As it stands, if I am only using the standard toolchains (single global
    GOPATH, no manipulation etc), and things work, and I "go get" a library
    that uses the vendor/ directory (but no custom build process), I will be
    *required* to use a 3rd party tool (such as govendor) as of Go 1.6.

    On Wed, Nov 11, 2015 at 3:56 PM Daniel Theophanes <kard...@gmail.com
    <javascript:>> wrote:
    Hi Travis,

    May I recommend using a tool to resolve that? If you used a tool like
    github.com/kardianos/govendor, it would take care of those diamond
    dependencies. I think other tools would do that as well.

    -Daniel


    On Wed, Nov 11, 2015 at 3:24 PM Travis Johnson <sna...@gmail.com
    <javascript:>> wrote:
    I use vendor/ for my "main" packages, but never with libraries, I do
    this by modifying my GOPATH to include the vendor/ directory of whatever
    project I'm presently working on.

    If I try to enable the vendor experiment instead of modifying my
    GOPATH, my projects break, because I use other people's libraries who also
    vendor with the vendor/ directory, and diamond dependencies immediately
    break the build.

    On Wed, Nov 11, 2015 at 2:36 PM Daniel Theophanes <kard...@gmail.com
    <javascript:>> wrote:
    Kyle, snafex,

    Now that go1.5 has been out for a while, what projects have you used
    it on and what tools did you use? Was your experience confusing or
    troubling and if so, what aspects were?

    Thanks, -Daniel


    On Wed, Nov 11, 2015 at 1:26 PM Kyle Wolfe <kyle.a...@gmail.com
    <javascript:>> wrote:
    On Wednesday, November 11, 2015 at 4:21:43 PM UTC-5, Dave Cheney
    wrote:
    If the argument is that people don't read design docs, or follow
    directions, then it seems unlikely they will heed a warning.
    So keep things hidden behind the scenes, then? While I agree,
    people should do their homework, there's no reason that things like this
    should not be presented to them.

    --
    You received this message because you are subscribed to a topic in
    the Google Groups "golang-dev" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/golang-dev/WebP4dLV1b0/unsubscribe
    .
    To unsubscribe from this group and all its topics, send an email to
    golang-dev+...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/d/optout.
    --
    -Travis
    617-549-3856
    --
    -Travis
    617-549-3856
    --
    -Travis
    617-549-3856
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniel Theophanes at Nov 12, 2015 at 3:34 am
    Hi Matt and Travis,

    I think what you are saying is that this allows packages to vendor their
    own dependencies. Thus if you go get a package into GOPATH that has a
    vendor folder, you could run into duplicates or conflicts.

    For example, package "dbmigrate/vendor/pgx" is imported by a main package
    that already imports "pgx". This would cause a runtime panic when they both
    try to register.

    Is this the concern you are raising? -Daniel
    On Wed, Nov 11, 2015 at 7:06 PM Matt Farina wrote:

    Daniel,

    The issue is with the default behavior. Defaults are important because a
    lot of people are only going to do that. In this case that's what we're
    getting at. The default behavior for a feature that's going to be on by
    default in 1.6 provides ample opportunity to create build breaking
    situations.

    This isn't good for developer experience. It's going to cause a lot of
    headaches.

    The fact that we talk about 3rd party tools to correct a common problem
    brought on by a default behavior isn't good. I'm not naming tools because
    it's not about using a tool to correct the behavior. We need to shine a
    light on the problems from the default behavior.

    Matt

    On Wednesday, November 11, 2015 at 9:58:02 PM UTC-5, Travis Johnson wrote:

    It doesn't, but it does download the vendor/ folder as well, and since
    the experiment is scheduled to be on by default in 1.6, "go build" will use
    that libraries vendor/ directory for imports.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Travis Johnson at Nov 12, 2015 at 5:15 am
    I agree with your first paragraph, but the second one doesn't quite make
    sense. If I import "dbmigrate" which imports "dbmigrate/vendor/pgx" then my
    "pgx" is incompatible with the types it uses/returns, however this is a
    compile time error, not a runtime one.
    On Wed, Nov 11, 2015 at 7:34 PM Daniel Theophanes wrote:

    Hi Matt and Travis,

    I think what you are saying is that this allows packages to vendor their
    own dependencies. Thus if you go get a package into GOPATH that has a
    vendor folder, you could run into duplicates or conflicts.

    For example, package "dbmigrate/vendor/pgx" is imported by a main package
    that already imports "pgx". This would cause a runtime panic when they both
    try to register.

    Is this the concern you are raising? -Daniel
    On Wed, Nov 11, 2015 at 7:06 PM Matt Farina wrote:

    Daniel,

    The issue is with the default behavior. Defaults are important because a
    lot of people are only going to do that. In this case that's what we're
    getting at. The default behavior for a feature that's going to be on by
    default in 1.6 provides ample opportunity to create build breaking
    situations.

    This isn't good for developer experience. It's going to cause a lot of
    headaches.

    The fact that we talk about 3rd party tools to correct a common problem
    brought on by a default behavior isn't good. I'm not naming tools because
    it's not about using a tool to correct the behavior. We need to shine a
    light on the problems from the default behavior.

    Matt

    On Wednesday, November 11, 2015 at 9:58:02 PM UTC-5, Travis Johnson wrote:

    It doesn't, but it does download the vendor/ folder as well, and since
    the experiment is scheduled to be on by default in 1.6, "go build" will use
    that libraries vendor/ directory for imports.
    --
    -Travis
    617-549-3856

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniel Theophanes at Nov 12, 2015 at 4:17 pm
    We agree that there would be an error. There may be compile time errors, if
    dbmigrate exposes the vendor/pgx package, but the panic would come from
    registering twice in "database/sql". We both agree that errors could occur
    in situations like this.

    I'm fairly certain this could occur if package copied dependencies locally
    with import path rewriting, but that is neither here nor there.

    I think this simply points to packages need to be careful what they vendor.
    Here is an example I am familiar with (many others exist I'm sure): package
    rdb vendors two packages:
    https://bitbucket.org/kardianos/rdb/src/9cb23bea5ac5ee09d98e06b1e30657ec6e93dae9/vendor/?at=default
    "x/net/context"
    and two packages from vitess. In this case the vitess packages are
    obviously safe; due to their construction even if there "were another copy"
    (unlikely) no harm would come of it and vitess has changed rapidly in the
    past that these packages need to be pulled out. "x/net/context" on the
    other hand make a good case to *not* be copied into the local "vendor"
    folder; it is both common, and directly used by other packages.

    I think we can agree on the above. Let me know if I'm catching something
    wrong.

    My take away is we need to focus on *what* message we want to send package
    and command authors. Is that also your take away? Or do you think that
    something with the vendor folder should be changed? What would you message
    package and command authors or what would you change with the vendor folder?

    On Wed, Nov 11, 2015 at 9:15 PM Travis Johnson wrote:

    I agree with your first paragraph, but the second one doesn't quite make
    sense. If I import "dbmigrate" which imports "dbmigrate/vendor/pgx" then my
    "pgx" is incompatible with the types it uses/returns, however this is a
    compile time error, not a runtime one.
    On Wed, Nov 11, 2015 at 7:34 PM Daniel Theophanes wrote:

    Hi Matt and Travis,

    I think what you are saying is that this allows packages to vendor their
    own dependencies. Thus if you go get a package into GOPATH that has a
    vendor folder, you could run into duplicates or conflicts.

    For example, package "dbmigrate/vendor/pgx" is imported by a main package
    that already imports "pgx". This would cause a runtime panic when they both
    try to register.

    Is this the concern you are raising? -Daniel

    On Wed, Nov 11, 2015 at 7:06 PM Matt Farina <matt.farina@gmail.com>
    wrote:
    Daniel,

    The issue is with the default behavior. Defaults are important because a
    lot of people are only going to do that. In this case that's what we're
    getting at. The default behavior for a feature that's going to be on by
    default in 1.6 provides ample opportunity to create build breaking
    situations.

    This isn't good for developer experience. It's going to cause a lot of
    headaches.

    The fact that we talk about 3rd party tools to correct a common problem
    brought on by a default behavior isn't good. I'm not naming tools because
    it's not about using a tool to correct the behavior. We need to shine a
    light on the problems from the default behavior.

    Matt


    On Wednesday, November 11, 2015 at 9:58:02 PM UTC-5, Travis Johnson
    wrote:
    It doesn't, but it does download the vendor/ folder as well, and since
    the experiment is scheduled to be on by default in 1.6, "go build" will use
    that libraries vendor/ directory for imports.
    --
    -Travis
    617-549-3856
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Snafex at Nov 11, 2015 at 9:58 pm

    On Wednesday, November 11, 2015 at 1:20:35 PM UTC-8, Kyle Wolfe wrote:
    I would say adding in a warning during build -v to show that a single
    import path is being resolved to two different real paths in the same
    project may be a helpful start to this issue. What if a new rule were added
    in to use the lowest level vendor path for the project? In the case of your
    test project, vendor/a.go would be used in place of vendor/b/vendor/a.go.
    This is definitely not something I'd suggest doing without the user knowing
    about it during build, though.
    This seems like a good idea at first glance, but if we allow different
    packages in the same executable to use different versions of the same
    dependencies (as the vendor experiment does), then there *will* be projects
    that depend on this, which means that legitimate projects will have a
    useless warning displayed on every build. I'm against this sort of noise
    (Like when I build C projects and watch the compiler warnings just fly
    past, no one pays attention to those, and thus they don't help).

    So keep things hidden behind the scenes, then? While I agree, people should
    do their homework, there's no reason that things like this should not be
    presented to them.
    I agree. But if the behavior warrants a warning, perhaps the behavior
    should be changed/removed?

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Russ Cox at Nov 12, 2015 at 4:40 pm

    On Wed, Nov 11, 2015 at 3:25 PM, wrote:

    I feel like this discussion has pointed out a problem with using the
    GO15VENDOREXPERIMENT. While I agree that 3rd-party vendor-management tools
    could solve this, if the experiment becomes enabled by default (as is
    planned for 1.6) then some interactions with packages that currently work
    (because the vendor is off by default, so everything uses my global
    versions instead of the vendored one), won't anymore.
    It is true that people who happen to have named a directory "vendor" and
    use it expecting the old behavior will find their code broken. We
    explicitly took this into account: there were very few instances of this in
    public code that we could find, and the point of the Go 1.5 transition
    period was to give people time to name that directory something else. If a
    source tree has no directory named "vendor" there is no difference between
    default Go 1.5 and default Go 1.6 behavior.

    That said, I don't quite understand what you mean here. If I didn't reply
    to your actual point, can you give a detailed example? In particular I
    don't know what you mean by "everything" and by "my global versions" and
    "the vendored one".

    Thanks.
    Russ

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Konstantin Shaposhnikov at Nov 12, 2015 at 5:11 pm
    I've tried to recreate the problem how I understand it here:
    https://github.com/kostya-sh/sandbox/tree/master/vendor-problem

    GO15VENDOREXPERIMENT=0 go get github.com/kostya-sh/sandbox/vendor-problem

    works, but

    GO15VENDOREXPERIMENT=1 go get github.com/kostya-sh/sandbox/vendor-problem

    fails with the following error:

    ./main.go:14: cannot use p.Status() (type
    *"github.com/derekparker/delve/vendor/golang.org/x/sys/unix".WaitStatus)
    as type *"golang.org/x/sys/unix".WaitStatus in assignment

    Basically with vendor experiment enabled by default a project that
    doesn't vendor its dependencies
    (github.com/kostya-sh/sandbox/vendor-problem in the example above)
    cannot be built if it depends on both
    1. a package that vendors its dependencies (github.com/derekparker/delve/proc)
    2 a package that is vendored by 1 (golang.org/x/sys/unix)

    I think the fix for this should make golang.org/x/sys/unix import path
    be resolved to github.com/derekparker/delve/vendor/golang.org/x/sys/unix
    in such case.

    On 13 November 2015 at 00:40, Russ Cox wrote:
    On Wed, Nov 11, 2015 at 3:25 PM, wrote:

    I feel like this discussion has pointed out a problem with using the
    GO15VENDOREXPERIMENT. While I agree that 3rd-party vendor-management tools
    could solve this, if the experiment becomes enabled by default (as is
    planned for 1.6) then some interactions with packages that currently work
    (because the vendor is off by default, so everything uses my global versions
    instead of the vendored one), won't anymore.

    It is true that people who happen to have named a directory "vendor" and use
    it expecting the old behavior will find their code broken. We explicitly
    took this into account: there were very few instances of this in public code
    that we could find, and the point of the Go 1.5 transition period was to
    give people time to name that directory something else. If a source tree has
    no directory named "vendor" there is no difference between default Go 1.5
    and default Go 1.6 behavior.

    That said, I don't quite understand what you mean here. If I didn't reply to
    your actual point, can you give a detailed example? In particular I don't
    know what you mean by "everything" and by "my global versions" and "the
    vendored one".

    Thanks.
    Russ

    --
    You received this message because you are subscribed to the Google Groups
    "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Russ Cox at Nov 12, 2015 at 5:35 pm

    On Thu, Nov 12, 2015 at 12:10 PM, Konstantin Shaposhnikov wrote:

    I've tried to recreate the problem how I understand it here:
    https://github.com/kostya-sh/sandbox/tree/master/vendor-problem

    GO15VENDOREXPERIMENT=0 go get github.com/kostya-sh/sandbox/vendor-problem

    works, but

    GO15VENDOREXPERIMENT=1 go get github.com/kostya-sh/sandbox/vendor-problem

    fails with the following error:

    ./main.go:14: cannot use p.Status() (type
    *"github.com/derekparker/delve/vendor/golang.org/x/sys/unix".WaitStatus)
    as type *"golang.org/x/sys/unix".WaitStatus in assignment

    Basically with vendor experiment enabled by default a project that
    doesn't vendor its dependencies
    (github.com/kostya-sh/sandbox/vendor-problem in the example above)
    cannot be built if it depends on both
    1. a package that vendors its dependencies (
    github.com/derekparker/delve/proc)
    2 a package that is vendored by 1 (golang.org/x/sys/unix)
    It's true, you can't do this. It looks like Delve does not intend for
    people to import pieces of it, and therefore you can't.

    I think the fix for this should make golang.org/x/sys/unix import path
    be resolved to github.com/derekparker/delve/vendor/golang.org/x/sys/unix
    in such case.
    I understand you to be saying you want
    github.com/kostya-sh/sandbox/vendor-problem's import "golang.org/x/sys/unix"
    to resolve to "github.com/derekparker/delve/vendor/golang.org/x/sys/unix"
    because it also happens to import "github.com/derekparker/delve/proc".
    That does seem the obvious fix, but it breaks down once there are multiple
    choices. What if vendor-problem also imported "rsc.io/foo/bar" and there
    was an "rsc.io/foo/vendor/golang.org/x/sys/unix"? Then what should import "
    golang.org/x/sys/unix" resolve to? What if you don't want derekparker's
    copy of sys/unix? How do you get the real one? The options seem to be "stop
    importing delve/proc" (seems unrelated) or "make Derek delete his copy of
    sys/unix" (seems excessive).

    The resolution of imports depends on the source location and not what the
    source imports precisely to avoid these problems.

    It really looks like Derek does not intend you or anyone else to import and
    use delve/proc directly, so he's doing everything right. If he did want you
    to import delve/proc directly, then he would need to use the real sys/unix
    types or else publish his copy of sys/unix (outside the vendor directory)
    for you to import.

    Also, vendored code is a private copy of something. That is, "vendor" is
    explicitly "internal" as well. We very much do not want one project
    importing another project's vendored copy of a third project.

    Russ

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Kyle Wolfe at Nov 12, 2015 at 5:51 pm

    On Thursday, November 12, 2015 at 12:35:22 PM UTC-5, rsc wrote:
    Also, vendored code is a private copy of something. That is, "vendor" is
    explicitly "internal" as well. We very much do not want one project
    importing another project's vendored copy of a third project.
    Russ, this is currently not the behaviour, as shown by the example project.
    My suggestion earlier was the the lowest level path should be resolved. In
    the meantime, I've opened a propsoal to push back the milestones for
    vendoring by one release until more of a consensus is reached
    (https://github.com/golang/go/issues/13218).

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Konstantin Shaposhnikov at Nov 12, 2015 at 5:52 pm
    I used delve/proc only as example to demonstrate the described issue.
    I agree that it is probably not intended to be used as a package.
    However I can imagine that putting applications (that usually will
    have vendored dependencies) and library packages in the same
    repository is quite common.

    I don't think that this particular issue is a big problem though. As
    you demonstrated there are potentially many ways to resolve a package
    in the described scenario so go get cannot do it automatically. The
    only valid solution in this case is to make a developer vendor the
    required dependencies manually.


    On 13 November 2015 at 01:35, Russ Cox wrote:
    On Thu, Nov 12, 2015 at 12:10 PM, Konstantin Shaposhnikov
    wrote:
    I've tried to recreate the problem how I understand it here:
    https://github.com/kostya-sh/sandbox/tree/master/vendor-problem

    GO15VENDOREXPERIMENT=0 go get github.com/kostya-sh/sandbox/vendor-problem

    works, but

    GO15VENDOREXPERIMENT=1 go get github.com/kostya-sh/sandbox/vendor-problem

    fails with the following error:

    ./main.go:14: cannot use p.Status() (type
    *"github.com/derekparker/delve/vendor/golang.org/x/sys/unix".WaitStatus)
    as type *"golang.org/x/sys/unix".WaitStatus in assignment

    Basically with vendor experiment enabled by default a project that
    doesn't vendor its dependencies
    (github.com/kostya-sh/sandbox/vendor-problem in the example above)
    cannot be built if it depends on both
    1. a package that vendors its dependencies
    (github.com/derekparker/delve/proc)
    2 a package that is vendored by 1 (golang.org/x/sys/unix)

    It's true, you can't do this. It looks like Delve does not intend for people
    to import pieces of it, and therefore you can't.
    I think the fix for this should make golang.org/x/sys/unix import path
    be resolved to github.com/derekparker/delve/vendor/golang.org/x/sys/unix
    in such case.

    I understand you to be saying you want
    github.com/kostya-sh/sandbox/vendor-problem's import "golang.org/x/sys/unix"
    to resolve to "github.com/derekparker/delve/vendor/golang.org/x/sys/unix"
    because it also happens to import "github.com/derekparker/delve/proc".
    That does seem the obvious fix, but it breaks down once there are multiple
    choices. What if vendor-problem also imported "rsc.io/foo/bar" and there was
    an "rsc.io/foo/vendor/golang.org/x/sys/unix"? Then what should import
    "golang.org/x/sys/unix" resolve to? What if you don't want derekparker's
    copy of sys/unix? How do you get the real one? The options seem to be "stop
    importing delve/proc" (seems unrelated) or "make Derek delete his copy of
    sys/unix" (seems excessive).

    The resolution of imports depends on the source location and not what the
    source imports precisely to avoid these problems.

    It really looks like Derek does not intend you or anyone else to import and
    use delve/proc directly, so he's doing everything right. If he did want you
    to import delve/proc directly, then he would need to use the real sys/unix
    types or else publish his copy of sys/unix (outside the vendor directory)
    for you to import.

    Also, vendored code is a private copy of something. That is, "vendor" is
    explicitly "internal" as well. We very much do not want one project
    importing another project's vendored copy of a third project.

    Russ
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Travis Johnson at Nov 14, 2015 at 12:09 am

    Also, vendored code is a private copy of something. That is, "vendor" is
    explicitly "internal" as well. We very much do not want one project
    importing another project's vendored copy of a third project.
    But isn't this what internal/ is for? Would it make sense to have internal/
    behave like vendor/ does for imports, and then eliminate vendor/ (this
    would also give us the relative imports that's been discussed in the past)?

    Though this may be seen as a breaking change, so maybe instead just
    expressly make vendor/ restricted like internal/ is (potentially with the
    additional compiler refusal to export types (I'd make an exception for
    interfaces), but there may be legitimate uses for this, and it sounds like
    a rabbit hole of complexity, so I'm hesitant to suggest it)? This would
    fall in line with this claim that vendor/ is "explicitly" internal (which
    while this may have been the intention, it isn't the pattern I see in
    projects).

    I do however believe that turning this on by default will break a lot of
    builds, but then people will very quickly realize incorrect usage and fix
    (not without backlash though), though previously stable versions of these
    libraries will be incompatible with 1.6 (not sure I'd call this a breaking
    change, as it's functionality depending on an off-by-default "experiment"),
    that is if you consider the vendor/ directory as "part of the project" (I
    do, as I think do many others).

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Daniel Theophanes at Nov 14, 2015 at 12:52 am
    Travis, The only way a package could access a type in the vendor folder
    *today* is through indirect access, just like the "internal" folder. Russ
    wasn't stating a policy, he was stating the technical fact as it stands
    today.
    On Fri, Nov 13, 2015 at 4:09 PM Travis Johnson wrote:

    Also, vendored code is a private copy of something. That is, "vendor" is
    explicitly "internal" as well. We very much do not want one project
    importing another project's vendored copy of a third project.
    But isn't this what internal/ is for? Would it make sense to have
    internal/ behave like vendor/ does for imports, and then eliminate vendor/
    (this would also give us the relative imports that's been discussed in the
    past)?

    Though this may be seen as a breaking change, so maybe instead just
    expressly make vendor/ restricted like internal/ is (potentially with the
    additional compiler refusal to export types (I'd make an exception for
    interfaces), but there may be legitimate uses for this, and it sounds like
    a rabbit hole of complexity, so I'm hesitant to suggest it)? This would
    fall in line with this claim that vendor/ is "explicitly" internal (which
    while this may have been the intention, it isn't the pattern I see in
    projects).

    I do however believe that turning this on by default will break a lot of
    builds, but then people will very quickly realize incorrect usage and fix
    (not without backlash though), though previously stable versions of these
    libraries will be incompatible with 1.6 (not sure I'd call this a breaking
    change, as it's functionality depending on an off-by-default "experiment"),
    that is if you consider the vendor/ directory as "part of the project" (I
    do, as I think do many others).

    --
    You received this message because you are subscribed to a topic in the
    Google Groups "golang-dev" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/golang-dev/WebP4dLV1b0/unsubscribe.
    To unsubscribe from this group and all its topics, send an email to
    golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-dev @
categoriesgo
postedNov 3, '15 at 5:28p
activeNov 16, '15 at 9:45a
posts74
users14
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase