FAQ
I've been looking at some tooling for deployment and one thing I'd really
like to have is the ability to use private repositories. The issue that
you quickly run into is that the "standard" for fetching Go dependencies is
either "go get" or "godep" (which, as far as I can tell, uses "go get" or
at least the "vcs" stuff beneath the surface).

So the bottom line is that I spent some time trying to figure out how to be
able to use private repositories. I found a lot of people with this issue,
but many claim it can't be done or even that the Go documentation is
incorrect.

Looking at the machinery, it seems that Go is perfectly capable of
supporting private repositories but there are some really minor hang ups.

The first thing is that people wanting to use private repositories are
generally expecting to use SSH. As far as I can tell, the main hangup here
is actually just the ordering of schemes in vcs.go. The "https" scheme
comes before the "git+ssh" scheme. As a result, when it gets a hit on "git
ls-remote https://..." it seems to think that is the preferred scheme. If
the authentication fails, it simply seems to give up. The sad thing is
that git+ssh is in the list (and this would be the best choice, as far as I
can tell, for private repos) but it just never gets there (at least if you
are using GitHub, since GitHub offers an https transport layer).

Was there a specific reason that https is "preferred" over git+ssh? To be
clear, GitHub supports the git+ssh scheme which means the current "go get"
infrastructure would probably work just fine for dealing with private
repositories (assuming you had the right SSH private key and perhaps a bit
of setup in ~/.ssh/config).

On a somewhat related note, it is a bit of a shame that the import path
stuff is so...rigid. It doesn't seem to support a general scheme for
turning import paths into URLs. The documentation seems to imply that this:

import "<something>/repo.git"

is transformed into a url of the form:

<scheme>://<something>/repo

or

<scheme>://<something>/repo

where several schemes are tried. However, there is clearly some logic
behind the scenes because this doesn't work for:

something=identifier (i.e. a non-dot qualified name, despite the presence
of .git explicitly in the import path)
something=hostname:port/path (i.e. if a port specification or even user
specification is present in <something>)

I know there is special code to look for patterns related to providers like
github and bitbucket. But it seems like this interferes with other
potentially legal URL components that are not from those providers.

Overall, I think that being able to easily utilize code in private
repositories is important (at least it is important to me). At the end of
the day, it seems very useful to me that it should be possible to import
(i.e. via "go get") code from private repositories. Am I missing
something? Is anybody working on this? Is there some reason (other than a
lack of pull requests) why this wasn't done?

Thanks.

--
Mike

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Ian Davis at Sep 18, 2014 at 2:53 pm
    On Thu, Sep 18, 2014, at 02:58 AM, [1]michael.tiller@gmail.com wrote:
    I've been looking at some tooling for deployment and one thing
    I'd really like to have is the ability to use private
    repositories. The issue that you quickly run into is that the
    "standard" for fetching Go dependencies is either "go get" or
    "godep" (which, as far as I can tell, uses "go get" or at least
    the "vcs" stuff beneath the surface).



    Go (the language) doesn't care where your code comes from as
    long as it can read it from your GOPATH. go get is a convenient
    tool that covers 90% of use cases. What's wrong with just using
    git clone on your private repositories?



    Ian

    References

    1. mailto:michael.tiller@gmail.com

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Michael Tiller at Sep 18, 2014 at 7:27 pm

    On Thursday, September 18, 2014 10:54:06 AM UTC-4, Ian Davis wrote:

    On Thu, Sep 18, 2014, at 02:58 AM, michael...@gmail.com <javascript:>
    wrote:

    I've been looking at some tooling for deployment and one thing I'd really
    like to have is the ability to use private repositories. The issue that
    you quickly run into is that the "standard" for fetching Go dependencies is
    either "go get" or "godep" (which, as far as I can tell, uses "go get" or
    at least the "vcs" stuff beneath the surface).


    Go (the language) doesn't care where your code comes from as long as it
    can read it from your GOPATH. go get is a convenient tool that covers 90%
    of use cases. What's wrong with just using git clone on your private
    repositories?
    There is nothing wrong with it. But "go get" and "godeps" exist for a
    reason. They make things very convenient. So there is that reason to not
    want to do it manually.

    But in addition (and this is what motivated me to ask the question), there
    are many ways to deploy Go applications to the cloud that ASSUME that all
    the dependencies can be fetched via "go get" or "go deps". In those
    environments, it isn't typically possible to manually checkout all the
    repositories. Of course, you still have the problem of getting your
    deployment keys added. But at least that is an accepted deployment
    practice that has a chance of being supported. I was using Dokku in my
    case where I probably have a chance to configure the Dokku host with the
    necessary privileges so that it could build the containers based on a "git
    push".

    --
    Mike

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Egon at Sep 18, 2014 at 8:02 pm

    On Thursday, 18 September 2014 22:27:39 UTC+3, Michael Tiller wrote:
    On Thursday, September 18, 2014 10:54:06 AM UTC-4, Ian Davis wrote:


    On Thu, Sep 18, 2014, at 02:58 AM, michael...@gmail.com wrote:

    I've been looking at some tooling for deployment and one thing I'd really
    like to have is the ability to use private repositories. The issue that
    you quickly run into is that the "standard" for fetching Go dependencies is
    either "go get" or "godep" (which, as far as I can tell, uses "go get" or
    at least the "vcs" stuff beneath the surface).


    Go (the language) doesn't care where your code comes from as long as it
    can read it from your GOPATH. go get is a convenient tool that covers 90%
    of use cases. What's wrong with just using git clone on your private
    repositories?
    There is nothing wrong with it. But "go get" and "godeps" exist for a
    reason. They make things very convenient. So there is that reason to not
    want to do it manually.

    But in addition (and this is what motivated me to ask the question), there
    are many ways to deploy Go applications to the cloud that ASSUME that all
    the dependencies can be fetched via "go get" or "go deps".
    One of the simplest solution would be to deploy only the binary instead.

    In those environments, it isn't typically possible to manually checkout
    all the repositories. Of course, you still have the problem of getting
    your deployment keys added. But at least that is an accepted deployment
    practice that has a chance of being supported. I was using Dokku in my
    case where I probably have a chance to configure the Dokku host with the
    necessary privileges so that it could build the containers based on a "git
    push".

    --
    Mike
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Eli Janssen at Sep 18, 2014 at 6:40 pm

    On Sep 17, 2014, at 6:58 PM, michael.tiller@gmail.com wrote:

    Am I missing something?
    `go get` supports .netrc just fine.
    see: https://coderwall.com/p/jtujgw


    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Michael Tiller at Sep 18, 2014 at 7:23 pm

    On Thursday, September 18, 2014 2:40:43 PM UTC-4, Eli Janssen wrote:
    On Sep 17, 2014, at 6:58 PM, michael...@gmail.com <javascript:> wrote:

    Am I missing something?
    `go get` supports .netrc just fine.
    see: https://coderwall.com/p/jtujgw
    True. In fact, that is what I settled on. However, the issue that I ran
    into was related to deployment keys. Both github and bitbucket offer
    deployment keys. Deployment keys are nice because they don't require you
    to expose your "root credentials" and they can be easily be revoked.

    So while the link you posted is absolutely a workable solution, only github
    allows their OAUTH api keys to be used via https:// (bitbucket allows them
    for teams, but not individuals). So if you want to go the .netrc route
    with other providers, you have to put real credentials in there.

    One approach would be to put SSH based authentication first. Maybe people
    are worried about that breaking something. On the other hand, it seems odd
    to me that https succeeds (as the "chosen" scheme) even when the request
    itself doesn't authenticate. This, in turn, prevents it from considering
    the SSH based schemes. This just doesn't seem optimal to me.

    --
    Mike

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Benjamin Measures at Sep 18, 2014 at 8:15 pm

    Was there a specific reason that https is "preferred" over git+ssh?
    Am I missing something?
    Why does "go get" use HTTPS when cloning a repository? <http://golang.org/doc/faq#git_https>

    Note git config also has insteadOf <https://www.kernel.org/pub/software/scm/git/docs/git-config.html>.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedSep 18, '14 at 2:45p
activeSep 18, '14 at 8:15p
posts7
users5
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase