FAQ
What is the recommended way to organize the source tree of a project
written in several programming languages (for example Go and Python)?

Ideally, we would want to keep everything in the same git repository
because 1) it makes easier to keep the Python and Go logic in sync and 2) some
assets are shared by the Python and Go code.

One obvious way is to checkout the project source in some arbitrary
directory like $HOME/projects/myproject, create a subdirectory go
containing all the Go code, and make it accessible in $GOPATH with ln -s
$HOME/projects/myproject/go $GOPATH/src/myproject. But this solution looks
like a dead-end because symbolic links were declared "not supported" many
times on golang-nuts and golang-dev.

Another way is to checkout the full project source (Go and Python) in
$GOPATH/src/myproject and import "myproject/go" instead of "myproject".
It's perfect and idiomatic from Go's point of view. But from Python's point
of view, it's weird to be forced to live in $GOPATH. Go requires the
project source to live in a directory src under $GOPATH. Now let's imagine
that Python would require the project source to be rooted in a directory
lib (for example) under its own $PYPATH. That would make Go's and Python's
requirements contradictory and impossible to satisfy at the same time.

Another way is to create a Go workspace directly in the project source tree
and add it to the $GOPATH. This implies myproject would have its own
src/bin/pkg directories and this approach was explicitly not recommended on
this mailing list in the past (
https://groups.google.com/forum/#!topic/golang-nuts/dxOFYPpJEXo).

I'm not sure how to solve this in a simple and clean way. What is your
experience solving this issue in your own projects?

Nicolas

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Kyle Lemons at Aug 15, 2013 at 4:51 am
    Since Go is almost always the primary language for such projects of mine, I
    just make all of the source live in $GOPATH/src/
    github.com/kylelemons/project. The Go code imports itself as
    github.com/kylelemons/project/whatever and the rest of the code doesn't
    really care where it lives.

    On Wed, Aug 14, 2013 at 4:01 PM, Nicolas Grilly wrote:

    What is the recommended way to organize the source tree of a project
    written in several programming languages (for example Go and Python)?

    Ideally, we would want to keep everything in the same git repository
    because 1) it makes easier to keep the Python and Go logic in sync and 2) some
    assets are shared by the Python and Go code.

    One obvious way is to checkout the project source in some arbitrary
    directory like $HOME/projects/myproject, create a subdirectory go
    containing all the Go code, and make it accessible in $GOPATH with ln -s
    $HOME/projects/myproject/go $GOPATH/src/myproject. But this solution looks
    like a dead-end because symbolic links were declared "not supported" many
    times on golang-nuts and golang-dev.

    Another way is to checkout the full project source (Go and Python) in
    $GOPATH/src/myproject and import "myproject/go" instead of "myproject".
    It's perfect and idiomatic from Go's point of view. But from Python's point
    of view, it's weird to be forced to live in $GOPATH. Go requires the
    project source to live in a directory src under $GOPATH. Now let's imagine
    that Python would require the project source to be rooted in a directory
    lib (for example) under its own $PYPATH. That would make Go's and Python's
    requirements contradictory and impossible to satisfy at the same time.

    Another way is to create a Go workspace directly in the project source
    tree and add it to the $GOPATH. This implies myproject would have its own
    src/bin/pkg directories and this approach was explicitly not recommended on
    this mailing list in the past (
    https://groups.google.com/forum/#!topic/golang-nuts/dxOFYPpJEXo).

    I'm not sure how to solve this in a simple and clean way. What is your
    experience solving this issue in your own projects?

    Nicolas

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nicolas Grilly at Aug 15, 2013 at 7:34 am

    On Thu, Aug 15, 2013 at 6:51 AM, Kyle Lemons wrote:

    Since Go is almost always the primary language for such projects of mine,
    I just make all of the source live in $GOPATH/src/
    github.com/kylelemons/project. The Go code imports itself as
    github.com/kylelemons/project/whatever and the rest of the code doesn't
    really care where it lives.
    Yes, that's a perfect layout when the project main language is Go. But what
    to do when Go is not (yet) the main language?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nicolas Grilly at Aug 24, 2013 at 1:10 pm

    On Thursday, August 15, 2013 6:51:13 AM UTC+2, Kyle Lemons wrote:

    Since Go is almost always the primary language for such projects of mine,
    I just make all of the source live in $GOPATH/src/
    github.com/kylelemons/project. The Go code imports itself as
    github.com/kylelemons/project/whatever and the rest of the code doesn't
    really care where it lives.
    Kyle, I like your answer because it's simple and it just works. As often,
    less is more :)

    And you're right about "the rest of the code doesn't really care where it
    lives".

    I noticed that other experienced Go developers gave the same advice in
    other discussions:

    *I recommend every beginner to set his $GOPATH=$HOME and forget about it
    forever. Just code.* -- Jan Mercl

    *I always had my source code in $HOME/src anyway. Yes, it's not only Go
    code, but so what? It's great that I can have my Go and non-Go code
    together in the same place. Another benefit is that it will put binaries in
    $HOME/bin, which many people already add to their $PATH.* -- Aram Hăvărneanu

    *On my machine, I have my GOPATH=$HOME, so my src in in $HOME/src, binaries
    get installed to $HOME/bin (which is in my $PATH), and $HOME/pkg is a junk
    cache directory of temporary object files (which I kinda ignore).* -- Brad
    Fitzpatrick

    I just followed these advices and it works well. For the record, I was
    afraid of godoc becoming slow because of having to scan all my projects,
    including the non-Go ones, but it appears it's really quick and not a
    problem in practice.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Fmccann23 at Jan 11, 2014 at 8:43 pm
    I'm having a similar problem with GOPATH and compiling. The issue I'm
    having is with multiple branches of code. A typical project looks like:

    project_foo/
         trunk/
         branch1/
         branch2/

    I don't have too many branches going at once, but even a single branch is
    causing problems because if I have this in the src/ directory of the
    GOPATH, then the branches are assumed to be part of the package names. It
    seems that the root location of a project or a branch of the project is
    tied to the package name. Is there any way around this?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Caleb Doxsey at Jan 11, 2014 at 9:43 pm
    I often use symlinks. They mostly work.

    Suppose you have:

    /Users/whoever/projects/project_foo/trunk

    You might want it here:

    /Users/whoever/go/src/code.google.com/p/project_foo

    And you can do:

    ln -s /Users/whoever/projects/project_foo/trunk /Users/whoever/go/src/
    code.google.com/p/project_foo

    In windows you can use junctions. (
    http://schinagl.priv.at/nt/hardlinkshellext/hardlinkshellext.html)

    On Sat, Jan 11, 2014 at 12:22 PM, wrote:

    I'm having a similar problem with GOPATH and compiling. The issue I'm
    having is with multiple branches of code. A typical project looks like:

    project_foo/
    trunk/
    branch1/
    branch2/

    I don't have too many branches going at once, but even a single branch is
    causing problems because if I have this in the src/ directory of the
    GOPATH, then the branches are assumed to be part of the package names. It
    seems that the root location of a project or a branch of the project is
    tied to the package name. Is there any way around this?

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Fmccann23 at May 18, 2015 at 2:10 am
    I realize that this is a very late response to this thread, but project
    layout has been something I’ve continued to struggle with since day one
    with Go.


    I’ve tried capturing the entire gopath in version control, but this is
    awkward and makes creating go-gettable libraries difficult. I’ve gone back
    and forth on this and I think the simple answer is if you can reasonably
    work inside a single gopath, that’s probably the best way of working.


    I can’t quite pull that off because of how I do branching, so I’ve been
    relying on home grown tools to mess around with the gopath. I’ve
    opensourced them today at:


    http://gojank.com


    I still consider this something of an experiment. If you can work inside a
    single gopath, that’s the way to go, but this is what i’m doing for now.
    On Saturday, January 11, 2014 at 2:22:27 PM UTC-5, fmcc...@gmail.com wrote:

    I'm having a similar problem with GOPATH and compiling. The issue I'm
    having is with multiple branches of code. A typical project looks like:

    project_foo/
    trunk/
    branch1/
    branch2/

    I don't have too many branches going at once, but even a single branch is
    causing problems because if I have this in the src/ directory of the
    GOPATH, then the branches are assumed to be part of the package names. It
    seems that the root location of a project or a branch of the project is
    tied to the package name. Is there any way around this?
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Frou at Aug 15, 2013 at 12:21 pm
    The symlink approach does work, doesn't it? If it stops working then you'll
    know about it and can find an alternative.
    On Thursday, 15 August 2013 00:01:01 UTC+1, Nicolas Grilly wrote:

    What is the recommended way to organize the source tree of a project
    written in several programming languages (for example Go and Python)?

    Ideally, we would want to keep everything in the same git repository
    because 1) it makes easier to keep the Python and Go logic in sync and 2) some
    assets are shared by the Python and Go code.

    One obvious way is to checkout the project source in some arbitrary
    directory like $HOME/projects/myproject, create a subdirectory go
    containing all the Go code, and make it accessible in $GOPATH with ln -s
    $HOME/projects/myproject/go $GOPATH/src/myproject. But this solution looks
    like a dead-end because symbolic links were declared "not supported" many
    times on golang-nuts and golang-dev.

    Another way is to checkout the full project source (Go and Python) in
    $GOPATH/src/myproject and import "myproject/go" instead of "myproject".
    It's perfect and idiomatic from Go's point of view. But from Python's point
    of view, it's weird to be forced to live in $GOPATH. Go requires the
    project source to live in a directory src under $GOPATH. Now let's imagine
    that Python would require the project source to be rooted in a directory
    lib (for example) under its own $PYPATH. That would make Go's and Python's
    requirements contradictory and impossible to satisfy at the same time.

    Another way is to create a Go workspace directly in the project source
    tree and add it to the $GOPATH. This implies myproject would have its own
    src/bin/pkg directories and this approach was explicitly not recommended on
    this mailing list in the past (
    https://groups.google.com/forum/#!topic/golang-nuts/dxOFYPpJEXo).

    I'm not sure how to solve this in a simple and clean way. What is your
    experience solving this issue in your own projects?

    Nicolas
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nicolas Grilly at Aug 15, 2013 at 1:50 pm

    On Thursday, August 15, 2013 2:21:21 PM UTC+2, frou wrote:

    The symlink approach does work, doesn't it? If it stops working then
    you'll know about it and can find an alternative.
    No, it does not.

    In my experience, many things don't work with symlinks:

    - go build/fmt/test myproject/... does not find packages rooted under the
    symlinked directory (this is because the related code in the Go standard
    library and tools uses filepath.Walk which does not follow symbolic links)
    - Packages rooted under the symlinked directory do not appear in godoc
    - go install myproject does not know in which directory to install the
    resulting binary
    - etc.

    And according to some of Go authors on this mailing list, symlinks are
    avoided by design because their usage solves some issues (like mine) but
    also brings a lot of problems (like the possibility of creating circular
    references and having several different paths leading to the same file,
    none of them being canonical).

    But maybe I'm missing something. Do you use symlinks in your own projects,
    without the problems I listed above?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Frou at Aug 15, 2013 at 2:26 pm
    go {build,install,fmt,test} proj/a/b

    work for me where proj is a symlink in GOPATH to outside of GOPATH.

    Indeed those packages don't show up in godoc.

    I may be missing something myself. I have only experimented with one such
    multi-language project and haven't read the ML discussions you mention.
    On Thursday, 15 August 2013 14:50:30 UTC+1, Nicolas Grilly wrote:
    On Thursday, August 15, 2013 2:21:21 PM UTC+2, frou wrote:

    The symlink approach does work, doesn't it? If it stops working then
    you'll know about it and can find an alternative.
    No, it does not.

    In my experience, many things don't work with symlinks:

    - go build/fmt/test myproject/... does not find packages rooted under the
    symlinked directory (this is because the related code in the Go standard
    library and tools uses filepath.Walk which does not follow symbolic links)
    - Packages rooted under the symlinked directory do not appear in godoc
    - go install myproject does not know in which directory to install the
    resulting binary
    - etc.

    And according to some of Go authors on this mailing list, symlinks are
    avoided by design because their usage solves some issues (like mine) but
    also brings a lot of problems (like the possibility of creating circular
    references and having several different paths leading to the same file,
    none of them being canonical).

    But maybe I'm missing something. Do you use symlinks in your own projects,
    without the problems I listed above?
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nicolas Grilly at Aug 15, 2013 at 3:35 pm

    On Thursday, August 15, 2013 4:26:32 PM UTC+2, frou wrote:

    go {build,install,fmt,test} proj/a/b

    work for me where proj is a symlink in GOPATH to outside of GOPATH.

    Indeed those packages don't show up in godoc.

    I may be missing something myself. I have only experimented with one such
    multi-language project and haven't read the ML discussions you mention.
    Yes, some things work, some don't.

    Works:
    - go {build,install,fmt,test,doc} proj/a/b
    - go install proj/a/b correctly installs the produced binaries in
    $GOPATH/bin and $GOPATH/pkg (contrary to what I wrote in my previous
    message in this thread)
    - cd $GOPATH/src && go {build,install,fmt,test} ./proj/...

    Doesn't work:
    - godoc -http=:6060
    - go {build,install,fmt,test} proj/... ("proj/..." matches no packages)
    - cd $HOME/myprojects/proj && go {build,install,fmt,test} ./... (no install
    location for directory $HOME/myprojects/proj/... outisde GOPATH)

    Conclusion:
    It's difficult to rely on Go tools while using symlinks. I especially miss
    godoc, which is one of Go's main selling points (but maybe there is some
    command lines arguments to make it work?). And some things don't work as
    expected. Because of this, I doubt symlinks are the way to go. But I'd be
    happy to be proven wrong :)

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jeremy Wall at Aug 15, 2013 at 1:55 pm
    You can have multiple GOPATHs. Why not just add an entry for the path where
    your go code in the larger project lives?
    On Aug 14, 2013 10:46 PM, "Nicolas Grilly" wrote:

    What is the recommended way to organize the source tree of a project
    written in several programming languages (for example Go and Python)?

    Ideally, we would want to keep everything in the same git repository
    because 1) it makes easier to keep the Python and Go logic in sync and 2) some
    assets are shared by the Python and Go code.

    One obvious way is to checkout the project source in some arbitrary
    directory like $HOME/projects/myproject, create a subdirectory go
    containing all the Go code, and make it accessible in $GOPATH with ln -s
    $HOME/projects/myproject/go $GOPATH/src/myproject. But this solution looks
    like a dead-end because symbolic links were declared "not supported" many
    times on golang-nuts and golang-dev.

    Another way is to checkout the full project source (Go and Python) in
    $GOPATH/src/myproject and import "myproject/go" instead of "myproject".
    It's perfect and idiomatic from Go's point of view. But from Python's point
    of view, it's weird to be forced to live in $GOPATH. Go requires the
    project source to live in a directory src under $GOPATH. Now let's imagine
    that Python would require the project source to be rooted in a directory
    lib (for example) under its own $PYPATH. That would make Go's and Python's
    requirements contradictory and impossible to satisfy at the same time.

    Another way is to create a Go workspace directly in the project source
    tree and add it to the $GOPATH. This implies myproject would have its own
    src/bin/pkg directories and this approach was explicitly not recommended on
    this mailing list in the past (
    https://groups.google.com/forum/#!topic/golang-nuts/dxOFYPpJEXo).

    I'm not sure how to solve this in a simple and clean way. What is your
    experience solving this issue in your own projects?

    Nicolas

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nicolas Grilly at Aug 15, 2013 at 2:24 pm

    On Thursday, August 15, 2013 3:55:47 PM UTC+2, Jeremy Wall wrote:

    You can have multiple GOPATHs. Why not just add an entry for the path
    where your go code in the larger project lives?
    Just to make sure I understand your suggestion, let's say the Go files of
    my project are in github.com/ngrilly/myproject/go/*.go. What would you add
    to GOPATH?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jeremy Wall at Aug 15, 2013 at 3:32 pm

    On Thu, Aug 15, 2013 at 9:24 AM, Nicolas Grilly wrote:
    On Thursday, August 15, 2013 3:55:47 PM UTC+2, Jeremy Wall wrote:

    You can have multiple GOPATHs. Why not just add an entry for the path
    where your go code in the larger project lives?
    Just to make sure I understand your suggestion, let's say the Go files of
    my project are in github.com/ngrilly/myproject/go/*.go. What would you
    add to GOPATH?
    Well first you'd have to do some rearranging. Add a gopath like directory
    with a src/ and pkg/ directory in your repo.
    github.com/ngrilly/myproject/go/{src,pkg} for instance

    Then move the *.go files into a reasonable directory in the src/ directory.
    .../go/src/myproject/*.go for instance.

    Then in your build script which you will need anyway since this is a multi
    language project add e.g. github.com/ngrilly/myproject/go/src/ to your
    GOPATH so all go builds of myproject/... will work.

    This will work for all your developers and packagers. It will not work for
    making the the go code go gettable but this is a multilanguage project
    where Go is not the primarly language so that is likely not really a goal
    in the first place.

    if go gettability *is* a goal then you should probably have the go code in
    a separate repo instead.

      --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nicolas Grilly at Aug 24, 2013 at 12:41 pm

    On Thursday, August 15, 2013 5:32:30 PM UTC+2, Jeremy Wall wrote:
    Well first you'd have to do some rearranging. Add a gopath like directory
    with a src/ and pkg/ directory in your repo.
    github.com/ngrilly/myproject/go/{src,pkg}<http://github.com/ngrilly/myproject/go/%7Bsrc,pkg%7D>for instance

    Then move the *.go files into a reasonable directory in the src/
    directory. .../go/src/myproject/*.go for instance.

    Then in your build script which you will need anyway since this is a multi
    language project add e.g. github.com/ngrilly/myproject/go/src/ to your
    GOPATH so all go builds of myproject/... will work.

    This will work for all your developers and packagers. It will not work for
    making the the go code go gettable but this is a multilanguage project
    where Go is not the primarly language so that is likely not really a goal
    in the first place.

    if go gettability *is* a goal then you should probably have the go code in
    a separate repo instead.

    Jeremy, thank you for the detailed explanation!

    I've eventually decided to not follow this way. After some tests, I prefer
    the "usual" approach of checking out the project source tree in
    $GOPATH/src/myproject, instead of hosting the Go workspace under the
    project source tree in $HOME/projects/myproject/go/src.

    I perceive the $GOPATH as a kind of temporary workspace, and I prefer it to
    live outside my source repository, instead on inside.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • TR NS at Aug 15, 2013 at 7:32 pm
    I deplore the design of a centralized workspace! I order my projects by
    many other criteria than what language they are written in. I think the
    designers of Go should rethink this design and come up with a more flexible
    solution.

    In the mean time, to work around I've had to do what you call the "obvious"
    way, but instead of using a symlink I have a small shell script in my
    project that reassigns the $GOPATH, which I can run before working on the
    project. Annoying but effective. Perhaps a more ideal solution would be to
    do the exporting when running a local build tool (e.g. Rake). By exporting
    the path in such a script it would only be temporary while the build is
    taking place, but then of course you have to write a bunch of build tasks
    to route to the go tool.


    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nicolas Grilly at Aug 15, 2013 at 7:47 pm

    On Thursday, August 15, 2013 9:32:33 PM UTC+2, TR NS wrote:

    In the mean time, to work around I've had to do what you call the
    "obvious" way, but instead of using a symlink I have a small shell script
    in my project that reassigns the $GOPATH, which I can run before working on
    the project. Annoying but effective. Perhaps a more ideal solution would be
    to do the exporting when running a local build tool (e.g. Rake). By
    exporting the path in such a script it would only be temporary while the
    build is taking place, but then of course you have to write a bunch of
    build tasks to route to the go tool.
    You're right: what I called the "obvious way" is more hacky than obvious :)

    Could you give some details about your setup? What is the layout of your
    source tree? Is the root of the Go workspace inside or outside your source
    tree? To which directory $GOPATH points?

    -- Nicolas

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Rodrigo Kochenburger at Aug 15, 2013 at 7:54 pm
    By the way, guys. If you need more flexibility you can always call the
    tools directly (compiler, linker, etc) instead of using the higher level
    tool and build your own build script.

    go tool 6c foo.go
    go tool 6l -o foo foo.8
    ./foo

    I definitely not recommend doing that for libraries because it's much more
    important for them to be go-getable but if it's a private application that
    you won't be sharing, you should be fine.

    - RK


    On Thu, Aug 15, 2013 at 12:47 PM, Nicolas Grilly
    wrote:
    On Thursday, August 15, 2013 9:32:33 PM UTC+2, TR NS wrote:

    In the mean time, to work around I've had to do what you call the
    "obvious" way, but instead of using a symlink I have a small shell script
    in my project that reassigns the $GOPATH, which I can run before working on
    the project. Annoying but effective. Perhaps a more ideal solution would be
    to do the exporting when running a local build tool (e.g. Rake). By
    exporting the path in such a script it would only be temporary while the
    build is taking place, but then of course you have to write a bunch of
    build tasks to route to the go tool.
    You're right: what I called the "obvious way" is more hacky than obvious :)

    Could you give some details about your setup? What is the layout of your
    source tree? Is the root of the Go workspace inside or outside your source
    tree? To which directory $GOPATH points?

    -- Nicolas

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nicolas Grilly at Aug 15, 2013 at 8:08 pm

    On Thursday, August 15, 2013 9:54:50 PM UTC+2, Rodrigo Kochenburger wrote:

    By the way, guys. If you need more flexibility you can always call the
    tools directly (compiler, linker, etc) instead of using the higher level
    tool and build your own build script.
    Yes, I'm aware of the possibility to call tools directly. It's okay for
    compiling and linking. But what about go test and godoc? Is it possible to
    use these two tools without conforming to the recommended directory layout?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Andrey mirtchovski at Aug 15, 2013 at 8:22 pm
    I don't understand the issue here. GOPATH is only really needed when
    you're doing "go get" or "go install". as long as you have a default
    GOPATH for the stuff you "go get", then the rest of your projects can
    be wherever you want. use relative imports to build library
    hierarchies completely outside the canon, but still use the go tool to
    compile them. I have plenty of projects outside GOPATH (including
    stuff with relative imports), the go tool manages them just fine*.

    $ echo $GOPATH
    /Users/aam
    $ cd work/logs/serve/
    $ go clean
    $ go build
    $ ls -l serve
    -rwxr-xr-x 1 aam aam 9232204 15 Aug 14:15 serve
    $ go install
    go install: no install location for directory
    /Users/aam/work/logs/serve outside GOPATH
    $

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nicolas Grilly at Aug 15, 2013 at 9:09 pm

    On Thursday, August 15, 2013 10:22:05 PM UTC+2, andrey mirtchovski wrote:

    I don't understand the issue here. GOPATH is only really needed when
    you're doing "go get" or "go install". as long as you have a default
    GOPATH for the stuff you "go get", then the rest of your projects can
    be wherever you want. use relative imports to build library
    hierarchies completely outside the canon, but still use the go tool to
    compile them. I have plenty of projects outside GOPATH (including
    stuff with relative imports), the go tool manages them just fine*.

    Putting packages outside GOPATH forces the use of relative import paths, as
    you wrote. The problem is that the use of relative import paths is heavily
    discouraged by most members of the Go team, according to what I've read in
    this list.

    As an example, here is a link to a related issue "cmd/go: Remove support
    for relative import paths":
    https://code.google.com/p/go/issues/detail?id=6147

    If I'm not mistaken, I see two other problems with this approach:
    - Every package is recompiled each time you build/test (binary packages are
    not stored in GOPATH/pkg as usual).
    - godoc -http is not usable to browse your package documentation.

    -- Nicolas

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Andrey mirtchovski at Aug 15, 2013 at 9:43 pm

    The problem is that the use of relative import paths is heavily
    discouraged by most members of the Go team, according to what I've read in
    this list.
    and putting stuff inside GOPATH is the recommended way of doing
    things. we shouldn't have this discussion then :)
    - Every package is recompiled each time you build/test (binary packages are
    not stored in GOPATH/pkg as usual).
    not a problem: go compiles fast.
    - godoc -http is not usable to browse your package documentation.
    godoc doesn't list a web browser in its requirements. use from the command line.

    that said, the relative import trick is not for everybody, i agree. if
    i understand your constraints correctly, what you really are asking
    for is to have your Go code live in two places at the same time:
    inside GOPATH when convenient, and outside GOPATH the rest of the
    time. you can do with some degree of success with union mounts on
    unix-y operating systems, or you can even use sshfs to do it. just
    mount at the right path and, even though you'll be editing outside
    GOPATH the go tool will find your code. that gives you a fine control
    over naming libraries as only the last part of the path must agree
    with the stated package name. here's an example:

    http://play.golang.org/p/twUCNd2UhJ

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Andrey mirtchovski at Aug 16, 2013 at 6:29 am
    Completely unrelated, but only just so: i discovered a Mark V Shaney
    quote (from his last posting on usenet) that is not only apt, but very
    appropriate. paraphrasing only slightly, replacing Unix with Go:

    "There is no problem and thus we disagree only if you don't feel like Go."

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nicolas Grilly at Aug 24, 2013 at 1:20 pm

    On Thursday, August 15, 2013 11:43:27 PM UTC+2, andrey mirtchovski wrote:

    that said, the relative import trick is not for everybody, i agree. if
    i understand your constraints correctly, what you really are asking
    for is to have your Go code live in two places at the same time:
    inside GOPATH when convenient, and outside GOPATH the rest of the
    time. you can do with some degree of success with union mounts on
    unix-y operating systems, or you can even use sshfs to do it. just
    mount at the right path and, even though you'll be editing outside
    GOPATH the go tool will find your code. that gives you a fine control
    over naming libraries as only the last part of the path must agree
    with the stated package name. here's an example:

    http://play.golang.org/p/twUCNd2UhJ
    Andrey, that's a very clever solution :)

    I would not recommend this as the default way to install the project I'm
    working on, because I don't want to impose the usage of union mounts and
    sshfs to my co-workers, but I could definitely use it for myself.

    Your suggestion reminds me of Plan 9 name spaces and the paper "Lexical
    File Names in Plan 9 or Getting Dot-Dot Right" by Rob Pike (
    http://plan9.bell-labs.com/sys/doc/lexnames.html).

    -- Nicolas

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Andrey mirtchovski at Aug 24, 2013 at 5:18 pm

    Your suggestion reminds me of Plan 9 name spaces and the paper "Lexical File
    Names in Plan 9 or Getting Dot-Dot Right" by Rob Pike
    (http://plan9.bell-labs.com/sys/doc/lexnames.html).
    that's where it comes from :)

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • TR NS at Aug 18, 2013 at 12:12 am

    On Thursday, August 15, 2013 3:47:48 PM UTC-4, Nicolas Grilly wrote:
    On Thursday, August 15, 2013 9:32:33 PM UTC+2, TR NS wrote:

    In the mean time, to work around I've had to do what you call the
    "obvious" way, but instead of using a symlink I have a small shell script
    in my project that reassigns the $GOPATH, which I can run before working on
    the project. Annoying but effective. Perhaps a more ideal solution would be
    to do the exporting when running a local build tool (e.g. Rake). By
    exporting the path in such a script it would only be temporary while the
    build is taking place, but then of course you have to write a bunch of
    build tasks to route to the go tool.
    You're right: what I called the "obvious way" is more hacky than obvious :)

    Could you give some details about your setup? What is the layout of your
    source tree? Is the root of the Go workspace inside or outside your source
    tree? To which directory $GOPATH points?
    First, let me qualify things by saying I am new to Go, so I am still
    learning how to do things too. Having said that, do you know about Ruby
    Gemsets? We'll its basically like that. By changing the $GOPATH to point to
    each project as I work on it, I have an isolated set of Go installs all
    geared for use by the project I am working on. Of course the downside it
    that It creates redundancy, but I can live with it.



    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nicolas Grilly at Aug 24, 2013 at 12:58 pm

    On Sunday, August 18, 2013 2:12:01 AM UTC+2, TR NS wrote:

    First, let me qualify things by saying I am new to Go, so I am still
    learning how to do things too. Having said that, do you know about Ruby
    Gemsets? We'll its basically like that. By changing the $GOPATH to point to
    each project as I work on it, I have an isolated set of Go installs all
    geared for use by the project I am working on. Of course the downside it
    that It creates redundancy, but I can live with it.
    Yes, I know about Ruby Gemsets (and Python virtualenv too) :)

    You're right, it's easy to get something similar in Go by having one
    workspace per project and changing the $GOPATH each time you switch to
    another project. But I would argue it's less necessary in Go than in Ruby
    and Python because Go produces executable binaries. You need the workspace
    only when you build and test your project, not when you simply execute it. It's
    very different in Ruby and Python where having the correct environment is
    essential to running your project.

    Because of this, and after a few weeks using Go, I now think that
    maintaining multiple workspaces is overkill in Go.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Benjamin Measures at Aug 18, 2013 at 12:55 am

    On Thursday, 15 August 2013 00:01:01 UTC+1, Nicolas Grilly wrote:

    Another way is to checkout the full project source (Go and Python) in
    $GOPATH/src/myproject and import "myproject/go" instead of "myproject".
    It's perfect and idiomatic from Go's point of view. But from Python's point
    of view, it's weird to be forced to live in $GOPATH. Go requires the
    project source to live in a directory src under $GOPATH. Now let's imagine
    that Python would require the project source to be rooted in a directory
    lib (for example) under its own $PYPATH. That would make Go's and Python's
    requirements contradictory and impossible to satisfy at the same time.
    It's only impossible if you've really completely ruled out symlinks. If you
    can't symlink go, why not python? Eg.
    ln -s $GOPATH/src/myproject/python $PYPATH/lib

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nicolas Grilly at Aug 18, 2013 at 10:06 am

    On Sun, Aug 18, 2013 at 2:55 AM, Benjamin Measures wrote:

    It's only impossible if you've really completely ruled out symlinks. If
    you can't symlink go, why not python? Eg.
    ln -s $GOPATH/src/myproject/python $PYPATH/lib
    You're right, it will work this way. Luckily, Python is more forgiving than
    Go with regards to symlinks :)

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Fatih Arslan at Jan 12, 2014 at 5:12 am
    We (Koding.com) use a single repository with several different
    programing languages in our source directory. Go code is a lot in our
    codebase, what we do is:

    1. We have a directory called "go" in which we put all our source code
    written in Go. This folder is inside our Git repo.
    2. This "go" directory is lied out as GOPATH, that means the inside
    there are directories like src/, pkg/, bin/ and so on.
    3. We have a custom build.sh script that lives in this "go" directory,
    when invoked it sets the GOPATH like:

    export GOPATH=$(cd "$(dirname "$0")"; pwd)

    and many other minor settings.

    4. The script also builds each our "main" packages with "go install".
    All the binaries will be copied by "go install" to "go/bin"

    It works perfectly and without any problem. If you want you can set
    the GOPATH in your bashrc/zshr and that way each "go get" package will
    also installed in that particular go directory (I'm working personally
    that way).

    However our repository is growing and we look to split out the Go
    directory into a separate git submodule. We are currently evaluating
    if this is going to work, because there are some conflicts we have to
    solve prior (we have config files that are shared and used by many
    other applications of us).

    On Wed, Aug 14, 2013 at 4:01 PM, Nicolas Grilly
    wrote:
    What is the recommended way to organize the source tree of a project written
    in several programming languages (for example Go and Python)?

    Ideally, we would want to keep everything in the same git repository because
    1) it makes easier to keep the Python and Go logic in sync and 2) some
    assets are shared by the Python and Go code.

    One obvious way is to checkout the project source in some arbitrary
    directory like $HOME/projects/myproject, create a subdirectory go containing
    all the Go code, and make it accessible in $GOPATH with ln -s
    $HOME/projects/myproject/go $GOPATH/src/myproject. But this solution looks
    like a dead-end because symbolic links were declared "not supported" many
    times on golang-nuts and golang-dev.

    Another way is to checkout the full project source (Go and Python) in
    $GOPATH/src/myproject and import "myproject/go" instead of "myproject". It's
    perfect and idiomatic from Go's point of view. But from Python's point of
    view, it's weird to be forced to live in $GOPATH. Go requires the project
    source to live in a directory src under $GOPATH. Now let's imagine that
    Python would require the project source to be rooted in a directory lib (for
    example) under its own $PYPATH. That would make Go's and Python's
    requirements contradictory and impossible to satisfy at the same time.

    Another way is to create a Go workspace directly in the project source tree
    and add it to the $GOPATH. This implies myproject would have its own
    src/bin/pkg directories and this approach was explicitly not recommended on
    this mailing list in the past
    (https://groups.google.com/forum/#!topic/golang-nuts/dxOFYPpJEXo).

    I'm not sure how to solve this in a simple and clean way. What is your
    experience solving this issue in your own projects?

    Nicolas

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.


    --
    Fatih Arslan

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedAug 15, '13 at 3:46a
activeMay 18, '15 at 2:10a
posts30
users11
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase