[go-nuts] gccgo + c++ library as a Golang package

The go tool supports .swig and .swigcxx files. The usual procedure
would be to write one of those files, and to also write a .go file.
The "go build" will run SWIG for you and give you a package that you
can import.

In order to use gccgo you currently need to install the go tool from
the gc sources (use tip for SWIG support) and use the -compiler=gccgo
option.

Ian

That is interesting. I don't see it documented anywhere. Could
you please point me to documentation?

Right now I am using swig's .i files and spend a lot of time figuring
out how it works. probably the new method with .swig is better.
where it is located?

Right now I have several external packages installed with the usual go
method - for example levigo is installed in go/pkg. Also I was able to
integrate
c++ library using gccgo _ g++ using swig as described on swig's page.
Now the question is - does the levigo go/pkg can work together with
the swig c++ app in one main app? This re-use is very important. It
would be very time consuming to make every c++ library to work through
swig - levigo is using leveldb which is c++ db library and I developed
my app with levigo and the traditional go. It would be nice if it can be
used with
c++ libraries too.
what is go tool?

I'm not aware of any. .swig files are mentioned at
http://tip.golang.org/cmd/go/ .
To get the tip sources for gc follow the instruction at
http://tip.golang.org/doc/install/source but omit "-u release" from
the "hg clone" command.
I'm sorry, I don't understand the question.
golang.org/cmd/go

Ian

I reworked my project according your recommendations, passed
the simple example with new math.

Now trying to use .swig file which is renamed .i file.
Ran the go command:


go build mygodir/mygoproject

But got the error message:

SWIG -go: -intgosize option required but not specified

How to pass a parameter to SWIG from go build command?

--Constantine

Could you open an issue for this please, at
http://code.google.com/p/go/issues? Thanks.

Ian

packages generated with gccgo + c++ + swig - can they be used
with other packages?

Yes.

Ian

1) Go installed from sources as you suggested:
http://tip.golang.org/doc/install/source but omit "-u release" from
the "hg clone" command.
2) .swig is not clear how it works so I used following command:

swig -v -Wall -go -intgosize 64 -gccgo -c++ myfunct.i

# generated ==> myfunct.i, myfunct.i_wrap.cxx

# Compile all these C/C++ and Go code:

g++ -c -g -O3 myfunct.cpp -o myfunct_impl.o # ==> myfunct_impl.o

g++ -c -g -O3 myfunct_wrap.cxx -o myfunct_wrap.o # ==> myfunct_wrap.o

gccgo -c myfunct.go -o myfunct.o # ==> myfunct.o


/* main.go

go fmt main.go

*/


/* main.go */

================================

*package* main


*import* (

"myfunct"

//"github.com/jmhodges/levigo"

)

================================

this command compiles:

gccgo -c main.go -o main.o # ==> main.go


now I want to use a package installed in go/pkg:

"github.com/jmhodges/levigo"

/* main.go */

================================

*package* main


*import* (

"myfunct"

"github.com/jmhodges/levigo"

)

================================

using command:

a) gccgo -c main.go -o main.o # ==> main.go

got error message:
main.go:11:28: error: import file ‘github.com/jmhodges/levigo’ not found

using command:

b) go build -gccgoflags '-L/usr/local/lib -lleveldb -Wl,-R,/usr/local/lib/'
-compiler gccgo main.go

got error message:

main.go:10:2: import "myfunct": cannot find package

So b) allows using existing package installed in go/pkg.

If it looks for "myfunct" package it follows i need to make "myfunct" a
package.

The question is how to do that?

When using the go tool you will rarely if ever invoke gccgo directly.
Instead use "go build -compiler=gccgo".

Set up your sources as per
http://blog.golang.org/2012/08/organizing-go-code.html .

Ian

got these files:

geoconvert_impl.o

geoconvert.o

geoconvert_wrap.o

how to link them and install them as a package?
e.g. how to use this command, I have three object files:

go install -gccgoflags '-v' -compiler=gccgo myproject/*mypackage*

The idea is that the go command should handle the details of creating
and using the package for you.

Ian

I am fighting a week with this and right now I don't see it working fully
automatically. It needs manual work which is OK because it can be automated
with
.bash script. I need it working until you figure out how to work
automatically.
There are a lot of details ad I doubt to can be automated 100%. This does
not
matter - I need it working.

I followed your recommendation for restructuring the code and I understand
better how go commands process is working. But still there some
missing steps to finish my task.

SWIG option for integer size prevents automatic code generation
using .swigxxx. I circumvented this by calling SWIG directly:

swig -v -Wall -go -intgosize 64 -gccgo -c++ geoconvert.i

This gives me:
1) geoconvert_wrap.cxx
2) geoconvert.go

if 1) is compiled directly returns errors - it need to be modified:

# edit geoconvert_wrap.cxx -> leave only:

#void SwigDoCgocall(void) __asm__("syscall.Cgocall");

#void SwigDoCgocallDone(void) __asm__("syscall.CgocallDone");

#void SwigDoCgocallBack(void) __asm__("syscall.CgocallBack");

#void SwigDoCgocallBackDone(void) __asm__("syscall.CgocallBackDone");


So it cannot be automated 100% because of SWIG code generation issue. Again

does not matter if I get the job done - it is one time task only


Then I compile all those C/C++ code:

g++ -c -g -O3 geoconvert.cpp -o geoconvert_impl.o # ==> geoconvert_impl.o

g++ -c -g -O3 geoconvert_wrap.cxx -o geoconvert_wrap.o # ==>
geoconvert_wrap.o

geoconvert.cpp is an interface I need to the library I am using.

then:

gccgo -c geoconvert.go -o geoconvert.o # ==> geoconvert.o

Now I have:
geoconvert.o
geoconvert_wrap.o
geoconvert_impl.o

Also I need two external libraries:
libGeographic.a
libstdc++.a

In order to have them in one place I combined them with archive:

ar cr *geoconvert.a* geoconvert.o geoconvert_wrap.o geoconvert_impl.o /usr
/local/lib/libGeographic.a /usr/lib/gcc/x86_64-linux-gnu/4.7.2/libstdc++.a

*geoconvert.a* is my final package as static library.

I suppose I have to copy it here manually:
/home/parallels/mygo/pkg/linux_amd64/*myproject*
*
*
*In order to call it from:*
*/* example.go */*

*import* (

"fmt"

"*myproject*/*geoconvert*"

)

Will this work?

--Constantine

ar cr geoconvert.a geoconvert.o geoconvert_wrap.o geoconvert_impl.o /usr
/local/lib/libGeographic.a /usr/lib/gcc/x86_64-linux-gnu/4.7.2/libstdc++.a

Please do file issues about these cases, or they will be forgotten. Thanks.


What errors?

The steps you are taking seem plausible.

Ian

I'm sorry, I'm often unclear on just what you are asking. Are you
looking for the -I or -L option?

http://gcc.gnu.org/onlinedocs/gccgo/Invoking-gccgo.html#Invoking-gccgo

Ian

Thank you for the links.

How to make this working:

I did these steps to install a go package:
1) go get github.com/jmhodges/levigo
2) package was installed at:
/home/parallels/mygo/pkg/linux_amd64/github.com/jmhodges
as levigo.a
3) I reference it in my:
------------
main.go
------------
import (
"github.com/jmhodges/levigo"
)

...
env := levigo.NewDefaultEnv()
...

I am trying to compile it:
gccgo -v -c main.go -o main.o
-I/home/parallels/mygo/pkg/linux_amd64/github.com/jmhodges
-fgo-pkgpath=levigo
-L-I/home/parallels/mygo/pkg/linux_amd64/github.com/jmhodges

got an error: reference to undefined name ‘levigo’

How to correctly form -I and -L paths?

--Constantine

When using gccgo you also need to explicitly use -l options to pull in
the libraries that you link against. That is not true when using gc.
The go tool will provide those -l options automatically, but when you
are not using you need to do it yourself.

Ian

go tool is not working in this case and I have to specify -I and -L myself

This is not working:

gccgo -v -c main.go -o main.o
-I/home/parallels/mygo/pkg/linux_amd64/github.com/jmhodges
-fgo-pkgpath=levigo
-L-I/home/parallels/mygo/pkg/linux_amd64/github.com/jmhodges

What happens?

What files are in /home/parallels/mygo/pkg/linux_amd64/github.com/jmhodges ?

Using -L -I does not make sense.

I'm sorry that this stuff doesn't just work. But it doesn't. And to
make it work you are going to have to do a little reading in how to
use the tools.

Ian

I described in:
2) package was installed at: /home/parallels/mygo/pkg/linux_amd64/
github.com/jmhodges
as levigo.a

In /home/parallels/mygo/pkg/linux_amd64/github.com/jmhodges
it is the package levigo.a

What happens is that after executing:

gccgo -v -c main.go -o main.o -I/home/parallels/mygo/pkg/linux_amd64/
github.com/jmhodges
-fgo-pkgpath=levigo
-L-I/home/parallels/mygo/pkg/linux_amd64/github.com/jmhodges

got an error: reference to undefined name ‘levigo’
levigo is the name of the just installed package

Here is a summary of the issue:

1) package installed in:
/home/parallels/mygo/pkg/linux_amd64/github.com/jmhodges/levigo.a

2) -Idir - Specify a directory to use when searching for an import package
at compile time.

3) -Idir Specify a directory to use when searching for an import package at
compile time.
gccgo -c dbgen.go
-I/home/parallels/mygo/pkg/linux_amd64/github.com/jmhodges
-L/home/parallels/mygo/pkg/linux_amd64/github.com/jmhodges

4) dbgen.go:63:28: error: import file ‘github.com/jmhodges/levigo’ not found

5) the line in dbgen.go:63 ==> import "github.com/jmhodges/levigo"

I need to be able to form valid -I and -L

compiling with -v option I see '-shared-libgcc'

the package levigo.a s static - with .a extension - I assume because of
-shared-libgcc' it is searched for dynamic library, not static:


COLLECT_GCC_OPTIONS='-v' '-c' '-I'
'/home/parallels/mygo/pkg/linux_amd64/github.com/jmhodges'
'-L/home/parallels/mygo/pkg/linux_amd64/github.com/jmhodges' '-o' 'dbgen.o'
'-shared-libgcc' '-mtune=generic' '-march=x86-64'
/usr/lib/gcc/x86_64-linux-gnu/4.7/go1 dbgen.go -quiet -dumpbase dbgen.go
-mtune=generic -march=x86-64 -auxbase-strip dbgen.o -version -I
/home/parallels/mygo/pkg/linux_amd64/github.com/jmhodges
-L/home/parallels/mygo/pkg/linux_amd64/github.com/jmhodges
-L/usr/lib/gcc/x86_64-linux-gnu/4.7
-L/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../x86_64-linux-gnu
-L/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../../lib -L/lib/x86_64-linux-gnu
-L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib
-L/usr/lib/gcc/x86_64-linux-gnu/4.7/../../.. -o /tmp/ccxUmgbY.s


--Constantine

I have only limited knowledge but, isn't the path above for a
gc-installed go package ?
therefore explaining why gccgo then fails to find that package for
import later on ?

IIRC, gccgo will install go packages under $GOPATH/pkg/gccgo.
e.g: $GOPATH/pkg/gccgo/github.com/jmhodges/levigo.a

-s

Just double checked:

1) my GOPATH="/home/parallels/mygo"

2) running from terminal: go get github.com/jmhodges/levigo

3) got the package levigo.a in:

/home/parallels/mygo/pkg/linux_amd64/github.com/jmhodges

--Constantine

A name like "levigo.a" is the right name to use when using the gc
compiler. It is not the right name to use when using gccgo. Sorry
for not considering that earlier.

Ian

I was installing it with "go get github.com/jmhodges/levigo"
You mean it needs to be installed with *gccgo*? How to do that?
Do I have to download the source. place it in $GOPATH/src/levigo
and install it from there with gccgo? How to install it with gccgo?

go get -compiler=gccgo github.com/jmhodges/levigo

-s

Thank you!

I confirm I got it installed in right place now:

1) go get github.com/jmhodges/levigo
2) package installed as:
/home/parallels/mygo/pkg/linux_amd64/github.com/jmhodges/levigo.a

Now how to compile it?
this is not working:
gccgo dbgen.go -I/home/parallels/mygo/pkg/gccgo/github.com/jmhodges

this command compiled it:

gccgo -c -dbgen.go -I/home/parallels/mygo/pkg/gccgo/github.com/jmhodges
-L/home/parallels/mygo/pkg/gccgo/github.com/jmhodges -llevigo

./dbgen.go is running it successfully!

sorry it was old executable.

running:
gccgo -dbgen.go -I/home/parallels/mygo/pkg/gccgo/github.com/jmhodges
-L/home/parallels/mygo/pkg/gccgo/github.com/jmhodges -llevigo

got the error:
/usr/lib/gcc/x86_64-linux-gnu/4.7/libgo.so: undefined reference to
`__go_init_main'
/usr/lib/gcc/x86_64-linux-gnu/4.7/libgo.so: undefined reference to
`main.main'

You aren't using -c, so you are linking the program. You don't have a
main package, or the main package doesn't have a main function.

Ian

No. -shared-libgcc is irrelevant here.

The compiler is not able to find the import file
github.com/jmhodges/levigo. In order for the compiler to find it,
somewhere on the search path (specified by -I and -L options) there
must be a directory github.com/jmhodges. In that directory there must
be a file levigo.gox, liblevigo.so, liblevigo.a, levigo.o. This is
documented at http://gcc.gnu.org/onlinedocs/gccgo/Import-and-Export.html
.

You indicated earlier that the file is named levigo.a. So I guess you
need to name it liblevigo.a. And it needs to be in a directory
github.gom/jmhodges.

Ian

You are using the -c option, so you are not linking. Somewhere in
your source file there is a reference to the name "levigo" and that
name is not defined.

What do your import statements look like?

Do you get any other error messages?

Does one of the packages you are importing include Go code that starts
with "package levigo"?

Ian

yes I am importing it here and error message is from here:

// dbgen.go
package main

import (
"bytes"
"encoding/csv"
"fmt"
"github.com/jmhodges/levigo"
"os"
"path/filepath"
"strconv"
"strings"
"time"
)

if the package is installed like this:
go install myappdir/dbgen

and imported package uses another package installed in a standard place
/usr/lical/lib, how at run time the app importing the myappdir/dbgen will
find this another package?

All Go packages are normally linked directly into the binary. They
are not loaded at runtime.

Ian

In theory yes. The package installed, the app using it compiled,
yet at run timer complained a dependent .so package cannot be found.
The .so extension suggests it is dynamic library searched at run time

Details?

Was this .so file generated from SWIG in some way?

Ian

no, this levigo is using leveled.so which needs to be installed in a
standard
place like /usr/local/lib. I made a package using levigo, it installs but
when running
from an app it complains for leveled.so.

I tried another way to embed levigo in my app, this time complaying
github.com/jmhodges/liblevigo cannot be found.

I installed it with gccgo:
#######
# gccgo
#######
# go get -compiler=gccgo github.com/jmhodges/levigo
# package installed as:
/home/parallels/mygo/pkg/gccgo/github.com/jmhodges/liblevigo.a

# -Idir - Specify a directory to use when searching for an import package
at compile time.
# -Idir Specify a directory to use when searching for an import package at
compile time.
gccgo -v -c dbgen.go
-I/home/parallels/mygo/pkg/linux_amd64/github.com/jmhodges
-L/home/parallels/mygo/pkg/linux_amd64/github.com/jmhodges -llevigo

it says github.com/jmhodges/liblevigo cannot be found

sorry copy./pasted wrong command

gccgo -v -c dbgen.go -static-libgcc
-I/home/parallels/mygo/pkg/gccgo/github.com/jmhodges
-L/home/parallels/mygo/pkg/gccgo/github.com/jmhodges -llevigo

it complains:
dbgen.go:18:28: error: import file ‘github.com/jmhodges/levigo’ not found

COLLECT_GCC_OPTIONS='-v' '-c' '-static-libgcc' '-I'
'/home/parallels/mygo/pkg/gccgo/github.com/jmhodges'
'-L/home/parallels/mygo/pkg/gccgo/github.com/jmhodges' '-o' 'dbgen.o'
'-mtune=generic' '-march=x86-64'
/usr/lib/gcc/x86_64-linux-gnu/4.7/go1 dbgen.go -quiet -dumpbase dbgen.go
-mtune=generic -march=x86-64 -auxbase-strip dbgen.o -version -I
/home/parallels/mygo/pkg/gccgo/github.com/jmhodges
-L/home/parallels/mygo/pkg/gccgo/github.com/jmhodges
-L/usr/lib/gcc/x86_64-linux-gnu/4.7
-L/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../x86_64-linux-gnu
-L/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../../lib -L/lib/x86_64-linux-gnu
-L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib
-L/usr/lib/gcc/x86_64-linux-gnu/4.7/../../.. -o /tmp/ccG8mSuO.s

You need to have an option -I DIR (or -L DIR) such that
DIR/github.com/jmhodges/levigo/liblevigo.a exists. Your earlier
message said that the file
/home/parallels/mygo/pkg/gccgo/github.com/jmhodges/liblevigo.a exists.
That implies that you should use -I /home/parallels/mygo/pkg/gccgo .

Ian

I see the file:
/home/parallels/mygo/pkg/gccgo/github.com/jmhodges/levigo.a
Yet:
gccgo dbgen.go -I/home/parallels/mygo/pkg/gccgo/github.com/jmhodges
gives me an error:
*dbgen.go*:18:28: error: import file ‘github.com/jmhodges/levigo’ not found

*dbgen.go:*

*import* (

"bytes"

"encoding/csv"

"fmt"

* "github.com/jmhodges/levigo"*

"io"

"os"

"path/filepath"

"strings"

)


The only thing working for me with gccgo is when I made an interface with
SWIG

with the 1st library.


But this gccgo project cannot be used with a 2nd library (in Golang)
installed as an external packages like l*"github.com/jmhodges/levigo"* probably
because it depends from another (C++) library in /usr/local/lib - *leveldb*.
levigo is C interface to leveldb.


For a plan B now I am exploring another option - to generate SWIG code
directly from *leveldb *and combine them together in one project because I
tried and cannot create a package using a SWIG project and gccgo. If
nothing works in plan C - I will create a web service running only the 1st
library which can be compiled standalone and called from the main project.

It would be slow but will work at least.


--Constantine


-

By the way after re-thinking I believe plan C is the best for me.

I want to isolate the gccgo based project - to compile it as a standalone
executable and run as a service. If is a service and called only from other
Golang apps on same physical machine what is best way to implement it?

It can be web service or socket service.but for intra-machine communication
probably there is a better way to communicate which to work with Go
from one side and gccgo from another?

--Constantine

Sockets or pipes would be best.

Ian

I guess I don't understand why you are using that -I option, when in
the e-mail to which you are replying I suggested using

-I /home/parallels/mygo/pkg/gccgo

Also, earlier you said you had the file liblevigo.a, and now you are
saying levigo.a, so I'm not sure which it is.

Ian

Sorry for misreading your suggestion. Now I can link with this command:
*gccgo dbgen.go -I /home/parallels/mygo/pkg/gccgo*

Now, referencing it. I had this statement in "go".
levigo.NewDefaultEnv

now gccgo compiler complains:
dbgen.go:(.text+0xa3e): undefined reference to `*github.com_jmhodges_levigo*
.NewDefaultEnv'

How is the proper way to reference the function in the library?

--Constantine

What is your exact link command line? This suggests that you are not
explicitly passing the library to the link command.

Ian

I added the -c option and now I got the* **dbgen.o*
*gccgo -c dbgen.go -I /home/parallels/mygo/pkg/gccgo*

Now what command to use to install it as a package?
With go it should be:
go install my_apps/dbgen

How to do it with gccgo?

The easiest way is to put dbgen.o in some directory DIR1/DIR2 should
that you use -I DIR1 on the command line and you import "DIR2/dbgen"
in your Go file.

You will also have to add it to the link line.

Ian

That does not sound like a Go specific issue at all. It sounds like
the usual issue with using any shared library. If you are using
GNU/Linux, you can edit /etc/ld.so.conf; for details, see "man
ldconfig". Or you can set the environment variable LD_LIBRARY_PATH to
/usr/local/lib. Or you can link with the option
"-Wl,-rpath,/usr/local/lib". None of these options are Go-specific in
any way.

Ian