FAQ
Spurred by Dmitri's work in cl/8793 I've been thinking about the
shortcomings in the gl package and where it should be going.

Some background: the gl package is designed for GL ES 2.0 on
Android/iOS. It also works on OS X and Linux/X11 for the purpose of
development (and eventually testing). This means you can write an app
designed for android, and compile it with the normal go command to get
a desktop binary. These desktop targets are and should remain a
secondary goal for now.

In cl/8793 there is a proposal for new build tag: js. This increases
the number of ways to configure the gl package, right now it selects
entirely on GOOS and GOARCH. I would like to avoid adding a build tag
mechanism because I think by solving another limit of the gl package
we can get to a place where backends for the GL package can be
registered by external packages.

There are two limits of the gl package I would like to address before
1.5 is released.

The first is that using it requires an understanding the
implementation details of goroutines. Each GL call maps directly onto
the C function which needs the GL context to be in thread-local
storage. This means careful use of runtime.LockOSThread is necessary
to use the package correctly.

The second is it will be very hard to add windows support without
importing code[1] under other copyright licenses and generating code.
These are completely acceptable copyrights, but their existence
complicates vendoring the mobile package, which I will want to
encourage when 1.5 is released. More generally: our only configuration
tool is #ifdef in cgo headers. If a software-based GL existed that
allowed us to write a good testing package, it would be very difficult
to integrate.

What I'm exploring is introducing a gl.Context interface with the
current functions hanging off it as methods. The gl.Context exported
by the package will forward function calls to a goroutine locked to an
OS thread, which will make the C calls. This removes the need to
understand goroutine implementation details to use the package.

Any performance lost due to the indirection can be recovered by
batching calls across the cgo boundary (each C function call in a
modern GL implementation appears to do little more than drop a task on
an asynchronous queue, so most of the cost of the function calls is
the cgo overhead).

There are several points I'm not sure about yet: does the gl package
continue to offer package-level functions, or export a package
variable containing the default gl.Context? Either way I intend to
change the function signatures very little, as I think there is a lot
of value in keeping the functions close to the C headers. OpenGL
knowledge is hard to come by and I don't want people to have to learn
even more than they have acquired elsewhere to use it.

Thoughts?

d.

[1] https://code.google.com/p/angleproject/

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Nigel Tao at May 22, 2015 at 11:40 am

    On Fri, May 22, 2015 at 8:20 PM, David Crawshaw wrote:
    This means careful use of runtime.LockOSThread is necessary
    to use the package correctly.
    Removing the need for careful use of runtime.LockOSThread seems like a
    great idea to me.

    Any performance lost due to the indirection can be recovered by
    batching calls across the cgo boundary
    Will batching require an explicit Flush call or perhaps a
    "ctx.StartBatch(); defer ctx.StopBatch()" or perhaps a bufgl.Context
    analogous to bufio.Writer?

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • David Crawshaw at May 22, 2015 at 11:52 am

    On Fri, May 22, 2015 at 7:40 AM, Nigel Tao wrote:
    On Fri, May 22, 2015 at 8:20 PM, David Crawshaw wrote:
    This means careful use of runtime.LockOSThread is necessary
    to use the package correctly.
    Removing the need for careful use of runtime.LockOSThread seems like a
    great idea to me.

    Any performance lost due to the indirection can be recovered by
    batching calls across the cgo boundary
    Will batching require an explicit Flush call or perhaps a
    "ctx.StartBatch(); defer ctx.StopBatch()" or perhaps a bufgl.Context
    analogous to bufio.Writer?
    I was experimenting with something like bufgl.Context. There are some
    tricky edge cases -- some functions pass in pointers to arrays and
    storing those means users cannot safely reuse the memory. This is
    often fine, but a semantic change, so I am thinking of making it
    something users have to opt in to. Also, it will come later.

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dmitri Shuralyov at May 24, 2015 at 3:41 am
    Hi David,

    The overall idea sounds great, and I think it's absolutely worth exploring!
    I don't think the current x/mobile/gl should go away until the replacement
    is ready and is deemed to completely supercede it.

    I have some questions and thoughts on some points you've made, I'll post
    them below.
    In cl/8793 there is a proposal for new build tag: js. This increases
    the number of ways to configure the gl package, right now it selects
    entirely on GOOS and GOARCH. I would like to avoid adding a build tag
    mechanism because I think by solving another limit of the gl package
    we can get to a place where backends for the GL package can be
    registered by external packages.
    Something like the image package and various image/{png,jpeg,...} backends.
    This sounds great, as it allows backends to come from different places, and
    users can pick and choose what they're interested in! All while having one
    common API, so user code is compatible.
    The first is that using it requires an understanding the
    implementation details of goroutines. Each GL call maps directly onto
    the C function which needs the GL context to be in thread-local
    storage. This means careful use of runtime.LockOSThread is necessary
    to use the package correctly.
    To what degree is this true?

    My understanding and use of runtime.LockOSThread has been very simple.
    Please let me know if the following is not correct.

    The way I approach this is by having:

    func init() {
    runtime.LockOSThread()
    }

    In every program where I use OpenGL, and issuing all OpenGL calls from the
    main goroutine. That's all. Is that correct? If so, that doesn't seem like
    "careful use of runtime.LockOSThread is necessary" is a large problem, is
    it? I admit that not even having to worry about that would be a plus (I'm
    just trying to undersand if I'm missing something, because you describe it
    as a bigger probelm than I thought it was).

    Part of my understanding is based on discussion in
    https://groups.google.com/d/msg/golang-nuts/6Rh4e0wvhn4/wQ7KqFVVX2gJ.
    What I'm exploring is introducing a gl.Context interface with the
    current functions hanging off it as methods. The gl.Context exported
    by the package will forward function calls to a goroutine locked to an
    OS thread, which will make the C calls. This removes the need to
    understand goroutine implementation details to use the package.

    Any performance lost due to the indirection can be recovered by
    batching calls across the cgo boundary (each C function call in a
    modern GL implementation appears to do little more than drop a task on
    an asynchronous queue, so most of the cost of the function calls is
    the cgo overhead).
    This sounds good to me.
    Either way I intend to
    change the function signatures very little, as I think there is a lot
    of value in keeping the functions close to the C headers.
    I support this *very* strongly! In addition to that, it will make updating
    existing code to use the new package a lot easier if existing code that
    looks like `gl.ClearColor(...)` continues to work, and only import paths
    and some inititalization code needs to be changed.
    There are several points I'm not sure about yet: does the gl package
    continue to offer package-level functions, or export a package
    variable containing the default gl.Context?
    One thought here: be very careful here, and consider how goimports will
    intersect with this.

    From my experience, my initial attempt at a GL binding with desktop and
    browser backends was at https://github.com/shurcooL/gogl.

    It creates a struct `Context` with methods for OpenGL calls.

    The package name `gogl` resulted in usage code being:

    import "github.com/shurcooL/gogl"

    // gl is the OpenGL context.
    var gl *gogl.Context

    var projMatrixUniform *gogl.Uniform

    func main() {
    ...
    gl = ... // Initialize the OpenGL context.

    gl.ClearColor(...)
    ...
    }

    Which is already not ideal because of things like `*gogl.Uniform` instead
    of cleaner `gl.Uniform`.

    However, the *largest problem with that scheme* was the following. If my Go
    package consists of multiple .go files, and some of them happen not to
    declare any `*gogl.Program` or other types, then that particular file did
    not import "github.com/shurcooL/gogl".

    Because goimports operates on a single .go file at at time, rather than
    full package, it didn't know that gl.BufferData refers to `var gl
    *gogl.Context` and it would try to add some other gl package import path
    statement. It was pretty bad and drove me away from that scheme. Be careful
    not to repeat this mistake.

    Thanks! :)

    On Friday, May 22, 2015 at 4:52:38 AM UTC-7, David Crawshaw wrote:

    On Fri, May 22, 2015 at 7:40 AM, Nigel Tao <nige...@golang.org
    <javascript:>> wrote:
    On Fri, May 22, 2015 at 8:20 PM, David Crawshaw <craw...@golang.org
    <javascript:>> wrote:
    This means careful use of runtime.LockOSThread is necessary
    to use the package correctly.
    Removing the need for careful use of runtime.LockOSThread seems like a
    great idea to me.

    Any performance lost due to the indirection can be recovered by
    batching calls across the cgo boundary
    Will batching require an explicit Flush call or perhaps a
    "ctx.StartBatch(); defer ctx.StopBatch()" or perhaps a bufgl.Context
    analogous to bufio.Writer?
    I was experimenting with something like bufgl.Context. There are some
    tricky edge cases -- some functions pass in pointers to arrays and
    storing those means users cannot safely reuse the memory. This is
    often fine, but a semantic change, so I am thinking of making it
    something users have to opt in to. Also, it will come later.
    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • David Crawshaw at May 27, 2015 at 8:09 pm
    Resending to golang-dev,
    On Sat, May 23, 2015 at 11:41 PM, Dmitri Shuralyov wrote:
    The first is that using it requires an understanding the
    implementation details of goroutines. Each GL call maps directly onto
    the C function which needs the GL context to be in thread-local
    storage. This means careful use of runtime.LockOSThread is necessary
    to use the package correctly.
    To what degree is this true?

    My understanding and use of runtime.LockOSThread has been very simple.
    It is simple in small programs written in a single thread style. But
    as soon as you try to use Go's natural concurrency model, everything
    breaks. Consider something mild:

    // on the the GL thread.
    work := make(chan *Job)
    go func() {
             for job := range work {
                 job.MakesGLCall()
             }
             <-done
    }()
    work <- ... // feed jobs to goroutine

    This piece of code doesn't work. The started goroutine does not have
    the context in TLS. That's surprising to a Go programmer, because we
    generally do not have a notion of thread-local storage. Turn the
    program around and it starts working:

    work := make(chan *Job)
    go func() {
       work <- ...
    }()
    for job := range work {
         job.MakesGLCall()
    }

    I would like to avoid these surprises.

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Nigel Tao at May 27, 2015 at 6:04 am

    On Fri, May 22, 2015 at 9:52 PM, David Crawshaw wrote:
    I was experimenting with something like bufgl.Context.
    Can you give some more detail on what you have in mind for the API? A
    bufio.Writer that wraps an io.Writer is much simpler because io.Writer
    has only one method, and bufio's Write can fill in sensible return
    values without waiting for the underlying io's Write. Also, a
    bufio.Writer doesn't prohibit callers from modifying input buffers
    after a Write call returns. I suspect that a bufgl.Context will have
    to be more complicated in API, not just implementation, if it wants to
    be efficient.

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Nigel Tao at May 27, 2015 at 6:31 am

    On Wed, May 27, 2015 at 4:04 PM, Nigel Tao wrote:
    On Fri, May 22, 2015 at 9:52 PM, David Crawshaw wrote:
    I was experimenting with something like bufgl.Context.
    Can you give some more detail on what you have in mind for the API?
    BTW some design discussion is also happening on
    https://go-review.googlesource.com/#/c/10396

    --
    You received this message because you are subscribed to the Google Groups "golang-dev" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-dev @
categoriesgo
postedMay 22, '15 at 10:20a
activeMay 27, '15 at 8:09p
posts7
users3
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase