FAQ
I've been thinking how to properly approach scientific computing with Go.

It seems that there are many cases which can be improved at the level of
compiler i.e. multi-dimensional arrays, matrices, sets. There are also
SIMD, GPGPUs, OpenCL, OpenMP, FPUs etc. basically hardware features and
libraries that can improve the speed of calculations. So the question is
how to properly target all of that?

Adding support for each of those special cases would make the compiler
slower, more complex and make it harder to maintain. So adding all of that
wouldn't probably be a good idea.

So what would be the alternative? One approach that would be viable is
runtime code generation, it's complex in it's internals, but idea is
simple. Take a string and convert it to a Go function at runtime. So
convert a string to an AST, run tons of optimizations and convert it to
specialized byte-code and finally run it.

When we are talking about scientific computing, this means we should do
aggressive inlining, optimizations that may take a lot of time... e.g. if
an optimization strategy takes 4min and it improves the computation by 20%
then with calculations running longer than 20min it's worthwhile to run
that optimization... but it may not be a good idea for general programming.

So, maybe, the better approach for scientific computation is to provide the
packages for DSLs, AST conversions, transformations, optimizations, targets
and nice runtime code generation support. I know the details will get
complicated, but maybe it's a better than adding every single thing to the
compiler. With Go compiler eventually being written in Go, it would mean
that we can use the same code in runtime code generation.

What are your thoughts on this?

+ egon

(PS. I'm probably not the best person for implementing it, but maybe
someone else gets inspired and runs with the idea :) )

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Egon at Mar 28, 2014 at 9:58 pm

    On Friday, March 28, 2014 11:31:54 PM UTC+2, egon wrote:
    I've been thinking how to properly approach scientific computing with Go.

    It seems that there are many cases which can be improved at the level of
    compiler i.e. multi-dimensional arrays, matrices, sets. There are also
    SIMD, GPGPUs, OpenCL, OpenMP, FPUs
    Sorry, brain typed the wrong thing... instead of FPU I really wanted to
    type FPGA... :)

    etc. basically hardware features and libraries that can improve the speed
    of calculations. So the question is how to properly target all of that?

    Adding support for each of those special cases would make the compiler
    slower, more complex and make it harder to maintain. So adding all of that
    wouldn't probably be a good idea.

    So what would be the alternative? One approach that would be viable is
    runtime code generation, it's complex in it's internals, but idea is
    simple. Take a string and convert it to a Go function at runtime. So
    convert a string to an AST, run tons of optimizations and convert it to
    specialized byte-code and finally run it.

    When we are talking about scientific computing, this means we should do
    aggressive inlining, optimizations that may take a lot of time... e.g. if
    an optimization strategy takes 4min and it improves the computation by 20%
    then with calculations running longer than 20min it's worthwhile to run
    that optimization... but it may not be a good idea for general programming.

    So, maybe, the better approach for scientific computation is to provide
    the packages for DSLs, AST conversions, transformations, optimizations,
    targets and nice runtime code generation support. I know the details will
    get complicated, but maybe it's a better than adding every single thing to
    the compiler. With Go compiler eventually being written in Go, it would
    mean that we can use the same code in runtime code generation.

    What are your thoughts on this?

    + egon

    (PS. I'm probably not the best person for implementing it, but maybe
    someone else gets inspired and runs with the idea :) )
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Carlos Castillo at Mar 28, 2014 at 11:07 pm
    I believe you just described OpenCL.
    On Friday, March 28, 2014 2:31:54 PM UTC-7, egon wrote:

    I've been thinking how to properly approach scientific computing with Go.

    It seems that there are many cases which can be improved at the level of
    compiler i.e. multi-dimensional arrays, matrices, sets. There are also
    SIMD, GPGPUs, OpenCL, OpenMP, FPUs etc. basically hardware features and
    libraries that can improve the speed of calculations. So the question is
    how to properly target all of that?

    Adding support for each of those special cases would make the compiler
    slower, more complex and make it harder to maintain. So adding all of that
    wouldn't probably be a good idea.

    So what would be the alternative? One approach that would be viable is
    runtime code generation, it's complex in it's internals, but idea is
    simple. Take a string and convert it to a Go function at runtime. So
    convert a string to an AST, run tons of optimizations and convert it to
    specialized byte-code and finally run it.

    When we are talking about scientific computing, this means we should do
    aggressive inlining, optimizations that may take a lot of time... e.g. if
    an optimization strategy takes 4min and it improves the computation by 20%
    then with calculations running longer than 20min it's worthwhile to run
    that optimization... but it may not be a good idea for general programming.

    So, maybe, the better approach for scientific computation is to provide
    the packages for DSLs, AST conversions, transformations, optimizations,
    targets and nice runtime code generation support. I know the details will
    get complicated, but maybe it's a better than adding every single thing to
    the compiler. With Go compiler eventually being written in Go, it would
    mean that we can use the same code in runtime code generation.

    What are your thoughts on this?

    + egon

    (PS. I'm probably not the best person for implementing it, but maybe
    someone else gets inspired and runs with the idea :) )
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Raul Mera at Mar 28, 2014 at 11:14 pm

    On Friday, March 28, 2014 10:31:54 PM UTC+1, egon wrote:
    I've been thinking how to properly approach scientific computing with Go.

    It seems that there are many cases which can be improved at the level of
    compiler i.e. multi-dimensional arrays, matrices, sets. There are also
    SIMD, GPGPUs, OpenCL, OpenMP, FPUs etc. basically hardware features and
    libraries that can improve the speed of calculations. So the question is
    how to properly target all of that?

    Adding support for each of those special cases would make the compiler
    slower, more complex and make it harder to maintain. So adding all of that
    wouldn't probably be a good idea.
    Please let's not mix hardware facilities or especial libraries, which are
    indeed special cases, with multi-dimensional slices (we already have
    multidimensional arrays) which are a natural way to represent matrices,
    which, in turn, are not quite an "special case" but pretty much the main
    thing you use for any code that deals with mathematics in some way (which
    is a lot of code, not only scientific). Multidimensional slices are not
    linked to a particular library, hardware or even algorithm. The proposal
    currently being discussed in another thread here is a natural extension of
    what exists, and is careful to keep complexity to a minimun.

    I very much agree with you about GPUGPU, OpenCL, etc. Those things should
    not be in the language. Also, noone in the scientific community has asked
    for sets, as far as I know (sorry if I am wrong here), and 2-dimensional
    slices are the same as matrices for our purposes. This means that while in
    your mail it appears that a long list of language additions is needed/asked
    for in order for Go to succeed in scientific programming, the community
    is actually asking for only one thing,

    Also, saying "multi-dimensional slices, matrices" gives a misleading
    impression of several issues, when both things are actually the same for
    our purposes. Since I have not seen anyone from the numerical community
    asking for sets, the whole list you present as language changes wanted by
    the scientific computing people is reduced to one item.


    So what would be the alternative? One approach that would be viable is
    runtime code generation, it's complex in it's internals, but idea is
    simple. Take a string and convert it to a Go function at runtime. So
    convert a string to an AST, run tons of optimizations and convert it to
    specialized byte-code and finally run it.

    When we are talking about scientific computing, this means we should do
    aggressive inlining, optimizations that may take a lot of time... e.g. if
    an optimization strategy takes 4min and it improves the computation by 20%
    then with calculations running longer than 20min it's worthwhile to run
    that optimization... but it may not be a good idea for general programming.

    I do not like that separation between "scientific" and "general"
    computing. We both want our code clean, and have to deal with large
    programs. Still, I agree that if you want to use special hardware and the
    like, or want the even the *last bit* of performance, no matter what, it is
    reasonable that you pay the price. I suspect the current approach of
    delegating those things to C is good enough, but I do not have a strong
    opinion (well, my strong opinion, like yours, is "leave them out of the
    language").


    So, maybe, the better approach for scientific computation is to provide
    the packages for DSLs, AST conversions, transformations, optimizations,
    targets and nice runtime code generation support. I know the details will
    get complicated, but maybe it's a better than adding every single thing to
    the compiler. With Go compiler eventually being written in Go, it would
    mean that we can use the same code in runtime code generation.

    What are your thoughts on this?
    I will not speak for them, but I don't see that the numerical/scientific Go
    comunity (mostly, the gonum people) has proposed or even thinks that we
    should add every single thing to the compiler, but are actually against
    that. That part sounds bit like a straw man to me. We want a natural way
    to represent a very common data structure. We do not want fotran-like
    matrix multiplication in the language, and we most certainly have not asked
    for language support for the other things you mention. For several of them,
    we can probably just work them out at library level, for some, like the GPU
    thing, we probably will always need to delegate to C (or, actually, CUDA or
    similar).

    I am not particularly against your proposal, but I don't like that it is
    presented as an alternative to what the scientific/numerical Go community
    are asking for.



    + egon
    (PS. I'm probably not the best person for implementing it, but maybe
    someone else gets inspired and runs with the idea :) )
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Raul Mera at Mar 28, 2014 at 11:55 pm
    As a clarification, Egon does not actually say numeric/scientific computing
    people are asking for the things he mention, and he was not actually
    presenting his idea as an alternative to the proposal in the other thread.
    It is important to state clearly that nobody is asking for those things in
    the language, and also to remark that Egon's idea is not an alternative
    to what we have asked for in the 2D-slices thread, but my reply was
    somewhat too jumpy and I read more than what Egon had written, sorry about
    that.
    On Saturday, March 29, 2014 12:14:50 AM UTC+1, Raul Mera wrote:


    On Friday, March 28, 2014 10:31:54 PM UTC+1, egon wrote:

    I've been thinking how to properly approach scientific computing with Go.

    It seems that there are many cases which can be improved at the level of
    compiler i.e. multi-dimensional arrays, matrices, sets. There are also
    SIMD, GPGPUs, OpenCL, OpenMP, FPUs etc. basically hardware features and
    libraries that can improve the speed of calculations. So the question is
    how to properly target all of that?

    Adding support for each of those special cases would make the compiler
    slower, more complex and make it harder to maintain. So adding all of that
    wouldn't probably be a good idea.
    Please let's not mix hardware facilities or especial libraries, which are
    indeed special cases, with multi-dimensional slices (we already have
    multidimensional arrays) which are a natural way to represent matrices,
    which, in turn, are not quite an "special case" but pretty much the main
    thing you use for any code that deals with mathematics in some way (which
    is a lot of code, not only scientific). Multidimensional slices are not
    linked to a particular library, hardware or even algorithm. The proposal
    currently being discussed in another thread here is a natural extension of
    what exists, and is careful to keep complexity to a minimun.

    I very much agree with you about GPUGPU, OpenCL, etc. Those things should
    not be in the language. Also, noone in the scientific community has asked
    for sets, as far as I know (sorry if I am wrong here), and 2-dimensional
    slices are the same as matrices for our purposes. This means that while in
    your mail it appears that a long list of language additions is needed/asked
    for in order for Go to succeed in scientific programming, the community
    is actually asking for only one thing,

    Also, saying "multi-dimensional slices, matrices" gives a misleading
    impression of several issues, when both things are actually the same for
    our purposes. Since I have not seen anyone from the numerical community
    asking for sets, the whole list you present as language changes wanted by
    the scientific computing people is reduced to one item.


    So what would be the alternative? One approach that would be viable is
    runtime code generation, it's complex in it's internals, but idea is
    simple. Take a string and convert it to a Go function at runtime. So
    convert a string to an AST, run tons of optimizations and convert it to
    specialized byte-code and finally run it.

    When we are talking about scientific computing, this means we should do
    aggressive inlining, optimizations that may take a lot of time... e.g. if
    an optimization strategy takes 4min and it improves the computation by 20%
    then with calculations running longer than 20min it's worthwhile to run
    that optimization... but it may not be a good idea for general programming.

    I do not like that separation between "scientific" and "general"
    computing. We both want our code clean, and have to deal with large
    programs. Still, I agree that if you want to use special hardware and the
    like, or want the even the *last bit* of performance, no matter what, it is
    reasonable that you pay the price. I suspect the current approach of
    delegating those things to C is good enough, but I do not have a strong
    opinion (well, my strong opinion, like yours, is "leave them out of the
    language").


    So, maybe, the better approach for scientific computation is to provide
    the packages for DSLs, AST conversions, transformations, optimizations,
    targets and nice runtime code generation support. I know the details will
    get complicated, but maybe it's a better than adding every single thing to
    the compiler. With Go compiler eventually being written in Go, it would
    mean that we can use the same code in runtime code generation.

    What are your thoughts on this?
    I will not speak for them, but I don't see that the numerical/scientific
    Go comunity (mostly, the gonum people) has proposed or even thinks that we
    should add every single thing to the compiler, but are actually against
    that. That part sounds bit like a straw man to me. We want a natural way
    to represent a very common data structure. We do not want fotran-like
    matrix multiplication in the language, and we most certainly have not asked
    for language support for the other things you mention. For several of them,
    we can probably just work them out at library level, for some, like the GPU
    thing, we probably will always need to delegate to C (or, actually, CUDA or
    similar).

    I am not particularly against your proposal, but I don't like that it is
    presented as an alternative to what the scientific/numerical Go community
    are asking for.



    + egon
    (PS. I'm probably not the best person for implementing it, but maybe
    someone else gets inspired and runs with the idea :) )
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Brendan Tracey at Mar 29, 2014 at 12:27 am

    I very much agree with you about GPUGPU, OpenCL, etc. Those things should
    not be in the language. Also, noone in the scientific community has asked
    for sets, as far as I know (sorry if I am wrong here), and 2-dimensional
    slices are the same as matrices for our purposes.
    They aren't the same. 2-dimensional slices are a language feature with
    special properties (generic, assignment operator, slicing). Gonum is likely
    to use a specific kind of 2-dimensional slice, [,]float64 as the underlying
    type for mat64.Matrix. In this case, Matrix is just a user defined type
    like any other in the language.


    I don't think egon was commenting on the proposal at all. I think he was
    suggesting alternate ways to think about the compiler in a heterogeneous
    environment. Is that correct egon?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Raul Mera at Mar 29, 2014 at 3:13 am

    They aren't the same. 2-dimensional slices are a language feature with
    special properties (generic, assignment operator, slicing). Gonum is likely
    to use a specific kind of 2-dimensional slice, [,]float64 as the underlying
    type for mat64.Matrix. In this case, Matrix is just a user defined type
    like any other in the language.
    What I meant is that you do not need 2D slices *and* matrices in the
    languages, as it is said in the original post by Egon (at least I
    understand that because they are listed together in the beginning of his
    post). Thus the "for our purposes" thing.

    I don't think egon was commenting on the proposal at all. I think he was
    suggesting alternate ways to think about the compiler in a heterogeneous
    environment. Is that correct egon?
    Neither do I now, actually. I still think It is important to make these
    points, as ND-slices are mentioned in his post. My response came out much
    more defensive than it should have, and that caused the
    clarification/apology afterwards.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Egon at Mar 29, 2014 at 5:08 am

    On Saturday, March 29, 2014 2:27:32 AM UTC+2, Brendan Tracey wrote:

    I very much agree with you about GPUGPU, OpenCL, etc. Those things should
    not be in the language. Also, noone in the scientific community has asked
    for sets, as far as I know (sorry if I am wrong here), and 2-dimensional
    slices are the same as matrices for our purposes.
    They aren't the same. 2-dimensional slices are a language feature with
    special properties (generic, assignment operator, slicing). Gonum is likely
    to use a specific kind of 2-dimensional slice, [,]float64 as the underlying
    type for mat64.Matrix. In this case, Matrix is just a user defined type
    like any other in the language.


    I don't think egon was commenting on the proposal at all. I think he was
    suggesting alternate ways to think about the compiler in a heterogeneous
    environment. Is that correct egon?
    The post was somewhat motivated by the proposal, but it's not commenting on
    it per se. I had the idea a while back and did some simple experiments with
    the idea, but I didn't get to a point where it properly worked.

    Also, I'm definitely not saying that we shouldn't add things to the
    compiler anymore, just that there might be better approaches that give a
    more flexible end-result.

    Basically there are a lot of things that we would like to have in Go
    (matrices, generics, algebra), but then again we don't want to complicate
    the compiler. So, why not just try to make a bridge that allows easily to
    move between DSLs, hardware and Go.

    + egon

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dan Kortschak at Mar 29, 2014 at 5:31 am
    I like this idea though I'm not competent to help build it. I think though that the table type adds significant utility with or without that kind of external tool.
    On 29/03/2014, at 3:38 PM, "egon" wrote:

    The post was somewhat motivated by the proposal, but it's not commenting on it per se. I had the idea a while back and did some simple experiments with the idea, but I didn't get to a point where it properly worked.

    Also, I'm definitely not saying that we shouldn't add things to the compiler anymore, just that there might be better approaches that give a more flexible end-result.

    Basically there are a lot of things that we would like to have in Go (matrices, generics, algebra), but then again we don't want to complicate the compiler. So, why not just try to make a bridge that allows easily to move between DSLs, hardware and Go.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Scott Pakin at Mar 30, 2014 at 11:56 pm

    On Friday, March 28, 2014 3:31:54 PM UTC-6, egon wrote:
    So what would be the alternative? One approach that would be viable is
    runtime code generation, it's complex in it's internals, but idea is
    simple. Take a string and convert it to a Go function at runtime. So
    convert a string to an AST, run tons of optimizations and convert it to
    specialized byte-code and finally run it.
    One solution that's a bit lower-level than what you're thinking of but that
    exists today is GoLLVM <https://github.com/axw/gollvm>. In particular,
    check out GoLLVM's factorial example<https://github.com/axw/gollvm/blob/master/examples/factorial/factorial.go>,
    which creates a factorial function instruction-by-instruction, JITs it at
    -O2, and runs it, all from within a single Go process.

    — Scott

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedMar 28, '14 at 9:31p
activeMar 30, '14 at 11:56p
posts10
users6
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase