FAQ
Hi,

I'm hitting a roadblock with package unsafe, maybe someone here can help.

I'd like to get a copy of a slice of structs from another struct passed as
an interface.
None of the structs are exported and I need the copy in another package.
This is probably a little hard to understand, so here's an example<http://play.golang.org/p/37pHFeTL_J>
.

Any help is highly appreciated!

Arne

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Minux at May 31, 2013 at 12:59 pm

    On Fri, May 31, 2013 at 8:48 PM, Arne Hormann wrote:

    I'm hitting a roadblock with package unsafe, maybe someone here can help.

    I'd like to get a copy of a slice of structs from another struct passed as
    an interface.
    None of the structs are exported and I need the copy in another package.
    This is probably a little hard to understand, so here's an example<http://play.golang.org/p/37pHFeTL_J>
    .

    Any help is highly appreciated!
    You can first dig the slice out using reflect, however, I'm not sure why
    you have such
    a strange requirement. Could you please elaborate your use case here?
    perhaps there
    are better solutions that the unsafe and hacky solutions.

    I have a feel that you're designing your solution against the language....
    just a feeling, though.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Arne Hormann at May 31, 2013 at 1:07 pm

    Am Freitag, 31. Mai 2013 14:59:06 UTC+2 schrieb minux:

    On Fri, May 31, 2013 at 8:48 PM, Arne Hormann <arneh...@gmail.com<javascript:>
    wrote:
    I'm hitting a roadblock with package unsafe, maybe someone here can help.

    I'd like to get a copy of a slice of structs from another struct passed
    as an interface.
    None of the structs are exported and I need the copy in another package.
    This is probably a little hard to understand, so here's an example<http://play.golang.org/p/37pHFeTL_J>
    .

    Any help is highly appreciated!
    You can first dig the slice out using reflect, however, I'm not sure why
    you have such
    a strange requirement. Could you please elaborate your use case here?
    perhaps there
    are better solutions that the unsafe and hacky solutions.

    I have a feel that you're designing your solution against the language....
    just a feeling, though.
    It's not just a feeling, you're right and this is not my prefered solution.
    I need column type information from database/sql drivers and everything in
    there is wrapped.
    I proposed a safe workaround in an earlier post<https://groups.google.com/forum/#!topic/golang-nuts/2aLctcVyp6Q>,
    but that requires changing the runtime.
    I don't know if I should have posted that to golang-dev instead of
    golang-nuts or maybe filed a bug (which I can still do),
    but I'd like to provide a way to use it with older Go version anyway.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Arne Hormann at May 31, 2013 at 1:15 pm

    Am Freitag, 31. Mai 2013 15:07:24 UTC+2 schrieb Arne Hormann:
    Am Freitag, 31. Mai 2013 14:59:06 UTC+2 schrieb minux:
    On Fri, May 31, 2013 at 8:48 PM, Arne Hormann wrote:

    I'm hitting a roadblock with package unsafe, maybe someone here can help.

    I'd like to get a copy of a slice of structs from another struct passed
    as an interface.
    None of the structs are exported and I need the copy in another package.
    This is probably a little hard to understand, so here's an example<http://play.golang.org/p/37pHFeTL_J>
    .

    Any help is highly appreciated!
    You can first dig the slice out using reflect, however, I'm not sure why
    you have such
    a strange requirement. Could you please elaborate your use case here?
    perhaps there
    are better solutions that the unsafe and hacky solutions.

    I have a feel that you're designing your solution against the language....
    just a feeling, though.
    It's not just a feeling, you're right and this is not my prefered solution.
    I need column type information from database/sql drivers and everything in
    there is wrapped.
    I proposed a safe workaround in an earlier post<https://groups.google.com/forum/#!topic/golang-nuts/2aLctcVyp6Q>,
    but that requires changing the runtime.
    I don't know if I should have posted that to golang-dev instead of
    golang-nuts or maybe filed a bug (which I can still do),
    but I'd like to provide a way to use it with older Go version anyway.
    A little more detail: my blockade is getting the internal rows
    implementation for github.com/go-sql-driver/mysql (mysqlRows) and extract
    the []mysqlField slice into my own copy.
    I get it as driver.Driver (an interface) and both structs aren't exported.
    Though I could probably get an implementation based on a safe way into the
    driver, Julien is strictly against anything using unsafe (and I think he's
    right).
    That's why I have to use an external package and that's why I can't access
    the two unexported structs.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • John Nagle at May 31, 2013 at 7:05 pm

    On 5/31/2013 6:07 AM, Arne Hormann wrote:
    Am Freitag, 31. Mai 2013 14:59:06 UTC+2 schrieb minux:
    I have a feel that you're designing your solution against the language....
    just a feeling, though.
    It's not just a feeling, you're right and this is not my preferred solution.
    I need column type information from database/sql drivers and everything in
    there is wrapped.
    I proposed a safe workaround in an earlier post
    <https://groups.google.com/forum/#!topic/golang-nuts/2aLctcVyp6Q>,
    but that requires changing the runtime.
         There's a lot of dissatisfaction with Go's SQL interface.
    That's why there are so many alternatives to it that do almost
    the same thing. The Go SQL interface itself isn't a good fit
    to Go; it's basically a warmed-over interface from another
    language.

         The fundamental problem is that what you get from a SQL
    query is a sequence of typed values, each of which has a
    type, a value, and a column name. In some languages,
    like Python, this is easily represented as a tuple.
    For Go, it would logically be a struct, but the form
    of that struct is query dependent and not known to the SQL
    package. So the Go SQL package returns an opaque type,
    a "Row", for which a few functions are provided.

    (The description for "func (*Row) Scan" at
    "http://golang.org/pkg/database/sql/#Row" makes
    no sense. It says: "Scan copies the columns from the matched row into
    the values pointed at by dest. If more than one row matches the query,
    Scan uses the first row and discards the rest. If no row matches the
    query, Scan returns ErrNoRows." A "Row" only contains one row.
    There's a type Rows, but that's different. This description may
    have been modified from that for Scan() for type Rows.)

    The original poster needs to convert a Row into a struct with
    the matching fields, and there's no general way to do that
    without reflection. Maybe there should be.

    The "...T" syntax in function calls turns a list of
    parameters into a slice of type []T. But you can only do
    that in function calls. It would be useful to have a general
    function that can turn a struct into a slice of interface{},
    and the reverse operation. Those are safe operations, and
    can be written (inefficiently) with reflection.

    There some other simple generic operations like that which
    would run fast if implemented at compile time but are
    painfully slow when done via reflections. Deep copy
    for nonrecursive types is an example.

         John Nagle

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Arne Hormann at May 31, 2013 at 7:09 pm


    The original poster needs to convert a Row into a struct with
    the matching fields, and there's no general way to do that
    without reflection. Maybe there should be.
    In my other post, I linked to Issue 5606<https://code.google.com/p/go/issues/detail?id=5606> created
    for exactly that purpose.


    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Arne Hormann at May 31, 2013 at 7:17 pm

    Am Freitag, 31. Mai 2013 21:09:11 UTC+2 schrieb Arne Hormann:
    The original poster needs to convert a Row into a struct with
    the matching fields, and there's no general way to do that
    without reflection. Maybe there should be.
    In my other post, I linked to Issue 5606<https://code.google.com/p/go/issues/detail?id=5606> created
    for exactly that purpose.
    Sorry, scrap that. I'm not interested in any row.
    I'm interested in the metadata for each column in a result - the type,
    flags, comments, whatever is available.
    What I want would only have to be extracted once per query.
    In some cases not even necessarily per query but per unique sql query
    string.

    My proposal is from a technical perspective:
    How do I add the functionality in a way that causes the least changes in
    the api and move/assign/provide as much responsibility and flexibility as
    possible to the driver implementations.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Arne Hormann at May 31, 2013 at 7:18 pm
    Ok, I probably confused the terminology. In the *standard library*, not in
    the runtime.

    Am Freitag, 31. Mai 2013 21:17:17 UTC+2 schrieb Arne Hormann:


    Am Freitag, 31. Mai 2013 21:09:11 UTC+2 schrieb Arne Hormann:
    The original poster needs to convert a Row into a struct with
    the matching fields, and there's no general way to do that
    without reflection. Maybe there should be.
    In my other post, I linked to Issue 5606<https://code.google.com/p/go/issues/detail?id=5606> created
    for exactly that purpose.
    Sorry, scrap that. I'm not interested in any row.
    I'm interested in the metadata for each column in a result - the type,
    flags, comments, whatever is available.
    What I want would only have to be extracted once per query.
    In some cases not even necessarily per query but per unique sql query
    string.

    My proposal is from a technical perspective:
    How do I add the functionality in a way that causes the least changes in
    the api and move/assign/provide as much responsibility and flexibility as
    possible to the driver implementations.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Kevin Gillette at May 31, 2013 at 8:28 pm
    Ah, then yes indeed. In Go the runtime refers to garbage collection,
    allocation, goroutine/channel scheduling, signal handling, etc (the stuff
    that's lower level than what the application or library programmer needs to
    be concerned with).
    On Friday, May 31, 2013 1:18:42 PM UTC-6, Arne Hormann wrote:

    Ok, I probably confused the terminology. In the *standard library*, not
    in the runtime.

    Am Freitag, 31. Mai 2013 21:17:17 UTC+2 schrieb Arne Hormann:


    Am Freitag, 31. Mai 2013 21:09:11 UTC+2 schrieb Arne Hormann:
    The original poster needs to convert a Row into a struct with
    the matching fields, and there's no general way to do that
    without reflection. Maybe there should be.
    In my other post, I linked to Issue 5606<https://code.google.com/p/go/issues/detail?id=5606> created
    for exactly that purpose.
    Sorry, scrap that. I'm not interested in any row.
    I'm interested in the metadata for each column in a result - the type,
    flags, comments, whatever is available.
    What I want would only have to be extracted once per query.
    In some cases not even necessarily per query but per unique sql query
    string.

    My proposal is from a technical perspective:
    How do I add the functionality in a way that causes the least changes in
    the api and move/assign/provide as much responsibility and flexibility as
    possible to the driver implementations.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • John Nagle at May 31, 2013 at 7:30 pm

    On 5/31/2013 12:17 PM, Arne Hormann wrote:

    Sorry, scrap that. I'm not interested in any row.
    I'm interested in the metadata for each column in a result - the type,
    flags, comments, whatever is available.
    What I want would only have to be extracted once per query.
    In some cases not even necessarily per query but per unique sql query
    string.
        You can do a SHOW COLUMNS query in SQL to get table structure
    and data type info.

         John Nagle


    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Arne Hormann at May 31, 2013 at 7:39 pm

    Am Freitag, 31. Mai 2013 21:30:01 UTC+2 schrieb John Nagle:
    On 5/31/2013 12:17 PM, Arne Hormann wrote:

    Sorry, scrap that. I'm not interested in any row.
    I'm interested in the metadata for each column in a result - the type,
    flags, comments, whatever is available.
    What I want would only have to be extracted once per query.
    In some cases not even necessarily per query but per unique sql query
    string.
    You can do a SHOW COLUMNS query in SQL to get table structure
    and data type info.
    Thanks, but that's not that easy for aggregated columns, joins, SHOW
    queries, ...

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jcnaylor at May 31, 2013 at 7:46 pm

    On Friday, May 31, 2013 12:39:16 PM UTC-7, Arne Hormann wrote:


    Am Freitag, 31. Mai 2013 21:30:01 UTC+2 schrieb John Nagle:
    On 5/31/2013 12:17 PM, Arne Hormann wrote:

    Sorry, scrap that. I'm not interested in any row.
    I'm interested in the metadata for each column in a result - the type,
    flags, comments, whatever is available.
    What I want would only have to be extracted once per query.
    In some cases not even necessarily per query but per unique sql query
    string.
    You can do a SHOW COLUMNS query in SQL to get table structure
    and data type info.
    Thanks, but that's not that easy for aggregated columns, joins, SHOW
    queries, ...
    It's also vendor-specific, I believe.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Kevin Gillette at May 31, 2013 at 8:26 pm
    SHOW COLUMNS is by no means standard SQL (or anywhere close). Besides
    which, it's reasonable that the OP would want to fetch this information
    from an arbitrary query result.
    On Friday, May 31, 2013 1:30:01 PM UTC-6, John Nagle wrote:
    On 5/31/2013 12:17 PM, Arne Hormann wrote:

    Sorry, scrap that. I'm not interested in any row.
    I'm interested in the metadata for each column in a result - the type,
    flags, comments, whatever is available.
    What I want would only have to be extracted once per query.
    In some cases not even necessarily per query but per unique sql query
    string.
    You can do a SHOW COLUMNS query in SQL to get table structure
    and data type info.

    John Nagle

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Carlos Castillo at May 31, 2013 at 11:09 pm
    Isn't this a very driver specific function? Different DB's have different
    types, different features for types, different ways of representing these
    features, and different ways those features can be used. Expecting a
    general package like database/sql to handle them all seems like a bad idea.
    You will either have a huge API, most of which won't apply, or a very small
    one, limited to features that mostly appear everywhere.

    Your previous proposal already put the main logic into the driver (ie: the
    user had to manually call driver Functions), so you already know the driver
    and have to use it's code. If you want to see how a driver deals with a
    given query you might as well ask it directly, instead of doing a back and
    forth with database/sql.

    Also you still haven't answered the question "What are you trying to do?".
    Are you trying to build an ORM? A tool for analyzing DB's?

    Here is an example of type and database agnostic code that converts a rows
    object into a []map[string]interface{} or
    [][]interface{}: http://play.golang.org/p/RJQgrLBX2K
    On Friday, May 31, 2013 12:17:17 PM UTC-7, Arne Hormann wrote:



    Am Freitag, 31. Mai 2013 21:09:11 UTC+2 schrieb Arne Hormann:
    The original poster needs to convert a Row into a struct with
    the matching fields, and there's no general way to do that
    without reflection. Maybe there should be.
    In my other post, I linked to Issue 5606<https://code.google.com/p/go/issues/detail?id=5606> created
    for exactly that purpose.
    Sorry, scrap that. I'm not interested in any row.
    I'm interested in the metadata for each column in a result - the type,
    flags, comments, whatever is available.
    What I want would only have to be extracted once per query.
    In some cases not even necessarily per query but per unique sql query
    string.

    My proposal is from a technical perspective:
    How do I add the functionality in a way that causes the least changes in
    the api and move/assign/provide as much responsibility and flexibility as
    possible to the driver implementations.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Arne Hormann at Jun 1, 2013 at 12:12 am

    Am Samstag, 1. Juni 2013 01:09:25 UTC+2 schrieb Carlos Castillo:
    Isn't this a very driver specific function? Different DB's have different
    types, different features for types, different ways of representing these
    features, and different ways those features can be used. Expecting a
    general package like database/sql to handle them all seems like a bad idea.
    You will either have a huge API, most of which won't apply, or a very small
    one, limited to features that mostly appear everywhere.

    Your previous proposal already put the main logic into the driver (ie: the
    user had to manually call driver Functions), so you already know the driver
    and have to use it's code. If you want to see how a driver deals with a
    given query you might as well ask it directly, instead of doing a back and
    forth with database/sql.

    Also you still haven't answered the question "What are you trying to do?".
    Are you trying to build an ORM? A tool for analyzing DB's?

    Here is an example of type and database agnostic code that converts a rows
    object into a []map[string]interface{} or [][]interface{}:
    http://play.golang.org/p/RJQgrLBX2K
    I only have one proposal, the one in the other thread. This one is an
    attempt at a workaround that is usable without an api change in Go.
    Your example will probably return a string representation as []byte (or
    sql.RawBytes) in most cases. That way you still lose the information
    available to the driver.

    On Friday, May 31, 2013 12:17:17 PM UTC-7, Arne Hormann wrote:



    Am Freitag, 31. Mai 2013 21:09:11 UTC+2 schrieb Arne Hormann:
    The original poster needs to convert a Row into a struct with
    the matching fields, and there's no general way to do that
    without reflection. Maybe there should be.
    In my other post, I linked to Issue 5606<https://code.google.com/p/go/issues/detail?id=5606> created
    for exactly that purpose.
    Sorry, scrap that. I'm not interested in any row.
    I'm interested in the metadata for each column in a result - the type,
    flags, comments, whatever is available.
    What I want would only have to be extracted once per query.
    In some cases not even necessarily per query but per unique sql query
    string.

    My proposal is from a technical perspective:
    How do I add the functionality in a way that causes the least changes in
    the api and move/assign/provide as much responsibility and flexibility as
    possible to the driver implementations.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Carlos Castillo at Jun 1, 2013 at 12:25 am
    That's the result of a choice made by the database/sql pakage, but there is
    no information lost if you convert a []byte to a string if that's what you
    wanted.


    On Fri, May 31, 2013 at 5:12 PM, Arne Hormann wrote:


    Am Samstag, 1. Juni 2013 01:09:25 UTC+2 schrieb Carlos Castillo:
    Isn't this a very driver specific function? Different DB's have different
    types, different features for types, different ways of representing these
    features, and different ways those features can be used. Expecting a
    general package like database/sql to handle them all seems like a bad idea.
    You will either have a huge API, most of which won't apply, or a very small
    one, limited to features that mostly appear everywhere.

    Your previous proposal already put the main logic into the driver (ie:
    the user had to manually call driver Functions), so you already know the
    driver and have to use it's code. If you want to see how a driver deals
    with a given query you might as well ask it directly, instead of doing a
    back and forth with database/sql.

    Also you still haven't answered the question "What are you trying to
    do?". Are you trying to build an ORM? A tool for analyzing DB's?

    Here is an example of type and database agnostic code that converts a
    rows object into a []map[string]interface{} or [][]interface{}:
    http://play.**golang.org/p/RJQgrLBX2K<http://play.golang.org/p/RJQgrLBX2K>
    I only have one proposal, the one in the other thread. This one is an
    attempt at a workaround that is usable without an api change in Go.
    Your example will probably return a string representation as []byte (or
    sql.RawBytes) in most cases. That way you still lose the information
    available to the driver.

    On Friday, May 31, 2013 12:17:17 PM UTC-7, Arne Hormann wrote:



    Am Freitag, 31. Mai 2013 21:09:11 UTC+2 schrieb Arne Hormann:
    The original poster needs to convert a Row into a struct with
    the matching fields, and there's no general way to do that
    without reflection. Maybe there should be.
    In my other post, I linked to Issue 5606<https://code.google.com/p/go/issues/detail?id=5606> created
    for exactly that purpose.
    Sorry, scrap that. I'm not interested in any row.
    I'm interested in the metadata for each column in a result - the type,
    flags, comments, whatever is available.
    What I want would only have to be extracted once per query.
    In some cases not even necessarily per query but per unique sql query
    string.

    My proposal is from a technical perspective:
    How do I add the functionality in a way that causes the least changes in
    the api and move/assign/provide as much responsibility and flexibility as
    possible to the driver implementations.
    --
    You received this message because you are subscribed to a topic in the
    Google Groups "golang-nuts" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/golang-nuts/_wig_BFKXC8/unsubscribe?hl=en-US
    .
    To unsubscribe from this group and all its topics, send an email to
    golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.



    --
    Carlos Castillo

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Arne Hormann at Jun 1, 2013 at 7:06 am

    Am Samstag, 1. Juni 2013 02:25:47 UTC+2 schrieb Carlos Castillo:
    That's the result of a choice made by the database/sql pakage, but there
    is no information lost if you convert a []byte to a string if that's what
    you wanted.
    I know that. But I'm not interested in a string or a []byte, I'm interested
    in getting the closest matching go type from the driver and causing the
    smallest amount of internal type conversions that is possible.
    Today, there's just no way to get there. If I use dynamic queries
    (specified by the user) or aggregated queries or anything synthetic (like
    SHOW in MySQL), I can't get the type from the database.
    Ok, maybe there is a way, but that means replicating a lot of
    infrastructure across projects to determine the column types and provide
    guesses for all synthetic stuff not using a table directly. Maybe there
    could be a companion support library for each driver, but who wants to
    write and maintain them?
    Additionaly, using []byte every time causes a lot of allocations and
    inspecting the table structure causes some queries and a slower startup,
    which in some cases can't be prevented by caching, the database may have
    changed. If I use SHOW COLUMNS or maybe parse the sql to get that info, I
    still have to construct something to store it and have to remember the
    matches between Go and the database types. Moving that to the driver makes
    life easier for everyone who wants to efficiently serialize rows in the
    smallest possible binary form, create ORM tools, build code generators (my
    own use case here), build tools for any database instead of the one one her
    own server, ...
    Having the ability to know the types would be a gift to everyone building
    tools. Maybe not the users with their own database and existing tables they
    already know, but I guess they won't complain when better tools are created.
    And saying better I also mean causing less queries and putting less strain
    on the GC. Or not having to implement tricks like a lazy buffer in every
    driver (which I didn't even know about until I saw it in a pull request
    from bradfitz).

    On Fri, May 31, 2013 at 5:12 PM, Arne Hormann <arneh...@gmail.com<javascript:>
    wrote:
    Am Samstag, 1. Juni 2013 01:09:25 UTC+2 schrieb Carlos Castillo:
    Isn't this a very driver specific function? Different DB's have
    different types, different features for types, different ways of
    representing these features, and different ways those features can be used.
    Expecting a general package like database/sql to handle them all seems like
    a bad idea. You will either have a huge API, most of which won't apply, or
    a very small one, limited to features that mostly appear everywhere.

    Your previous proposal already put the main logic into the driver (ie:
    the user had to manually call driver Functions), so you already know the
    driver and have to use it's code. If you want to see how a driver deals
    with a given query you might as well ask it directly, instead of doing a
    back and forth with database/sql.

    Also you still haven't answered the question "What are you trying to
    do?". Are you trying to build an ORM? A tool for analyzing DB's?

    Here is an example of type and database agnostic code that converts a
    rows object into a []map[string]interface{} or [][]interface{}:
    http://play.**golang.org/p/RJQgrLBX2K<http://play.golang.org/p/RJQgrLBX2K>
    I only have one proposal, the one in the other thread. This one is an
    attempt at a workaround that is usable without an api change in Go.
    Your example will probably return a string representation as []byte (or
    sql.RawBytes) in most cases. That way you still lose the information
    available to the driver.

    On Friday, May 31, 2013 12:17:17 PM UTC-7, Arne Hormann wrote:



    Am Freitag, 31. Mai 2013 21:09:11 UTC+2 schrieb Arne Hormann:
    The original poster needs to convert a Row into a struct with
    the matching fields, and there's no general way to do that
    without reflection. Maybe there should be.
    In my other post, I linked to Issue 5606<https://code.google.com/p/go/issues/detail?id=5606> created
    for exactly that purpose.
    Sorry, scrap that. I'm not interested in any row.
    I'm interested in the metadata for each column in a result - the type,
    flags, comments, whatever is available.
    What I want would only have to be extracted once per query.
    In some cases not even necessarily per query but per unique sql query
    string.

    My proposal is from a technical perspective:
    How do I add the functionality in a way that causes the least changes
    in the api and move/assign/provide as much responsibility and flexibility
    as possible to the driver implementations.
    --
    You received this message because you are subscribed to a topic in the
    Google Groups "golang-nuts" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/golang-nuts/_wig_BFKXC8/unsubscribe?hl=en-US
    .
    To unsubscribe from this group and all its topics, send an email to
    golang-nuts...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/groups/opt_out.



    --
    Carlos Castillo
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Arne Hormann at Jun 1, 2013 at 7:58 am
    If you came here to discuss my reflect problem, feel free to post here.
    If you want to discuss column type retrieval in database/sql, please reply
    in the thread with my proposal:
    https://groups.google.com/forum/#!topic/golang-nuts/2aLctcVyp6Q
    I linked the other thread in Issue 5606, anything here may be overlooked.


    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jlmoiron at Jun 1, 2013 at 10:56 pm
    @John Nagle

    Your description of why sql.Row[s] exists is very concise, I'll definitely
    use it at some point in the future. I actually quite like database/sql
    interface, since I think anything more sophisticated would have a worse
    user-side interface. The only bags we have for holding a bag of stuff of
    various types are []interface{} and structs, and only one of those can be
    created at runtime.

    I think people misunderstand QueryRow & Row. Their implementation is
    strictly in terms of Query and Rows and is extremely short and easy to
    understand:

    http://golang.org/src/pkg/database/sql/sql.go?s=19667:19729#L768
    http://golang.org/src/pkg/database/sql/sql.go?s=34857:34974#L1379
    http://golang.org/src/pkg/database/sql/sql.go?s=35211:35256#L1389

    QueryRow uses Query and returns the rows & err wrapped in a Row.
      Row.Scan() returns that error if set, otherwise scans once and throws the
    rest away, returning any scanning error. The documentation for Row.Scan()
    is completely correct, and Row.Scan() itself is written in terms of
    Rows.Scan().

    The source of confusion here is mostly in the naming: is that there is no
    situation in which a Query or manipulation of its resultant Rows becomes a
    Row. I'm sure most people expect Rows.Next() to give them a Row, or for
    Row to be some kind of component of a Rows, where in fact the opposite is
    true. There's nothing much that can be done about this after go1, but it's
    merely awkward, not incorrect.

    An additional comment, just in general, is that database/sql is very
    protective about its interface. In my attempts to extend it (with sqlx), I
    had to actually re-implement sql.Row & the various QueryRow functions to be
    able to provide access to the underlying Rows.Columns() function when using
    QueryRow:

    https://github.com/jmoiron/sqlx/blob/master/sqlx.go#L80-L85

    I think requiring the use of reflect is absolutely fine, since this is what
    it's for (runtime type manipulation). Requiring the use of unsafe on the
    other hand...

    @Arne

    As for this proposal, I think it's an awkward way of doing it, but I agree
    that it's beneficial to avoid adding public driver-only stuff to `sql.go`,
    and that the drivers should have some way of accessing the information that
    they put into an sql.Rows in the first place. I worry a bit about how much
    database/sql is becoming fractured across driver lines (already DSNs and
    query string interpolation will differ among drivers, even for the same
    database), but wherever this is possible to fix it's probably a cultural
    issue rather than a technical one. It's access to efficient APIs which
    differ when needed rather than to pretend everything is the same and suffer
    with a poor abstraction.


    On Friday, May 31, 2013 3:05:17 PM UTC-4, John Nagle wrote:
    On 5/31/2013 6:07 AM, Arne Hormann wrote:
    Am Freitag, 31. Mai 2013 14:59:06 UTC+2 schrieb minux:
    I have a feel that you're designing your solution against the
    language....
    just a feeling, though.
    It's not just a feeling, you're right and this is not my preferred solution.
    I need column type information from database/sql drivers and everything in
    there is wrapped.
    I proposed a safe workaround in an earlier post
    <https://groups.google.com/forum/#!topic/golang-nuts/2aLctcVyp6Q>,
    but that requires changing the runtime.
    There's a lot of dissatisfaction with Go's SQL interface.
    That's why there are so many alternatives to it that do almost
    the same thing. The Go SQL interface itself isn't a good fit
    to Go; it's basically a warmed-over interface from another
    language.

    The fundamental problem is that what you get from a SQL
    query is a sequence of typed values, each of which has a
    type, a value, and a column name. In some languages,
    like Python, this is easily represented as a tuple.
    For Go, it would logically be a struct, but the form
    of that struct is query dependent and not known to the SQL
    package. So the Go SQL package returns an opaque type,
    a "Row", for which a few functions are provided.

    (The description for "func (*Row) Scan" at
    "http://golang.org/pkg/database/sql/#Row" makes
    no sense. It says: "Scan copies the columns from the matched row into
    the values pointed at by dest. If more than one row matches the query,
    Scan uses the first row and discards the rest. If no row matches the
    query, Scan returns ErrNoRows." A "Row" only contains one row.
    There's a type Rows, but that's different. This description may
    have been modified from that for Scan() for type Rows.)

    The original poster needs to convert a Row into a struct with
    the matching fields, and there's no general way to do that
    without reflection. Maybe there should be.

    The "...T" syntax in function calls turns a list of
    parameters into a slice of type []T. But you can only do
    that in function calls. It would be useful to have a general
    function that can turn a struct into a slice of interface{},
    and the reverse operation. Those are safe operations, and
    can be written (inefficiently) with reflection.

    There some other simple generic operations like that which
    would run fast if implemented at compile time but are
    painfully slow when done via reflections. Deep copy
    for nonrecursive types is an example.

    John Nagle
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Kevin Gillette at May 31, 2013 at 7:13 pm

    On Friday, May 31, 2013 7:07:24 AM UTC-6, Arne Hormann wrote:

    I proposed a safe workaround in an earlier post<https://groups.google.com/forum/#!topic/golang-nuts/2aLctcVyp6Q>,
    but that requires changing the runtime.

    Why would it require changing the runtime? database/sql hardly does
    anything deep in the runtime. Doing what you want in an official way may
    require awkward API changes, but should be fully achievable given the
    current *runtime* implementation.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedMay 31, '13 at 12:48p
activeJun 1, '13 at 10:56p
posts20
users7
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase