FAQ
Note to the group: This message is intended for future people who like me
are coming to golang after spending a good deal of time out in nodejs land.

I've been a programmer for a lot of years, C/C++, Java, Node.js you name it
I've probably put it to good use at some point.

Lately I've found that I like Go alot and I really want to keep liking Go,
but having used node.js as my go to webdev language for the last couple of
years has spoiled me in the way I interact with loosely structured data.

This is most obvious when I start dealing with JSON being passed back and
forth across the wire.
I really do hate Javascript & callback hell and all the other reasons I
decided this current project would not be built with node, but I love
dealing with JSON in node because it's native and feels natural.

I saw that Go advertises strong support for JSON, and I'll admit that the
language features a very good parser.
However if you need data from an arbitrary field, it's just not able to do
it and messing with the idiomatic methods of doing it just looks ugly in my
opinion.

I spent a few days struggling with this, because I guess most folks who use
Go are used to dealing with predictable structured data so most of the JSON
examples don't really go into any great depth of how to deal with data that
doesn't fit into structures that can be built at compile time.

In my case actual data interaction is minimal, but when I need a field it
may be buried 10 layers deep in a JSON object, and I don't even get to
control whether or not that field is present.
I just need the language to get out of my way and let me have the data at
field x.y.z.a.n.w.t.f or tell me something along the lines of "Sorry that
field couldn't be found."

This is the one place where Javascript really shines, I can simply say
something like

if(someObject.hasOwnProperty('some.field')){
     doSomethingWith(someObject)
}


And it just works!

As far as I can tell, go has no equivalent.

So I was faced with a few choices.
#1 Ditch Go and run back node.js (not going to happen even if I have to
hard code every possible data structure, lives will literally depend on
this code)
#2 Hard code structs for the expected types merely for the purpose of
checking for field values, (that's a LOT of code and I am a very lazy
person)
#3 Use one of the Javascript interpreters for Go (too much overhead just
for this, and leaning on JS could lead to very bad things)
#4 Take a close look at my previous node projects and see what features
were actually being used the most and try to replicate them in Go.

I opted for number 4.

The most common things I do with server side javascript
#1 Marshall Strings into JSON
#2 Check JSON for presence of fields
#3 Take logical actions based on data contained in said fields
#4 Ship JSON out somewhere else

Actual data manipulation is very, very rare in my past projects. Usually
once data has been JSONized somewhere, the primary purpose of said data is
to tell some controller somewhere to take some action.
Thus I did not include any way to actually manipulate JSON Objects.

The result is here, https://play.golang.org/p/cRgh7j2TW2

I've given you two functions.
The first one is json.Parse which takes a string and gives you a
map[string]interface{} which is the closest thing to an actual JSON object
that you are going to find in Go.
The second function is json.HasOwnProperty which takes a field label and
the json object created by Parse and will return to you either the value at
said label or nil

It does differ from Javascript here in that it will return the actual value
of a label rather than a boolean indicating whether it was present or not.
I did that because asking if a field is present and then asking it to
retrieve that field are two steps that really should be just one, imho.

You can use the code like this...

First get a JSON string and parse it into a JSON object

const B = `{
     "allowedtypes" :{
         "medical": {
             "sources":{
                 "myself": ["*"],
                 "mylocation":["r"],
                 "other": ["a"]
             }
         }
     }
}`

var objB := json.Parse(B)


Next query and assign the field to a variable so you can take action on
it...
permissions := json.HasOwnProperty("allowedtypes.medical.sources.myself",
objB)
     fmt.Printf("Permissions for allowedtypes.medical.sources.myself %+v\n",
permissions)

That's it!
Thanks for taking the time to read this and I hope someone who stumbles on
this later can put it to good use.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Egon at Jan 22, 2015 at 7:46 am

    On Thursday, 22 January 2015 08:57:03 UTC+2, maxpow...@gmail.com wrote:
    Note to the group: This message is intended for future people who like me
    are coming to golang after spending a good deal of time out in nodejs land.

    I've been a programmer for a lot of years, C/C++, Java, Node.js you name
    it I've probably put it to good use at some point.

    Lately I've found that I like Go alot and I really want to keep liking Go,
    but having used node.js as my go to webdev language for the last couple of
    years has spoiled me in the way I interact with loosely structured data.

    This is most obvious when I start dealing with JSON being passed back and
    forth across the wire.
    I really do hate Javascript & callback hell and all the other reasons I
    decided this current project would not be built with node, but I love
    dealing with JSON in node because it's native and feels natural.

    I saw that Go advertises strong support for JSON, and I'll admit that the
    language features a very good parser.
    However if you need data from an arbitrary field, it's just not able to do
    it and messing with the idiomatic methods of doing it just looks ugly in my
    opinion.

    I spent a few days struggling with this, because I guess most folks who
    use Go are used to dealing with predictable structured data so most of the
    JSON examples don't really go into any great depth of how to deal with data
    that doesn't fit into structures that can be built at compile time.

    In my case actual data interaction is minimal, but when I need a field it
    may be buried 10 layers deep in a JSON object, and I don't even get to
    control whether or not that field is present.
    I just need the language to get out of my way and let me have the data at
    field x.y.z.a.n.w.t.f or tell me something along the lines of "Sorry that
    field couldn't be found."

    This is the one place where Javascript really shines, I can simply say
    something like

    if(someObject.hasOwnProperty('some.field')){
    doSomethingWith(someObject)
    }


    And it just works!

    As far as I can tell, go has no equivalent.

    So I was faced with a few choices.
    #1 Ditch Go and run back node.js (not going to happen even if I have to
    hard code every possible data structure, lives will literally depend on
    this code)
    #2 Hard code structs for the expected types merely for the purpose of
    checking for field values, (that's a LOT of code and I am a very lazy
    person)
    #3 Use one of the Javascript interpreters for Go (too much overhead just
    for this, and leaning on JS could lead to very bad things)
    #4 Take a close look at my previous node projects and see what features
    were actually being used the most and try to replicate them in Go.

    I opted for number 4.

    The most common things I do with server side javascript
    #1 Marshall Strings into JSON
    #2 Check JSON for presence of fields
    #3 Take logical actions based on data contained in said fields
    #4 Ship JSON out somewhere else

    Actual data manipulation is very, very rare in my past projects. Usually
    once data has been JSONized somewhere, the primary purpose of said data is
    to tell some controller somewhere to take some action.
    Thus I did not include any way to actually manipulate JSON Objects.

    The result is here, https://play.golang.org/p/cRgh7j2TW2

    I've given you two functions.
    The first one is json.Parse which takes a string and gives you a
    map[string]interface{} which is the closest thing to an actual JSON object
    that you are going to find in Go.
    The second function is json.HasOwnProperty which takes a field label and
    the json object created by Parse and will return to you either the value at
    said label or nil

    It does differ from Javascript here in that it will return the actual
    value of a label rather than a boolean indicating whether it was present or
    not.
    I did that because asking if a field is present and then asking it to
    retrieve that field are two steps that really should be just one, imho.

    You can use the code like this...

    First get a JSON string and parse it into a JSON object

    const B = `{
    "allowedtypes" :{
    "medical": {
    "sources":{
    "myself": ["*"],
    "mylocation":["r"],
    "other": ["a"]
    }
    }
    }
    }`

    var objB := json.Parse(B)


    Next query and assign the field to a variable so you can take action on
    it...
    permissions := json.HasOwnProperty("allowedtypes.medical.sources.myself",
    objB)
    fmt.Printf("Permissions for allowedtypes.medical.sources.myself %+v\n"
    , permissions)

    That's it!
    Thanks for taking the time to read this and I hope someone who stumbles on
    this later can put it to good use.
    For arbitrary data there is also:
    https://godoc.org/github.com/bitly/go-simplejson

      + Egon

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Klaus Post at Jan 22, 2015 at 11:31 am
    Hi!

    Good call by Egon - you should go for that.

    Also it is trivial to implement what you need, see:

    http://play.golang.org/p/5oeZxliMyn

    Of course you would need to type assert the results you get, but that is
    the deal when dealing with loosely structured data.


    /Klaus
    On Thursday, 22 January 2015 07:57:03 UTC+1, maxpow...@gmail.com wrote:

    Note to the group: This message is intended for future people who like me
    are coming to golang after spending a good deal of time out in nodejs land.

    I've been a programmer for a lot of years, C/C++, Java, Node.js you name
    it I've probably put it to good use at some point.

    Lately I've found that I like Go alot and I really want to keep liking Go,
    but having used node.js as my go to webdev language for the last couple of
    years has spoiled me in the way I interact with loosely structured data.

    This is most obvious when I start dealing with JSON being passed back and
    forth across the wire.
    I really do hate Javascript & callback hell and all the other reasons I
    decided this current project would not be built with node, but I love
    dealing with JSON in node because it's native and feels natural.

    I saw that Go advertises strong support for JSON, and I'll admit that the
    language features a very good parser.
    However if you need data from an arbitrary field, it's just not able to do
    it and messing with the idiomatic methods of doing it just looks ugly in my
    opinion.

    I spent a few days struggling with this, because I guess most folks who
    use Go are used to dealing with predictable structured data so most of the
    JSON examples don't really go into any great depth of how to deal with data
    that doesn't fit into structures that can be built at compile time.

    In my case actual data interaction is minimal, but when I need a field it
    may be buried 10 layers deep in a JSON object, and I don't even get to
    control whether or not that field is present.
    I just need the language to get out of my way and let me have the data at
    field x.y.z.a.n.w.t.f or tell me something along the lines of "Sorry that
    field couldn't be found."

    This is the one place where Javascript really shines, I can simply say
    something like

    if(someObject.hasOwnProperty('some.field')){
    doSomethingWith(someObject)
    }


    And it just works!

    As far as I can tell, go has no equivalent.

    So I was faced with a few choices.
    #1 Ditch Go and run back node.js (not going to happen even if I have to
    hard code every possible data structure, lives will literally depend on
    this code)
    #2 Hard code structs for the expected types merely for the purpose of
    checking for field values, (that's a LOT of code and I am a very lazy
    person)
    #3 Use one of the Javascript interpreters for Go (too much overhead just
    for this, and leaning on JS could lead to very bad things)
    #4 Take a close look at my previous node projects and see what features
    were actually being used the most and try to replicate them in Go.

    I opted for number 4.

    The most common things I do with server side javascript
    #1 Marshall Strings into JSON
    #2 Check JSON for presence of fields
    #3 Take logical actions based on data contained in said fields
    #4 Ship JSON out somewhere else

    Actual data manipulation is very, very rare in my past projects. Usually
    once data has been JSONized somewhere, the primary purpose of said data is
    to tell some controller somewhere to take some action.
    Thus I did not include any way to actually manipulate JSON Objects.

    The result is here, https://play.golang.org/p/cRgh7j2TW2

    I've given you two functions.
    The first one is json.Parse which takes a string and gives you a
    map[string]interface{} which is the closest thing to an actual JSON object
    that you are going to find in Go.
    The second function is json.HasOwnProperty which takes a field label and
    the json object created by Parse and will return to you either the value at
    said label or nil

    It does differ from Javascript here in that it will return the actual
    value of a label rather than a boolean indicating whether it was present or
    not.
    I did that because asking if a field is present and then asking it to
    retrieve that field are two steps that really should be just one, imho.

    You can use the code like this...

    First get a JSON string and parse it into a JSON object

    const B = `{
    "allowedtypes" :{
    "medical": {
    "sources":{
    "myself": ["*"],
    "mylocation":["r"],
    "other": ["a"]
    }
    }
    }
    }`

    var objB := json.Parse(B)


    Next query and assign the field to a variable so you can take action on
    it...
    permissions := json.HasOwnProperty("allowedtypes.medical.sources.myself",
    objB)
    fmt.Printf("Permissions for allowedtypes.medical.sources.myself %+v\n"
    , permissions)

    That's it!
    Thanks for taking the time to read this and I hope someone who stumbles on
    this later can put it to good use.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Egon at Jan 22, 2015 at 11:49 am

    On Thursday, 22 January 2015 13:31:16 UTC+2, Klaus Post wrote:
    Hi!

    Good call by Egon - you should go for that.
    Actually, my recommended approach for that problem was
    http://play.golang.org/p/4SqbLxr_lg. But for dealing random arbitrary data
    is simplejson is better. For something inbetween, there are multiple
    approaches, i.e. use map[string]interface{} directly, or properly marshal
    into a nice type. Using a proper type simplifies code else-where while
    keeping the problematic part in a single place.

    + Egon

    Also it is trivial to implement what you need, see:

    http://play.golang.org/p/5oeZxliMyn

    Of course you would need to type assert the results you get, but that is
    the deal when dealing with loosely structured data.


    /Klaus
    On Thursday, 22 January 2015 07:57:03 UTC+1, maxpow...@gmail.com wrote:

    Note to the group: This message is intended for future people who like
    me are coming to golang after spending a good deal of time out in nodejs
    land.

    I've been a programmer for a lot of years, C/C++, Java, Node.js you name
    it I've probably put it to good use at some point.

    Lately I've found that I like Go alot and I really want to keep liking
    Go, but having used node.js as my go to webdev language for the last couple
    of years has spoiled me in the way I interact with loosely structured data.

    This is most obvious when I start dealing with JSON being passed back and
    forth across the wire.
    I really do hate Javascript & callback hell and all the other reasons I
    decided this current project would not be built with node, but I love
    dealing with JSON in node because it's native and feels natural.

    I saw that Go advertises strong support for JSON, and I'll admit that the
    language features a very good parser.
    However if you need data from an arbitrary field, it's just not able to
    do it and messing with the idiomatic methods of doing it just looks ugly in
    my opinion.

    I spent a few days struggling with this, because I guess most folks who
    use Go are used to dealing with predictable structured data so most of the
    JSON examples don't really go into any great depth of how to deal with data
    that doesn't fit into structures that can be built at compile time.

    In my case actual data interaction is minimal, but when I need a field it
    may be buried 10 layers deep in a JSON object, and I don't even get to
    control whether or not that field is present.
    I just need the language to get out of my way and let me have the data at
    field x.y.z.a.n.w.t.f or tell me something along the lines of "Sorry that
    field couldn't be found."

    This is the one place where Javascript really shines, I can simply say
    something like

    if(someObject.hasOwnProperty('some.field')){
    doSomethingWith(someObject)
    }


    And it just works!

    As far as I can tell, go has no equivalent.

    So I was faced with a few choices.
    #1 Ditch Go and run back node.js (not going to happen even if I have to
    hard code every possible data structure, lives will literally depend on
    this code)
    #2 Hard code structs for the expected types merely for the purpose of
    checking for field values, (that's a LOT of code and I am a very lazy
    person)
    #3 Use one of the Javascript interpreters for Go (too much overhead just
    for this, and leaning on JS could lead to very bad things)
    #4 Take a close look at my previous node projects and see what features
    were actually being used the most and try to replicate them in Go.

    I opted for number 4.

    The most common things I do with server side javascript
    #1 Marshall Strings into JSON
    #2 Check JSON for presence of fields
    #3 Take logical actions based on data contained in said fields
    #4 Ship JSON out somewhere else

    Actual data manipulation is very, very rare in my past projects. Usually
    once data has been JSONized somewhere, the primary purpose of said data is
    to tell some controller somewhere to take some action.
    Thus I did not include any way to actually manipulate JSON Objects.

    The result is here, https://play.golang.org/p/cRgh7j2TW2

    I've given you two functions.
    The first one is json.Parse which takes a string and gives you a
    map[string]interface{} which is the closest thing to an actual JSON object
    that you are going to find in Go.
    The second function is json.HasOwnProperty which takes a field label and
    the json object created by Parse and will return to you either the value at
    said label or nil

    It does differ from Javascript here in that it will return the actual
    value of a label rather than a boolean indicating whether it was present or
    not.
    I did that because asking if a field is present and then asking it to
    retrieve that field are two steps that really should be just one, imho.

    You can use the code like this...

    First get a JSON string and parse it into a JSON object

    const B = `{
    "allowedtypes" :{
    "medical": {
    "sources":{
    "myself": ["*"],
    "mylocation":["r"],
    "other": ["a"]
    }
    }
    }
    }`

    var objB := json.Parse(B)


    Next query and assign the field to a variable so you can take action on
    it...
    permissions := json.HasOwnProperty("allowedtypes.medical.sources.myself",
    objB)
    fmt.Printf("Permissions for allowedtypes.medical.sources.myself
    %+v\n", permissions)

    That's it!
    Thanks for taking the time to read this and I hope someone who stumbles
    on this later can put it to good use.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Maxpowers477 at Jan 22, 2015 at 1:12 pm
    Yeah Egon has some great ideas and reading his code replies to my previous
    posts is where I noticed this particular opportunity.

    I agree that there are a lot of approaches, to dealing with unstructured
    data.

    My intent was more narrow and was intended to show how easy it is to
    reimplement something akin to hasOwnProperty from the JS world in Go.

    This is not to say that either approach mentioned by Egon is in anyway
    deficient, just the particular use case of needing an object or value at
    some arbitrary depth in unstructured data does not appear to be addressed
    any where that I have been able to find.

    I showed an example using permissions since that was the thing that was
    buggered up when I started, but consider a different use case for a moment.

    When you store a document in Mongo you tend to store the whole document,
    these things do not lend themselves to relational structuring very easily
    and can get heavy if you're not careful.

    Again I'm not addressing the stupidity of the idea of trying to store
    *.world+dog , but merely, what do you do when you are presented with a need
    to extract something particular from an unstructured document containing
    dog+world and at completely arbitrary depth? You can say things like "just
    make an struct that addresses it" or "you shouldn't do it that way in the
    first place!". But that's not helpful when what you have has been handed
    to you and your job is just to make it work.

    Use case...
    There are custom forms created using a custom tool that runs clientside,
    built on top of angular.
    Any form, can contain any field those fields can and do contain subforms to
    any arbitrary depth.

    This allows for the creation of business forms that meet regulatory
    compliance by stringing them together from other forms that also meet
    compliance. In otherwords the person creating the forms does not want to
    have to re-invent the wheel. To their mind they are just adding new pages
    to existing forms.

    These were persisted wholesale in node and now the "legacy" node app is
    being replaced, we need to ensure that anything that node may have returned
    previously is also returned by go.

    In this particular case, the User has a custom reporting system that needs
    to extract data from field z of subform y of form x, which itself is
    embedded in form n (and n could be embedded in l etc).

    What you have as far as information about the thing you need to return is a
    document id generated by mongo during the last upsert, and a command coming
    from the browser that looks like
    [query: {collection: 'forms', _id: 100, path:'n.x.y.z' },...]

    The server neither knows, nor cares about the type of data stored there,
    it's job is to isolate requested field(s), extract the data from those
    fields and pass it back to the caller assuming they had permission to read
    it :).

    This was handled in node with nothing more than(pseudocode follows)
    document = db.fetchOne(query._id)
    if(document.hasOwnProperty(query.path)){
        res.write(JSON.stringify(document[query.path]))
    }else{
       res.send(404)
    }



    I can't fathom what that would need to look like in Go, using the
    recommended or idiomatic approach, but replacing that lovely little tidbit
    was a task, way up high on my todo list.
    The replacement code now looks about the same
    (again, psuedo code here)

    collection.Find(bson.M{'_id': query.ID}).One(&document)
    section := json.HasOwnProperty(query.path,&document)
    if section !=nil {
         fmt.Fprintf(w, "%s",json.Marshall(&document))
    }else{
        http.Error(404)
    }



    Obviously the "correct" answer here is to dedup and normalize the data ,
    however the goal of the project is to unify disparate information systems
    and to do so while having the smallest impact possible to existing systems
    all the while maintaining current levels of data integrity. Easiest path
    to maintaining data integrity is to leave legacy data laying where it sits
    while gradually phasing in replacment front ends and ensuring new data ends
    up stored in a more coherent fashion. Eventually the old data ends up in
    long term storage anyways, but the retention period on some of these
    records is 10 years before they go to archives.
    I'm in a position where I didn't make the data that I'm expected to deal
    with, but I have to ensure that every user of that data has at least the
    same quality of experience they do now.
    On the other hand I have no desire to take 1,000 lines of node and convert
    it into 100k lines of Go just to cover every edge case.

    My instincts which are informed by over a decade of coming in and cleaning
    up after 1st year college freshman, H1Bs who couldn't spell SQL without
    "My" in front of it and *shudder* PHP devs, tell me that things like I've
    described tend to be the norm after any "super pumped, agile, web 2.0,
    html11, kitchen sink" project reaches more than about 6 months of
    development.

    I have a feeling that as node and other "web server" tech becomes
    supplanted by Go, things like this are going to come to light pretty
    frequently, and my goal with this posting was to leave a trace for anyone
    in the future bumping up against similar things. Hope it's helpful to
    someone.


    On Thursday, January 22, 2015 at 3:47:13 AM UTC-8, Egon wrote:


    On Thursday, 22 January 2015 13:31:16 UTC+2, Klaus Post wrote:

    Hi!

    Good call by Egon - you should go for that.
    Actually, my recommended approach for that problem was
    http://play.golang.org/p/4SqbLxr_lg. But for dealing random arbitrary
    data is simplejson is better. For something inbetween, there are multiple
    approaches, i.e. use map[string]interface{} directly, or properly marshal
    into a nice type. Using a proper type simplifies code else-where while
    keeping the problematic part in a single place.

    + Egon

    Also it is trivial to implement what you need, see:

    http://play.golang.org/p/5oeZxliMyn

    Of course you would need to type assert the results you get, but that is
    the deal when dealing with loosely structured data.


    /Klaus
    On Thursday, 22 January 2015 07:57:03 UTC+1, maxpow...@gmail.com wrote:

    Note to the group: This message is intended for future people who like
    me are coming to golang after spending a good deal of time out in nodejs
    land.

    I've been a programmer for a lot of years, C/C++, Java, Node.js you name
    it I've probably put it to good use at some point.

    Lately I've found that I like Go alot and I really want to keep liking
    Go, but having used node.js as my go to webdev language for the last couple
    of years has spoiled me in the way I interact with loosely structured data.

    This is most obvious when I start dealing with JSON being passed back
    and forth across the wire.
    I really do hate Javascript & callback hell and all the other reasons I
    decided this current project would not be built with node, but I love
    dealing with JSON in node because it's native and feels natural.

    I saw that Go advertises strong support for JSON, and I'll admit that
    the language features a very good parser.
    However if you need data from an arbitrary field, it's just not able to
    do it and messing with the idiomatic methods of doing it just looks ugly in
    my opinion.

    I spent a few days struggling with this, because I guess most folks who
    use Go are used to dealing with predictable structured data so most of the
    JSON examples don't really go into any great depth of how to deal with data
    that doesn't fit into structures that can be built at compile time.

    In my case actual data interaction is minimal, but when I need a field
    it may be buried 10 layers deep in a JSON object, and I don't even get to
    control whether or not that field is present.
    I just need the language to get out of my way and let me have the data
    at field x.y.z.a.n.w.t.f or tell me something along the lines of "Sorry
    that field couldn't be found."

    This is the one place where Javascript really shines, I can simply say
    something like

    if(someObject.hasOwnProperty('some.field')){
    doSomethingWith(someObject)
    }


    And it just works!

    As far as I can tell, go has no equivalent.

    So I was faced with a few choices.
    #1 Ditch Go and run back node.js (not going to happen even if I have to
    hard code every possible data structure, lives will literally depend on
    this code)
    #2 Hard code structs for the expected types merely for the purpose of
    checking for field values, (that's a LOT of code and I am a very lazy
    person)
    #3 Use one of the Javascript interpreters for Go (too much overhead just
    for this, and leaning on JS could lead to very bad things)
    #4 Take a close look at my previous node projects and see what features
    were actually being used the most and try to replicate them in Go.

    I opted for number 4.

    The most common things I do with server side javascript
    #1 Marshall Strings into JSON
    #2 Check JSON for presence of fields
    #3 Take logical actions based on data contained in said fields
    #4 Ship JSON out somewhere else

    Actual data manipulation is very, very rare in my past projects.
    Usually once data has been JSONized somewhere, the primary purpose of said
    data is to tell some controller somewhere to take some action.
    Thus I did not include any way to actually manipulate JSON Objects.

    The result is here, https://play.golang.org/p/cRgh7j2TW2

    I've given you two functions.
    The first one is json.Parse which takes a string and gives you a
    map[string]interface{} which is the closest thing to an actual JSON object
    that you are going to find in Go.
    The second function is json.HasOwnProperty which takes a field label and
    the json object created by Parse and will return to you either the value at
    said label or nil

    It does differ from Javascript here in that it will return the actual
    value of a label rather than a boolean indicating whether it was present or
    not.
    I did that because asking if a field is present and then asking it to
    retrieve that field are two steps that really should be just one, imho.

    You can use the code like this...

    First get a JSON string and parse it into a JSON object

    const B = `{
    "allowedtypes" :{
    "medical": {
    "sources":{
    "myself": ["*"],
    "mylocation":["r"],
    "other": ["a"]
    }
    }
    }
    }`

    var objB := json.Parse(B)


    Next query and assign the field to a variable so you can take action on
    it...
    permissions := json.HasOwnProperty("allowedtypes.medical.sources.myself"
    , objB)
    fmt.Printf("Permissions for allowedtypes.medical.sources.myself
    %+v\n", permissions)

    That's it!
    Thanks for taking the time to read this and I hope someone who stumbles
    on this later can put it to good use.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Egon at Jan 22, 2015 at 3:00 pm

    On Thursday, 22 January 2015 15:12:23 UTC+2, maxpow...@gmail.com wrote:
    Yeah Egon has some great ideas and reading his code replies to my previous
    posts is where I noticed this particular opportunity.

    I agree that there are a lot of approaches, to dealing with unstructured
    data.

    My intent was more narrow and was intended to show how easy it is to
    reimplement something akin to hasOwnProperty from the JS world in Go.

    This is not to say that either approach mentioned by Egon is in anyway
    deficient, just the particular use case of needing an object or value at
    some arbitrary depth in unstructured data does not appear to be addressed
    any where that I have been able to find.

    I showed an example using permissions since that was the thing that was
    buggered up when I started, but consider a different use case for a moment.

    When you store a document in Mongo you tend to store the whole document,
    these things do not lend themselves to relational structuring very easily
    and can get heavy if you're not careful.

    Again I'm not addressing the stupidity of the idea of trying to store
    *.world+dog , but merely, what do you do when you are presented with a need
    to extract something particular from an unstructured document containing
    dog+world and at completely arbitrary depth? You can say things like "just
    make an struct that addresses it" or "you shouldn't do it that way in the
    first place!". But that's not helpful when what you have has been handed
    to you and your job is just to make it work.

    Use case...
    There are custom forms created using a custom tool that runs clientside,
    built on top of angular.
    Any form, can contain any field those fields can and do contain subforms
    to any arbitrary depth.

    This allows for the creation of business forms that meet regulatory
    compliance by stringing them together from other forms that also meet
    compliance. In otherwords the person creating the forms does not want to
    have to re-invent the wheel. To their mind they are just adding new pages
    to existing forms.

    These were persisted wholesale in node and now the "legacy" node app is
    being replaced, we need to ensure that anything that node may have returned
    previously is also returned by go.

    In this particular case, the User has a custom reporting system that needs
    to extract data from field z of subform y of form x, which itself is
    embedded in form n (and n could be embedded in l etc).

    What you have as far as information about the thing you need to return is
    a document id generated by mongo during the last upsert, and a command
    coming from the browser that looks like
    [query: {collection: 'forms', _id: 100, path:'n.x.y.z' },...]

    The server neither knows, nor cares about the type of data stored there,
    it's job is to isolate requested field(s), extract the data from those
    fields and pass it back to the caller assuming they had permission to read
    it :).

    This was handled in node with nothing more than(pseudocode follows)
    document = db.fetchOne(query._id)
    if(document.hasOwnProperty(query.path)){
    res.write(JSON.stringify(document[query.path]))
    }else{
    res.send(404)
    }



    I can't fathom what that would need to look like in Go, using the
    recommended or idiomatic approach, but replacing that lovely little tidbit
    was a task, way up high on my todo list.
    The replacement code now looks about the same
    (again, psuedo code here)

    collection.Find(bson.M{'_id': query.ID}).One(&document)
    section := json.HasOwnProperty(query.path,&document)
    if section !=nil {
    fmt.Fprintf(w, "%s",json.Marshall(&document))
    }else{
    http.Error(404)
    }

    Also a possibility: https://play.golang.org/p/eSw9PMgQrF

    collection.Find(bson.M{'_id': query.ID}).One(&document)
    if section, ok := document.Get(query.path); ok {
         fmt.Fprintf(w, "%s",json.Marshal(&document))
    }else{
        http.Error(404)
    }

    Obviously the "correct" answer here is to dedup and normalize the data ,
    however the goal of the project is to unify disparate information systems
    and to do so while having the smallest impact possible to existing systems
    all the while maintaining current levels of data integrity. Easiest path
    to maintaining data integrity is to leave legacy data laying where it sits
    while gradually phasing in replacment front ends and ensuring new data ends
    up stored in a more coherent fashion. Eventually the old data ends up in
    long term storage anyways, but the retention period on some of these
    records is 10 years before they go to archives.
    I'm in a position where I didn't make the data that I'm expected to deal
    with, but I have to ensure that every user of that data has at least the
    same quality of experience they do now.
    On the other hand I have no desire to take 1,000 lines of node and convert
    it into 100k lines of Go just to cover every edge case.
    In that situation your approach makes perfectly sense.

    My instincts which are informed by over a decade of coming in and cleaning
    up after 1st year college freshman, H1Bs who couldn't spell SQL without
    "My" in front of it and *shudder* PHP devs, tell me that things like I've
    described tend to be the norm after any "super pumped, agile, web 2.0,
    html11, kitchen sink" project reaches more than about 6 months of
    development.

    I have a feeling that as node and other "web server" tech becomes
    supplanted by Go, things like this are going to come to light pretty
    frequently, and my goal with this posting was to leave a trace for anyone
    in the future bumping up against similar things. Hope it's helpful to
    someone.


    On Thursday, January 22, 2015 at 3:47:13 AM UTC-8, Egon wrote:


    On Thursday, 22 January 2015 13:31:16 UTC+2, Klaus Post wrote:

    Hi!

    Good call by Egon - you should go for that.
    Actually, my recommended approach for that problem was
    http://play.golang.org/p/4SqbLxr_lg. But for dealing random arbitrary
    data is simplejson is better. For something inbetween, there are multiple
    approaches, i.e. use map[string]interface{} directly, or properly marshal
    into a nice type. Using a proper type simplifies code else-where while
    keeping the problematic part in a single place.

    + Egon

    Also it is trivial to implement what you need, see:

    http://play.golang.org/p/5oeZxliMyn

    Of course you would need to type assert the results you get, but that is
    the deal when dealing with loosely structured data.


    /Klaus
    On Thursday, 22 January 2015 07:57:03 UTC+1, maxpow...@gmail.com wrote:

    Note to the group: This message is intended for future people who like
    me are coming to golang after spending a good deal of time out in nodejs
    land.

    I've been a programmer for a lot of years, C/C++, Java, Node.js you
    name it I've probably put it to good use at some point.

    Lately I've found that I like Go alot and I really want to keep liking
    Go, but having used node.js as my go to webdev language for the last couple
    of years has spoiled me in the way I interact with loosely structured data.

    This is most obvious when I start dealing with JSON being passed back
    and forth across the wire.
    I really do hate Javascript & callback hell and all the other reasons I
    decided this current project would not be built with node, but I love
    dealing with JSON in node because it's native and feels natural.

    I saw that Go advertises strong support for JSON, and I'll admit that
    the language features a very good parser.
    However if you need data from an arbitrary field, it's just not able to
    do it and messing with the idiomatic methods of doing it just looks ugly in
    my opinion.

    I spent a few days struggling with this, because I guess most folks who
    use Go are used to dealing with predictable structured data so most of the
    JSON examples don't really go into any great depth of how to deal with data
    that doesn't fit into structures that can be built at compile time.

    In my case actual data interaction is minimal, but when I need a field
    it may be buried 10 layers deep in a JSON object, and I don't even get to
    control whether or not that field is present.
    I just need the language to get out of my way and let me have the data
    at field x.y.z.a.n.w.t.f or tell me something along the lines of "Sorry
    that field couldn't be found."

    This is the one place where Javascript really shines, I can simply say
    something like

    if(someObject.hasOwnProperty('some.field')){
    doSomethingWith(someObject)
    }


    And it just works!

    As far as I can tell, go has no equivalent.

    So I was faced with a few choices.
    #1 Ditch Go and run back node.js (not going to happen even if I have to
    hard code every possible data structure, lives will literally depend on
    this code)
    #2 Hard code structs for the expected types merely for the purpose of
    checking for field values, (that's a LOT of code and I am a very lazy
    person)
    #3 Use one of the Javascript interpreters for Go (too much overhead
    just for this, and leaning on JS could lead to very bad things)
    #4 Take a close look at my previous node projects and see what features
    were actually being used the most and try to replicate them in Go.

    I opted for number 4.

    The most common things I do with server side javascript
    #1 Marshall Strings into JSON
    #2 Check JSON for presence of fields
    #3 Take logical actions based on data contained in said fields
    #4 Ship JSON out somewhere else

    Actual data manipulation is very, very rare in my past projects.
    Usually once data has been JSONized somewhere, the primary purpose of said
    data is to tell some controller somewhere to take some action.
    Thus I did not include any way to actually manipulate JSON Objects.

    The result is here, https://play.golang.org/p/cRgh7j2TW2

    I've given you two functions.
    The first one is json.Parse which takes a string and gives you a
    map[string]interface{} which is the closest thing to an actual JSON object
    that you are going to find in Go.
    The second function is json.HasOwnProperty which takes a field label
    and the json object created by Parse and will return to you either the
    value at said label or nil

    It does differ from Javascript here in that it will return the actual
    value of a label rather than a boolean indicating whether it was present or
    not.
    I did that because asking if a field is present and then asking it to
    retrieve that field are two steps that really should be just one, imho.

    You can use the code like this...

    First get a JSON string and parse it into a JSON object

    const B = `{
    "allowedtypes" :{
    "medical": {
    "sources":{
    "myself": ["*"],
    "mylocation":["r"],
    "other": ["a"]
    }
    }
    }
    }`

    var objB := json.Parse(B)


    Next query and assign the field to a variable so you can take action on
    it...
    permissions := json.HasOwnProperty(
    "allowedtypes.medical.sources.myself", objB)
    fmt.Printf("Permissions for allowedtypes.medical.sources.myself
    %+v\n", permissions)

    That's it!
    Thanks for taking the time to read this and I hope someone who stumbles
    on this later can put it to good use.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Maxpowers477 at Jan 23, 2015 at 12:37 am
    Ooh I really like that one Egon. You never cease to amaze me.
    One thing though, I would always treat words like class & object as though
    they were reserved words (for later maintenance, it can get confusing
    especially if like me you need to language hop a lot).
    Instead of

    type Object map[string]interface{}

    It might be clearer in the future if it were

    type JSONObject map[string]interface{}

    I say this after having worked on a project where somebody decided to
    redefine boolean into 3 states, true, false & null/never set
    Not sure if they were trying to create Job security by being the only one
    "in the know" or if they had some legit reason, but eventually they left
    and my team was tasked with fixing some oddities that had crept up.

    Just imagine how much fun that caused for us at debug time :)

    On Thursday, January 22, 2015 at 7:00:18 AM UTC-8, Egon wrote:


    On Thursday, 22 January 2015 15:12:23 UTC+2, maxpow...@gmail.com wrote:

    Yeah Egon has some great ideas and reading his code replies to my
    previous posts is where I noticed this particular opportunity.

    I agree that there are a lot of approaches, to dealing with unstructured
    data.

    My intent was more narrow and was intended to show how easy it is to
    reimplement something akin to hasOwnProperty from the JS world in Go.

    This is not to say that either approach mentioned by Egon is in anyway
    deficient, just the particular use case of needing an object or value at
    some arbitrary depth in unstructured data does not appear to be addressed
    any where that I have been able to find.

    I showed an example using permissions since that was the thing that was
    buggered up when I started, but consider a different use case for a moment.

    When you store a document in Mongo you tend to store the whole document,
    these things do not lend themselves to relational structuring very easily
    and can get heavy if you're not careful.

    Again I'm not addressing the stupidity of the idea of trying to store
    *.world+dog , but merely, what do you do when you are presented with a need
    to extract something particular from an unstructured document containing
    dog+world and at completely arbitrary depth? You can say things like "just
    make an struct that addresses it" or "you shouldn't do it that way in the
    first place!". But that's not helpful when what you have has been handed
    to you and your job is just to make it work.

    Use case...
    There are custom forms created using a custom tool that runs clientside,
    built on top of angular.
    Any form, can contain any field those fields can and do contain subforms
    to any arbitrary depth.

    This allows for the creation of business forms that meet regulatory
    compliance by stringing them together from other forms that also meet
    compliance. In otherwords the person creating the forms does not want to
    have to re-invent the wheel. To their mind they are just adding new pages
    to existing forms.

    These were persisted wholesale in node and now the "legacy" node app is
    being replaced, we need to ensure that anything that node may have returned
    previously is also returned by go.

    In this particular case, the User has a custom reporting system that
    needs to extract data from field z of subform y of form x, which itself is
    embedded in form n (and n could be embedded in l etc).

    What you have as far as information about the thing you need to return is
    a document id generated by mongo during the last upsert, and a command
    coming from the browser that looks like
    [query: {collection: 'forms', _id: 100, path:'n.x.y.z' },...]

    The server neither knows, nor cares about the type of data stored there,
    it's job is to isolate requested field(s), extract the data from those
    fields and pass it back to the caller assuming they had permission to read
    it :).

    This was handled in node with nothing more than(pseudocode follows)
    document = db.fetchOne(query._id)
    if(document.hasOwnProperty(query.path)){
    res.write(JSON.stringify(document[query.path]))
    }else{
    res.send(404)
    }



    I can't fathom what that would need to look like in Go, using the
    recommended or idiomatic approach, but replacing that lovely little tidbit
    was a task, way up high on my todo list.
    The replacement code now looks about the same
    (again, psuedo code here)

    collection.Find(bson.M{'_id': query.ID}).One(&document)
    section := json.HasOwnProperty(query.path,&document)
    if section !=nil {
    fmt.Fprintf(w, "%s",json.Marshall(&document))
    }else{
    http.Error(404)
    }

    Also a possibility: https://play.golang.org/p/eSw9PMgQrF

    collection.Find(bson.M{'_id': query.ID}).One(&document)
    if section, ok := document.Get(query.path); ok {
    fmt.Fprintf(w, "%s",json.Marshal(&document))
    }else{
    http.Error(404)
    }

    Obviously the "correct" answer here is to dedup and normalize the data ,
    however the goal of the project is to unify disparate information systems
    and to do so while having the smallest impact possible to existing systems
    all the while maintaining current levels of data integrity. Easiest path
    to maintaining data integrity is to leave legacy data laying where it sits
    while gradually phasing in replacment front ends and ensuring new data ends
    up stored in a more coherent fashion. Eventually the old data ends up in
    long term storage anyways, but the retention period on some of these
    records is 10 years before they go to archives.
    I'm in a position where I didn't make the data that I'm expected to deal
    with, but I have to ensure that every user of that data has at least the
    same quality of experience they do now.
    On the other hand I have no desire to take 1,000 lines of node and
    convert it into 100k lines of Go just to cover every edge case.
    In that situation your approach makes perfectly sense.

    My instincts which are informed by over a decade of coming in and
    cleaning up after 1st year college freshman, H1Bs who couldn't spell SQL
    without "My" in front of it and *shudder* PHP devs, tell me that things
    like I've described tend to be the norm after any "super pumped, agile, web
    2.0, html11, kitchen sink" project reaches more than about 6 months of
    development.

    I have a feeling that as node and other "web server" tech becomes
    supplanted by Go, things like this are going to come to light pretty
    frequently, and my goal with this posting was to leave a trace for anyone
    in the future bumping up against similar things. Hope it's helpful to
    someone.


    On Thursday, January 22, 2015 at 3:47:13 AM UTC-8, Egon wrote:


    On Thursday, 22 January 2015 13:31:16 UTC+2, Klaus Post wrote:

    Hi!

    Good call by Egon - you should go for that.
    Actually, my recommended approach for that problem was
    http://play.golang.org/p/4SqbLxr_lg. But for dealing random arbitrary
    data is simplejson is better. For something inbetween, there are multiple
    approaches, i.e. use map[string]interface{} directly, or properly marshal
    into a nice type. Using a proper type simplifies code else-where while
    keeping the problematic part in a single place.

    + Egon

    Also it is trivial to implement what you need, see:

    http://play.golang.org/p/5oeZxliMyn

    Of course you would need to type assert the results you get, but that
    is the deal when dealing with loosely structured data.


    /Klaus
    On Thursday, 22 January 2015 07:57:03 UTC+1, maxpow...@gmail.com wrote:

    Note to the group: This message is intended for future people who
    like me are coming to golang after spending a good deal of time out in
    nodejs land.

    I've been a programmer for a lot of years, C/C++, Java, Node.js you
    name it I've probably put it to good use at some point.

    Lately I've found that I like Go alot and I really want to keep liking
    Go, but having used node.js as my go to webdev language for the last couple
    of years has spoiled me in the way I interact with loosely structured data.

    This is most obvious when I start dealing with JSON being passed back
    and forth across the wire.
    I really do hate Javascript & callback hell and all the other reasons
    I decided this current project would not be built with node, but I love
    dealing with JSON in node because it's native and feels natural.

    I saw that Go advertises strong support for JSON, and I'll admit that
    the language features a very good parser.
    However if you need data from an arbitrary field, it's just not able
    to do it and messing with the idiomatic methods of doing it just looks ugly
    in my opinion.

    I spent a few days struggling with this, because I guess most folks
    who use Go are used to dealing with predictable structured data so most of
    the JSON examples don't really go into any great depth of how to deal with
    data that doesn't fit into structures that can be built at compile time.

    In my case actual data interaction is minimal, but when I need a field
    it may be buried 10 layers deep in a JSON object, and I don't even get to
    control whether or not that field is present.
    I just need the language to get out of my way and let me have the data
    at field x.y.z.a.n.w.t.f or tell me something along the lines of "Sorry
    that field couldn't be found."

    This is the one place where Javascript really shines, I can simply say
    something like

    if(someObject.hasOwnProperty('some.field')){
    doSomethingWith(someObject)
    }


    And it just works!

    As far as I can tell, go has no equivalent.

    So I was faced with a few choices.
    #1 Ditch Go and run back node.js (not going to happen even if I have
    to hard code every possible data structure, lives will literally depend on
    this code)
    #2 Hard code structs for the expected types merely for the purpose of
    checking for field values, (that's a LOT of code and I am a very lazy
    person)
    #3 Use one of the Javascript interpreters for Go (too much overhead
    just for this, and leaning on JS could lead to very bad things)
    #4 Take a close look at my previous node projects and see what
    features were actually being used the most and try to replicate them in Go.

    I opted for number 4.

    The most common things I do with server side javascript
    #1 Marshall Strings into JSON
    #2 Check JSON for presence of fields
    #3 Take logical actions based on data contained in said fields
    #4 Ship JSON out somewhere else

    Actual data manipulation is very, very rare in my past projects.
    Usually once data has been JSONized somewhere, the primary purpose of said
    data is to tell some controller somewhere to take some action.
    Thus I did not include any way to actually manipulate JSON Objects.

    The result is here, https://play.golang.org/p/cRgh7j2TW2

    I've given you two functions.
    The first one is json.Parse which takes a string and gives you a
    map[string]interface{} which is the closest thing to an actual JSON object
    that you are going to find in Go.
    The second function is json.HasOwnProperty which takes a field label
    and the json object created by Parse and will return to you either the
    value at said label or nil

    It does differ from Javascript here in that it will return the actual
    value of a label rather than a boolean indicating whether it was present or
    not.
    I did that because asking if a field is present and then asking it to
    retrieve that field are two steps that really should be just one, imho.

    You can use the code like this...

    First get a JSON string and parse it into a JSON object

    const B = `{
    "allowedtypes" :{
    "medical": {
    "sources":{
    "myself": ["*"],
    "mylocation":["r"],
    "other": ["a"]
    }
    }
    }
    }`

    var objB := json.Parse(B)


    Next query and assign the field to a variable so you can take action
    on it...
    permissions := json.HasOwnProperty(
    "allowedtypes.medical.sources.myself", objB)
    fmt.Printf("Permissions for allowedtypes.medical.sources.myself
    %+v\n", permissions)

    That's it!
    Thanks for taking the time to read this and I hope someone who
    stumbles on this later can put it to good use.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Egon at Jan 23, 2015 at 7:23 am

    On Friday, 23 January 2015 02:36:56 UTC+2, maxpow...@gmail.com wrote:
    Ooh I really like that one Egon. You never cease to amaze me.
    One thing though, I would always treat words like class & object as though
    they were reserved words (for later maintenance, it can get confusing
    especially if like me you need to language hop a lot).
    Instead of

    type Object map[string]interface{}

    It might be clearer in the future if it were

    type JSONObject map[string]interface{}
    Name the package "js" then the fully qualified name will be "js.Object"

    + Egon

    I say this after having worked on a project where somebody decided to
    redefine boolean into 3 states, true, false & null/never set
    Not sure if they were trying to create Job security by being the only one
    "in the know" or if they had some legit reason, but eventually they left
    and my team was tasked with fixing some oddities that had crept up.

    Just imagine how much fun that caused for us at debug time :)

    On Thursday, January 22, 2015 at 7:00:18 AM UTC-8, Egon wrote:


    On Thursday, 22 January 2015 15:12:23 UTC+2, maxpow...@gmail.com wrote:

    Yeah Egon has some great ideas and reading his code replies to my
    previous posts is where I noticed this particular opportunity.

    I agree that there are a lot of approaches, to dealing with unstructured
    data.

    My intent was more narrow and was intended to show how easy it is to
    reimplement something akin to hasOwnProperty from the JS world in Go.

    This is not to say that either approach mentioned by Egon is in anyway
    deficient, just the particular use case of needing an object or value at
    some arbitrary depth in unstructured data does not appear to be addressed
    any where that I have been able to find.

    I showed an example using permissions since that was the thing that was
    buggered up when I started, but consider a different use case for a moment.

    When you store a document in Mongo you tend to store the whole document,
    these things do not lend themselves to relational structuring very easily
    and can get heavy if you're not careful.

    Again I'm not addressing the stupidity of the idea of trying to store
    *.world+dog , but merely, what do you do when you are presented with a need
    to extract something particular from an unstructured document containing
    dog+world and at completely arbitrary depth? You can say things like "just
    make an struct that addresses it" or "you shouldn't do it that way in the
    first place!". But that's not helpful when what you have has been handed
    to you and your job is just to make it work.

    Use case...
    There are custom forms created using a custom tool that runs clientside,
    built on top of angular.
    Any form, can contain any field those fields can and do contain subforms
    to any arbitrary depth.

    This allows for the creation of business forms that meet regulatory
    compliance by stringing them together from other forms that also meet
    compliance. In otherwords the person creating the forms does not want to
    have to re-invent the wheel. To their mind they are just adding new pages
    to existing forms.

    These were persisted wholesale in node and now the "legacy" node app is
    being replaced, we need to ensure that anything that node may have returned
    previously is also returned by go.

    In this particular case, the User has a custom reporting system that
    needs to extract data from field z of subform y of form x, which itself is
    embedded in form n (and n could be embedded in l etc).

    What you have as far as information about the thing you need to return
    is a document id generated by mongo during the last upsert, and a command
    coming from the browser that looks like
    [query: {collection: 'forms', _id: 100, path:'n.x.y.z' },...]

    The server neither knows, nor cares about the type of data stored there,
    it's job is to isolate requested field(s), extract the data from those
    fields and pass it back to the caller assuming they had permission to read
    it :).

    This was handled in node with nothing more than(pseudocode follows)
    document = db.fetchOne(query._id)
    if(document.hasOwnProperty(query.path)){
    res.write(JSON.stringify(document[query.path]))
    }else{
    res.send(404)
    }



    I can't fathom what that would need to look like in Go, using the
    recommended or idiomatic approach, but replacing that lovely little tidbit
    was a task, way up high on my todo list.
    The replacement code now looks about the same
    (again, psuedo code here)

    collection.Find(bson.M{'_id': query.ID}).One(&document)
    section := json.HasOwnProperty(query.path,&document)
    if section !=nil {
    fmt.Fprintf(w, "%s",json.Marshall(&document))
    }else{
    http.Error(404)
    }

    Also a possibility: https://play.golang.org/p/eSw9PMgQrF

    collection.Find(bson.M{'_id': query.ID}).One(&document)
    if section, ok := document.Get(query.path); ok {
    fmt.Fprintf(w, "%s",json.Marshal(&document))
    }else{
    http.Error(404)
    }

    Obviously the "correct" answer here is to dedup and normalize the data ,
    however the goal of the project is to unify disparate information systems
    and to do so while having the smallest impact possible to existing systems
    all the while maintaining current levels of data integrity. Easiest path
    to maintaining data integrity is to leave legacy data laying where it sits
    while gradually phasing in replacment front ends and ensuring new data ends
    up stored in a more coherent fashion. Eventually the old data ends up in
    long term storage anyways, but the retention period on some of these
    records is 10 years before they go to archives.
    I'm in a position where I didn't make the data that I'm expected to deal
    with, but I have to ensure that every user of that data has at least the
    same quality of experience they do now.
    On the other hand I have no desire to take 1,000 lines of node and
    convert it into 100k lines of Go just to cover every edge case.
    In that situation your approach makes perfectly sense.

    My instincts which are informed by over a decade of coming in and
    cleaning up after 1st year college freshman, H1Bs who couldn't spell SQL
    without "My" in front of it and *shudder* PHP devs, tell me that things
    like I've described tend to be the norm after any "super pumped, agile, web
    2.0, html11, kitchen sink" project reaches more than about 6 months of
    development.

    I have a feeling that as node and other "web server" tech becomes
    supplanted by Go, things like this are going to come to light pretty
    frequently, and my goal with this posting was to leave a trace for anyone
    in the future bumping up against similar things. Hope it's helpful to
    someone.


    On Thursday, January 22, 2015 at 3:47:13 AM UTC-8, Egon wrote:


    On Thursday, 22 January 2015 13:31:16 UTC+2, Klaus Post wrote:

    Hi!

    Good call by Egon - you should go for that.
    Actually, my recommended approach for that problem was
    http://play.golang.org/p/4SqbLxr_lg. But for dealing random arbitrary
    data is simplejson is better. For something inbetween, there are multiple
    approaches, i.e. use map[string]interface{} directly, or properly marshal
    into a nice type. Using a proper type simplifies code else-where while
    keeping the problematic part in a single place.

    + Egon

    Also it is trivial to implement what you need, see:

    http://play.golang.org/p/5oeZxliMyn

    Of course you would need to type assert the results you get, but that
    is the deal when dealing with loosely structured data.


    /Klaus

    On Thursday, 22 January 2015 07:57:03 UTC+1, maxpow...@gmail.com
    wrote:
    Note to the group: This message is intended for future people who
    like me are coming to golang after spending a good deal of time out in
    nodejs land.

    I've been a programmer for a lot of years, C/C++, Java, Node.js you
    name it I've probably put it to good use at some point.

    Lately I've found that I like Go alot and I really want to keep
    liking Go, but having used node.js as my go to webdev language for the last
    couple of years has spoiled me in the way I interact with loosely
    structured data.

    This is most obvious when I start dealing with JSON being passed back
    and forth across the wire.
    I really do hate Javascript & callback hell and all the other reasons
    I decided this current project would not be built with node, but I love
    dealing with JSON in node because it's native and feels natural.

    I saw that Go advertises strong support for JSON, and I'll admit that
    the language features a very good parser.
    However if you need data from an arbitrary field, it's just not able
    to do it and messing with the idiomatic methods of doing it just looks ugly
    in my opinion.

    I spent a few days struggling with this, because I guess most folks
    who use Go are used to dealing with predictable structured data so most of
    the JSON examples don't really go into any great depth of how to deal with
    data that doesn't fit into structures that can be built at compile time.

    In my case actual data interaction is minimal, but when I need a
    field it may be buried 10 layers deep in a JSON object, and I don't even
    get to control whether or not that field is present.
    I just need the language to get out of my way and let me have the
    data at field x.y.z.a.n.w.t.f or tell me something along the lines of
    "Sorry that field couldn't be found."

    This is the one place where Javascript really shines, I can simply
    say something like

    if(someObject.hasOwnProperty('some.field')){
    doSomethingWith(someObject)
    }


    And it just works!

    As far as I can tell, go has no equivalent.

    So I was faced with a few choices.
    #1 Ditch Go and run back node.js (not going to happen even if I have
    to hard code every possible data structure, lives will literally depend on
    this code)
    #2 Hard code structs for the expected types merely for the purpose of
    checking for field values, (that's a LOT of code and I am a very lazy
    person)
    #3 Use one of the Javascript interpreters for Go (too much overhead
    just for this, and leaning on JS could lead to very bad things)
    #4 Take a close look at my previous node projects and see what
    features were actually being used the most and try to replicate them in Go.

    I opted for number 4.

    The most common things I do with server side javascript
    #1 Marshall Strings into JSON
    #2 Check JSON for presence of fields
    #3 Take logical actions based on data contained in said fields
    #4 Ship JSON out somewhere else

    Actual data manipulation is very, very rare in my past projects.
    Usually once data has been JSONized somewhere, the primary purpose of said
    data is to tell some controller somewhere to take some action.
    Thus I did not include any way to actually manipulate JSON Objects.

    The result is here, https://play.golang.org/p/cRgh7j2TW2

    I've given you two functions.
    The first one is json.Parse which takes a string and gives you a
    map[string]interface{} which is the closest thing to an actual JSON object
    that you are going to find in Go.
    The second function is json.HasOwnProperty which takes a field label
    and the json object created by Parse and will return to you either the
    value at said label or nil

    It does differ from Javascript here in that it will return the actual
    value of a label rather than a boolean indicating whether it was present or
    not.
    I did that because asking if a field is present and then asking it to
    retrieve that field are two steps that really should be just one, imho.

    You can use the code like this...

    First get a JSON string and parse it into a JSON object

    const B = `{
    "allowedtypes" :{
    "medical": {
    "sources":{
    "myself": ["*"],
    "mylocation":["r"],
    "other": ["a"]
    }
    }
    }
    }`

    var objB := json.Parse(B)


    Next query and assign the field to a variable so you can take action
    on it...
    permissions := json.HasOwnProperty(
    "allowedtypes.medical.sources.myself", objB)
    fmt.Printf("Permissions for allowedtypes.medical.sources.myself
    %+v\n", permissions)

    That's it!
    Thanks for taking the time to read this and I hope someone who
    stumbles on this later can put it to good use.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • C Banning at Jan 25, 2015 at 1:07 pm
    http://godoc.org/github.com/clbanning/mxj#Map.NewMapJson and
      http://godoc.org/github.com/clbanning/mxj#Map.ValuesForKey would have
    worked too - but if that's all you need the mxi package is probably
    overkill.

    On Thursday, January 22, 2015 at 12:57:03 AM UTC-6, maxpow...@gmail.com
    wrote:
    Note to the group: This message is intended for future people who like me
    are coming to golang after spending a good deal of time out in nodejs land.

    I've been a programmer for a lot of years, C/C++, Java, Node.js you name
    it I've probably put it to good use at some point.

    Lately I've found that I like Go alot and I really want to keep liking Go,
    but having used node.js as my go to webdev language for the last couple of
    years has spoiled me in the way I interact with loosely structured data.

    This is most obvious when I start dealing with JSON being passed back and
    forth across the wire.
    I really do hate Javascript & callback hell and all the other reasons I
    decided this current project would not be built with node, but I love
    dealing with JSON in node because it's native and feels natural.

    I saw that Go advertises strong support for JSON, and I'll admit that the
    language features a very good parser.
    However if you need data from an arbitrary field, it's just not able to do
    it and messing with the idiomatic methods of doing it just looks ugly in my
    opinion.

    I spent a few days struggling with this, because I guess most folks who
    use Go are used to dealing with predictable structured data so most of the
    JSON examples don't really go into any great depth of how to deal with data
    that doesn't fit into structures that can be built at compile time.

    In my case actual data interaction is minimal, but when I need a field it
    may be buried 10 layers deep in a JSON object, and I don't even get to
    control whether or not that field is present.
    I just need the language to get out of my way and let me have the data at
    field x.y.z.a.n.w.t.f or tell me something along the lines of "Sorry that
    field couldn't be found."

    This is the one place where Javascript really shines, I can simply say
    something like

    if(someObject.hasOwnProperty('some.field')){
    doSomethingWith(someObject)
    }


    And it just works!

    As far as I can tell, go has no equivalent.

    So I was faced with a few choices.
    #1 Ditch Go and run back node.js (not going to happen even if I have to
    hard code every possible data structure, lives will literally depend on
    this code)
    #2 Hard code structs for the expected types merely for the purpose of
    checking for field values, (that's a LOT of code and I am a very lazy
    person)
    #3 Use one of the Javascript interpreters for Go (too much overhead just
    for this, and leaning on JS could lead to very bad things)
    #4 Take a close look at my previous node projects and see what features
    were actually being used the most and try to replicate them in Go.

    I opted for number 4.

    The most common things I do with server side javascript
    #1 Marshall Strings into JSON
    #2 Check JSON for presence of fields
    #3 Take logical actions based on data contained in said fields
    #4 Ship JSON out somewhere else

    Actual data manipulation is very, very rare in my past projects. Usually
    once data has been JSONized somewhere, the primary purpose of said data is
    to tell some controller somewhere to take some action.
    Thus I did not include any way to actually manipulate JSON Objects.

    The result is here, https://play.golang.org/p/cRgh7j2TW2

    I've given you two functions.
    The first one is json.Parse which takes a string and gives you a
    map[string]interface{} which is the closest thing to an actual JSON object
    that you are going to find in Go.
    The second function is json.HasOwnProperty which takes a field label and
    the json object created by Parse and will return to you either the value at
    said label or nil

    It does differ from Javascript here in that it will return the actual
    value of a label rather than a boolean indicating whether it was present or
    not.
    I did that because asking if a field is present and then asking it to
    retrieve that field are two steps that really should be just one, imho.

    You can use the code like this...

    First get a JSON string and parse it into a JSON object

    const B = `{
    "allowedtypes" :{
    "medical": {
    "sources":{
    "myself": ["*"],
    "mylocation":["r"],
    "other": ["a"]
    }
    }
    }
    }`

    var objB := json.Parse(B)


    Next query and assign the field to a variable so you can take action on
    it...
    permissions := json.HasOwnProperty("allowedtypes.medical.sources.myself",
    objB)
    fmt.Printf("Permissions for allowedtypes.medical.sources.myself %+v\n"
    , permissions)

    That's it!
    Thanks for taking the time to read this and I hope someone who stumbles on
    this later can put it to good use.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedJan 22, '15 at 6:57a
activeJan 25, '15 at 1:07p
posts9
users4
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase