Grokbase Groups Avro user March 2011
FAQ
I have create a schema as

{

"namespace": "avro.test",

"name": "testTransferObject",

"type": "map",

"values": {

"type": "array",

"items" : {

"type" : "map",

"values" : {

"namespace":"avro.test",

"name" : "entry",

"type" : "record",

"fields" : [

{"name":"Id", "type":"int"},

{"name":"Category1", "type":"long"},

{"name":"Category2", "type":"long"},

{"name":"Segment1", "type":"long"},

{"name":"Segment2", "type":"long"},

{"name":"index", "type":"double"}

]

}

}

}

}



The code generated by pecificCompiler.compileSchema only contains entry
under the package avro.test. I am wondering where is the Map and Array go?
And the wrapper object testTransferObject does not seems to be generated.


Thanks,


Felix

Search Discussions

  • Doug Cutting at Apr 1, 2011 at 4:39 pm
    Maps are not a named type in Avro. The "name" field you specify in the
    "map" schemas is ignored. Only record, enum, and fixed are named types.
    If you wish to refer to a map by name in schemas, define a record with
    a map as its single field. This adds no storage overhead: it will
    serialize to the same bytes as the map alone.

    Doug
    On 03/31/2011 04:56 PM, felix gao wrote:
    I have create a schema as

    {

    "namespace": "avro.test",

    "name": "testTransferObject",

    "type": "map",

    "values": {

    "type": "array",

    "items" : {

    "type" : "map",

    "values" : {

    "namespace":"avro.test",

    "name" : "entry",

    "type" : "record",

    "fields" : [

    {"name":"Id", "type":"int"},

    {"name":"Category1", "type":"long"},

    {"name":"Category2", "type":"long"},

    {"name":"Segment1", "type":"long"},

    {"name":"Segment2", "type":"long"},

    {"name":"index", "type":"double"}

    ]

    }

    }

    }

    }



    The code generated by pecificCompiler.compileSchema only contains entry
    under the package avro.test. I am wondering where is the Map and Array
    go? And the wrapper object testTransferObject does not seems to be
    generated.


    Thanks,


    Felix

  • Scott Carey at Apr 1, 2011 at 5:12 pm
    On 3/31/11 4:56 PM, "felix gao" wrote:

    I have create a schema as

    {

    "namespace": "avro.test",

    "name": "testTransferObject",

    "type": "map",

    "values": {

    "type": "array",

    "items" : {

    "type" : "map",

    "values" : {

    "namespace":"avro.test",

    "name" : "entry",

    "type" : "record",

    "fields" : [

    {"name":"Id", "type":"int"},

    {"name":"Category1", "type":"long"},

    {"name":"Category2", "type":"long"},

    {"name":"Segment1", "type":"long"},

    {"name":"Segment2", "type":"long"},

    {"name":"index", "type":"double"}

    ]

    }

    }

    }

    }



    The code generated by pecificCompiler.compileSchema only contains entry under the package avro.test. I am wondering where is the Map and Array go?

    Avro maps and arrays become java Map and List objects respectively. There is no reason to compile them. The SpecificRecord for the array above is:
    List<Map<Utf8, avro.test.entry>>
    As Doug points out, the top level map is not a named data type, so it is not generated. Only record, enum, and fixed result in generated classes. In code it would be of type:
    Map<Utf8, List<Map<Utf8, avro.test.entry>>>



    And the wrapper object testTransferObject does not seems to be generated.


    Thanks,


    Felix
  • Felix gao at Apr 1, 2011 at 7:07 pm
    Thanks for the explanation. so when I construct my data, I just need to put
    them into the concrete java Map or List collection and it should work
    right?

    Felix
    On Fri, Apr 1, 2011 at 10:15 AM, Scott Carey wrote:

    On 3/31/11 4:56 PM, "felix gao" wrote:

    I have create a schema as

    {

    "namespace": "avro.test",

    "name": "testTransferObject",

    "type": "map",

    "values": {

    "type": "array",

    "items" : {

    "type" : "map",

    "values" : {

    "namespace":"avro.test",

    "name" : "entry",

    "type" : "record",

    "fields" : [

    {"name":"Id", "type":"int"},

    {"name":"Category1", "type":"long"},

    {"name":"Category2", "type":"long"},

    {"name":"Segment1", "type":"long"},

    {"name":"Segment2", "type":"long"},

    {"name":"index", "type":"double"}

    ]

    }

    }

    }

    }



    The code generated by pecificCompiler.compileSchema only contains entry
    under the package avro.test. I am wondering where is the Map and Array go?


    Avro maps and arrays become java Map and List objects respectively. There
    is no reason to compile them. The SpecificRecord for the array above is:
    List<Map<Utf8, avro.test.entry>>
    As Doug points out, the top level map is not a named data type, so it is
    not generated. Only record, enum, and fixed result in generated classes.
    In code it would be of type:
    Map<Utf8, List<Map<Utf8, avro.test.entry>>>


    And the wrapper object testTransferObject does not seems to be generated.


    Thanks,


    Felix


Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriesavro
postedMar 31, '11 at 11:56p
activeApr 1, '11 at 7:07p
posts4
users3
websiteavro.apache.org
irc#avro

People

Translate

site design / logo © 2021 Grokbase