Grokbase Groups Avro user

Search Discussions

933 discussions - 3,279 posts

  • I am trying to implement a JSON validation class, that takes in JSON (as a std::string) and validates it against the established schema. I have managed to create the JSON schema and get avrogencpp to ...
    Kyle CummingsKyle Cummings
    Oct 29, 2014 at 6:19 pm
    Oct 29, 2014 at 6:19 pm
  • Hi All, I am trying to use avro's protocol definition to provide the basis for an RPC system. In this case, I'll be handling the handshaking and transport layer rather the avro's IPC package. Are the ...
    Camp, JonathanCamp, Jonathan
    Oct 26, 2014 at 7:23 pm
    Oct 27, 2014 at 7:40 pm
  • Hi I would like to understand if there is a way to use the Avro libraries to serialize data from a .NET input stream. The input stream will contain JSON strings. I want to convert the JSON strings ...
    Oct 23, 2014 at 7:18 pm
    Oct 23, 2014 at 7:18 pm
  • HI , I have complicated nested POJOs. I am able to generate the schema using ReflectData tool. I want to convert them to Generic Record preferably or Indexed Records. I dnt want to hand code and set ...
    Oct 21, 2014 at 5:11 pm
    Oct 24, 2014 at 6:15 pm
  • Hi, Imagine a scenario where one reads avro file and distributes individual records (aka datums). Suppose the schema is known, and we just need to pass records as bytes to another system for further ...
    Ivan BalashovIvan Balashov
    Oct 15, 2014 at 11:48 am
    Oct 24, 2014 at 6:18 pm
  • Thought I’d ask for a second set of eyes. I’m working with a large schema, a basic translation of some tab separated data with many fields (over 300). It appears that the output of Random can not be ...
    Pritchard, Charles X. -NDPritchard, Charles X. -ND
    Oct 15, 2014 at 12:34 am
    Oct 24, 2014 at 6:44 pm
  • Hi, a problem arised with the Avro Builder in a MapReduce-program I am writing. According to this issue should have been fixed a while ago. However, I ...
    Tobias LangTobias Lang
    Oct 9, 2014 at 2:19 pm
    Oct 9, 2014 at 2:19 pm
  • Has anyone run into the problem where the generated java class for an enum has a dollar sign for one enum value? The schema {"type": "enum", "name": "ButtonTypeID", "symbols": ["default", ...
    Lukas SteiblysLukas Steiblys
    Oct 8, 2014 at 7:06 pm
    Oct 8, 2014 at 8:07 pm
  • Hi, Don't know since when, the avro-maven-plugin cannot compile the Java code generated from the avro.avsc file any more. Here is my pom.xml file with the avro-maven-plugin: <plugin <groupId ...
    Oct 7, 2014 at 7:33 pm
    Oct 7, 2014 at 7:33 pm
  • You can only get that kind of error from an avro_value_get_current_branch call. `branch` and `leftNode` should be records, so avro_value_copy shouldn't try to call that function on `branch` or ...
    Douglas CreagerDouglas Creager
    Oct 6, 2014 at 7:25 pm
    Oct 6, 2014 at 7:25 pm
  • I'm working on a project that involves reading in records that may be one of a large number of different types. Here's a sample schema to give you the basic idea: protocol AvroTestProtocol { ...
    Brad SmithBrad Smith
    Oct 6, 2014 at 5:45 pm
    Oct 6, 2014 at 5:45 pm
  • Good Day all, I’m hoping to get some insight(s) if possible... with regards to Data Validation... We are replacing our std POJO's with AVRO backed Schema POJO's.. We currently utilized Hibernate's ...
    Al PivonkaAl Pivonka
    Oct 6, 2014 at 1:10 pm
    Oct 6, 2014 at 6:21 pm
  • I'm looking for a complete Map Reduce example for Python. Particularly interested in how you connect to a Hadoop instance, and how you write to and read from it. Any pointers welcome. Cheers, Fred
    Frederic LaruelleFrederic Laruelle
    Sep 27, 2014 at 4:01 pm
    Sep 27, 2014 at 7:01 pm
  • Hi I am wondering why avro-cpp JsonGenerator encodeBinary encode every byte as unicode? 132 class AVRO_DECL JsonGenerator { 157 158 void escapeCtl(char c) { 159 out_.write('\\'); 160 ...
    Patrick NipPatrick Nip
    Sep 27, 2014 at 3:01 am
    Sep 27, 2014 at 3:01 am
  • According to the documentation: avro::GenericDatum::GenericDatum ( const ...
    Patrick NipPatrick Nip
    Sep 25, 2014 at 6:48 pm
    Sep 25, 2014 at 6:48 pm
  • Hi Folks, Apologies for cross posting. The Apache Gora team are pleased to announce the immediate availability of Apache Gora 0.5. The Apache Gora open source framework provides an in-memory data ...
    Lewis john mcgibbneyLewis john mcgibbney
    Sep 20, 2014 at 7:26 pm
    Sep 20, 2014 at 7:26 pm
  • I've been experimenting with MapReduce job using CSV and avro format. What I find it strange is that Avro format is larger than CSV. For example, I exported some data in CSV, which is about 1.6GB. I ...
    diplomatic Gurudiplomatic Guru
    Sep 19, 2014 at 10:31 am
    Sep 19, 2014 at 2:21 pm
  • I am trying to use the c++ generic interface to update the following record: { "type": "record", "namespace": "", "name": “def", "fields": [ { "name": "id", "type": [ "null", "bytes" ] } ] ...
    Patrick NipPatrick Nip
    Sep 15, 2014 at 4:46 am
    Sep 15, 2014 at 5:13 am
  • Hi All, Is the following legal schema: { { "metadata" : { "schema" : { "family" : "search", "version" : "v1", "attrs" : [ "srch" ] } } }{ "metadata" : { "schema" : { "family" : "UDB", "version" ...
    Patrick NipPatrick Nip
    Sep 15, 2014 at 12:29 am
    Sep 15, 2014 at 12:29 am
  • Hi, I am new to avro and I am trying to use it for RPC calls from Python to Python. I follow the same structure shown in When ...
    Marcelo Elias Del ValleMarcelo Elias Del Valle
    Sep 10, 2014 at 5:27 pm
    Sep 10, 2014 at 5:27 pm
  • Does Avro (Java API) have support for Lazy Deserial? I see some tickets suggesting that it does. --
    Grant Overby (groverby)Grant Overby (groverby)
    Sep 10, 2014 at 4:37 pm
    Sep 10, 2014 at 4:37 pm
  • Good evening. This is snippet of avro schema field that I need to build data for and populate in php *{"name": "meta__kvpairs", "type": [ "null", { "type": "map", "values": ["null", "string"] } ] ...
    Vadim KeylisVadim Keylis
    Sep 10, 2014 at 6:27 am
    Sep 10, 2014 at 11:01 pm
  • I'm pretty new to avro and am trying to evaluate its potential as a "container" for hdfs file contents. I do know about SequenceFiles but would like a language agnostic file format if possible. which ...
    Sep 9, 2014 at 5:15 pm
    Sep 9, 2014 at 5:15 pm
  • Hi, I wanted to see how much overhead would be involved if we ship an avro schema along with message in a messaging context. Seems like it might simplify things to not always need a schema registry ...
    Roger HooverRoger Hoover
    Sep 8, 2014 at 10:07 pm
    Sep 8, 2014 at 10:07 pm
  • [] More specifically if I have the following payload with 2 records, "metadata" and "data". Can avro cpp library parse each record successively? If ...
    Patrick NipPatrick Nip
    Sep 5, 2014 at 11:36 pm
    Sep 5, 2014 at 11:36 pm
  • We've recently upgraded our Avro dependency to 1.7.7 from 1.7.5. The avdl for Avro used to look like this: record Event{ long creationTime; /** * The optional payload of the event. */ union{null, ...
    Micah WhitacreMicah Whitacre
    Aug 27, 2014 at 6:08 pm
    Aug 28, 2014 at 7:49 pm
  • Hi Users, I am trying to debug a rather unusual bug where Avro tool fails to read a simple file. I compiled Avro on Windows as explained in README.maintaining_win32.txt. I am using this library with ...
    Ujjwal WadhawanUjjwal Wadhawan
    Aug 26, 2014 at 8:40 pm
    Aug 26, 2014 at 8:40 pm
  • Hi, Does anyone know how I can avoid an exception for the following code: ------------------------------------------------------------------- public class EntityWrapper<T { T obj; } public class ...
    S GS G
    Aug 22, 2014 at 6:46 pm
    Aug 25, 2014 at 6:57 pm
  • Hi all, I have a couple of questions - I have read through some related discussions on the list and also through the code, so this is mainly to confirm my thinking. Is there any capability, or is ...
    Sam LawranceSam Lawrance
    Aug 22, 2014 at 1:34 pm
    Aug 22, 2014 at 6:05 pm
  • Hi list, I'm working on a project in Java where we have a DSL working on GenericRecord objects, over which we define record transformation operations like projections, filters and so. This implies ...
    Juan Rodríguez HortaláJuan Rodríguez Hortalá
    Aug 19, 2014 at 6:24 am
    Aug 21, 2014 at 9:39 am
  • Hi, I'm writing a MR 2 job in which I'm reading plain text as input and producing avro output. On running the job in local mode, the output is being serialized into json format. What can I do so that ...
    Anand NalyaAnand Nalya
    Aug 16, 2014 at 9:38 am
    Aug 16, 2014 at 2:41 pm
  • Hi, I'm sorry, I'm feeling kind of dense at the moment. I just upgraded to Avro 1.7.7 to hopefully use the new Decimal type ( ), however I can't seem ...
    Michael PigottMichael Pigott
    Aug 11, 2014 at 5:53 pm
    Aug 12, 2014 at 4:41 am
  • Hi, I wonder if I'm running into a bug with builders for specific records here. I'm using the Builder of a specific record to create a new instance of that record with the same properties. See the ...
    Xander UiterlindenXander Uiterlinden
    Aug 6, 2014 at 2:18 pm
    Aug 7, 2014 at 4:18 pm
  • Hello, I am trying to (i) read avro files in pig on Amazon EMR which I have created in my local cluster from JSONs (complex nested including arrays) and uploaded to S3 (ii) Create avro files in EMR ...
    Klüber, RalfKlüber, Ralf
    Aug 6, 2014 at 1:27 pm
    Aug 21, 2014 at 3:58 pm
  • Hi, I would like to know what is the best way to convert POJO into Avro type object and send it as an response from REST web service? I have managed to do following things : 1. Write avsc files and ...
    Chandrashekhar KotekarChandrashekhar Kotekar
    Aug 6, 2014 at 7:46 am
    Aug 8, 2014 at 6:15 pm
  • Greetings, I am desiring to read and write Avro files (such as those manipulated by MapReduce applications) from a C++ program. While there are higher-level wrappers (such as Hive), I am interested ...
    John LilleyJohn Lilley
    Aug 5, 2014 at 8:58 pm
    Aug 18, 2014 at 11:41 pm
  • Hi, I looked through the JIRA tickets but I did not find one that matched what I just ran into - unions can't have multiple maps, even if each one has a different value type? Automatic-map-generation ...
    Michael PigottMichael Pigott
    Aug 5, 2014 at 7:28 pm
    Aug 5, 2014 at 9:55 pm
  • Hello, We recently refactored our large avro schema into separate schemas to make them more composable, since many of them get incorporated into other objects. We were glad to see that the ...
    Peterson, MichaelPeterson, Michael
    Aug 4, 2014 at 7:29 pm
    Aug 4, 2014 at 10:13 pm
  • Hi, I was trying to use Avro IDL to organize a complicated schema and ran into a problem with mutually recursive types. Here is a simple example: { "type" : "record", "name" : "Outer", "fields" : [ { ...
    Shafquat RahmanShafquat Rahman
    Aug 1, 2014 at 6:44 pm
    Aug 1, 2014 at 6:44 pm
  • I had some issues with the cmakefile when I built avro c++ for windows a month or two ago. If I remebered correctly it did not find or possibly figure out the configuration of boost. I ended up doing ...
    Svante karlssonSvante karlsson
    Aug 1, 2014 at 12:38 pm
    Aug 1, 2014 at 5:55 pm
  • We used mapreduce for ETL and storing results in Avro files, which are loaded to hive/impala for query. Now we are trying to migrate to spark, but didn't find a way to write resulting RDD to Avro ...
    Fengyun RAOFengyun RAO
    Jul 30, 2014 at 9:15 am
    Jul 30, 2014 at 3:38 pm
  • I'd like to announce the availability of Avro release 1.7.7. Changes are listed at: This release can be downloaded from: Java ...
    Doug CuttingDoug Cutting
    Jul 24, 2014 at 9:27 pm
    Jul 24, 2014 at 9:27 pm
  • Is there support for encoding/decoding with snappy compression in avro-cpp? Any tips are appreciated. Best, Robert
    Robert VoyerRobert Voyer
    Jul 24, 2014 at 7:34 pm
    Jul 24, 2014 at 7:34 pm
  • I have successfully stream Avro data file to Python mrjobs using the library AvroAsTextInputFormat -inputformat org.apache.avro.mapred.AvroAsTextInputFormat However, unlike text file, it does not ...
    Jul 24, 2014 at 12:08 am
    Jul 25, 2014 at 12:22 am
  • Hello, I'm new here, hope I can get help from you guys. Basically I have an avro file with union/many schemas and mixed records. I will need to split it to many avro file, one schema per file. All ...
    Echo LiEcho Li
    Jul 22, 2014 at 7:07 pm
    Jul 25, 2014 at 12:13 am
  • Hi, I woud like to know if it is possible to send data in avro format to and from jersey based web services. Regards, Chandrash3khar Kotekar Mobile - +91 8600011455
    Chandrashekhar KotekarChandrashekhar Kotekar
    Jul 21, 2014 at 2:24 pm
    Jul 22, 2014 at 5:07 am
  • Hi List, As part of my freelance work supporting a Java team, I have been asked to provide them with Flume messages using the Avro serialisation. First of all, there are some problems with the trunk ...
    Lee HambleyLee Hambley
    Jul 17, 2014 at 2:04 pm
    Jul 21, 2014 at 9:01 pm
  • Hi everyone I am trying to convert an avro file to csv. Are there any tools that support this conversion? Avro totext tool only supports a generic schema and fails if my avro file schema does not ...
    Bhuvana BellalaBhuvana Bellala
    Jul 14, 2014 at 6:52 pm
    Jul 14, 2014 at 7:20 pm
  • Hi, I'm exploring the human-readable avro options in the avro-tools jar, namely `tojson` and `totext`. `tojson` works fine, but I try `totext` with: `$ java -jar avro-tools-1.7.6.jar totext ...
    Jul 14, 2014 at 5:23 am
    Jul 21, 2014 at 4:59 pm
  • How do I just serialize single message using php? It appears php uses Container Files serialization approach and for my project I need to serialize single message without attaching schema as header ...
    Vadim KeylisVadim Keylis
    Jul 10, 2014 at 10:29 pm
    Jul 11, 2014 at 5:31 am
Group Navigation
period‹ prev | Latest | first ›
Group Overview
groupuser @

Top users

Doug Cutting: 457 posts Scott Carey: 148 posts Scott Carey: 144 posts Harsh J: 62 posts Lewis John Mcgibbney: 61 posts Yang: 52 posts Russell Jurney: 45 posts Ey-chih chow: 37 posts Jeremy Lewi: 37 posts James Baldassari: 36 posts Jeff Hammerbacher: 34 posts Martin Kleppmann: 34 posts Tatu Saloranta: 31 posts Philip Zeyliger: 29 posts Weishung Chung: 25 posts Jonathan Coveney: 24 posts Miki Tebeka: 23 posts Mark: 23 posts Koert Kuipers: 22 posts Harsh J: 22 posts
show more