Search Discussions

24 discussions - 60 posts

  • Folks, I am involved in a project to extract data from a large number of files (to be provided at some point), in numerous formats, among which is some Avro files (both binary and JSON-encoded), and ...
    Public Network ServicesPublic Network Services
    Jan 17, 2013 at 10:11 pm
    Jan 19, 2013 at 4:36 am
  • I am able to serialize with binary encoding to a file using following : FileOutputStream outputStream = new FileOutputStream(file); Encoder e = EncoderFactory.get().binaryEncoder(outputStream, ...
    Pratyush ChandraPratyush Chandra
    Jan 7, 2013 at 12:42 pm
    Jan 9, 2013 at 7:48 pm
  • Does Avro have an API to allow you to tell whether two schemas are a match, statically? i.e., schema1.canRead(schema2) /** return true iff schema1 can be used as a reader schema for schema2 */ there ...
    Aaron KimballAaron Kimball
    Jan 30, 2013 at 11:18 pm
    Feb 4, 2013 at 10:30 pm
  • I've gone through the documentation, but haven't been able to get a definite answer: is Avro, or specifically Trevni, only for read-only data? Is it possible to update or delete records? If records ...
    Jan 28, 2013 at 8:05 pm
    Jan 30, 2013 at 11:21 pm
  • I have an .avro file that I'm trying to use within a Map/Reduce job. I believe it was corrupted when I appended one file to another by mistake. Are there any tools to repair this?
    Terry HealyTerry Healy
    Jan 15, 2013 at 6:00 pm
    Jan 23, 2013 at 7:46 pm
  • How do I serialize a record containing a nested record? There doesn't seem to be any way to create a DataFileWriter without tieing it to a single record type. Here's the scenario: I've defined an ...
    Jan 21, 2013 at 2:15 pm
    Jan 22, 2013 at 6:35 pm
  • As I understand it, Avro container files contain synchronization markers every so often to support splitting the file. See ...
    Josh SpiegelJosh Spiegel
    Jan 23, 2013 at 9:10 pm
    Jan 24, 2013 at 3:25 pm
  • I have a log collection application that writes .avro files within HDFS. Ideally I would like to include the current days (open for append) file as one of the input files for a periodic M/R job. I ...
    Terry HealyTerry Healy
    Jan 14, 2013 at 7:22 pm
    Jan 18, 2013 at 2:51 pm
  • Hi, I'm new Avro Data serialization system. I want something hands on so that I can understand better about the Avro concepts. I searched on the net and got some examples but they're bit complex for ...
    Pradeep KanchgarPradeep Kanchgar
    Jan 14, 2013 at 11:21 am
    Jan 15, 2013 at 9:30 am
  • Hi, I have a simple schema say: { "namespace": "example.proto", "name": "Message", "type": "record", "fields": [ {"name": "user", "type": ["string","null"]} ] } When I am running map reduce with ...
    Pratyush ChandraPratyush Chandra
    Jan 9, 2013 at 7:46 pm
    Jan 14, 2013 at 7:07 pm
  • Hello- I'm upgrading a logging program to append GenericRecords to a .avro file instead of text (.tsv). I have a working schema that is used to convert existing .tsv of the same format into .avro and ...
    Terry HealyTerry Healy
    Jan 3, 2013 at 8:37 pm
    Jan 9, 2013 at 4:22 pm
  • Hi, I have a schema with an array of records (I'm open to other suggestions too) field called ifnet to store misc attribute name/values for a host's network interfaces. e.g. { "type": "record", ...
    Alan MillerAlan Miller
    Jan 7, 2013 at 4:35 pm
    Jan 8, 2013 at 11:48 am
  • Hi, I am new to Avro. I was going through examples and figured out that GenericRecord can be appended to DataFileWriter and then serialized. Example: record.avsc is { "namespace": "example.proto", ...
    Pratyush ChandraPratyush Chandra
    Jan 7, 2013 at 11:46 am
    Jan 8, 2013 at 10:33 am
  • We are trying to recover, report bad record, and move to the next record of an Avro file in PIG-3015 and PIG-3059. It seems that sync blocks don't exist between files, however. How should we recover ...
    Russell JurneyRussell Jurney
    Jan 7, 2013 at 3:39 am
    Jan 8, 2013 at 10:31 am
  • Is it necessary to write setters and getters for all member variables for a class that is going to be serialized by Avro? Thanks -Tanya
    Tanya BansalTanya Bansal
    Jan 7, 2013 at 7:22 pm
    Jan 8, 2013 at 9:42 am
  • Hi! Maybe I'm the only one ever used this :D. Adding namedOutputs with AvroMultipleOutputs.addNamedOutput just adds them to a static map which is of course not available on the cluster during reduce ...
    Johannes SchulteJohannes Schulte
    Jan 30, 2013 at 10:33 pm
    Jan 30, 2013 at 10:33 pm
  • We are using Apache Avro as a JSON interface between our Python application and some third party Java libraries that we run within a Tomcat service. We decided to simply extend the ...
    Satwik HebbarSatwik Hebbar
    Jan 28, 2013 at 7:19 am
    Jan 28, 2013 at 7:19 am
  • Hi, I'm trying to familiarize myself with avro-rpc by running and tweaking the example at https://github.com/jbaldassari/Avro-RPC. By the way, thanks for creating this example James. I'm using the ...
    Matt CorganMatt Corgan
    Jan 28, 2013 at 12:17 am
    Jan 28, 2013 at 12:17 am
  • Hi, I'm trying the example from http://avro.apache.org/docs/1.7.3/gettingstartedjava.html site. Serializing and deserializing with code generation Schema : user.avsc {"namespace": "example.avro", ...
    Pradeep KanchgarPradeep Kanchgar
    Jan 22, 2013 at 5:34 am
    Jan 22, 2013 at 5:34 am
  • Hi, We found that the method set(String string) in the class Utf8 of v. 1.7.3 was not supported in the corresponding jar file. Is this a bug? Thanks. Ey-Chih Chow
    Ey-chih chowEy-chih chow
    Jan 21, 2013 at 7:23 pm
    Jan 21, 2013 at 7:23 pm
  • Have contributed a Hamcrest matcher to GitHub which we find very useful in unit tests to check the equality of complex Avro objects in asserts and expectations: https://github.com/Byhiras/avro-utils ...
    Jacob MetcalfJacob Metcalf
    Jan 11, 2013 at 8:26 am
    Jan 11, 2013 at 8:26 am
  • Hi, Can somebody please paste here c++ sample code, how to validate json serialized data against given valid schema. I know nothing about data and schema, I have just two files on input - first ...
    Martin kourilMartin kouril
    Jan 10, 2013 at 2:22 pm
    Jan 10, 2013 at 2:22 pm
  • Hi, I have a complex schema consisting of primitives, multiple records, and multiple array of records fields.That's all great and everything works fine when I process these avro data files via ...
    Alan MillerAlan Miller
    Jan 10, 2013 at 2:01 pm
    Jan 10, 2013 at 2:01 pm
  • Hi, The issue is that it does not allow additional outputs with a different schema. I am using "org.apache.avro.mapred.AvroMultipleOutputs" following the Javadoc and pass in a "newSchema" that is ...
    Luke LiuLuke Liu
    Jan 9, 2013 at 12:02 am
    Jan 9, 2013 at 12:02 am
Group Navigation
period‹ prev | Jan 2013 | next ›
Group Overview
groupuser @

24 users for January 2013

Terry Healy: 9 posts Scott Carey: 8 posts Alan Miller: 6 posts Pratyush Chandra: 6 posts Doug Cutting: 3 posts Public Network Services: 3 posts Aaron Kimball: 2 posts Josh Spiegel: 2 posts Miki Tebeka: 2 posts Pradeep Kanchgar: 2 posts Russell Jurney: 2 posts Tanya Bansal: 2 posts Yimca: 2 posts Ccleve: 1 post Dave Beech: 1 post Ey-chih chow: 1 post Jacob Metcalf: 1 post Johannes Schulte: 1 post Luke Liu: 1 post Martin Kleppmann: 1 post
show more