Grokbase Groups Avro user May 2013
Making the DatumReader/Writers configurable would be a welcome addition.

Ideally, much more of what goes on there could be:
  1. configuration driven
  2. pre-computed to avoid repeated work during decoding/encoding

We do some of both already. The trick is to do #1 without impacting
performance and #2 requires a bigger overhaul.

If you would like, a contribution including a Clojure related maven module
or two that depends on the Java stuff would be a welcome addition and
allow us to identify compatibility issues as we change the Java library
over time.

On 5/8/13 3:33 PM, "Marshall Bockrath-Vandegrift" wrote:

Hi all:

Is there a reason Avro¹s Hadoop serialization classes don¹t allow
configuration of the DatumReader and DatumWriter classes?

My use-case is that I¹m implementing Clojure DatumReader and -Writer
classes which produce and consume Clojure¹s data structures directly.
I¹d like to then extend that to Hadoop MapReduce jobs which operate in
terms of Clojure data, with Avro handling all de/serialization directly
to/from that Clojure data.

Am I going around this in a backwards fashion, or would a patch to allow
configuration of the Hadoop serialization DatumReader/Writers be


Search Discussions

Discussion Posts


Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 2 of 3 | next ›
Discussion Overview
groupuser @
postedMay 8, '13 at 10:35p
activeMay 13, '13 at 11:23p



site design / logo © 2021 Grokbase