FAQ
Hey everyone,

I'm trying to follow along with the new Big Data book while trying to write
the code in Clojure.
I'm finding examples of using Pail from Clojure pretty sparse. I'd also
like to be able to use
JSON or EDN as a serialization format rather than Thrift. Can someone
provide either a bare
bones example of this kind of integration, or an explanation of how the
pieces should fit together?

Thanks,
--Mike

--
You received this message because you are subscribed to the Google Groups "cascalog-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascalog-user+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Mike Drogalis at Jun 13, 2013 at 5:05 pm
    I should point out that I want to append to files in Hadoop, and thus am
    using Pail.
    It looks like Pail requires an out-of-band schema, which clashes with
    EDN/JSON's in-band schema definitions.
    That's kind of where the problem lies for me at the moment.
    On Thursday, June 13, 2013 12:45:48 PM UTC-4, Mike Drogalis wrote:

    Hey everyone,

    I'm trying to follow along with the new Big Data book while trying to
    write the code in Clojure.
    I'm finding examples of using Pail from Clojure pretty sparse. I'd also
    like to be able to use
    JSON or EDN as a serialization format rather than Thrift. Can someone
    provide either a bare
    bones example of this kind of integration, or an explanation of how the
    pieces should fit together?

    Thanks,
    --Mike
    --
    You received this message because you are subscribed to the Google Groups "cascalog-user" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to cascalog-user+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • David Kincaid at Jun 13, 2013 at 6:45 pm
    An excellent example of using Pail from Clojure is the forma-clj project (
    https://github.com/reddmetrics/forma-clj<https://github.com/reddmetrics/forma-clj/blob/develop/src/clj/forma/hadoop/pail.clj>/).
    Especially look at the pail.clj code:
    https://github.com/reddmetrics/forma-clj/blob/develop/src/clj/forma/hadoop/pail.clj

    I've used that project as a reference quite a bit lately.
    On Thursday, June 13, 2013 1:05:43 PM UTC-4, Mike Drogalis wrote:

    I should point out that I want to append to files in Hadoop, and thus am
    using Pail.
    It looks like Pail requires an out-of-band schema, which clashes with
    EDN/JSON's in-band schema definitions.
    That's kind of where the problem lies for me at the moment.
    On Thursday, June 13, 2013 12:45:48 PM UTC-4, Mike Drogalis wrote:

    Hey everyone,

    I'm trying to follow along with the new Big Data book while trying to
    write the code in Clojure.
    I'm finding examples of using Pail from Clojure pretty sparse. I'd also
    like to be able to use
    JSON or EDN as a serialization format rather than Thrift. Can someone
    provide either a bare
    bones example of this kind of integration, or an explanation of how the
    pieces should fit together?

    Thanks,
    --Mike
    --
    You received this message because you are subscribed to the Google Groups "cascalog-user" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to cascalog-user+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Mike Drogalis at Jun 13, 2013 at 7:22 pm
    Thanks David. I'd actually looked at that project, and it's still fuzzy to
    me how to use Pail from Clojure. Any way you can pare it down to its
    essentials?
    On Thursday, June 13, 2013 2:45:50 PM UTC-4, David Kincaid wrote:

    An excellent example of using Pail from Clojure is the forma-clj project (
    https://github.com/reddmetrics/forma-clj<https://github.com/reddmetrics/forma-clj/blob/develop/src/clj/forma/hadoop/pail.clj>/).
    Especially look at the pail.clj code:
    https://github.com/reddmetrics/forma-clj/blob/develop/src/clj/forma/hadoop/pail.clj

    I've used that project as a reference quite a bit lately.
    On Thursday, June 13, 2013 1:05:43 PM UTC-4, Mike Drogalis wrote:

    I should point out that I want to append to files in Hadoop, and thus am
    using Pail.
    It looks like Pail requires an out-of-band schema, which clashes with
    EDN/JSON's in-band schema definitions.
    That's kind of where the problem lies for me at the moment.
    On Thursday, June 13, 2013 12:45:48 PM UTC-4, Mike Drogalis wrote:

    Hey everyone,

    I'm trying to follow along with the new Big Data book while trying to
    write the code in Clojure.
    I'm finding examples of using Pail from Clojure pretty sparse. I'd also
    like to be able to use
    JSON or EDN as a serialization format rather than Thrift. Can someone
    provide either a bare
    bones example of this kind of integration, or an explanation of how the
    pieces should fit together?

    Thanks,
    --Mike
    --
    You received this message because you are subscribed to the Google Groups "cascalog-user" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to cascalog-user+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcascalog-user @
categoriesclojure, hadoop
postedJun 13, '13 at 4:45p
activeJun 13, '13 at 7:22p
posts4
users2
websiteclojure.org
irc#clojure

2 users in discussion

Mike Drogalis: 3 posts David Kincaid: 1 post

People

Translate

site design / logo © 2021 Grokbase