Hello, Nathan et al,

Could you please help me figure out how to pass parameters to aggregators?
Let's say I want to do something non-trivial in reduce - I can write a
custom aggregator or a bufferop, but I can't seem to find a way to pass a
parameter to either of those.

(In this context, a "parameter" is something that configures aggregator
behavior, not the aggregator's input data)

Thanks,
Artem.

Search Discussions

  • Sam Ritchie at Nov 18, 2011 at 3:40 am
    Hey Artem,

    Do you have an example of what you're trying to do? We're working on a new
    syntax that will give you access to the JobConf, but here's how Cascalog
    aggregators currently take parameters:

    The usual syntax for a defaggregateop is

    (defaggregateop agg-name
    ([] ...initialize-state...)
    ([state arg1 arg2] ...returns next state iteration...)
    ([state] ...final aggregator return...))

    Used like (agg-name ?arg1 ?arg2 :> ?outvar)

    To send in a parameter, use this syntax:

    (defaggregateop [agg-name [param-1 param-2]] ;; as many as you want
    ([] ...initialize-state...)
    ([state arg1 arg2] ...returns next state iteration...)
    ([state] ...final aggregator return...))

    Pass params in like this: (agg-name ["param1" #{1 2 3}] ?arg1 ?arg2 :>
    ?outvar)

    Does this help?

    Cheers,
    Sam

    On Thu, Nov 17, 2011 at 4:14 PM, Artem Boytsov
    wrote:
    Hello, Nathan et al,

    Could you please help me figure out how to pass parameters to aggregators?
    Let's say I want to do something non-trivial in reduce - I can write a
    custom aggregator or a bufferop, but I can't seem to find a way to pass a
    parameter to either of those.

    (In this context, a "parameter" is something that configures aggregator
    behavior, not the aggregator's input data)

    Thanks,
    Artem.


    --
    Sam Ritchie, Twitter Inc
    703.662.1337
    @sritchie09

    (Too brief? Here's why! http://emailcharter.org)

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcascalog-user @
categoriesclojure, hadoop
postedNov 17, '11 at 11:15p
activeNov 18, '11 at 3:40a
posts2
users2
websiteclojure.org
irc#clojure

2 users in discussion

Artem Boytsov: 1 post Sam Ritchie: 1 post

People

Translate

site design / logo © 2022 Grokbase