FAQ
In my application I am running set of mappers and aggregator using
jcascalog. I am executing these in multiple iterations. Now I want to
implement "stop the iteration if we are done" feature. Means each mapper to
set global flag shouldContinue = shouldContinue || false if it identifies
the end of its processing. So start the next iteration only
if shouldContinue ==true else stop the processing.

Ideally I am looking for a global boolean variable that can be accessed and
modified by mappers or CascalogFunctions.

Can we achieve something like this in jcascalog?

Thanks
Sourabh

--
You received this message because you are subscribed to the Google Groups "cascalog-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascalog-user+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Sam Ritchie at Aug 6, 2013 at 3:18 pm
    Unfortunately, no -- global state is a bit difficult in a distributed
    world :)

    That said, a single process is responsible for launching your MapReduce
    jobs. If you're talking about iterations of MapReduce jobs, you can keep
    the switch in your local code, and decide whether or not to launch the
    next job within the same process.

    If you need to coordinate actually between mappers (which I suspect you
    can get around), you can store global state in an external database like
    Memcached or Redis and poll during your operations.
    sourabh.chaki@corp.247customer.com
    August 6, 2013 4:14 AM
    In my application I am running set of mappers and aggregator using
    jcascalog. I am executing these in multiple iterations. Now I want to
    implement "stop the iteration if we are done" feature. Means each
    mapper to set global flag shouldContinue = shouldContinue || false if
    it identifies the end of its processing. So start the next iteration
    only if shouldContinue ==true else stop the processing.

    Ideally I am looking for a global boolean variable that can be
    accessed and modified by mappers or CascalogFunctions.

    Can we achieve something like this in jcascalog?

    Thanks
    Sourabh
    --
    You received this message because you are subscribed to the Google
    Groups "cascalog-user" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to cascalog-user+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    Sam Ritchie, Twitter Inc
    703.662.1337
    @sritchie

    --
    You received this message because you are subscribed to the Google Groups "cascalog-user" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to cascalog-user+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcascalog-user @
categoriesclojure, hadoop
postedAug 6, '13 at 11:14a
activeAug 6, '13 at 3:18p
posts2
users2
websiteclojure.org
irc#clojure

2 users in discussion

Sourabh Chaki: 1 post Sam Ritchie: 1 post

People

Translate

site design / logo © 2022 Grokbase