FAQ
Unfortunately, no -- global state is a bit difficult in a distributed
world :)

That said, a single process is responsible for launching your MapReduce
jobs. If you're talking about iterations of MapReduce jobs, you can keep
the switch in your local code, and decide whether or not to launch the
next job within the same process.

If you need to coordinate actually between mappers (which I suspect you
can get around), you can store global state in an external database like
Memcached or Redis and poll during your operations.
sourabh.chaki@corp.247customer.com
August 6, 2013 4:14 AM
In my application I am running set of mappers and aggregator using
jcascalog. I am executing these in multiple iterations. Now I want to
implement "stop the iteration if we are done" feature. Means each
mapper to set global flag shouldContinue = shouldContinue || false if
it identifies the end of its processing. So start the next iteration
only if shouldContinue ==true else stop the processing.

Ideally I am looking for a global boolean variable that can be
accessed and modified by mappers or CascalogFunctions.

Can we achieve something like this in jcascalog?

Thanks
Sourabh
--
You received this message because you are subscribed to the Google
Groups "cascalog-user" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to cascalog-user+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
Sam Ritchie, Twitter Inc
703.662.1337
@sritchie

--
You received this message because you are subscribed to the Google Groups "cascalog-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascalog-user+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

Discussion Posts

Previous

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 2 of 2 | next ›
Discussion Overview
groupcascalog-user @
categoriesclojure, hadoop
postedAug 6, '13 at 11:14a
activeAug 6, '13 at 3:18p
posts2
users2
websiteclojure.org
irc#clojure

2 users in discussion

Sourabh Chaki: 1 post Sam Ritchie: 1 post

People

Translate

site design / logo © 2022 Grokbase