FAQ
I'm trying to figure out how to achieve the following from a Java client,
1. My app (which is a web server) starts up
2. As part of startup my jar file, which includes my map reduce classes are distributed to hadoop nodes
3. My web app uses map reduce to extract data without the performance overhead of each job deploying a jar file, via setJar(), setJarByClass()

It looks like DistributedCache() has potential but the need for commands like 'hadoop fs -copyFromLocal ...' and the API methods like '.getLocalCacheArchives()' look to be at odds with my scenario. Any thoughts?

-Peter

Search Discussions

  • Pete Tyler at Sep 13, 2010 at 1:46 pm
    I'm trying to figure out how to achieve the following from a Java client,
    1. My app (which is a web server) starts up
    2. As part of startup my jar file, which includes my map reduce classes are distributed to hadoop nodes
    3. My web app uses map reduce to extract data without the performance overhead of each job deploying a jar file, via setJar(), setJarByClass()

    It looks like DistributedCache() has potential but the need for commands like 'hadoop fs -copyFromLocal ...' and the API methods like '.getLocalCacheArchives()' look to be at odds with my scenario. Any thoughts?

    -Peter

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedSep 13, '10 at 3:20a
activeSep 13, '10 at 1:46p
posts2
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Pete Tyler: 2 posts

People

Translate

site design / logo © 2022 Grokbase