FAQ
Hello,

I have run into a problem when attempting to run jobs that require
code from multiple jars. The JobConf only allows me to set one jar
file, so the other jars are not copied to the task trackers, and hence
the jobs fail due to the dependencies. Besides merging the jars or
manually copying them to each machine, is there a solution for this?

Thanks in advance,
Albert

Search Discussions

  • Darek Zbik at Oct 6, 2006 at 8:54 am

    Albert Chern wrote:
    Hello,

    I have run into a problem when attempting to run jobs that require
    code from multiple jars. The JobConf only allows me to set one jar
    file, so the other jars are not copied to the task trackers, and hence
    the jobs fail due to the dependencies. Besides merging the jars or
    manually copying them to each machine, is there a solution for this?
    Try to create lib directory in your jar and put your dependency into
    this directory. Contents of the jar's lib directory is added to
    classloader during starting map and reduce tasks. Unfortunately it does
    not work if you try to use additional libraries in InputReader in this
    case classlaoder is not modified.

    Retards,

    darek

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedOct 6, '06 at 7:35a
activeOct 6, '06 at 8:54a
posts2
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Albert Chern: 1 post Darek Zbik: 1 post

People

Translate

site design / logo © 2022 Grokbase