FAQ
Hi All:

I am tearing out my hair trying to get a simple valueaggregator to run. This seems like an
easy enough thing, but I am consistently getting hit with ClassNotFoundException which
makes no sense to me. Any help would be immensely appreciated. Note that I've had
valueaggregators running on 0.13.*. I'm a little concerned there is a bug in 0.14.1. On the
other hand, maybe I'm just blind to something obvious and just need a boot to the head...

The environment is hadoop-0.14.1. I am running Java 1.6:
$ java -version
java version "1.6.0_02"
Java(TM) SE Runtime Environment (build 1.6.0_02-b05)
Java HotSpot(TM) Server VM (build 1.6.0_02-b05, mixed mode)

The class name is called testAggregator, and exists in my jar file:
$ jar -tvf hadoop-0.14.1-dev-company.jar | grep testAggregator
1826 Fri Oct 12 16:30:14 EDT 2007 com/company/hadoop/metrics/testAggregator.class

My XML file is testAggregator.xml and looks like:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>aggregator.descriptor.num</name>
<value>1</value>
</property>
<property>
<name>aggregator.descriptor.0</name>
<value>UserDefined,com.company.hadoop.metrics.testAggregator</value>
</property>
</configuration>

My code couldn't be simpler:

package com.company.hadoop.metrics;
// snip a bunch of imports
public class testAggregator extends ValueAggregatorBaseDescriptor {
public ArrayList<Entry> generateKeyValPairs(Object key, Object val) {
String fields[] = val.toString().split("\t");
ArrayList<Entry> retv = new ArrayList<Entry>();
retv.add(generateEntry(LONG_VALUE_SUM, fields[ 6 ], ONE));
retv.add(generateEntry(UNIQ_VALUE_COUNT, fields[ 6 ], new Text( fields[ 2 ])));
return retv;
}
public void configure(JobConf job) {}
}

I invoke things this way:

bin/hadoop jar hadoop-0.14.1-dev-company.jar org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJob /input/set1 /output/set1 8 textinputformat testAggregator.xml

All my jobs fail with a ClassNotFoundException like the following traceback:

2007-10-12 16:56:14,227 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=MAP, sessionId=
2007-10-12 16:56:14,277 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 8
2007-10-12 16:56:14,299 WARN org.apache.hadoop.mapred.TaskTracker: Error running child
java.lang.RuntimeException: java.lang.ClassNotFoundException:
com.company.hadoop.metrics.testAggregator
at
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.createInstance
(UserDefinedValueAggregatorDescriptor.java:56)
at org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.createAggregator(UserDefinedValueAggregatorDescriptor.java:63)
at org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.(ValueAggregatorJobBase.java:49)
at org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorDescriptors(ValueAggregatorJobBase.java:60)
at org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySpec(ValueAggregatorJobBase.java:69)
at org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(ValueAggregatorJobBase.java:37)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:32)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:185)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1777)
Caused by: java.lang.ClassNotFoundException: com.company.hadoop.metrics.testAggregator
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.createInstance
(UserDefinedValueAggregatorDescriptor.java:51)
... 13 more

As near as I can tell I am doing everything correctly. Other code existing in this jar file is
running just fine - in fact it was used to produce the input to this test. I've checked my
systems to make I've got the jar file (and XML file) onto each of the 4 compute nodes in the
system.

Can anybody spot my problem or tell me what to do to get this going?

Thanks,
C G


---------------------------------
Tonight's top picks. What will you watch tonight? Preview the hottest shows on Yahoo! TV.

Search Discussions

  • Runping Qi at Oct 12, 2007 at 10:16 pm
    Hadoop ignored the jar file you specified.
    This is a known problem andhopefully HADOOP-1622
    (https://issues.apache.org/jira/browse/HADOOP-1622) will fix the problem.

    In the meantime, you can create a class like below and add it to your jar:

    package your_package;


    public class MyValueAggregatorJob{

    public static void main(String args[]) throws IOException {
    JobConf job = ValueAggregatorJob.createValueAggregatorJob(args);
    JobClient.runJob(job);
    }
    }


    And use the above class as the main class when you submit hadoop job:
    bin/hadoop jar hadoop-0.14.1-dev-company.jar
    your_package.MyValueAggregatorJob /input/set1 /output/set1 8 textinputformat
    testAggregator.xml


    Hope this solve your problem.

    Runping



    -----Original Message-----
    From: C G
    Sent: Friday, October 12, 2007 2:43 PM
    To: hadoop-user@lucene.apache.org
    Subject: Question about valueaggregators in 0.14.1...

    Hi All:

    I am tearing out my hair trying to get a simple valueaggregator to run.
    This seems like an
    easy enough thing, but I am consistently getting hit with
    ClassNotFoundException which
    makes no sense to me. Any help would be immensely appreciated. Note
    that I've had
    valueaggregators running on 0.13.*. I'm a little concerned there is a bug
    in 0.14.1. On the
    other hand, maybe I'm just blind to something obvious and just need a
    boot to the head...

    The environment is hadoop-0.14.1. I am running Java 1.6:
    $ java -version
    java version "1.6.0_02"
    Java(TM) SE Runtime Environment (build 1.6.0_02-b05)
    Java HotSpot(TM) Server VM (build 1.6.0_02-b05, mixed mode)

    The class name is called testAggregator, and exists in my jar file:
    $ jar -tvf hadoop-0.14.1-dev-company.jar | grep testAggregator
    1826 Fri Oct 12 16:30:14 EDT 2007
    com/company/hadoop/metrics/testAggregator.class

    My XML file is testAggregator.xml and looks like:
    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <configuration>
    <property>
    <name>aggregator.descriptor.num</name>
    <value>1</value>
    </property>
    <property>
    <name>aggregator.descriptor.0</name>
    <value>UserDefined,com.company.hadoop.metrics.testAggregator</value>
    </property>
    </configuration>

    My code couldn't be simpler:

    package com.company.hadoop.metrics;
    // snip a bunch of imports
    public class testAggregator extends ValueAggregatorBaseDescriptor {
    public ArrayList<Entry> generateKeyValPairs(Object key, Object val) {
    String fields[] = val.toString().split("\t");
    ArrayList<Entry> retv = new ArrayList<Entry>();
    retv.add(generateEntry(LONG_VALUE_SUM, fields[ 6 ], ONE));
    retv.add(generateEntry(UNIQ_VALUE_COUNT, fields[ 6 ], new Text(
    fields[ 2 ])));
    return retv;
    }
    public void configure(JobConf job) {}
    }

    I invoke things this way:

    bin/hadoop jar hadoop-0.14.1-dev-company.jar
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJob /input/set1
    /output/set1 8 textinputformat testAggregator.xml

    All my jobs fail with a ClassNotFoundException like the following
    traceback:

    2007-10-12 16:56:14,227 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
    Initializing JVM Metrics with processName=MAP, sessionId=
    2007-10-12 16:56:14,277 INFO org.apache.hadoop.mapred.MapTask:
    numReduceTasks: 8
    2007-10-12 16:56:14,299 WARN org.apache.hadoop.mapred.TaskTracker: Error
    running child
    java.lang.RuntimeException: java.lang.ClassNotFoundException:
    com.company.hadoop.metrics.testAggregator
    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.createInstance
    (UserDefinedValueAggregatorDescriptor.java:56)
    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.createAggregator(UserDefinedValueAggregatorDescriptor.java:63)
    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.<init>(UserDefinedValueAggregatorDescriptor.java:75)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggr
    egatorDescriptor(ValueAggregatorJobBase.java:49)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregato
    rDescriptors(ValueAggregatorJobBase.java:60)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMy
    Spec(ValueAggregatorJobBase.java:69)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Va
    lueAggregatorJobBase.java:37)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82
    )
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:32)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82
    )
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:185)
    at
    org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1777)
    Caused by: java.lang.ClassNotFoundException:
    com.company.hadoop.metrics.testAggregator
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.createInstance
    (UserDefinedValueAggregatorDescriptor.java:51)
    ... 13 more

    As near as I can tell I am doing everything correctly. Other code
    existing in this jar file is
    running just fine - in fact it was used to produce the input to this
    test. I've checked my
    systems to make I've got the jar file (and XML file) onto each of the 4
    compute nodes in the
    system.

    Can anybody spot my problem or tell me what to do to get this going?

    Thanks,
    C G


    ---------------------------------
    Tonight's top picks. What will you watch tonight? Preview the hottest
    shows on Yahoo! TV.
  • C G at Oct 13, 2007 at 3:06 am
    Hi Runping and All:

    I took your suggestion, renamed my original code to testAggregatorPlugin and wrote a class testAggregator which contained the main as you show here. It also fails with the same traceback...so it seems something is profoundly wrong here.

    Is anybody else using the aggregation classes in 0.14.1 with success?

    This is ironic and frustrating...I have several other programs which use map/reduce that are quite complex and those are running just fine. This little plugin is all that stands in the way of delivering the system :-(.

    Any other comments/help, etc. most welcome...

    Thanks,
    C G

    Runping Qi wrote:

    Hadoop ignored the jar file you specified.
    This is a known problem andhopefully HADOOP-1622
    (https://issues.apache.org/jira/browse/HADOOP-1622) will fix the problem.

    In the meantime, you can create a class like below and add it to your jar:

    package your_package;


    public class MyValueAggregatorJob{

    public static void main(String args[]) throws IOException {
    JobConf job = ValueAggregatorJob.createValueAggregatorJob(args);
    JobClient.runJob(job);
    }
    }


    And use the above class as the main class when you submit hadoop job:
    bin/hadoop jar hadoop-0.14.1-dev-company.jar
    your_package.MyValueAggregatorJob /input/set1 /output/set1 8 textinputformat
    testAggregator.xml


    Hope this solve your problem.

    Runping



    -----Original Message-----
    From: C G
    Sent: Friday, October 12, 2007 2:43 PM
    To: hadoop-user@lucene.apache.org
    Subject: Question about valueaggregators in 0.14.1...

    Hi All:

    I am tearing out my hair trying to get a simple valueaggregator to run.
    This seems like an
    easy enough thing, but I am consistently getting hit with
    ClassNotFoundException which
    makes no sense to me. Any help would be immensely appreciated. Note
    that I've had
    valueaggregators running on 0.13.*. I'm a little concerned there is a bug
    in 0.14.1. On the
    other hand, maybe I'm just blind to something obvious and just need a
    boot to the head...

    The environment is hadoop-0.14.1. I am running Java 1.6:
    $ java -version
    java version "1.6.0_02"
    Java(TM) SE Runtime Environment (build 1.6.0_02-b05)
    Java HotSpot(TM) Server VM (build 1.6.0_02-b05, mixed mode)

    The class name is called testAggregator, and exists in my jar file:
    $ jar -tvf hadoop-0.14.1-dev-company.jar | grep testAggregator
    1826 Fri Oct 12 16:30:14 EDT 2007
    com/company/hadoop/metrics/testAggregator.class

    My XML file is testAggregator.xml and looks like:



    aggregator.descriptor.num
    1 >
    aggregator.descriptor.0
    UserDefined,com.company.hadoop.metrics.testAggregator

    My code couldn't be simpler:

    package com.company.hadoop.metrics;
    // snip a bunch of imports
    public class testAggregator extends ValueAggregatorBaseDescriptor {
    public ArrayList generateKeyValPairs(Object key, Object val) {
    String fields[] = val.toString().split("\t");
    ArrayList retv = new ArrayList();
    retv.add(generateEntry(LONG_VALUE_SUM, fields[ 6 ], ONE));
    retv.add(generateEntry(UNIQ_VALUE_COUNT, fields[ 6 ], new Text(
    fields[ 2 ])));
    return retv;
    }
    public void configure(JobConf job) {}
    }

    I invoke things this way:

    bin/hadoop jar hadoop-0.14.1-dev-company.jar
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJob /input/set1
    /output/set1 8 textinputformat testAggregator.xml

    All my jobs fail with a ClassNotFoundException like the following
    traceback:

    2007-10-12 16:56:14,227 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
    Initializing JVM Metrics with processName=MAP, sessionId=
    2007-10-12 16:56:14,277 INFO org.apache.hadoop.mapred.MapTask:
    numReduceTasks: 8
    2007-10-12 16:56:14,299 WARN org.apache.hadoop.mapred.TaskTracker: Error
    running child
    java.lang.RuntimeException: java.lang.ClassNotFoundException:
    com.company.hadoop.metrics.testAggregator
    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.createInstance
    (UserDefinedValueAggregatorDescriptor.java:56)
    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.createAggregator(UserDefinedValueAggregatorDescriptor.java:63)
    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.(UserDefinedValueAggregatorDescriptor.java:75)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggr
    egatorDescriptor(ValueAggregatorJobBase.java:49)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregato
    rDescriptors(ValueAggregatorJobBase.java:60)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMy
    Spec(ValueAggregatorJobBase.java:69)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Va
    lueAggregatorJobBase.java:37)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82
    )
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:32)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82
    )
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:185)
    at
    org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1777)
    Caused by: java.lang.ClassNotFoundException:
    com.company.hadoop.metrics.testAggregator
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.createInstance
    (UserDefinedValueAggregatorDescriptor.java:51)
    ... 13 more

    As near as I can tell I am doing everything correctly. Other code
    existing in this jar file is
    running just fine - in fact it was used to produce the input to this
    test. I've checked my
    systems to make I've got the jar file (and XML file) onto each of the 4
    compute nodes in the
    system.

    Can anybody spot my problem or tell me what to do to get this going?

    Thanks,
    C G


    ---------------------------------
    Tonight's top picks. What will you watch tonight? Preview the hottest
    shows on Yahoo! TV.



    ---------------------------------
    Be a better Heartthrob. Get better relationship answers from someone who knows.
    Yahoo! Answers - Check it out.
  • Dennis Kubes at Oct 13, 2007 at 3:10 am
    HADOOP-1622 has a new patch available. That may solve your problem.

    Dennis Kubes

    C G wrote:
    Hi Runping and All:

    I took your suggestion, renamed my original code to testAggregatorPlugin and wrote a class testAggregator which contained the main as you show here. It also fails with the same traceback...so it seems something is profoundly wrong here.

    Is anybody else using the aggregation classes in 0.14.1 with success?

    This is ironic and frustrating...I have several other programs which use map/reduce that are quite complex and those are running just fine. This little plugin is all that stands in the way of delivering the system :-(.

    Any other comments/help, etc. most welcome...

    Thanks,
    C G

    Runping Qi wrote:

    Hadoop ignored the jar file you specified.
    This is a known problem andhopefully HADOOP-1622
    (https://issues.apache.org/jira/browse/HADOOP-1622) will fix the problem.

    In the meantime, you can create a class like below and add it to your jar:

    package your_package;


    public class MyValueAggregatorJob{

    public static void main(String args[]) throws IOException {
    JobConf job = ValueAggregatorJob.createValueAggregatorJob(args);
    JobClient.runJob(job);
    }
    }


    And use the above class as the main class when you submit hadoop job:
    bin/hadoop jar hadoop-0.14.1-dev-company.jar
    your_package.MyValueAggregatorJob /input/set1 /output/set1 8 textinputformat
    testAggregator.xml


    Hope this solve your problem.

    Runping



    -----Original Message-----
    From: C G
    Sent: Friday, October 12, 2007 2:43 PM
    To: hadoop-user@lucene.apache.org
    Subject: Question about valueaggregators in 0.14.1...

    Hi All:

    I am tearing out my hair trying to get a simple valueaggregator to run.
    This seems like an
    easy enough thing, but I am consistently getting hit with
    ClassNotFoundException which
    makes no sense to me. Any help would be immensely appreciated. Note
    that I've had
    valueaggregators running on 0.13.*. I'm a little concerned there is a bug
    in 0.14.1. On the
    other hand, maybe I'm just blind to something obvious and just need a
    boot to the head...

    The environment is hadoop-0.14.1. I am running Java 1.6:
    $ java -version
    java version "1.6.0_02"
    Java(TM) SE Runtime Environment (build 1.6.0_02-b05)
    Java HotSpot(TM) Server VM (build 1.6.0_02-b05, mixed mode)

    The class name is called testAggregator, and exists in my jar file:
    $ jar -tvf hadoop-0.14.1-dev-company.jar | grep testAggregator
    1826 Fri Oct 12 16:30:14 EDT 2007
    com/company/hadoop/metrics/testAggregator.class

    My XML file is testAggregator.xml and looks like:



    aggregator.descriptor.num
    1
    aggregator.descriptor.0
    UserDefined,com.company.hadoop.metrics.testAggregator
    My code couldn't be simpler:

    package com.company.hadoop.metrics;
    // snip a bunch of imports
    public class testAggregator extends ValueAggregatorBaseDescriptor {
    public ArrayList generateKeyValPairs(Object key, Object val) {
    String fields[] = val.toString().split("\t");
    ArrayList retv = new ArrayList();
    retv.add(generateEntry(LONG_VALUE_SUM, fields[ 6 ], ONE));
    retv.add(generateEntry(UNIQ_VALUE_COUNT, fields[ 6 ], new Text(
    fields[ 2 ])));
    return retv;
    }
    public void configure(JobConf job) {}
    }

    I invoke things this way:

    bin/hadoop jar hadoop-0.14.1-dev-company.jar
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJob /input/set1
    /output/set1 8 textinputformat testAggregator.xml

    All my jobs fail with a ClassNotFoundException like the following
    traceback:

    2007-10-12 16:56:14,227 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
    Initializing JVM Metrics with processName=MAP, sessionId=
    2007-10-12 16:56:14,277 INFO org.apache.hadoop.mapred.MapTask:
    numReduceTasks: 8
    2007-10-12 16:56:14,299 WARN org.apache.hadoop.mapred.TaskTracker: Error
    running child
    java.lang.RuntimeException: java.lang.ClassNotFoundException:
    com.company.hadoop.metrics.testAggregator
    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.createInstance
    (UserDefinedValueAggregatorDescriptor.java:56)
    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.createAggregator(UserDefinedValueAggregatorDescriptor.java:63)
    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.(UserDefinedValueAggregatorDescriptor.java:75)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggr
    egatorDescriptor(ValueAggregatorJobBase.java:49)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregato
    rDescriptors(ValueAggregatorJobBase.java:60)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMy
    Spec(ValueAggregatorJobBase.java:69)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Va
    lueAggregatorJobBase.java:37)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82
    )
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:32)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82
    )
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:185)
    at
    org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1777)
    Caused by: java.lang.ClassNotFoundException:
    com.company.hadoop.metrics.testAggregator
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.createInstance
    (UserDefinedValueAggregatorDescriptor.java:51)
    ... 13 more

    As near as I can tell I am doing everything correctly. Other code
    existing in this jar file is
    running just fine - in fact it was used to produce the input to this
    test. I've checked my
    systems to make I've got the jar file (and XML file) onto each of the 4
    compute nodes in the
    system.

    Can anybody spot my problem or tell me what to do to get this going?

    Thanks,
    C G


    ---------------------------------
    Tonight's top picks. What will you watch tonight? Preview the hottest
    shows on Yahoo! TV.



    ---------------------------------
    Be a better Heartthrob. Get better relationship answers from someone who knows.
    Yahoo! Answers - Check it out.
  • Runping Qi at Oct 13, 2007 at 3:30 am
    I am sorry I overlooked something.
    Try to add the following:

    job.setJarByClass(MyValueAggregatorJob.class);
    JobClient.runJob(job);

    To your main.

    Runping
    -----Original Message-----
    From: C G
    Sent: Friday, October 12, 2007 8:05 PM
    To: hadoop-user@lucene.apache.org
    Subject: RE: Question about valueaggregators in 0.14.1...

    Hi Runping and All:

    I took your suggestion, renamed my original code to testAggregatorPlugin
    and wrote a class testAggregator which contained the main as you show
    here. It also fails with the same traceback...so it seems something is
    profoundly wrong here.

    Is anybody else using the aggregation classes in 0.14.1 with success?

    This is ironic and frustrating...I have several other programs which use
    map/reduce that are quite complex and those are running just fine. This
    little plugin is all that stands in the way of delivering the system :-(.

    Any other comments/help, etc. most welcome...

    Thanks,
    C G

    Runping Qi wrote:

    Hadoop ignored the jar file you specified.
    This is a known problem andhopefully HADOOP-1622
    (https://issues.apache.org/jira/browse/HADOOP-1622) will fix the problem.

    In the meantime, you can create a class like below and add it to your jar:

    package your_package;


    public class MyValueAggregatorJob{

    public static void main(String args[]) throws IOException {
    JobConf job = ValueAggregatorJob.createValueAggregatorJob(args);
    JobClient.runJob(job);
    }
    }


    And use the above class as the main class when you submit hadoop job:
    bin/hadoop jar hadoop-0.14.1-dev-company.jar
    your_package.MyValueAggregatorJob /input/set1 /output/set1 8
    textinputformat
    testAggregator.xml


    Hope this solve your problem.

    Runping



    -----Original Message-----
    From: C G
    Sent: Friday, October 12, 2007 2:43 PM
    To: hadoop-user@lucene.apache.org
    Subject: Question about valueaggregators in 0.14.1...

    Hi All:

    I am tearing out my hair trying to get a simple valueaggregator to run.
    This seems like an
    easy enough thing, but I am consistently getting hit with
    ClassNotFoundException which
    makes no sense to me. Any help would be immensely appreciated. Note
    that I've had
    valueaggregators running on 0.13.*. I'm a little concerned there is a bug
    in 0.14.1. On the
    other hand, maybe I'm just blind to something obvious and just need a
    boot to the head...

    The environment is hadoop-0.14.1. I am running Java 1.6:
    $ java -version
    java version "1.6.0_02"
    Java(TM) SE Runtime Environment (build 1.6.0_02-b05)
    Java HotSpot(TM) Server VM (build 1.6.0_02-b05, mixed mode)

    The class name is called testAggregator, and exists in my jar file:
    $ jar -tvf hadoop-0.14.1-dev-company.jar | grep testAggregator
    1826 Fri Oct 12 16:30:14 EDT 2007
    com/company/hadoop/metrics/testAggregator.class

    My XML file is testAggregator.xml and looks like:



    aggregator.descriptor.num
    1

    aggregator.descriptor.0
    UserDefined,com.company.hadoop.metrics.testAggregator

    My code couldn't be simpler:

    package com.company.hadoop.metrics;
    // snip a bunch of imports
    public class testAggregator extends ValueAggregatorBaseDescriptor {
    public ArrayList generateKeyValPairs(Object key, Object val) {
    String fields[] = val.toString().split("\t");
    ArrayList retv = new ArrayList();
    retv.add(generateEntry(LONG_VALUE_SUM, fields[ 6 ], ONE));
    retv.add(generateEntry(UNIQ_VALUE_COUNT, fields[ 6 ], new Text(
    fields[ 2 ])));
    return retv;
    }
    public void configure(JobConf job) {}
    }

    I invoke things this way:

    bin/hadoop jar hadoop-0.14.1-dev-company.jar
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJob /input/set1
    /output/set1 8 textinputformat testAggregator.xml

    All my jobs fail with a ClassNotFoundException like the following
    traceback:

    2007-10-12 16:56:14,227 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
    Initializing JVM Metrics with processName=MAP, sessionId=
    2007-10-12 16:56:14,277 INFO org.apache.hadoop.mapred.MapTask:
    numReduceTasks: 8
    2007-10-12 16:56:14,299 WARN org.apache.hadoop.mapred.TaskTracker: Error
    running child
    java.lang.RuntimeException: java.lang.ClassNotFoundException:
    com.company.hadoop.metrics.testAggregator
    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.createInstance
    (UserDefinedValueAggregatorDescriptor.java:56)
    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.createAggregator(UserDefinedValueAggregatorDescriptor.java:63)
    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.(UserDefinedValueAggregatorDescriptor.java:75)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggr
    egatorDescriptor(ValueAggregatorJobBase.java:49)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregato
    rDescriptors(ValueAggregatorJobBase.java:60)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMy
    Spec(ValueAggregatorJobBase.java:69)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Va
    lueAggregatorJobBase.java:37)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82
    )
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:32)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82
    )
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:185)
    at
    org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1777)
    Caused by: java.lang.ClassNotFoundException:
    com.company.hadoop.metrics.testAggregator
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.createInstance
    (UserDefinedValueAggregatorDescriptor.java:51)
    ... 13 more

    As near as I can tell I am doing everything correctly. Other code
    existing in this jar file is
    running just fine - in fact it was used to produce the input to this
    test. I've checked my
    systems to make I've got the jar file (and XML file) onto each of the 4
    compute nodes in the
    system.

    Can anybody spot my problem or tell me what to do to get this going?

    Thanks,
    C G


    ---------------------------------
    Tonight's top picks. What will you watch tonight? Preview the hottest
    shows on Yahoo! TV.



    ---------------------------------
    Be a better Heartthrob. Get better relationship answers from someone who
    knows.
    Yahoo! Answers - Check it out.
  • C G at Oct 13, 2007 at 3:50 am
    Hi Runping and All:

    That fixed the problem. Of course my aggregator is now failing for a different reason, but that's an error in my code that I can fix.

    I am extremely grateful for your assistance!

    Thanks,
    C G

    Runping Qi wrote:


    I am sorry I overlooked something.
    Try to add the following:

    job.setJarByClass(MyValueAggregatorJob.class);
    JobClient.runJob(job);

    To your main.

    Runping
    -----Original Message-----
    From: C G
    Sent: Friday, October 12, 2007 8:05 PM
    To: hadoop-user@lucene.apache.org
    Subject: RE: Question about valueaggregators in 0.14.1...

    Hi Runping and All:

    I took your suggestion, renamed my original code to testAggregatorPlugin
    and wrote a class testAggregator which contained the main as you show
    here. It also fails with the same traceback...so it seems something is
    profoundly wrong here.

    Is anybody else using the aggregation classes in 0.14.1 with success?

    This is ironic and frustrating...I have several other programs which use
    map/reduce that are quite complex and those are running just fine. This
    little plugin is all that stands in the way of delivering the system :-(.

    Any other comments/help, etc. most welcome...

    Thanks,
    C G

    Runping Qi wrote:

    Hadoop ignored the jar file you specified.
    This is a known problem andhopefully HADOOP-1622
    (https://issues.apache.org/jira/browse/HADOOP-1622) will fix the problem.

    In the meantime, you can create a class like below and add it to your jar:

    package your_package;


    public class MyValueAggregatorJob{

    public static void main(String args[]) throws IOException {
    JobConf job = ValueAggregatorJob.createValueAggregatorJob(args);
    JobClient.runJob(job);
    }
    }


    And use the above class as the main class when you submit hadoop job:
    bin/hadoop jar hadoop-0.14.1-dev-company.jar
    your_package.MyValueAggregatorJob /input/set1 /output/set1 8
    textinputformat
    testAggregator.xml


    Hope this solve your problem.

    Runping



    -----Original Message-----
    From: C G
    Sent: Friday, October 12, 2007 2:43 PM
    To: hadoop-user@lucene.apache.org
    Subject: Question about valueaggregators in 0.14.1...

    Hi All:

    I am tearing out my hair trying to get a simple valueaggregator to run.
    This seems like an
    easy enough thing, but I am consistently getting hit with
    ClassNotFoundException which
    makes no sense to me. Any help would be immensely appreciated. Note
    that I've had
    valueaggregators running on 0.13.*. I'm a little concerned there is a bug
    in 0.14.1. On the
    other hand, maybe I'm just blind to something obvious and just need a
    boot to the head...

    The environment is hadoop-0.14.1. I am running Java 1.6:
    $ java -version
    java version "1.6.0_02"
    Java(TM) SE Runtime Environment (build 1.6.0_02-b05)
    Java HotSpot(TM) Server VM (build 1.6.0_02-b05, mixed mode)

    The class name is called testAggregator, and exists in my jar file:
    $ jar -tvf hadoop-0.14.1-dev-company.jar | grep testAggregator
    1826 Fri Oct 12 16:30:14 EDT 2007
    com/company/hadoop/metrics/testAggregator.class

    My XML file is testAggregator.xml and looks like:



    aggregator.descriptor.num
    1

    aggregator.descriptor.0
    UserDefined,com.company.hadoop.metrics.testAggregator

    My code couldn't be simpler:

    package com.company.hadoop.metrics;
    // snip a bunch of imports
    public class testAggregator extends ValueAggregatorBaseDescriptor {
    public ArrayList generateKeyValPairs(Object key, Object val) {
    String fields[] = val.toString().split("\t");
    ArrayList retv = new ArrayList();
    retv.add(generateEntry(LONG_VALUE_SUM, fields[ 6 ], ONE));
    retv.add(generateEntry(UNIQ_VALUE_COUNT, fields[ 6 ], new Text(
    fields[ 2 ])));
    return retv;
    }
    public void configure(JobConf job) {}
    }

    I invoke things this way:

    bin/hadoop jar hadoop-0.14.1-dev-company.jar
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJob /input/set1
    /output/set1 8 textinputformat testAggregator.xml

    All my jobs fail with a ClassNotFoundException like the following
    traceback:

    2007-10-12 16:56:14,227 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
    Initializing JVM Metrics with processName=MAP, sessionId=
    2007-10-12 16:56:14,277 INFO org.apache.hadoop.mapred.MapTask:
    numReduceTasks: 8
    2007-10-12 16:56:14,299 WARN org.apache.hadoop.mapred.TaskTracker: Error
    running child
    java.lang.RuntimeException: java.lang.ClassNotFoundException:
    com.company.hadoop.metrics.testAggregator
    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.createInstance
    (UserDefinedValueAggregatorDescriptor.java:56)
    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.createAggregator(UserDefinedValueAggregatorDescriptor.java:63)
    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.(UserDefinedValueAggregatorDescriptor.java:75)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggr
    egatorDescriptor(ValueAggregatorJobBase.java:49)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregato
    rDescriptors(ValueAggregatorJobBase.java:60)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMy
    Spec(ValueAggregatorJobBase.java:69)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Va
    lueAggregatorJobBase.java:37)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82
    )
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:32)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82
    )
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:185)
    at
    org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1777)
    Caused by: java.lang.ClassNotFoundException:
    com.company.hadoop.metrics.testAggregator
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.createInstance
    (UserDefinedValueAggregatorDescriptor.java:51)
    ... 13 more

    As near as I can tell I am doing everything correctly. Other code
    existing in this jar file is
    running just fine - in fact it was used to produce the input to this
    test. I've checked my
    systems to make I've got the jar file (and XML file) onto each of the 4
    compute nodes in the
    system.

    Can anybody spot my problem or tell me what to do to get this going?

    Thanks,
    C G


    ---------------------------------
    Tonight's top picks. What will you watch tonight? Preview the hottest
    shows on Yahoo! TV.



    ---------------------------------
    Be a better Heartthrob. Get better relationship answers from someone who
    knows.
    Yahoo! Answers - Check it out.



    ---------------------------------
    Catch up on fall's hot new shows on Yahoo! TV. Watch previews, get listings, and more!
  • Runping Qi at Oct 13, 2007 at 4:02 am
    Glad it works.

    HADOOP-1622 should fix the problem properly.
    Until then, the users have to use this kind of hacky obscure workaround:)

    Runping
    -----Original Message-----
    From: C G
    Sent: Friday, October 12, 2007 8:48 PM
    To: hadoop-user@lucene.apache.org
    Subject: RE: Question about valueaggregators in 0.14.1...

    Hi Runping and All:

    That fixed the problem. Of course my aggregator is now failing for a
    different reason, but that's an error in my code that I can fix.

    I am extremely grateful for your assistance!

    Thanks,
    C G

    Runping Qi wrote:


    I am sorry I overlooked something.
    Try to add the following:

    job.setJarByClass(MyValueAggregatorJob.class);
    JobClient.runJob(job);

    To your main.

    Runping
    -----Original Message-----
    From: C G
    Sent: Friday, October 12, 2007 8:05 PM
    To: hadoop-user@lucene.apache.org
    Subject: RE: Question about valueaggregators in 0.14.1...

    Hi Runping and All:

    I took your suggestion, renamed my original code to testAggregatorPlugin
    and wrote a class testAggregator which contained the main as you show
    here. It also fails with the same traceback...so it seems something is
    profoundly wrong here.

    Is anybody else using the aggregation classes in 0.14.1 with success?

    This is ironic and frustrating...I have several other programs which use
    map/reduce that are quite complex and those are running just fine. This
    little plugin is all that stands in the way of delivering the system :- (.
    Any other comments/help, etc. most welcome...

    Thanks,
    C G

    Runping Qi wrote:

    Hadoop ignored the jar file you specified.
    This is a known problem andhopefully HADOOP-1622
    (https://issues.apache.org/jira/browse/HADOOP-1622) will fix the problem.
    In the meantime, you can create a class like below and add it to your jar:
    package your_package;


    public class MyValueAggregatorJob{

    public static void main(String args[]) throws IOException {
    JobConf job = ValueAggregatorJob.createValueAggregatorJob(args);
    JobClient.runJob(job);
    }
    }


    And use the above class as the main class when you submit hadoop job:
    bin/hadoop jar hadoop-0.14.1-dev-company.jar
    your_package.MyValueAggregatorJob /input/set1 /output/set1 8
    textinputformat
    testAggregator.xml


    Hope this solve your problem.

    Runping



    -----Original Message-----
    From: C G
    Sent: Friday, October 12, 2007 2:43 PM
    To: hadoop-user@lucene.apache.org
    Subject: Question about valueaggregators in 0.14.1...

    Hi All:

    I am tearing out my hair trying to get a simple valueaggregator to
    run.
    This seems like an
    easy enough thing, but I am consistently getting hit with
    ClassNotFoundException which
    makes no sense to me. Any help would be immensely appreciated. Note
    that I've had
    valueaggregators running on 0.13.*. I'm a little concerned there is a bug
    in 0.14.1. On the
    other hand, maybe I'm just blind to something obvious and just need a
    boot to the head...

    The environment is hadoop-0.14.1. I am running Java 1.6:
    $ java -version
    java version "1.6.0_02"
    Java(TM) SE Runtime Environment (build 1.6.0_02-b05)
    Java HotSpot(TM) Server VM (build 1.6.0_02-b05, mixed mode)

    The class name is called testAggregator, and exists in my jar file:
    $ jar -tvf hadoop-0.14.1-dev-company.jar | grep testAggregator
    1826 Fri Oct 12 16:30:14 EDT 2007
    com/company/hadoop/metrics/testAggregator.class

    My XML file is testAggregator.xml and looks like:



    aggregator.descriptor.num
    1

    aggregator.descriptor.0
    UserDefined,com.company.hadoop.metrics.testAggregator

    My code couldn't be simpler:

    package com.company.hadoop.metrics;
    // snip a bunch of imports
    public class testAggregator extends ValueAggregatorBaseDescriptor {
    public ArrayList generateKeyValPairs(Object key, Object val) {
    String fields[] = val.toString().split("\t");
    ArrayList retv = new ArrayList();
    retv.add(generateEntry(LONG_VALUE_SUM, fields[ 6 ], ONE));
    retv.add(generateEntry(UNIQ_VALUE_COUNT, fields[ 6 ], new Text(
    fields[ 2 ])));
    return retv;
    }
    public void configure(JobConf job) {}
    }

    I invoke things this way:

    bin/hadoop jar hadoop-0.14.1-dev-company.jar
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJob /input/set1
    /output/set1 8 textinputformat testAggregator.xml

    All my jobs fail with a ClassNotFoundException like the following
    traceback:

    2007-10-12 16:56:14,227 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
    Initializing JVM Metrics with processName=MAP, sessionId=
    2007-10-12 16:56:14,277 INFO org.apache.hadoop.mapred.MapTask:
    numReduceTasks: 8
    2007-10-12 16:56:14,299 WARN org.apache.hadoop.mapred.TaskTracker:
    Error
    running child
    java.lang.RuntimeException: java.lang.ClassNotFoundException:
    com.company.hadoop.metrics.testAggregator
    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.createInstance
    (UserDefinedValueAggregatorDescriptor.java:56)
    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.createAggregator(UserDefinedValueAggregatorDescriptor.java:63)
    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.(UserDefinedValueAggregatorDescriptor.java:75)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggr
    egatorDescriptor(ValueAggregatorJobBase.java:49)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregato
    rDescriptors(ValueAggregatorJobBase.java:60)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMy
    Spec(ValueAggregatorJobBase.java:69)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Va
    lueAggregatorJobBase.java:37)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82
    )
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:32)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82
    )
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:185)
    at
    org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1777)
    Caused by: java.lang.ClassNotFoundException:
    com.company.hadoop.metrics.testAggregator
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescripto
    r.createInstance
    (UserDefinedValueAggregatorDescriptor.java:51)
    ... 13 more

    As near as I can tell I am doing everything correctly. Other code
    existing in this jar file is
    running just fine - in fact it was used to produce the input to this
    test. I've checked my
    systems to make I've got the jar file (and XML file) onto each of the
    4
    compute nodes in the
    system.

    Can anybody spot my problem or tell me what to do to get this going?

    Thanks,
    C G


    ---------------------------------
    Tonight's top picks. What will you watch tonight? Preview the hottest
    shows on Yahoo! TV.



    ---------------------------------
    Be a better Heartthrob. Get better relationship answers from someone who
    knows.
    Yahoo! Answers - Check it out.



    ---------------------------------
    Catch up on fall's hot new shows on Yahoo! TV. Watch previews, get
    listings, and more!

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedOct 12, '07 at 9:43p
activeOct 13, '07 at 4:02a
posts7
users3
websitehadoop.apache.org...
irc#hadoop

3 users in discussion

C G: 3 posts Runping Qi: 3 posts Dennis Kubes: 1 post

People

Translate

site design / logo © 2022 Grokbase