FAQ
Hi Group,



I have trouble running couple of examples provided by Hadoop. Below are the
error messages I have from the console, could you please advise what could
be the problem and probable solution?



09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
: 1

09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
: 1

09/06/17 16:30:29 INFO mapred.JobClient: Running job: job_200906171601_0009

09/06/17 16:30:30 INFO mapred.JobClient: map 0% reduce 0%

09/06/17 16:30:38 INFO mapred.JobClient: Task Id :
attempt_200906171601_0009_m_000000_0, Status : FAILED

java.io.IOException: hdfs://localhost:9000/user/root/words not a
SequenceFile

at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1458)

at
org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1420)

at
org.apache.hadoop.io.SequenceFile$Reader.(SequenceFileRecordR
eader.java:43)

at
org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFil
eInputFormat.java:54)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)

at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)



09/06/17 16:30:39 INFO mapred.JobClient: Task Id :
attempt_200906171601_0009_r_000000_0, Status : FAILED

java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass

at
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
createInstance(UserDefinedValueAggregatorDescriptor.java:57)

at
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
createAggregator(UserDefinedValueAggregatorDescriptor.java:64)

at
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
<init>(UserDefinedValueAggregatorDescriptor.java:76)

at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggreg
atorDescriptor(ValueAggregatorJobBase.java:54)

at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorD
escriptors(ValueAggregatorJobBase.java:65)

at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySp
ec(ValueAggregatorJobBase.java:74)

at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Valu
eAggregatorJobBase.java:42)

at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)

at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)

at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:240)

at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)

Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass

at java.net.URLClassLoader$1.run(URLClassLoader.java:200)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:188)

at java.lang.ClassLoader.loadClass(ClassLoader.java:306)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)

at java.lang.ClassLoader.loadClass(ClassLoader.java:251)

at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)

at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Class.java:242)

at
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
createInstance(UserDefinedValueAggregatorDescriptor.java:52)

... 10 more



Thank You,

Shravan Kumar. M

Catalytic Software Ltd. [SEI-CMMI Level 5 Company]

-----------------------------

This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed.
If you have received this email in error please notify the system
administrator -
netopshelpdesk@catalytic.com

Search Discussions

  • Nick Cen at Jun 17, 2009 at 1:20 pm
    I guest you have set SequenceFileFormat as your inputformat in the
    configuration object, but the file you provide is not a sequence file.

    2009/6/17 Shravan Mahankali <shravan.mahankali@catalytic.com>
    Hi Group,



    I have trouble running couple of examples provided by Hadoop. Below are the
    error messages I have from the console, could you please advise what could
    be the problem and probable solution?



    09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
    : 1

    09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
    : 1

    09/06/17 16:30:29 INFO mapred.JobClient: Running job: job_200906171601_0009

    09/06/17 16:30:30 INFO mapred.JobClient: map 0% reduce 0%

    09/06/17 16:30:38 INFO mapred.JobClient: Task Id :
    attempt_200906171601_0009_m_000000_0, Status : FAILED

    java.io.IOException: hdfs://localhost:9000/user/root/words not a
    SequenceFile

    at
    org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1458)

    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)

    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1420)

    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1415)

    at

    org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordR
    eader.java:43)

    at

    org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFil
    eInputFormat.java:54)

    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)

    at
    org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)



    09/06/17 16:30:39 INFO mapred.JobClient: Task Id :
    attempt_200906171601_0009_r_000000_0, Status : FAILED

    java.lang.RuntimeException: java.lang.ClassNotFoundException:
    org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass

    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createInstance(UserDefinedValueAggregatorDescriptor.java:57)

    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createAggregator(UserDefinedValueAggregatorDescriptor.java:64)

    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    <init>(UserDefinedValueAggregatorDescriptor.java:76)

    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggreg
    atorDescriptor(ValueAggregatorJobBase.java:54)

    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorD
    escriptors(ValueAggregatorJobBase.java:65)

    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySp
    ec(ValueAggregatorJobBase.java:74)

    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Valu
    eAggregatorJobBase.java:42)

    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)

    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)

    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:240)

    at
    org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)

    Caused by: java.lang.ClassNotFoundException:
    org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass

    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)

    at java.security.AccessController.doPrivileged(Native Method)

    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)

    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)

    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)

    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)

    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)

    at java.lang.Class.forName0(Native Method)

    at java.lang.Class.forName(Class.java:242)

    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createInstance(UserDefinedValueAggregatorDescriptor.java:52)

    ... 10 more



    Thank You,

    Shravan Kumar. M

    Catalytic Software Ltd. [SEI-CMMI Level 5 Company]

    -----------------------------

    This email and any files transmitted with it are confidential and intended
    solely for the use of the individual or entity to whom they are addressed.
    If you have received this email in error please notify the system
    administrator - netopshelpdesk@catalytic.com





    --
    http://daily.appspot.com/food/
  • Shravan Mahankali at Jun 18, 2009 at 5:02 am
    Hi Nick,



    Thanks for your response.



    I am very new to Hadoop. I was trying to execute the "AggregateWordCount"
    example program provided by Hadoop in the distribution from the linux but
    was having the issue as stated in my earlier email.



    I have also tried executing the MultiFetch example with no success.



    If this example program needs the input file to be a sequence file. how
    should I create one, please advice?



    Thank You,

    Shravan Kumar. M

    Catalytic Software Ltd. [SEI-CMMI Level 5 Company]

    -----------------------------

    This email and any files transmitted with it are confidential and intended
    solely for the use of the individual or entity to whom they are addressed.
    If you have received this email in error please notify the system
    administrator -
    netopshelpdesk@catalytic.com

    _____

    From: Nick Cen
    Sent: Wednesday, June 17, 2009 6:50 PM
    To: core-user@hadoop.apache.org; shravan.mahankali@catalytic.com
    Subject: Re: not a SequenceFile?



    I guest you have set SequenceFileFormat as your inputformat in the
    configuration object, but the file you provide is not a sequence file.

    2009/6/17 Shravan Mahankali <shravan.mahankali@catalytic.com>

    Hi Group,



    I have trouble running couple of examples provided by Hadoop. Below are the
    error messages I have from the console, could you please advise what could
    be the problem and probable solution?



    09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
    : 1

    09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
    : 1

    09/06/17 16:30:29 INFO mapred.JobClient: Running job: job_200906171601_0009

    09/06/17 16:30:30 INFO mapred.JobClient: map 0% reduce 0%

    09/06/17 16:30:38 INFO mapred.JobClient: Task Id :
    attempt_200906171601_0009_m_000000_0, Status : FAILED

    java.io.IOException: hdfs://localhost:9000/user/root/words not a
    SequenceFile

    at
    org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1458)

    at
    org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1420)

    at
    org.apache.hadoop.io.SequenceFile$Reader.(SequenceFileRecordR
    eader.java:43)

    at
    org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFil
    eInputFormat.java:54)

    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)

    at
    org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)



    09/06/17 16:30:39 INFO mapred.JobClient: Task Id :
    attempt_200906171601_0009_r_000000_0, Status : FAILED

    java.lang.RuntimeException: java.lang.ClassNotFoundException:
    org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass

    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createInstance(UserDefinedValueAggregatorDescriptor.java:57)

    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createAggregator(UserDefinedValueAggregatorDescriptor.java:64)

    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    <init>(UserDefinedValueAggregatorDescriptor.java:76)

    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggreg
    atorDescriptor(ValueAggregatorJobBase.java:54)

    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorD
    escriptors(ValueAggregatorJobBase.java:65)

    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySp
    ec(ValueAggregatorJobBase.java:74)

    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Valu
    eAggregatorJobBase.java:42)

    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)

    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)

    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:240)

    at
    org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)

    Caused by: java.lang.ClassNotFoundException:
    org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass

    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)

    at java.security.AccessController.doPrivileged(Native Method)

    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)

    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)

    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)

    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)

    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)

    at java.lang.Class.forName0(Native Method)

    at java.lang.Class.forName(Class.java:242)

    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createInstance(UserDefinedValueAggregatorDescriptor.java:52)

    ... 10 more



    Thank You,

    Shravan Kumar. M

    Catalytic Software Ltd. [SEI-CMMI Level 5 Company]

    -----------------------------

    This email and any files transmitted with it are confidential and intended
    solely for the use of the individual or entity to whom they are addressed.
    If you have received this email in error please notify the system
    administrator -
    netopshelpdesk@catalytic.com
  • Eason.Lee at Jun 18, 2009 at 5:57 am
    you'd better run it like this:

    bin/hadoop jar hadoop-*-examples.jar aggregatewordcount input output
    numOfReducers *textinputformat*
    *
    *
    input textinputformat to use textinputformat instand of SequenceFile

    hope it is helpful!

    2009/6/18 Shravan Mahankali <shravan.mahankali@catalytic.com>
    Hi Nick,



    Thanks for your response.



    I am very new to Hadoop. I was trying to execute the "AggregateWordCount"
    example program provided by Hadoop in the distribution from the linux but
    was having the issue as stated in my earlier email.



    I have also tried executing the MultiFetch example with no success.



    If this example program needs the input file to be a sequence file. how
    should I create one, please advice?



    Thank You,

    Shravan Kumar. M

    Catalytic Software Ltd. [SEI-CMMI Level 5 Company]

    -----------------------------

    This email and any files transmitted with it are confidential and intended
    solely for the use of the individual or entity to whom they are addressed.
    If you have received this email in error please notify the system
    administrator - netopshelpdesk@catalytic.com

    _____

    From: Nick Cen
    Sent: Wednesday, June 17, 2009 6:50 PM
    To: core-user@hadoop.apache.org; shravan.mahankali@catalytic.com
    Subject: Re: not a SequenceFile?



    I guest you have set SequenceFileFormat as your inputformat in the
    configuration object, but the file you provide is not a sequence file.

    2009/6/17 Shravan Mahankali <shravan.mahankali@catalytic.com>

    Hi Group,



    I have trouble running couple of examples provided by Hadoop. Below are the
    error messages I have from the console, could you please advise what could
    be the problem and probable solution?



    09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
    : 1

    09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
    : 1

    09/06/17 16:30:29 INFO mapred.JobClient: Running job: job_200906171601_0009

    09/06/17 16:30:30 INFO mapred.JobClient: map 0% reduce 0%

    09/06/17 16:30:38 INFO mapred.JobClient: Task Id :
    attempt_200906171601_0009_m_000000_0, Status : FAILED

    java.io.IOException: hdfs://localhost:9000/user/root/words not a
    SequenceFile

    at
    org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1458)

    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)

    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1420)

    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1415)

    at

    org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordR
    eader.java:43)

    at

    org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFil
    eInputFormat.java:54)

    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)

    at
    org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)



    09/06/17 16:30:39 INFO mapred.JobClient: Task Id :
    attempt_200906171601_0009_r_000000_0, Status : FAILED

    java.lang.RuntimeException: java.lang.ClassNotFoundException:
    org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass

    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createInstance(UserDefinedValueAggregatorDescriptor.java:57)

    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createAggregator(UserDefinedValueAggregatorDescriptor.java:64)

    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    <init>(UserDefinedValueAggregatorDescriptor.java:76)

    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggreg
    atorDescriptor(ValueAggregatorJobBase.java:54)

    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorD
    escriptors(ValueAggregatorJobBase.java:65)

    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySp
    ec(ValueAggregatorJobBase.java:74)

    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Valu
    eAggregatorJobBase.java:42)

    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)

    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)

    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:240)

    at
    org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)

    Caused by: java.lang.ClassNotFoundException:
    org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass

    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)

    at java.security.AccessController.doPrivileged(Native Method)

    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)

    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)

    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)

    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)

    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)

    at java.lang.Class.forName0(Native Method)

    at java.lang.Class.forName(Class.java:242)

    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createInstance(UserDefinedValueAggregatorDescriptor.java:52)

    ... 10 more



    Thank You,

    Shravan Kumar. M

    Catalytic Software Ltd. [SEI-CMMI Level 5 Company]

    -----------------------------

    This email and any files transmitted with it are confidential and intended
    solely for the use of the individual or entity to whom they are addressed.
    If you have received this email in error please notify the system
    administrator - netopshelpdesk@catalytic.com









    --
    http://daily.appspot.com/food/
  • Shravan Mahankali at Jun 18, 2009 at 6:33 am
    Thanks for your suggestion Eason.

    I think sequence file issue is resolved, how ever success is far on the hill
    top!!!

    *Below is the command I have executed in the terminal:*
    #/bin/hadoop jar AggregateWordCount.jar
    org.apache.hadoop.examples.AggregateWordCount words/*
    aggregatewordcount_output 2 textinputformat

    *Error:*

    09/06/18 11:51:42 INFO mapred.JobClient: Task Id :
    attempt_200906181145_0005_r_000001_1, Status : FAILED
    java.lang.RuntimeException: java.lang.ClassNotFoundException:
    org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createInstance(UserDefinedValueAggregatorDescriptor.java:57)
    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createAggregator(UserDefinedValueAggregatorDescriptor.java:64)
    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    <init>(UserDefinedValueAggregatorDescriptor.java:76)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggreg
    atorDescriptor(ValueAggregatorJobBase.java:54)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorD
    escriptors(ValueAggregatorJobBase.java:65)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySp
    ec(ValueAggregatorJobBase.java:74)
    at
    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Valu
    eAggregatorJobBase.java:42)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:240)
    at
    org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
    Caused by: java.lang.ClassNotFoundException:
    org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:242)
    at
    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createInstance(UserDefinedValueAggregatorDescriptor.java:52)
    ... 10 more


    *However, I could see that
    org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass class is
    available in the AggregateWordCount.jar, below is the listing of this jar: *

    # jar -tvf AggregateWordCount.jar
    0 Tue Jun 09 18:11:42 IST 2009 META-INF/
    71 Tue Jun 09 18:11:42 IST 2009 META-INF/MANIFEST.MF
    0 Tue Jun 09 18:10:58 IST 2009 org/
    0 Tue Jun 09 18:10:58 IST 2009 org/apache/
    0 Tue Jun 09 18:10:58 IST 2009 org/apache/hadoop/
    0 Tue Jun 09 18:10:58 IST 2009 org/apache/hadoop/examples/
    1298 Tue Jun 09 18:11:32 IST 2009
    org/apache/hadoop/examples/AggregateWordCount$WordCountPlugInClass.class
    846 Tue Jun 09 18:11:32 IST 2009
    org/apache/hadoop/examples/AggregateWordCount.class


    Could you please advice what could be the problem here?

    Thank You,
    Shravan Kumar. M
    Catalytic Software Ltd. [SEI-CMMI Level 5 Company]
    -----------------------------
    This email and any files transmitted with it are confidential and intended
    solely for the use of the individual or entity to whom they are addressed.
    If you have received this email in error please notify the system
    administrator - netopshelpdesk@catalytic.com
    -----Original Message-----
    From: Eason.Lee
    Sent: Thursday, June 18, 2009 11:27 AM
    To: core-user@hadoop.apache.org; shravan.mahankali@catalytic.com
    Subject: Re: not a SequenceFile?

    you'd better run it like this:

    bin/hadoop jar hadoop-*-examples.jar aggregatewordcount input output
    numOfReducers *textinputformat*
    *
    *
    input textinputformat to use textinputformat instand of SequenceFile

    hope it is helpful!

    2009/6/18 Shravan Mahankali <shravan.mahankali@catalytic.com>
    Hi Nick,



    Thanks for your response.



    I am very new to Hadoop. I was trying to execute the "AggregateWordCount"
    example program provided by Hadoop in the distribution from the linux but
    was having the issue as stated in my earlier email.



    I have also tried executing the MultiFetch example with no success.



    If this example program needs the input file to be a sequence file. how
    should I create one, please advice?



    Thank You,

    Shravan Kumar. M

    Catalytic Software Ltd. [SEI-CMMI Level 5 Company]

    -----------------------------

    This email and any files transmitted with it are confidential and intended
    solely for the use of the individual or entity to whom they are addressed.
    If you have received this email in error please notify the system
    administrator - netopshelpdesk@catalytic.com

    _____

    From: Nick Cen
    Sent: Wednesday, June 17, 2009 6:50 PM
    To: core-user@hadoop.apache.org; shravan.mahankali@catalytic.com
    Subject: Re: not a SequenceFile?



    I guest you have set SequenceFileFormat as your inputformat in the
    configuration object, but the file you provide is not a sequence file.

    2009/6/17 Shravan Mahankali <shravan.mahankali@catalytic.com>

    Hi Group,



    I have trouble running couple of examples provided by Hadoop. Below are the
    error messages I have from the console, could you please advise what could
    be the problem and probable solution?



    09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
    : 1

    09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
    : 1

    09/06/17 16:30:29 INFO mapred.JobClient: Running job:
    job_200906171601_0009
    09/06/17 16:30:30 INFO mapred.JobClient: map 0% reduce 0%

    09/06/17 16:30:38 INFO mapred.JobClient: Task Id :
    attempt_200906171601_0009_m_000000_0, Status : FAILED

    java.io.IOException: hdfs://localhost:9000/user/root/words not a
    SequenceFile

    at
    org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1458)

    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)

    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1420)

    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1415)

    at

    org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordR
    eader.java:43)

    at

    org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFil
    eInputFormat.java:54)

    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)

    at
    org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)



    09/06/17 16:30:39 INFO mapred.JobClient: Task Id :
    attempt_200906171601_0009_r_000000_0, Status : FAILED

    java.lang.RuntimeException: java.lang.ClassNotFoundException:
    org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass

    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createInstance(UserDefinedValueAggregatorDescriptor.java:57)

    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createAggregator(UserDefinedValueAggregatorDescriptor.java:64)

    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    <init>(UserDefinedValueAggregatorDescriptor.java:76)

    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggreg
    atorDescriptor(ValueAggregatorJobBase.java:54)

    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorD
    escriptors(ValueAggregatorJobBase.java:65)

    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySp
    ec(ValueAggregatorJobBase.java:74)

    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Valu
    eAggregatorJobBase.java:42)

    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)

    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:240)

    at
    org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)

    Caused by: java.lang.ClassNotFoundException:
    org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass

    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)

    at java.security.AccessController.doPrivileged(Native Method)

    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)

    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)

    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)

    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)

    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)

    at java.lang.Class.forName0(Native Method)

    at java.lang.Class.forName(Class.java:242)

    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createInstance(UserDefinedValueAggregatorDescriptor.java:52)

    ... 10 more



    Thank You,

    Shravan Kumar. M

    Catalytic Software Ltd. [SEI-CMMI Level 5 Company]

    -----------------------------

    This email and any files transmitted with it are confidential and intended
    solely for the use of the individual or entity to whom they are addressed.
    If you have received this email in error please notify the system
    administrator - netopshelpdesk@catalytic.com









    --
    http://daily.appspot.com/food/
  • Eason.Lee at Jun 18, 2009 at 7:10 am
    http://www.nabble.com/ClassNotFoundException-td23441528.html<http://www.nabble.com/ClassNotFoundException-td23441528.html>
    <http://www.nabble.com/ClassNotFoundException-td23441528.html>You must make
    all of the required jars available to all of your tasks. You
    can either install them all the tasktracker machines and setup the
    tasktracker classpath to include them, or distributed them via the
    distributed cache.


    2009/6/18 Shravan Mahankali <shravan.mahankali@catalytic.com>
    Thanks for your suggestion Eason.

    I think sequence file issue is resolved, how ever success is far on the
    hill
    top!!!

    *Below is the command I have executed in the terminal:*
    #/bin/hadoop jar AggregateWordCount.jar
    org.apache.hadoop.examples.AggregateWordCount words/*
    aggregatewordcount_output 2 textinputformat

    *Error:*

    09/06/18 11:51:42 INFO mapred.JobClient: Task Id :
    attempt_200906181145_0005_r_000001_1, Status : FAILED
    java.lang.RuntimeException: java.lang.ClassNotFoundException:
    org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createInstance(UserDefinedValueAggregatorDescriptor.java:57)
    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createAggregator(UserDefinedValueAggregatorDescriptor.java:64)
    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    <init>(UserDefinedValueAggregatorDescriptor.java:76)
    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggreg
    atorDescriptor(ValueAggregatorJobBase.java:54)
    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorD
    escriptors(ValueAggregatorJobBase.java:65)
    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySp
    ec(ValueAggregatorJobBase.java:74)
    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Valu
    eAggregatorJobBase.java:42)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:240)
    at
    org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
    Caused by: java.lang.ClassNotFoundException:
    org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:242)
    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createInstance(UserDefinedValueAggregatorDescriptor.java:52)
    ... 10 more


    *However, I could see that
    org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass class is
    available in the AggregateWordCount.jar, below is the listing of this jar:
    *

    # jar -tvf AggregateWordCount.jar
    0 Tue Jun 09 18:11:42 IST 2009 META-INF/
    71 Tue Jun 09 18:11:42 IST 2009 META-INF/MANIFEST.MF
    0 Tue Jun 09 18:10:58 IST 2009 org/
    0 Tue Jun 09 18:10:58 IST 2009 org/apache/
    0 Tue Jun 09 18:10:58 IST 2009 org/apache/hadoop/
    0 Tue Jun 09 18:10:58 IST 2009 org/apache/hadoop/examples/
    1298 Tue Jun 09 18:11:32 IST 2009
    org/apache/hadoop/examples/AggregateWordCount$WordCountPlugInClass.class
    846 Tue Jun 09 18:11:32 IST 2009
    org/apache/hadoop/examples/AggregateWordCount.class


    Could you please advice what could be the problem here?

    Thank You,
    Shravan Kumar. M
    Catalytic Software Ltd. [SEI-CMMI Level 5 Company]
    -----------------------------
    This email and any files transmitted with it are confidential and intended
    solely for the use of the individual or entity to whom they are addressed.
    If you have received this email in error please notify the system
    administrator - netopshelpdesk@catalytic.com
    -----Original Message-----
    From: Eason.Lee
    Sent: Thursday, June 18, 2009 11:27 AM
    To: core-user@hadoop.apache.org; shravan.mahankali@catalytic.com
    Subject: Re: not a SequenceFile?

    you'd better run it like this:

    bin/hadoop jar hadoop-*-examples.jar aggregatewordcount input output
    numOfReducers *textinputformat*
    *
    *
    input textinputformat to use textinputformat instand of SequenceFile

    hope it is helpful!

    2009/6/18 Shravan Mahankali <shravan.mahankali@catalytic.com>
    Hi Nick,



    Thanks for your response.



    I am very new to Hadoop. I was trying to execute the "AggregateWordCount"
    example program provided by Hadoop in the distribution from the linux but
    was having the issue as stated in my earlier email.



    I have also tried executing the MultiFetch example with no success.



    If this example program needs the input file to be a sequence file. how
    should I create one, please advice?



    Thank You,

    Shravan Kumar. M

    Catalytic Software Ltd. [SEI-CMMI Level 5 Company]

    -----------------------------

    This email and any files transmitted with it are confidential and intended
    solely for the use of the individual or entity to whom they are
    addressed.
    If you have received this email in error please notify the system
    administrator - > netopshelpdesk@catalytic.com

    _____

    From: Nick Cen
    Sent: Wednesday, June 17, 2009 6:50 PM
    To: core-user@hadoop.apache.org; shravan.mahankali@catalytic.com
    Subject: Re: not a SequenceFile?



    I guest you have set SequenceFileFormat as your inputformat in the
    configuration object, but the file you provide is not a sequence file.

    2009/6/17 Shravan Mahankali <shravan.mahankali@catalytic.com>

    Hi Group,



    I have trouble running couple of examples provided by Hadoop. Below are the
    error messages I have from the console, could you please advise what could
    be the problem and probable solution?



    09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
    : 1

    09/06/17 16:30:29 INFO mapred.FileInputFormat: Total input paths to process
    : 1

    09/06/17 16:30:29 INFO mapred.JobClient: Running job:
    job_200906171601_0009
    09/06/17 16:30:30 INFO mapred.JobClient: map 0% reduce 0%

    09/06/17 16:30:38 INFO mapred.JobClient: Task Id :
    attempt_200906171601_0009_m_000000_0, Status : FAILED

    java.io.IOException: hdfs://localhost:9000/user/root/words not a
    SequenceFile

    at
    org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1458)

    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)

    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1420)

    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1415)

    at

    org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordR
    eader.java:43)

    at

    org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFil
    eInputFormat.java:54)

    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)

    at
    org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)



    09/06/17 16:30:39 INFO mapred.JobClient: Task Id :
    attempt_200906171601_0009_r_000000_0, Status : FAILED

    java.lang.RuntimeException: java.lang.ClassNotFoundException:
    org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass

    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createInstance(UserDefinedValueAggregatorDescriptor.java:57)

    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createAggregator(UserDefinedValueAggregatorDescriptor.java:64)

    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    <init>(UserDefinedValueAggregatorDescriptor.java:76)

    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggreg
    atorDescriptor(ValueAggregatorJobBase.java:54)

    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorD
    escriptors(ValueAggregatorJobBase.java:65)

    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySp
    ec(ValueAggregatorJobBase.java:74)

    at

    org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(Valu
    eAggregatorJobBase.java:42)

    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)

    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:240)

    at
    org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)

    Caused by: java.lang.ClassNotFoundException:
    org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass

    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)

    at java.security.AccessController.doPrivileged(Native Method)

    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)

    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)

    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)

    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)

    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)

    at java.lang.Class.forName0(Native Method)

    at java.lang.Class.forName(Class.java:242)

    at

    org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.
    createInstance(UserDefinedValueAggregatorDescriptor.java:52)

    ... 10 more



    Thank You,

    Shravan Kumar. M

    Catalytic Software Ltd. [SEI-CMMI Level 5 Company]

    -----------------------------

    This email and any files transmitted with it are confidential and intended
    solely for the use of the individual or entity to whom they are
    addressed.
    If you have received this email in error please notify the system
    administrator - > netopshelpdesk@catalytic.com









    --
    http://daily.appspot.com/food/

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedJun 17, '09 at 12:36p
activeJun 18, '09 at 7:10a
posts6
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase