FAQ
hi,

I try to use contrib/index/hadoop-0.17.1-index.jar build lucene index, but Directory NoClassDefFoundError.

How solve? Thinks.

Additional, hadoop run Pseudo-Distributed Mode. and can run "bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'"

---------------------------error----------------------------------------------------------------------------------
[chenlb@master hadoop]$ bin/hadoop jar contrib/index/hadoop-0.17.1-index.jar -inputPaths src/contrib/index/sample -outputPath index-msg-out -indexPath index -conf src/contrib/index/conf/index-config.xml
08/10/09 21:41:28 INFO main.UpdateIndex: inputPaths = src/contrib/index/sample
08/10/09 21:41:28 INFO main.UpdateIndex: outputPath = index-msg-out
08/10/09 21:41:28 INFO main.UpdateIndex: shards = null
08/10/09 21:41:28 INFO main.UpdateIndex: indexPath = index
08/10/09 21:41:28 INFO main.UpdateIndex: numShards = -1
08/10/09 21:41:28 INFO main.UpdateIndex: numMapTasks= -1
08/10/09 21:41:28 INFO main.UpdateIndex: confPath = src/contrib/index/conf/index-config.xml
08/10/09 21:41:28 INFO main.UpdateIndex: Elapsed time is 0s
Elapsed time is 0s
java.lang.NoClassDefFoundError: org/apache/lucene/store/Directory
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
at org.apache.hadoop.contrib.index.mapred.IndexUpdateConfiguration.getIndexUpdaterClass(IndexUpdateConfiguration.java:145)
at org.apache.hadoop.contrib.index.main.UpdateIndex.main(UpdateIndex.java:257)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)
Caused by: java.lang.ClassNotFoundException: org.apache.lucene.store.Directory
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
... 16 more

-----------------------------------------------------------------------------------------------------------------

2008-10-10



chenlbspace

Search Discussions

  • Imcaptor at Oct 10, 2008 at 8:39 am
    U must download the lucene binary jar file,and set the jar file in the
    classpath.

    chenlbspace 写道:
    hi,

    I try to use contrib/index/hadoop-0.17.1-index.jar build lucene index, but Directory NoClassDefFoundError.

    How solve? Thinks.

    Additional, hadoop run Pseudo-Distributed Mode. and can run "bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'"

    ---------------------------error----------------------------------------------------------------------------------
    [chenlb@master hadoop]$ bin/hadoop jar contrib/index/hadoop-0.17.1-index.jar -inputPaths src/contrib/index/sample -outputPath index-msg-out -indexPath index -conf src/contrib/index/conf/index-config.xml
    08/10/09 21:41:28 INFO main.UpdateIndex: inputPaths = src/contrib/index/sample
    08/10/09 21:41:28 INFO main.UpdateIndex: outputPath = index-msg-out
    08/10/09 21:41:28 INFO main.UpdateIndex: shards = null
    08/10/09 21:41:28 INFO main.UpdateIndex: indexPath = index
    08/10/09 21:41:28 INFO main.UpdateIndex: numShards = -1
    08/10/09 21:41:28 INFO main.UpdateIndex: numMapTasks= -1
    08/10/09 21:41:28 INFO main.UpdateIndex: confPath = src/contrib/index/conf/index-config.xml
    08/10/09 21:41:28 INFO main.UpdateIndex: Elapsed time is 0s
    Elapsed time is 0s
    java.lang.NoClassDefFoundError: org/apache/lucene/store/Directory
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
    at org.apache.hadoop.contrib.index.mapred.IndexUpdateConfiguration.getIndexUpdaterClass(IndexUpdateConfiguration.java:145)
    at org.apache.hadoop.contrib.index.main.UpdateIndex.main(UpdateIndex.java:257)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
    at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)
    Caused by: java.lang.ClassNotFoundException: org.apache.lucene.store.Directory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    ... 16 more

    -----------------------------------------------------------------------------------------------------------------

    2008-10-10



    chenlbspace
  • Imcaptor at Oct 10, 2008 at 8:55 am
    The jar file is in the src/contrib/index/lib/lucene-core-2.3.1.jar, u
    must set the file in classpath.

    imcaptor 写道:
    U must download the lucene binary jar file,and set the jar file in the
    classpath.

    chenlbspace 写道:
    hi,

    I try to use contrib/index/hadoop-0.17.1-index.jar build lucene index, but Directory NoClassDefFoundError.

    How solve? Thinks.

    Additional, hadoop run Pseudo-Distributed Mode. and can run "bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'"

    ---------------------------error----------------------------------------------------------------------------------
    [chenlb@master hadoop]$ bin/hadoop jar contrib/index/hadoop-0.17.1-index.jar -inputPaths src/contrib/index/sample -outputPath index-msg-out -indexPath index -conf src/contrib/index/conf/index-config.xml
    08/10/09 21:41:28 INFO main.UpdateIndex: inputPaths = src/contrib/index/sample
    08/10/09 21:41:28 INFO main.UpdateIndex: outputPath = index-msg-out
    08/10/09 21:41:28 INFO main.UpdateIndex: shards = null
    08/10/09 21:41:28 INFO main.UpdateIndex: indexPath = index
    08/10/09 21:41:28 INFO main.UpdateIndex: numShards = -1
    08/10/09 21:41:28 INFO main.UpdateIndex: numMapTasks= -1
    08/10/09 21:41:28 INFO main.UpdateIndex: confPath = src/contrib/index/conf/index-config.xml
    08/10/09 21:41:28 INFO main.UpdateIndex: Elapsed time is 0s
    Elapsed time is 0s
    java.lang.NoClassDefFoundError: org/apache/lucene/store/Directory
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
    at org.apache.hadoop.contrib.index.mapred.IndexUpdateConfiguration.getIndexUpdaterClass(IndexUpdateConfiguration.java:145)
    at org.apache.hadoop.contrib.index.main.UpdateIndex.main(UpdateIndex.java:257)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
    at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)
    Caused by: java.lang.ClassNotFoundException: org.apache.lucene.store.Directory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    ... 16 more

    -----------------------------------------------------------------------------------------------------------------

    2008-10-10



    chenlbspace



  • Imcaptor at Oct 10, 2008 at 9:08 am
    Try do this.

    In the hadoop path,
    -bash-3.00$ pwd
    /data/hadoop/hadoop-0.18.1
    cp src/contrib/index/lib/lucene-core-2.3.1.jar lib


    Then you can run the task.

    chenlbspace 写道:
    hi,

    I try to use contrib/index/hadoop-0.17.1-index.jar build lucene index, but Directory NoClassDefFoundError.

    How solve? Thinks.

    Additional, hadoop run Pseudo-Distributed Mode. and can run "bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'"

    ---------------------------error----------------------------------------------------------------------------------
    [chenlb@master hadoop]$ bin/hadoop jar contrib/index/hadoop-0.17.1-index.jar -inputPaths src/contrib/index/sample -outputPath index-msg-out -indexPath index -conf src/contrib/index/conf/index-config.xml
    08/10/09 21:41:28 INFO main.UpdateIndex: inputPaths = src/contrib/index/sample
    08/10/09 21:41:28 INFO main.UpdateIndex: outputPath = index-msg-out
    08/10/09 21:41:28 INFO main.UpdateIndex: shards = null
    08/10/09 21:41:28 INFO main.UpdateIndex: indexPath = index
    08/10/09 21:41:28 INFO main.UpdateIndex: numShards = -1
    08/10/09 21:41:28 INFO main.UpdateIndex: numMapTasks= -1
    08/10/09 21:41:28 INFO main.UpdateIndex: confPath = src/contrib/index/conf/index-config.xml
    08/10/09 21:41:28 INFO main.UpdateIndex: Elapsed time is 0s
    Elapsed time is 0s
    java.lang.NoClassDefFoundError: org/apache/lucene/store/Directory
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
    at org.apache.hadoop.contrib.index.mapred.IndexUpdateConfiguration.getIndexUpdaterClass(IndexUpdateConfiguration.java:145)
    at org.apache.hadoop.contrib.index.main.UpdateIndex.main(UpdateIndex.java:257)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
    at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)
    Caused by: java.lang.ClassNotFoundException: org.apache.lucene.store.Directory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    ... 16 more

    -----------------------------------------------------------------------------------------------------------------

    2008-10-10



    chenlbspace
  • Chenlbspace at Oct 10, 2008 at 9:50 am
    hi, imcaptor

    I try to cp lucene-core-2.3.1.jar to lib, but error also at map task.

    -------------------lib-----------------
    [chenlb@master hadoop]$ pwd
    /home/chenlb/hadoop
    [chenlb@master hadoop]$ ls lib
    commons-cli-2.0-SNAPSHOT.jar commons-logging-1.0.4.jar jetty-5.1.4.jar junit-3.8.1.jar kfs-0.1.LICENSE.txt native
    commons-codec-1.3.jar commons-logging-api-1.0.4.jar jetty-5.1.4.LICENSE.txt junit-3.8.1.LICENSE.txt log4j-1.2.13.jar servlet-api.jar
    commons-httpclient-3.0.1.jar jets3t-0.5.0.jar jetty-ext kfs-0.1.jar lucene-core-2.3.1.jar xmlenc-0.52.jar

    -------------------hadoop-site.xml---------------------------------
    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <!-- Put site-specific property overrides in this file. -->
    <configuration>
    <property>
    <name>fs.default.name</name>
    <value>hdfs://master:9000/</value>
    </property>
    <property>
    <name>mapred.job.tracker</name>
    <value>hdfs://master:9001/</value>
    </property>
    <property>
    <name>dfs.replication</name>
    <value>1</value>
    </property>
    <property>
    <name>hadoop.tmp.dir</name>
    <value>/home/chenlb/hadoop/tmp/</value>
    </property>
    </configuration>
    ----------------------------

    Thinks!

    =====================error=======================
    [chenlb@master hadoop]$ bin/hadoop jar hadoop-0.17.1-index.jar -inputPaths data-in -outputPath index-msg-out -indexPath index -conf index-config.xml
    08/10/09 22:33:36 INFO main.UpdateIndex: inputPaths = data-in
    08/10/09 22:33:36 INFO main.UpdateIndex: outputPath = index-msg-out
    08/10/09 22:33:36 INFO main.UpdateIndex: shards = null
    08/10/09 22:33:36 INFO main.UpdateIndex: indexPath = index
    08/10/09 22:33:36 INFO main.UpdateIndex: numShards = -1
    08/10/09 22:33:36 INFO main.UpdateIndex: numMapTasks= -1
    08/10/09 22:33:36 INFO main.UpdateIndex: confPath = index-config.xml
    08/10/09 22:33:37 INFO main.UpdateIndex: sea.index.updater = org.apache.hadoop.contrib.index.mapred.IndexUpdater
    08/10/09 22:33:37 INFO mapred.IndexUpdater: mapred.input.dir = hdfs://master:9000/user/chenlb/data-in
    08/10/09 22:33:37 INFO mapred.IndexUpdater: mapred.output.dir = hdfs://master:9000/user/chenlb/index-msg-out
    08/10/09 22:33:37 INFO mapred.IndexUpdater: mapred.map.tasks = 2
    08/10/09 22:33:37 INFO mapred.IndexUpdater: mapred.reduce.tasks = 1
    08/10/09 22:33:37 INFO mapred.IndexUpdater: 1 shards = -1@index/00000@-1
    08/10/09 22:33:37 INFO mapred.IndexUpdater: mapred.input.format.class = org.apache.hadoop.contrib.index.example.LineDocInputFormat
    08/10/09 22:33:38 INFO mapred.FileInputFormat: Total input paths to process : 2
    08/10/09 22:33:39 INFO mapred.JobClient: Running job: job_200810092201_0010
    08/10/09 22:33:40 INFO mapred.JobClient: map 0% reduce 0%
    08/10/09 22:33:48 INFO mapred.JobClient: Task Id : task_200810092201_0010_m_000000_0, Status : FAILED
    java.lang.NoClassDefFoundError: org/apache/lucene/store/Directory
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
    at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:552)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:185)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
    Caused by: java.lang.ClassNotFoundException: org.apache.lucene.store.Directory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    ... 9 more
    08/10/09 22:33:50 INFO mapred.JobClient: Task Id : task_200810092201_0010_m_000001_0, Status : FAILED
    java.lang.NoClassDefFoundError: org/apache/lucene/store/Directory
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
    at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:552)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:185)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
    Caused by: java.lang.ClassNotFoundException: org.apache.lucene.store.Directory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    ... 9 more
    08/10/09 22:33:55 INFO mapred.JobClient: Task Id : task_200810092201_0010_r_000000_0, Status : FAILED
    java.lang.NoClassDefFoundError: org/apache/lucene/store/Directory
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
    at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:552)
    at org.apache.hadoop.mapred.ReduceTask$ReduceCopier.(ReduceTask.java:327)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
    Caused by: java.lang.ClassNotFoundException: org.apache.lucene.store.Directory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    ... 9 more
    08/10/09 22:33:56 INFO mapred.JobClient: Task Id : task_200810092201_0010_m_000000_1, Status : FAILED
    java.lang.NoClassDefFoundError: org/apache/lucene/store/Directory
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
    at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:552)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:185)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
    Caused by: java.lang.ClassNotFoundException: org.apache.lucene.store.Directory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    ... 9 more
    08/10/09 22:33:59 INFO mapred.JobClient: Task Id : task_200810092201_0010_m_000001_1, Status : FAILED
    java.lang.NoClassDefFoundError: org/apache/lucene/store/Directory
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
    at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:552)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:185)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
    Caused by: java.lang.ClassNotFoundException: org.apache.lucene.store.Directory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    ... 9 more
    08/10/09 22:34:03 INFO mapred.JobClient: Task Id : task_200810092201_0010_r_000000_1, Status : FAILED
    java.lang.NoClassDefFoundError: org/apache/lucene/store/Directory
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
    at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:552)
    at org.apache.hadoop.mapred.ReduceTask$ReduceCopier.(ReduceTask.java:327)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
    Caused by: java.lang.ClassNotFoundException: org.apache.lucene.store.Directory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    ... 9 more
    08/10/09 22:34:05 INFO mapred.JobClient: Task Id : task_200810092201_0010_m_000000_2, Status : FAILED
    java.lang.NoClassDefFoundError: org/apache/lucene/store/Directory
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
    at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:552)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:185)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
    Caused by: java.lang.ClassNotFoundException: org.apache.lucene.store.Directory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    ... 9 more
    08/10/09 22:34:08 INFO mapred.JobClient: Task Id : task_200810092201_0010_m_000001_2, Status : FAILED
    java.lang.NoClassDefFoundError: org/apache/lucene/store/Directory
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
    at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:552)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:185)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
    Caused by: java.lang.ClassNotFoundException: org.apache.lucene.store.Directory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    ... 9 more
    08/10/09 22:34:10 INFO mapred.JobClient: Task Id : task_200810092201_0010_r_000000_2, Status : FAILED
    java.lang.NoClassDefFoundError: org/apache/lucene/store/Directory
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
    at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:552)
    at org.apache.hadoop.mapred.ReduceTask$ReduceCopier.(ReduceTask.java:327)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
    Caused by: java.lang.ClassNotFoundException: org.apache.lucene.store.Directory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    ... 9 more
    08/10/09 22:34:13 INFO mapred.JobClient: map 100% reduce 100%
    08/10/09 22:34:14 INFO main.UpdateIndex: Elapsed time is 37s
    Elapsed time is 37s
    java.io.IOException: Job failed!
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1062)
    at org.apache.hadoop.contrib.index.mapred.IndexUpdater.run(IndexUpdater.java:53)
    at org.apache.hadoop.contrib.index.main.UpdateIndex.main(UpdateIndex.java:263)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
    at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)

    2008-10-10



    chenlbspace



    发件人: imcaptor
    发送时间: 2008-10-10 17:07:39
    收件人: core-user@hadoop.apache.org
    抄送:
    主题: Re: hadoop index org/apache/lucene/store/Directory NoClassDefFoundError

    Try do this.
    In the hadoop path,
    -bash-3.00$ pwd
    /data/hadoop/hadoop-0.18.1
    cp src/contrib/index/lib/lucene-core-2.3.1.jar lib
    Then you can run the task.
    chenlbspace 写道:
    hi,

    I try to use contrib/index/hadoop-0.17.1-index.jar build lucene index, but Directory NoClassDefFoundError.

    How solve? Thinks.

    Additional, hadoop run Pseudo-Distributed Mode. and can run "bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'"

    ---------------------------error----------------------------------------------------------------------------------
    [chenlb@master hadoop]$ bin/hadoop jar contrib/index/hadoop-0.17.1-index.jar -inputPaths src/contrib/index/sample -outputPath index-msg-out -indexPath index -conf src/contrib/index/conf/index-config.xml
    08/10/09 21:41:28 INFO main.UpdateIndex: inputPaths = src/contrib/index/sample
    08/10/09 21:41:28 INFO main.UpdateIndex: outputPath = index-msg-out
    08/10/09 21:41:28 INFO main.UpdateIndex: shards = null
    08/10/09 21:41:28 INFO main.UpdateIndex: indexPath = index
    08/10/09 21:41:28 INFO main.UpdateIndex: numShards = -1
    08/10/09 21:41:28 INFO main.UpdateIndex: numMapTasks= -1
    08/10/09 21:41:28 INFO main.UpdateIndex: confPath = src/contrib/index/conf/index-config.xml
    08/10/09 21:41:28 INFO main.UpdateIndex: Elapsed time is 0s
    Elapsed time is 0s
    java.lang.NoClassDefFoundError: org/apache/lucene/store/Directory
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
    at org.apache.hadoop.contrib.index.mapred.IndexUpdateConfiguration.getIndexUpdaterClass(IndexUpdateConfiguration.java:145)
    at org.apache.hadoop.contrib.index.main.UpdateIndex.main(UpdateIndex.java:257)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
    at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)
    Caused by: java.lang.ClassNotFoundException: org.apache.lucene.store.Directory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    ... 16 more

    -----------------------------------------------------------------------------------------------------------------

    2008-10-10



    chenlbspace
    __________ NOD32 3510 (20081010) Information __________
    This message was checked by NOD32 antivirus system.
    http://www.nod32cn.com
  • Kyle Lau at Oct 16, 2008 at 1:51 am
    I ran into the same problem today. Found a solution that works:

    bin/hadoop jar -libjars <comma separated jars> hadoop-xxx-index.jar
    -inputPaths...

    E.g., in your particular case, you command will look like:

    bin/hadoop jar -libjars lib/lucene-core-2.3.1.jar
    contrib/index/hadoop-0.17.1-index.jar -inputPaths
    src/contrib/index/sample -outputPath index-msg-out -indexPath index
    -conf src/contrib/index/conf/index-config.xml


    Kyle


    On Fri, 2008-10-10 at 16:35 +0800, chenlbspace wrote:
    hi,

    I try to use contrib/index/hadoop-0.17.1-index.jar build lucene index, but Directory NoClassDefFoundError.

    How solve? Thinks.

    Additional, hadoop run Pseudo-Distributed Mode. and can run "bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'"

    ---------------------------error----------------------------------------------------------------------------------
    [chenlb@master hadoop]$ bin/hadoop jar contrib/index/hadoop-0.17.1-index.jar -inputPaths src/contrib/index/sample -outputPath index-msg-out -indexPath index -conf src/contrib/index/conf/index-config.xml
    08/10/09 21:41:28 INFO main.UpdateIndex: inputPaths = src/contrib/index/sample
    08/10/09 21:41:28 INFO main.UpdateIndex: outputPath = index-msg-out
    08/10/09 21:41:28 INFO main.UpdateIndex: shards = null
    08/10/09 21:41:28 INFO main.UpdateIndex: indexPath = index
    08/10/09 21:41:28 INFO main.UpdateIndex: numShards = -1
    08/10/09 21:41:28 INFO main.UpdateIndex: numMapTasks= -1
    08/10/09 21:41:28 INFO main.UpdateIndex: confPath = src/contrib/index/conf/index-config.xml
    08/10/09 21:41:28 INFO main.UpdateIndex: Elapsed time is 0s
    Elapsed time is 0s
    java.lang.NoClassDefFoundError: org/apache/lucene/store/Directory
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
    at org.apache.hadoop.contrib.index.mapred.IndexUpdateConfiguration.getIndexUpdaterClass(IndexUpdateConfiguration.java:145)
    at org.apache.hadoop.contrib.index.main.UpdateIndex.main(UpdateIndex.java:257)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
    at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)
    Caused by: java.lang.ClassNotFoundException: org.apache.lucene.store.Directory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    ... 16 more

    -----------------------------------------------------------------------------------------------------------------

    2008-10-10



    chenlbspace
  • Chenlbspace at Oct 16, 2008 at 5:44 am
    Thinks, imcaptor and Kyle.

    I add lucene-core-2.3.1.jar to lib, It can't correct run, but, suddenly no any error the day of tomorrow


    2008-10-16



    chenlbspace



    发件人: chenlbspace
    发送时间: 2008-10-10 16:35:40
    收件人: core-user@hadoop.apache.org
    抄送:
    主题: hadoop index org/apache/lucene/store/Directory NoClassDefFoundError

    hi,
    I try to use contrib/index/hadoop-0.17.1-index.jar build lucene index, but Directory NoClassDefFoundError.
    How solve? Thinks.
    Additional, hadoop run Pseudo-Distributed Mode. and can run "bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'"
    ---------------------------error----------------------------------------------------------------------------------
    [chenlb@master hadoop]$ bin/hadoop jar contrib/index/hadoop-0.17.1-index.jar -inputPaths src/contrib/index/sample -outputPath index-msg-out -indexPath index -conf src/contrib/index/conf/index-config.xml
    08/10/09 21:41:28 INFO main.UpdateIndex: inputPaths = src/contrib/index/sample
    08/10/09 21:41:28 INFO main.UpdateIndex: outputPath = index-msg-out
    08/10/09 21:41:28 INFO main.UpdateIndex: shards = null
    08/10/09 21:41:28 INFO main.UpdateIndex: indexPath = index
    08/10/09 21:41:28 INFO main.UpdateIndex: numShards = -1
    08/10/09 21:41:28 INFO main.UpdateIndex: numMapTasks= -1
    08/10/09 21:41:28 INFO main.UpdateIndex: confPath = src/contrib/index/conf/index-config.xml
    08/10/09 21:41:28 INFO main.UpdateIndex: Elapsed time is 0s
    Elapsed time is 0s
    java.lang.NoClassDefFoundError: org/apache/lucene/store/Directory
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
    at org.apache.hadoop.contrib.index.mapred.IndexUpdateConfiguration.getIndexUpdaterClass(IndexUpdateConfiguration.java:145)
    at org.apache.hadoop.contrib.index.main.UpdateIndex.main(UpdateIndex.java:257)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
    at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)
    Caused by: java.lang.ClassNotFoundException: org.apache.lucene.store.Directory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
    ... 16 more
    -----------------------------------------------------------------------------------------------------------------
    2008-10-10
    chenlbspace
    __________ NOD32 3510 (20081010) Information __________
    This message was checked by NOD32 antivirus system.
    http://www.nod32cn.com

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedOct 10, '08 at 8:36a
activeOct 16, '08 at 5:44a
posts7
users3
websitehadoop.apache.org...
irc#hadoop

3 users in discussion

Imcaptor: 3 posts Chenlbspace: 3 posts Kyle Lau: 1 post

People

Translate

site design / logo © 2022 Grokbase