FAQ
Hi,

I’m running Nutch 2 on a Hadoop 1.2.1 cluster with 2 nodes. I’m running Solr 4 separately on a box and I replaced Solr’s schema with Nutch’s Solr-4 schema. When I run a crawl, I get the following error at the end of the job

14/05/26 14:08:32 INFO solr.SolrDeleteDuplicates: SolrDeleteDuplicates: starting...
14/05/26 14:08:32 INFO solr.SolrDeleteDuplicates: SolrDeleteDuplicates: Solr url: http://10.130.231.16:8983/solr/nutch
14/05/26 14:08:33 WARN mapred.JobClient: No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
14/05/26 14:08:33 INFO mapred.JobClient: Running job: job_201405261214_0014
14/05/26 14:08:34 INFO mapred.JobClient:  map 0% reduce 0%
14/05/26 14:08:43 INFO mapred.JobClient: Task Id : attempt_201405261214_0014_m_000000_0, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat
  at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
  at org.apache.hadoop.mapreduce.JobContext.getInputFormatClass(JobContext.java:187)
  at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
  at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
  at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.ClassNotFoundException: org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat
  at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
  at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
  at java.security.AccessController.doPrivileged(Native Method)
  at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
  at java.lang.Class.forName0(Native Method)
  at java.lang.Class.forName(Class.java:270)
  at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
  at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855)
  ... 8 more

14/05/26 14:08:43 WARN mapred.JobClient: Error reading task outputnutch-two-qontifi
14/05/26 14:08:43 WARN mapred.JobClient: Error reading task outputnutch-two-qontifi
14/05/26 14:08:44 INFO mapred.JobClient: Task Id : attempt_201405261214_0014_m_000001_0, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat
  at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
  at org.apache.hadoop.mapreduce.JobContext.getInputFormatClass(JobContext.java:187)
  at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
  at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
  at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.ClassNotFoundException: org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat
  at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
  at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
  at java.security.AccessController.doPrivileged(Native Method)
  at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
  at java.lang.Class.forName0(Native Method)
  at java.lang.Class.forName(Class.java:270)
  at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
  at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855)
  ... 8 more

14/05/26 14:08:50 INFO mapred.JobClient: Task Id : attempt_201405261214_0014_m_000001_1, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat
  at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
  at org.apache.hadoop.mapreduce.JobContext.getInputFormatClass(JobContext.java:187)
  at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
  at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
  at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.ClassNotFoundException: org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat
  at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
  at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
  at java.security.AccessController.doPrivileged(Native Method)
  at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
  at java.lang.Class.forName0(Native Method)
  at java.lang.Class.forName(Class.java:270)
  at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
  at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855)
  ... 8 more

14/05/26 14:08:50 WARN mapred.JobClient: Error reading task outputnutch-two-qontifi
14/05/26 14:08:50 WARN mapred.JobClient: Error reading task outputnutch-two-qontifi
14/05/26 14:08:51 INFO mapred.JobClient: Task Id : attempt_201405261214_0014_m_000000_1, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat
  at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
  at org.apache.hadoop.mapreduce.JobContext.getInputFormatClass(JobContext.java:187)
  at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
  at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
  at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.ClassNotFoundException: org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat
  at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
  at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
  at java.security.AccessController.doPrivileged(Native Method)
  at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
  at java.lang.Class.forName0(Native Method)
  at java.lang.Class.forName(Class.java:270)
  at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
  at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855)
  ... 8 more

14/05/26 14:08:56 INFO mapred.JobClient: Task Id : attempt_201405261214_0014_m_000001_2, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat
  at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
  at org.apache.hadoop.mapreduce.JobContext.getInputFormatClass(JobContext.java:187)
  at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
  at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
  at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.ClassNotFoundException: org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat
  at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
  at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
  at java.security.AccessController.doPrivileged(Native Method)
  at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
  at java.lang.Class.forName0(Native Method)
  at java.lang.Class.forName(Class.java:270)
  at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
  at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855)
  ... 8 more

14/05/26 14:08:56 WARN mapred.JobClient: Error reading task outputnutch-two-qontifi
14/05/26 14:08:56 WARN mapred.JobClient: Error reading task outputnutch-two-qontifi
14/05/26 14:08:57 INFO mapred.JobClient: Task Id : attempt_201405261214_0014_m_000000_2, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat
  at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
  at org.apache.hadoop.mapreduce.JobContext.getInputFormatClass(JobContext.java:187)
  at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
  at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
  at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.ClassNotFoundException: org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat
  at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
  at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
  at java.security.AccessController.doPrivileged(Native Method)
  at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
  at java.lang.Class.forName0(Native Method)
  at java.lang.Class.forName(Class.java:270)
  at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
  at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855)

And finally, nothing is being indexed to Solr. Am I missing anything? Please help.

Regards,
Manikandan

Search Discussions

  • Shawn Heisey at May 26, 2014 at 7:11 pm

    On 5/26/2014 12:20 PM, Manikandan Saravanan wrote:
    I’m running Nutch 2 on a Hadoop 1.2.1 cluster with 2 nodes. I’m running Solr 4 separately on a box and I replaced Solr’s schema with Nutch’s Solr-4 schema. When I run a crawl, I get the following error at the end of the job

    14/05/26 14:08:32 INFO solr.SolrDeleteDuplicates: SolrDeleteDuplicates: starting...
    14/05/26 14:08:32 INFO solr.SolrDeleteDuplicates: SolrDeleteDuplicates: Solr url: http://10.130.231.16:8983/solr/nutch
    14/05/26 14:08:33 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
    14/05/26 14:08:33 INFO mapred.JobClient: Running job: job_201405261214_0014
    14/05/26 14:08:34 INFO mapred.JobClient: map 0% reduce 0%
    14/05/26 14:08:43 INFO mapred.JobClient: Task Id : attempt_201405261214_0014_m_000000_0, Status : FAILED
    java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
    at org.apache.hadoop.mapreduce.JobContext.getInputFormatClass(JobContext.java:187)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)
    Caused by: java.lang.ClassNotFoundException: org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat
    I am not subscribed to the nutch mailing list, so I have removed that
    list from the recipients here.

    If you look at the last line that I quoted above, you'll see that the
    exception is caused by the inability of Java to locate a class, and that
    the class is a Nutch class. I just built Nutch 2.2.1 on my server, and
    the strange thing here is that this class seems to be part of the main
    apache nutch jar, so I have no idea how you are using nutch without this
    class being present.

    Because this is a nutch class that is missing and not a Solr class, the
    Solr mailing list can't really provide much help.

    Thanks,
    Shawn

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupsolr-user @
categorieslucene
postedMay 26, '14 at 6:20p
activeMay 26, '14 at 7:11p
posts2
users2
websitelucene.apache.org...

People

Translate

site design / logo © 2018 Grokbase