Hello, guys,
I am very new to hadoop. I was trying to read nutch data files using a
script i found on http://wiki.apache.org/nutch/Getting_Started . And after 2
days of trying, I still cannot get it to work. now the error i got is
"java.lang.RuntimeException: java.io.IOException: WritableName can't load
class".
Below is my script:
/*
* To change this template, choose Tools | Templates
* and open the template in the editor.
*/
package test;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.MapFile;
import org.apache.hadoop.io.Writable;
import org.apache.hadoop.io.WritableComparable;
/**
*
* @author mudong
*/
public class Main {
/**
* @param args the command line arguments
*/
public static void main(String[] args) {
// TODO code application logic here
try{
Configuration conf = new Configuration();
conf.addResource(new
Path("/home/mudong/programming/java/hadoop-0.17.2.1/conf/hadoop-default.xml"));
//conf.addResource(new
Path("/home/mudong/programming/java/hadoop-0.18.1/conf/hadoop-default.xml"));
FileSystem fs= FileSystem.get(conf);
String seqFile = new
String("/home/mudong/programming/java/nutch-0.9/crawl/segments/20081021075837/content/part-00000");
MapFile.Reader reader;
reader = new MapFile.Reader (fs, seqFile, conf);
Class keyC = reader.getKeyClass();
Class valueC = reader.getValueClass();
while (true) {
WritableComparable key = null;
Writable value = null;
try {
key = (WritableComparable)keyC.newInstance();
value = (Writable)valueC.newInstance();
} catch (Exception ex) {
ex.printStackTrace();
System.exit(-1);
}
try {
if (!reader.next(key, value)) {
break;
}
System.out.println(key);
System.out.println(value);
} catch (Exception e) {
e.printStackTrace();
System.out.println("Exception occured. " + e);
break;
}
}
}catch(Exception e){
e.printStackTrace();
System.out.println("Exception occured. " + e);
}
}
}
And when I running the script above, I got error messages like below.
java.lang.RuntimeException: java.io.IOException: WritableName can't load
class
at
org.apache.hadoop.io.SequenceFile$Reader.getValueClass(SequenceFile.java:1612)
Exception occured. java.lang.RuntimeException: java.io.IOException:
WritableName can't load class
at
org.apache.hadoop.io.MapFile$Reader.getValueClass(MapFile.java:248)
at test.Main.main(Main.java:36)
Caused by: java.io.IOException: WritableName can't load class
at org.apache.hadoop.io.WritableName.getClass(WritableName.java:74)
at
org.apache.hadoop.io.SequenceFile$Reader.getValueClass(SequenceFile.java:1610)
... 2 more
Caused by: java.lang.ClassNotFoundException:
org.apache.nutch.protocol.Content
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
at org.apache.hadoop.io.WritableName.getClass(WritableName.java:72)
I've tried a lot of things, but it's just not working. I use
hadoop-0.17.2.1. Thanks a lot, guys,
Rongdong
