FAQ
Hi,

The following was working fine with Hadoop Writables.
Now, I'm using my custom Writable class called "TermFreqArrayWritable" to produce a Sequence File with key=LongWritable and value=TermFreqArrayWritable.

However, when I try to read the produced Sequence File using its Reader, I get the following:

java.lang.RuntimeException: java.io.IOException: WritableName can't load class: TermFreqArrayWritable
at org.apache.hadoop.io.SequenceFile$Reader.getValueClass(SequenceFile.java:1615)
at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1555)
at org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1417)
at org.apache.hadoop.io.SequenceFile$Reader.(SequenceFileReader_HadoopJob.java:46)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.io.IOException: WritableName can't load class: TermFreqArrayWritable
at org.apache.hadoop.io.WritableName.getClass(WritableName.java:73)
at org.apache.hadoop.io.SequenceFile$Reader.getValueClass(SequenceFile.java:1613)
... 10 more
Caused by: java.lang.ClassNotFoundException: TermFreqArrayWritable
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
at org.apache.hadoop.io.WritableName.getClass(WritableName.java:71)
... 11 more

By the way, my SequenceFileReader has those lines:

LongWritable key = new LongWritable();
TermFreqArrayWritable value = new TermFreqArrayWritable();
while(reader.next(key,value)){
System.out.println("key: "+ key.toString());
System.out.println("value: "+ value.toString());
}
and TermFreqArrayWritable is inside the same project under a default package.


Has any one tried their custom Writable with SequenceFiles ?

Thank you,
Maha

Search Discussions

Discussion Posts

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 1 of 3 | next ›
Discussion Overview
groupcommon-user @
categorieshadoop
postedMar 18, '11 at 11:24p
activeMar 20, '11 at 4:46a
posts3
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Maha: 2 posts Simon: 1 post

People

Translate

site design / logo © 2022 Grokbase