[Hadoop-common-user] problems reading compressed sequencefiles in streaming (0.13.1)

-----Original Message-----
From: Joydeep Sen Sarma
Sent: Friday, October 26, 2007 12:30 AM
To: hadoop-user@lucene.apache.org
Subject: problems reading compressed sequencefiles in streaming (0.13.1)

I was hoping to use -inputformat SequenceFileAsTextInputFormat to process
compressed sequencefiles in streaming jobs



However, using a python mapper that just echoes out each line as it gets,
and numreducetasks=0 - here's what the streaming job output looks like:



SEQ^F
org.apache.hadoop.io.IntWritable^Yorg.apache.hadoop.io.Text^A^A'org.apache
.hadoop.io.compress.GzipCodec^@^@^@^@Z+r������^F�



So seems like the input file was not treated as sequencefile.



I must be missing some args - except don't understand what. Help
appreciated ..



Thx,



Joydeep

-----Original Message-----
From: Joydeep Sen Sarma
Sent: Friday, October 26, 2007 12:30 AM
To: hadoop-user@lucene.apache.org
Subject: problems reading compressed sequencefiles in streaming (0.13.1)

I was hoping to use -inputformat SequenceFileAsTextInputFormat to process
compressed sequencefiles in streaming jobs



However, using a python mapper that just echoes out each line as it gets,
and numreducetasks=0 - here's what the streaming job output looks like:



SEQ^F
org.apache.hadoop.io.IntWritable^Yorg.apache.hadoop.io.Text^A^A'org.apache
.hadoop.io.compress.GzipCodec^@^@^@^@Z+r������^F�



So seems like the input file was not treated as sequencefile.



I must be missing some args - except don't understand what. Help
appreciated ..



Thx,



Joydeep