FAQ
Hi,
I tried to use SequenceFileInputFormat, for this I appended "SEQ" as first
bytes of my "binary" files (with hex editor).
but I get this exception:

A record version mismatch occured. Expecting v6, found v32
at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1460)
at
org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1417)
at
org.apache.hadoop.io.SequenceFile$Reader.(SequenceFileRecordReader.java:43)
at
org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFileInputFormat.java:58)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:321)
at org.apache.hadoop.mapred.Child.main(Child.java:155)

What could it be? Is it not enough just to add "SEQ" to binary files?
I use Hadoop v.0.19.0 .

Thanks in advance..
Rasit


different *version* of *Hadoop* between your server and your client.

--
M. Raşit ÖZDAŞ

Search Discussions

  • Rasit OZDAS at Feb 2, 2009 at 3:21 pm
    I tried to use SequenceFile.Writer to convert my binaries into Sequence
    Files,
    I read the binary data with FileInputStream, getting all bytes with
    reader.read(byte[]) , wrote it to a file with SequenceFile.Writer, with
    parameters NullWritable as key, BytesWritable as value. But the content
    changes,
    (I can see that by converting to Base64)

    Binary File:
    73 65 65 65 81 65 65 65 65 65 81 81 65 119 84 81 65 111 67 81 65 52 57 81 65
    103 54 81 65 65 97 81 65 65 65 81 ...

    Sequence File:
    73 65 65 65 65 69 65 65 65 65 65 65 65 69 66 65 65 77 66 77 81 103 67 103 67
    69 77 65 52 80 86 67 65 73 68 114 ...

    Thanks for any points..
    Rasit

    2009/2/2 Rasit OZDAS <rasitozdas@gmail.com>
    Hi,
    I tried to use SequenceFileInputFormat, for this I appended "SEQ" as first
    bytes of my "binary" files (with hex editor).
    but I get this exception:

    A record version mismatch occured. Expecting v6, found v32
    at
    org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1460)
    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1428)
    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1417)
    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1412)
    at
    org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordReader.java:43)
    at
    org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFileInputFormat.java:58)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:321)
    at org.apache.hadoop.mapred.Child.main(Child.java:155)

    What could it be? Is it not enough just to add "SEQ" to binary files?
    I use Hadoop v.0.19.0 .

    Thanks in advance..
    Rasit


    different *version* of *Hadoop* between your server and your client.

    --
    M. Raşit ÖZDAŞ


    --
    M. Raşit ÖZDAŞ
  • Tom White at Feb 2, 2009 at 3:58 pm
    The SequenceFile format is described here:
    http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/io/SequenceFile.html.
    The format of the keys and values depends on the serialization classes
    used. For example, BytesWritable writes out the length of its byte
    array followed by the actual bytes in the array (see the write()
    method in BytesWritable).

    Hope this helps.
    Tom
    On Mon, Feb 2, 2009 at 3:21 PM, Rasit OZDAS wrote:
    I tried to use SequenceFile.Writer to convert my binaries into Sequence
    Files,
    I read the binary data with FileInputStream, getting all bytes with
    reader.read(byte[]) , wrote it to a file with SequenceFile.Writer, with
    parameters NullWritable as key, BytesWritable as value. But the content
    changes,
    (I can see that by converting to Base64)

    Binary File:
    73 65 65 65 81 65 65 65 65 65 81 81 65 119 84 81 65 111 67 81 65 52 57 81 65
    103 54 81 65 65 97 81 65 65 65 81 ...

    Sequence File:
    73 65 65 65 65 69 65 65 65 65 65 65 65 69 66 65 65 77 66 77 81 103 67 103 67
    69 77 65 52 80 86 67 65 73 68 114 ...

    Thanks for any points..
    Rasit

    2009/2/2 Rasit OZDAS <rasitozdas@gmail.com>
    Hi,
    I tried to use SequenceFileInputFormat, for this I appended "SEQ" as first
    bytes of my "binary" files (with hex editor).
    but I get this exception:

    A record version mismatch occured. Expecting v6, found v32
    at
    org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1460)
    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1428)
    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1417)
    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1412)
    at
    org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordReader.java:43)
    at
    org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFileInputFormat.java:58)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:321)
    at org.apache.hadoop.mapred.Child.main(Child.java:155)

    What could it be? Is it not enough just to add "SEQ" to binary files?
    I use Hadoop v.0.19.0 .

    Thanks in advance..
    Rasit


    different *version* of *Hadoop* between your server and your client.

    --
    M. Raşit ÖZDAŞ


    --
    M. Raşit ÖZDAŞ
  • Rasit OZDAS at Feb 3, 2009 at 7:23 am
    Thanks, Tom
    The problem that content was different was that
    I converted one sample to Base64 byte-by-byte, and converted the other
    from-byte-array to-byte-array (Strange, that they cause different outputs).
    Thanks for good points.

    Rasit

    2009/2/2 Tom White <tom@cloudera.com>
    The SequenceFile format is described here:

    http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/io/SequenceFile.html
    .
    The format of the keys and values depends on the serialization classes
    used. For example, BytesWritable writes out the length of its byte
    array followed by the actual bytes in the array (see the write()
    method in BytesWritable).

    Hope this helps.
    Tom
    On Mon, Feb 2, 2009 at 3:21 PM, Rasit OZDAS wrote:
    I tried to use SequenceFile.Writer to convert my binaries into Sequence
    Files,
    I read the binary data with FileInputStream, getting all bytes with
    reader.read(byte[]) , wrote it to a file with SequenceFile.Writer, with
    parameters NullWritable as key, BytesWritable as value. But the content
    changes,
    (I can see that by converting to Base64)

    Binary File:
    73 65 65 65 81 65 65 65 65 65 81 81 65 119 84 81 65 111 67 81 65 52 57 81 65
    103 54 81 65 65 97 81 65 65 65 81 ...

    Sequence File:
    73 65 65 65 65 69 65 65 65 65 65 65 65 69 66 65 65 77 66 77 81 103 67 103 67
    69 77 65 52 80 86 67 65 73 68 114 ...

    Thanks for any points..
    Rasit

    2009/2/2 Rasit OZDAS <rasitozdas@gmail.com>
    Hi,
    I tried to use SequenceFileInputFormat, for this I appended "SEQ" as
    first
    bytes of my "binary" files (with hex editor).
    but I get this exception:

    A record version mismatch occured. Expecting v6, found v32
    at
    org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1460)
    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1428)
    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1417)
    at
    org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1412)
    at
    org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordReader.java:43)
    at
    org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFileInputFormat.java:58)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:321)
    at org.apache.hadoop.mapred.Child.main(Child.java:155)

    What could it be? Is it not enough just to add "SEQ" to binary files?
    I use Hadoop v.0.19.0 .

    Thanks in advance..
    Rasit


    different *version* of *Hadoop* between your server and your client.

    --
    M. Raşit ÖZDAŞ


    --
    M. Raşit ÖZDAŞ


    --
    M. Raşit ÖZDAŞ

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedFeb 2, '09 at 3:08p
activeFeb 3, '09 at 7:23a
posts4
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Rasit OZDAS: 3 posts Tom White: 1 post

People

Translate

site design / logo © 2022 Grokbase