FAQ
Hi,

In my hadoop running example, the data ouput is compressed using gzip. I would
like to create a small java program that decompress the output. Can anyone
give an example on how to decompress the output in java using the hadoop API?



--
Best regards,

-----------------------

Search Discussions

  • Real great.. at Jul 5, 2011 at 3:26 pm
    would not a shell program be better suited?
    just inquisitive.
    On Tue, Jul 5, 2011 at 8:54 PM, Pedro Sa Costa wrote:


    Hi,

    In my hadoop running example, the data ouput is compressed using gzip. I
    would
    like to create a small java program that decompress the output. Can anyone
    give an example on how to decompress the output in java using the hadoop
    API?



    --
    Best regards,

    -----------------------

    --
    Regards,
    R.V.
  • Pedro Sa Costa at Jul 5, 2011 at 3:37 pm
    first of ll, I would like to create a java program and not using shell
    commands.

    But your question brings me another one. If java already contains classes to
    compress data, why hadoop mapreduce created their own class like
    ./org/apache/hadoop/io/compress/GzipCodec.java?

    would not a shell program be better suited?
    just inquisitive.
    On Tue, Jul 5, 2011 at 8:54 PM, Pedro Sa Costa wrote:
    Hi,

    In my hadoop running example, the data ouput is compressed using gzip. I
    would
    like to create a small java program that decompress the output. Can
    anyone give an example on how to decompress the output in java using the
    hadoop API?



    --
    Best regards,

    -----------------------
    --
    Best regards,

    -----------------------
  • David Rosenstrauch at Jul 5, 2011 at 3:34 pm

    On 07/05/2011 11:24 AM, Pedro Sa Costa wrote:
    Hi,

    In my hadoop running example, the data ouput is compressed using gzip. I would
    like to create a small java program that decompress the output. Can anyone
    give an example on how to decompress the output in java using the hadoop API?
    Write a Java app which read from an input stream (until EOF), where the
    input stream is like so:

    FileSystem fs = FileSystem.getLocal(conf);
    InputStream hdfsIn = fs.open(filePath);
    InputStream in = new GZIPInputStream(hdfsIn);

    DR
  • Harsh J at Jul 5, 2011 at 4:51 pm
    I think the 'hadoop fs -text' command may just be what you need.

    It decompresses various formats by itself, and gzip for both Sequence
    and Text files is included in its feature set.
    On Tue, Jul 5, 2011 at 8:54 PM, Pedro Sa Costa wrote:

    Hi,

    In my hadoop running example, the data ouput is compressed using gzip. I would
    like to create a small java program that decompress the output.  Can anyone
    give an example on how to decompress the output in java using the hadoop API?



    --
    Best regards,

    -----------------------


    --
    Harsh J

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmapreduce-user @
categorieshadoop
postedJul 5, '11 at 3:24p
activeJul 5, '11 at 4:51p
posts5
users4
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase