FAQ
Compression test utility
------------------------

Key: HBASE-1875
URL: https://issues.apache.org/jira/browse/HBASE-1875
Project: Hadoop HBase
Issue Type: New Feature
Components: util
Affects Versions: 0.20.0
Reporter: elsif
Priority: Trivial
Fix For: 0.20.1


Attached is a stand alone utility to test compression compatibility
before starting hbase:

# ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
Usage: HFileCompressionTest hdfs compression
compression must be one of [ none, gz, lzo ]

# ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
hdfs://localhost:8020/ lzo
09/09/29 13:39:55 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
09/09/29 13:39:55 INFO lzo.LzoCodec: Successfully loaded & initialized
native-lzo library
09/09/29 13:39:55 INFO compress.CodecPool: Got brand-new compressor
OK

The utility creates a temporary HFile named '.hfile-comp-test' at the
specified HDFS path and outputs either 'OK' or 'FAILED'.


--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • elsif (JIRA) at Sep 30, 2009 at 3:16 pm
    [ https://issues.apache.org/jira/browse/HBASE-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    elsif updated HBASE-1875:
    --------------------------

    Attachment: HFileCompressionTest.java
    Compression test utility
    ------------------------

    Key: HBASE-1875
    URL: https://issues.apache.org/jira/browse/HBASE-1875
    Project: Hadoop HBase
    Issue Type: New Feature
    Components: util
    Affects Versions: 0.20.0
    Reporter: elsif
    Priority: Trivial
    Fix For: 0.20.1

    Attachments: HFileCompressionTest.java


    Attached is a stand alone utility to test compression compatibility
    before starting hbase:
    # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
    Usage: HFileCompressionTest hdfs compression
    compression must be one of [ none, gz, lzo ]
    # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
    hdfs://localhost:8020/ lzo
    09/09/29 13:39:55 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
    09/09/29 13:39:55 INFO lzo.LzoCodec: Successfully loaded & initialized
    native-lzo library
    09/09/29 13:39:55 INFO compress.CodecPool: Got brand-new compressor
    OK
    The utility creates a temporary HFile named '.hfile-comp-test' at the
    specified HDFS path and outputs either 'OK' or 'FAILED'.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Sep 30, 2009 at 4:01 pm
    [ https://issues.apache.org/jira/browse/HBASE-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12760940#action_12760940 ]

    stack commented on HBASE-1875:
    ------------------------------

    Comment from Stephen Mulcahy:

    {code}

    Hi,

    Thanks for this. I found it very useful this morning. Can we get this included in the standard distribution?

    For those not so familiar with building and running code against hbase, the following detailed steps may be useful,

    1. Download compression test util from
    http://mail-archives.apache.org/mod_mbox/hadoop-hbase-user/200909.mbox/%3C4AC272
    AE.3080105@gmail.com%3E

    2. Compile

    (change .jar's below to match your hadoop and hbase versions)

    javac -cp hadoop/hadoop-0.20.1-core.jar:hbase/hbase-0.20.0.jar HFileCompressionTest.java

    3. "Install"
    mkdir -p build/org/apache/hadoop/hbase/test
    mv HFileCompressionTest.class build/org/apache/hadoop/hbase/test

    4. Run
    export HBASE_CLASSPATH=build
    hbase/bin/hbase org.apache.hadoop.hbase.test.HFileCompressionTest hdfs://hbase.example.org:9000/ lzo

    -stephen
    {code}
    Compression test utility
    ------------------------

    Key: HBASE-1875
    URL: https://issues.apache.org/jira/browse/HBASE-1875
    Project: Hadoop HBase
    Issue Type: New Feature
    Components: util
    Affects Versions: 0.20.0
    Reporter: elsif
    Priority: Trivial
    Fix For: 0.20.1

    Attachments: HFileCompressionTest.java


    Attached is a stand alone utility to test compression compatibility
    before starting hbase:
    # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
    Usage: HFileCompressionTest hdfs compression
    compression must be one of [ none, gz, lzo ]
    # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
    hdfs://localhost:8020/ lzo
    09/09/29 13:39:55 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
    09/09/29 13:39:55 INFO lzo.LzoCodec: Successfully loaded & initialized
    native-lzo library
    09/09/29 13:39:55 INFO compress.CodecPool: Got brand-new compressor
    OK
    The utility creates a temporary HFile named '.hfile-comp-test' at the
    specified HDFS path and outputs either 'OK' or 'FAILED'.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Oct 3, 2009 at 5:19 am
    [ https://issues.apache.org/jira/browse/HBASE-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    stack updated HBASE-1875:
    -------------------------

    Status: Patch Available (was: Open)
    Compression test utility
    ------------------------

    Key: HBASE-1875
    URL: https://issues.apache.org/jira/browse/HBASE-1875
    Project: Hadoop HBase
    Issue Type: New Feature
    Components: util
    Affects Versions: 0.20.0
    Reporter: elsif
    Priority: Trivial
    Fix For: 0.20.1

    Attachments: HFileCompressionTest.java


    Attached is a stand alone utility to test compression compatibility
    before starting hbase:
    # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
    Usage: HFileCompressionTest hdfs compression
    compression must be one of [ none, gz, lzo ]
    # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
    hdfs://localhost:8020/ lzo
    09/09/29 13:39:55 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
    09/09/29 13:39:55 INFO lzo.LzoCodec: Successfully loaded & initialized
    native-lzo library
    09/09/29 13:39:55 INFO compress.CodecPool: Got brand-new compressor
    OK
    The utility creates a temporary HFile named '.hfile-comp-test' at the
    specified HDFS path and outputs either 'OK' or 'FAILED'.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Oct 5, 2009 at 9:07 pm
    [ https://issues.apache.org/jira/browse/HBASE-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    stack updated HBASE-1875:
    -------------------------

    Attachment: 1875-v2.patch

    I added a license, formatted it to adhere to rough rules used elsewhere in the codebase, renamed it CompressionTest, and added it to o.a.h.h.util. Hope you don't mind elsif.
    Compression test utility
    ------------------------

    Key: HBASE-1875
    URL: https://issues.apache.org/jira/browse/HBASE-1875
    Project: Hadoop HBase
    Issue Type: New Feature
    Components: util
    Affects Versions: 0.20.0
    Reporter: elsif
    Priority: Trivial
    Fix For: 0.20.1

    Attachments: 1875-v2.patch, HFileCompressionTest.java


    Attached is a stand alone utility to test compression compatibility
    before starting hbase:
    # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
    Usage: HFileCompressionTest hdfs compression
    compression must be one of [ none, gz, lzo ]
    # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
    hdfs://localhost:8020/ lzo
    09/09/29 13:39:55 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
    09/09/29 13:39:55 INFO lzo.LzoCodec: Successfully loaded & initialized
    native-lzo library
    09/09/29 13:39:55 INFO compress.CodecPool: Got brand-new compressor
    OK
    The utility creates a temporary HFile named '.hfile-comp-test' at the
    specified HDFS path and outputs either 'OK' or 'FAILED'.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Oct 5, 2009 at 9:24 pm
    [ https://issues.apache.org/jira/browse/HBASE-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    stack updated HBASE-1875:
    -------------------------

    Resolution: Fixed
    Fix Version/s: 0.21.0
    Hadoop Flags: [Reviewed]
    Status: Resolved (was: Patch Available)

    I also made it so we exit with code 0 if successfully wrote compressed file and != 0 if anything else. Updated wiki compression page to reference this new tool.

    Thanks for the patch elsif. Applied branch and trunk.
    Compression test utility
    ------------------------

    Key: HBASE-1875
    URL: https://issues.apache.org/jira/browse/HBASE-1875
    Project: Hadoop HBase
    Issue Type: New Feature
    Components: util
    Affects Versions: 0.20.0
    Reporter: elsif
    Priority: Trivial
    Fix For: 0.20.1, 0.21.0

    Attachments: 1875-v2.patch, HFileCompressionTest.java


    Attached is a stand alone utility to test compression compatibility
    before starting hbase:
    # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
    Usage: HFileCompressionTest hdfs compression
    compression must be one of [ none, gz, lzo ]
    # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
    hdfs://localhost:8020/ lzo
    09/09/29 13:39:55 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
    09/09/29 13:39:55 INFO lzo.LzoCodec: Successfully loaded & initialized
    native-lzo library
    09/09/29 13:39:55 INFO compress.CodecPool: Got brand-new compressor
    OK
    The utility creates a temporary HFile named '.hfile-comp-test' at the
    specified HDFS path and outputs either 'OK' or 'FAILED'.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categorieshbase, hadoop
postedSep 30, '09 at 3:07p
activeOct 5, '09 at 9:24p
posts6
users1
websitehbase.apache.org

1 user in discussion

stack (JIRA): 6 posts

People

Translate

site design / logo © 2022 Grokbase