Grokbase Groups HBase dev June 2009
FAQ
Benchmark Tool for 0.20 API
---------------------------

Key: HBASE-1501
URL: https://issues.apache.org/jira/browse/HBASE-1501
Project: Hadoop HBase
Issue Type: Test
Affects Versions: 0.20.0
Reporter: Jonathan Gray
Assignee: Jonathan Gray
Priority: Trivial


I wrote a set of classes to aid in benchmarking puts, gets, and scans for various table schemas and patterns.

We can turn this in to something as part of contrib down the road, but for now this is just an internal tool I'm sharing to help debug.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Jonathan Gray (JIRA) at Jun 9, 2009 at 12:26 am
    [ https://issues.apache.org/jira/browse/HBASE-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jonathan Gray updated HBASE-1501:
    ---------------------------------

    Attachment: HBench-v1.tar.gz

    Contains 5 assorted tests, each with timings for sequential inserts, random reads, and full scans.

    There are a couple static finals in HBench.java that define how long to sleep between calls to HBA flush and major compact (after puts, before gets/scans). There is also a boolean to enable/disable that from happening. Right now I believe it is disabled.

    When enabled, you will sleep the specified amount of time and then after the HBA calls and sleeps the program holds for stdin input (return) before continuing (if you want to watch the logs and ensure things are over).

    To uncover some of the compaction weirdness and halfhfile stuff, play with enabling and disabling the flush/compact boolean and the sleeping. With no sleep and no manual flushes, I have issues reading from any of the tables which have enough inserts to cause a split. Also, calling the flush/compact and continuing before it's done does some interesting things as well.

    Specifically, the avgFat() and wide() cases have both given me problems.
    Benchmark Tool for 0.20 API
    ---------------------------

    Key: HBASE-1501
    URL: https://issues.apache.org/jira/browse/HBASE-1501
    Project: Hadoop HBase
    Issue Type: Test
    Affects Versions: 0.20.0
    Reporter: Jonathan Gray
    Assignee: Jonathan Gray
    Priority: Trivial
    Attachments: HBench-v1.tar.gz


    I wrote a set of classes to aid in benchmarking puts, gets, and scans for various table schemas and patterns.
    We can turn this in to something as part of contrib down the road, but for now this is just an internal tool I'm sharing to help debug.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jonathan Gray (JIRA) at Jun 9, 2009 at 5:26 am
    [ https://issues.apache.org/jira/browse/HBASE-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jonathan Gray updated HBASE-1501:
    ---------------------------------

    Attachment: HBench-v2.tar.gz

    Extracts into subdir HBench now.

    Adds Millions.java which is a simple test for a table with millions of columns in each row. I don't have a full cluster of trunk available to me tonight so I haven't even tested this yet, though I have done million column tests on earlier 1304s with success.
    Benchmark Tool for 0.20 API
    ---------------------------

    Key: HBASE-1501
    URL: https://issues.apache.org/jira/browse/HBASE-1501
    Project: Hadoop HBase
    Issue Type: Test
    Affects Versions: 0.20.0
    Reporter: Jonathan Gray
    Assignee: Jonathan Gray
    Priority: Trivial
    Attachments: HBench-v1.tar.gz, HBench-v2.tar.gz


    I wrote a set of classes to aid in benchmarking puts, gets, and scans for various table schemas and patterns.
    We can turn this in to something as part of contrib down the road, but for now this is just an internal tool I'm sharing to help debug.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Andrew Purtell (JIRA) at Jun 9, 2009 at 6:00 pm
    [ https://issues.apache.org/jira/browse/HBASE-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717768#action_12717768 ]

    Andrew Purtell commented on HBASE-1501:
    ---------------------------------------

    Thanks Jon. I'll add this to my local testing repertoire.
    Benchmark Tool for 0.20 API
    ---------------------------

    Key: HBASE-1501
    URL: https://issues.apache.org/jira/browse/HBASE-1501
    Project: Hadoop HBase
    Issue Type: Test
    Affects Versions: 0.20.0
    Reporter: Jonathan Gray
    Assignee: Jonathan Gray
    Priority: Trivial
    Attachments: HBench-v1.tar.gz, HBench-v2.tar.gz


    I wrote a set of classes to aid in benchmarking puts, gets, and scans for various table schemas and patterns.
    We can turn this in to something as part of contrib down the road, but for now this is just an internal tool I'm sharing to help debug.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categorieshbase, hadoop
postedJun 9, '09 at 12:12a
activeJun 9, '09 at 6:00p
posts4
users1
websitehbase.apache.org

1 user in discussion

Andrew Purtell (JIRA): 4 posts

People

Translate

site design / logo © 2022 Grokbase