FAQ
Hi

Before filling a jira, I would like to check this with you.
It seems that the major compaction behavior changed between 0.90.x and 0.92
(not taking about all the nice stuff you added there...)

Effectively when i trigger a major compaction (via hbase shell or ui) when
other compactions (minor) are running (because i had a lot of storefiles
and a big compactionQueueSize), the major compaction is not executed. While
in 0.90.X major_compact command was executed immediately after any current
compaction.


Mikael.S

Search Discussions

  • Jean-Daniel Cryans at Feb 24, 2012 at 12:06 am
    Can you show us some logs that exhibit this behavior?

    Thx,

    J-D
    On Thu, Feb 23, 2012 at 2:58 PM, Mikael Sitruk wrote:
    Hi

    Before filling a jira, I  would like to check this with you.
    It seems that the major compaction behavior changed between 0.90.x and 0.92
    (not taking about all the nice stuff you added there...)

    Effectively when i trigger a major compaction (via hbase shell or ui) when
    other compactions (minor) are running (because i had a lot of storefiles
    and a big compactionQueueSize), the major compaction is not executed. While
    in 0.90.X major_compact command was executed immediately after any current
    compaction.


    Mikael.S
  • Mikael Sitruk at Feb 24, 2012 at 9:32 am
    I triggered several time major_compaction between 22:30 and the time i
    wrote the original email 00:58
    From the log of one of the severs between those time no major compaction
    was running a major_compaction ended at 22:23 on the same server but this
    is because the number of file was high (after a load phase) and not
    manually triggered.
    On the system major_compaction are disabled.
    From the logs:
    1. 22:23:42 --> 22:23:48 - single major compaction runs
    2. 22:23:48 --> 23:44:27 - minor compaction runs
    3. 23:44:27 --> 23:44:32 - single major compaction runs
    5. 23:44:32 --> 01:13:49 - minor compaction are run
    5. 01:13:49 the cycle of major compaction began to run (heavily working
    only from 01:40:).
    -------------------------------
    *below a grep of compaction (ends, with major and minor to illustrate the
    above)*

    2012-02-23 22:23:48,272 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 1 file(s) in PI of
    TC,00000000000012921999,1329399349811.8dcc952ff57e87c6d82840e84eb870a6.
    into 7400a662b5924d3baf3ac520b1eb7ad9, size=49.0m; total size for store is
    49.0m
    2012-02-23 22:25:38,477 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000010451999,1329399349799.38739d7acb5f10198540bcfa0d4b5aef.
    into 77c43127e38846f78e7ada302d399732, size=540.8m; total size for store is
    2.1g
    2012-02-23 22:27:41,930 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000003951999,1329399349764.f55cfda6067ad1425ba68d20d82c945b.
    into f9c31ac78b3d4feb848b1e8d78975e34, size=532.0m; total size for store is
    2.1g
    2012-02-23 22:29:43,260 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000010971999,1329399349802.15720106584fe32f58287b6174970f31.
    into af2472f30a9f4c558323a3268f98183c, size=517.9m; total size for store is
    2.6g
    2012-02-23 22:30:19,476 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000010061999,1329399349797.27a2f37d5d6ac48c860cca5251be5731.
    into 3ac255c439ab4c518f4a938d542c9220, size=335.4m; total size for store is
    2.1g
    2012-02-23 22:31:15,051 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000008501999,1329399349789.fa0ac64d8ac9050f4acf2b0deb6e2dc7.
    into b8c733a27d02468a82f8a2c4d84db4ff, size=523.0m; total size for store is
    2.1g
    2012-02-23 22:32:33,430 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000004991999,1329399349770.46a2a16d484faacfcda36bc52bb218ba.
    into 5344911081424de69299128412d41573, size=538.0m; total size for store is
    2.2g
    2012-02-23 22:33:28,882 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000004081999,1329399349765.a5ca0a6a0a5561af5b6964e458615493.
    into d15c1e5b089d46beb2c4d01c796367a7, size=520.7m; total size for store is
    2.1g
    2012-02-23 22:34:46,714 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000007591999,1329399349784.711b032969e1ae6cce8e16bb9b
    ...
    ..
    ..
    2012-02-23 23:44:32,215 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 1 file(s) in PI of
    TC,00000000000000051999,1329399349734.e7ba4a35ee8524fbf4e6f9f9ba845206.
    into 2c701bd613314198889d8b3d79b82992, size=40.8m; total size for store is
    40.8m
    2012-02-23 23:45:00,838 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000005251999,1329399349771.50df7d847b5e36be9982b47eb3234ad4.
    into 701d49ca010a41169ca495917b4ce3d4, size=1.3g; total size for store is
    2.1g
    2012-02-23 23:47:12,191 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000002261999,1329399349751.ffb8f2f3d9d5a23aee280359793557ae.
    into d2b8336a36b1451d9a7d38c6bf3e79ee, size=1.2g; total size for store is
    2.1g
    2012-02-23 23:49:28,440 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000011881999,1329399349806.81f2bb0de10974b617521b62b9b85f09.
    into 19208d50fe8543a682e3351c60801eb9, size=1.2g; total size for store is
    2.1g
    2012-02-23 23:52:12,523 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000000051999,1329399349734.e7ba4a35ee8524fbf4e6f9f9ba845206.
    into 639ffb3d8c434363aac0d007b6f78ce9, size=1.5g; total size for store is
    2.1g
    2012-02-23 23:56:14,058 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000005901999,1329399349775.48aa14a1feea4793ec68bfe076570257.
    into 3e2f293a33f14612b8fa1ad570799feb, size=1.6g; total size for store is
    2.1g
    2012-02-23 23:59:08,211 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000010451999,1329399349799.38739d7acb5f10198540bcfa0d4b5aef.
    into dd7f92dd786d4e3ea9cb733cb3a6d2f8, size=1.6g; total size for store is
    2.1g
    ...
    ...
    ...
    ...
    2012-02-24 01:17:15,611 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 6 file(s) in EVENT of
    TC,00000000000012401999,1329399349808.03a3d34ac2c0c7d09619
    8032a151120f. into eff7b5bda7bf4653a9f229c1b40e0e4f, size=1.3g; total size
    for store is 2.1g
    2012-02-24 01:17:51,057 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000000441999,1329399349737.02354040d2650db1d59de0e01b
    0edd1d. into ac1d5e5f1ab746229bad223f56534054, size=329.8m; total size for
    store is 2.2g
    2012-02-24 01:20:40,454 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 5 file(s) in EVENT of
    TC,00000000000012921999,1329399349811.8dcc952ff57e87c6d828
    40e84eb870a6. into ae82b7832eed4bafb9e64ed45574831c, size=978.9m; total
    size for store is 2.5g
    2012-02-24 01:21:13,635 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000007981999,1329399349786.7e26a6440421112a0a15ed78ac
    33b168. into 94aa6f0328834b0896a05cda6a6a9a0f, size=309.1m; total size for
    store is 2.1g
    2012-02-24 01:22:50,578 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000002391999,1329399349752.05a0d235efa8135618628e5124
    dc9507. into 0ca4a3b8221c42f992649431e7182a6d, size=329.6m; total size for
    store is 2.2g
    2012-02-24 01:25:03,055 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 5 file(s) in EVENT of
    TC,00000000000001221999,1329399349743.56d16361d17dce747fb0
    23ab15ded16b. into 47e9f68efb8a475dbce7db2c08b46f9c, size=1.0g; total size
    for store is 2.5g
    2012-02-24 01:25:38,876 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000001091999,1329399349742.5907e62062b4edc83033b4fda0
    fc105f. into d5148c6b4dbc4d3c8339f151b987e51a, size=326.4m; total size for
    store is 2.2g
    2012-02-24 01:26:10,242 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000005381999,1329399349772.15d2f2fb1ad338ab3b5e57077d
    874601. into bacf547a2b4f4627b630a18b88ec8ec6, size=306.5m; total size for
    store is 2.1g
    2012-02-24 01:27:47,690 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000010581999,1329399349800.8144106ed55b1571a720e46569
    af8ea6. into 827d75a8b3144ee3a7e53dae5fdecf3a, size=296.5m; total size for
    store is 2.1g
    2012-02-24 01:28:23,907 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000006681999,1329399349779.9e0dc98e734715e66702419dc2
    1e23ed. into 4a93423a056645d7a6f869f66c76fd0e, size=341.4m; total size for
    store is 2.1g
    2012-02-24 01:29:11,192 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000012661999,1329399349810.76715703e8bf5f3f960564834f
    d9417d. into 3ef0ae62a13a4ed98cd772eeb1e55fd0, size=438.1m; total size for
    store is 2.1g
    2012-02-24 01:29:49,930 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000011101999,1329399349802.a63a43695fdcc0be243cfe3ea6
    263981. into 2a542d20b2ed49b5964777c97f5f2a5b, size=355.2m; total size for
    store is 2.1g
    2012-02-24 01:33:08,747 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 5 file(s) in EVENT of
    TC,00000000000008761999,1329399349791.7c0b5c9e5c98e9e39ae7
    a3afcfaad4d9. into 93fb6ab4de7a4672aceed0e23cc9effc, size=1.2g; total size
    for store is 2.1g
    2012-02-24 01:35:21,194 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 5 file(s) in EVENT of
    TC,00000000000006811999,1329399349780.6051d2a9230460194d41
    f6249d087f6f. into 3adba15b5ae04bd1a18a18afeb7cf107, size=1.2g; total size
    for store is 2.1g
    2012-02-24 01:37:08,338 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000011621999,1329399349805.4d2bb3d00b0fa089163d6a0491dedadb.
    into 26ac704d826a495080e93f07b3c6e24d, size=395.6m; total size for store is
    2.6g
    2012-02-24 01:37:51,454 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000012011999,1329399349807.99d15c956a3897276d0fdeaa33611417.
    into 28aac65938bf493187f84e7019ca126d, size=393.8m; total size for store is
    2.1g
    2012-02-24 01:38:33,930 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000004861999,1329399349769.dce64ca59ca897a34aa6416de4f32dea.
    into 5dda42912eb64701b20639065a7612cb, size=393.6m; total size for store is
    2.1g
    2012-02-24 01:39:23,057 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000010841999,1329399349801.c7f801818067718327858c10b0f3ded2.
    into 39ce298723bc4f66a5bfe1f0729143d4, size=468.0m; total size for store is
    2.2g
    2012-02-24 01:40:16,369 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 4 file(s) in EVENT of
    TC,00000000000001611999,1329399349746.eb6f8b45363b6d3f254d4b6d3dab3617.
    into 7821cb65510840afaff12c1e008b28c4, size=491.1m; total size for store is
    2.1g
    2012-02-24 01:42:18,657 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 4 file(s) in EVENT of
    TC,00000000000002651999,1329399349754.5c75673b41370540e3c758e937181651.
    into a7a12651ff3d414bb722945157686c11, size=505.1m; total size for store is
    2.1g
    2012-02-24 01:43:10,404 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 4 file(s) in EVENT of
    TC,00000000000004211999,1329399349766.041d7f3c86c4d5657c6eeded7584fad8.
    into 7b8d8075a4d04873a50e517431935e03, size=482.2m; total size for store is
    2.1g
    2012-02-24 01:44:03,388 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 4 file(s) in EVENT of
    TC,00000000000011751999,1329399349805.21ac7408f58fa1b022b6a38b2f197915.
    into f2906f916c16442a89161fdf9551bc0c, size=507.9m; total size for store is
    2.1g
    2012-02-24 01:44:59,108 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 4 file(s) in EVENT of
    TC,00000000000010061999,1329399349797.27a2f37d5d6ac48c860cca5251be5731.
    into 60defcf94fce43488fffc619c509a9d3, size=524.2m; total size for store is
    2.1g
    2012-02-24 01:46:17,683 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 4 file(s) in EVENT of
    TC,00000000000011491999,1329399349804.c613c26eff42730acd1f
    ..
    ..
    ..
    ..


    Mikael.S

    On Fri, Feb 24, 2012 at 2:05 AM, Jean-Daniel Cryans wrote:

    Can you show us some logs that exhibit this behavior?

    Thx,

    J-D
    On Thu, Feb 23, 2012 at 2:58 PM, Mikael Sitruk wrote:
    Hi

    Before filling a jira, I would like to check this with you.
    It seems that the major compaction behavior changed between 0.90.x and 0.92
    (not taking about all the nice stuff you added there...)

    Effectively when i trigger a major compaction (via hbase shell or ui) when
    other compactions (minor) are running (because i had a lot of storefiles
    and a big compactionQueueSize), the major compaction is not executed. While
    in 0.90.X major_compact command was executed immediately after any current
    compaction.


    Mikael.S


    --
    Mikael.S
  • Nicolas Spiegelberg at Feb 24, 2012 at 4:36 pm
    In 0.92, we can handle multiple compactions but we wait on major compact
    if minor compactions are ongoing for the same store. It should be
    enqueued and work after the minor compaction finishes, it's just not
    immediately enqueued. See HBASE-5330 for some discussion on this. There
    should be a log message in the regionserver to verify the the
    MAJOR_COMPACT request was received. The existing contract on major
    compaction requests are that they are asynchronous & the request has
    in-memory persistence.
    On 2/24/12 4:32 AM, "Mikael Sitruk" wrote:

    I triggered several time major_compaction between 22:30 and the time i
    wrote the original email 00:58
    From the log of one of the severs between those time no major compaction
    was running a major_compaction ended at 22:23 on the same server but this
    is because the number of file was high (after a load phase) and not
    manually triggered.
    On the system major_compaction are disabled.

    From the logs:
    1. 22:23:42 --> 22:23:48 - single major compaction runs
    2. 22:23:48 --> 23:44:27 - minor compaction runs
    3. 23:44:27 --> 23:44:32 - single major compaction runs
    5. 23:44:32 --> 01:13:49 - minor compaction are run
    5. 01:13:49 the cycle of major compaction began to run (heavily working
    only from 01:40:).
    -------------------------------
    *below a grep of compaction (ends, with major and minor to illustrate the
    above)*

    2012-02-23 22:23:48,272 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 1 file(s) in PI of
    TC,00000000000012921999,1329399349811.8dcc952ff57e87c6d82840e84eb870a6.
    into 7400a662b5924d3baf3ac520b1eb7ad9, size=49.0m; total size for store is
    49.0m
    2012-02-23 22:25:38,477 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000010451999,1329399349799.38739d7acb5f10198540bcfa0d4b5aef.
    into 77c43127e38846f78e7ada302d399732, size=540.8m; total size for store
    is
    2.1g
    2012-02-23 22:27:41,930 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000003951999,1329399349764.f55cfda6067ad1425ba68d20d82c945b.
    into f9c31ac78b3d4feb848b1e8d78975e34, size=532.0m; total size for store
    is
    2.1g
    2012-02-23 22:29:43,260 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000010971999,1329399349802.15720106584fe32f58287b6174970f31.
    into af2472f30a9f4c558323a3268f98183c, size=517.9m; total size for store
    is
    2.6g
    2012-02-23 22:30:19,476 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000010061999,1329399349797.27a2f37d5d6ac48c860cca5251be5731.
    into 3ac255c439ab4c518f4a938d542c9220, size=335.4m; total size for store
    is
    2.1g
    2012-02-23 22:31:15,051 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000008501999,1329399349789.fa0ac64d8ac9050f4acf2b0deb6e2dc7.
    into b8c733a27d02468a82f8a2c4d84db4ff, size=523.0m; total size for store
    is
    2.1g
    2012-02-23 22:32:33,430 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000004991999,1329399349770.46a2a16d484faacfcda36bc52bb218ba.
    into 5344911081424de69299128412d41573, size=538.0m; total size for store
    is
    2.2g
    2012-02-23 22:33:28,882 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000004081999,1329399349765.a5ca0a6a0a5561af5b6964e458615493.
    into d15c1e5b089d46beb2c4d01c796367a7, size=520.7m; total size for store
    is
    2.1g
    2012-02-23 22:34:46,714 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000007591999,1329399349784.711b032969e1ae6cce8e16bb9b
    ...
    ..
    ..
    2012-02-23 23:44:32,215 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 1 file(s) in PI of
    TC,00000000000000051999,1329399349734.e7ba4a35ee8524fbf4e6f9f9ba845206.
    into 2c701bd613314198889d8b3d79b82992, size=40.8m; total size for store is
    40.8m
    2012-02-23 23:45:00,838 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000005251999,1329399349771.50df7d847b5e36be9982b47eb3234ad4.
    into 701d49ca010a41169ca495917b4ce3d4, size=1.3g; total size for store is
    2.1g
    2012-02-23 23:47:12,191 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000002261999,1329399349751.ffb8f2f3d9d5a23aee280359793557ae.
    into d2b8336a36b1451d9a7d38c6bf3e79ee, size=1.2g; total size for store is
    2.1g
    2012-02-23 23:49:28,440 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000011881999,1329399349806.81f2bb0de10974b617521b62b9b85f09.
    into 19208d50fe8543a682e3351c60801eb9, size=1.2g; total size for store is
    2.1g
    2012-02-23 23:52:12,523 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000000051999,1329399349734.e7ba4a35ee8524fbf4e6f9f9ba845206.
    into 639ffb3d8c434363aac0d007b6f78ce9, size=1.5g; total size for store is
    2.1g
    2012-02-23 23:56:14,058 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000005901999,1329399349775.48aa14a1feea4793ec68bfe076570257.
    into 3e2f293a33f14612b8fa1ad570799feb, size=1.6g; total size for store is
    2.1g
    2012-02-23 23:59:08,211 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000010451999,1329399349799.38739d7acb5f10198540bcfa0d4b5aef.
    into dd7f92dd786d4e3ea9cb733cb3a6d2f8, size=1.6g; total size for store is
    2.1g
    ...
    ...
    ...
    ...
    2012-02-24 01:17:15,611 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 6 file(s) in EVENT of
    TC,00000000000012401999,1329399349808.03a3d34ac2c0c7d09619
    8032a151120f. into eff7b5bda7bf4653a9f229c1b40e0e4f, size=1.3g; total size
    for store is 2.1g
    2012-02-24 01:17:51,057 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000000441999,1329399349737.02354040d2650db1d59de0e01b
    0edd1d. into ac1d5e5f1ab746229bad223f56534054, size=329.8m; total size for
    store is 2.2g
    2012-02-24 01:20:40,454 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 5 file(s) in EVENT of
    TC,00000000000012921999,1329399349811.8dcc952ff57e87c6d828
    40e84eb870a6. into ae82b7832eed4bafb9e64ed45574831c, size=978.9m; total
    size for store is 2.5g
    2012-02-24 01:21:13,635 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000007981999,1329399349786.7e26a6440421112a0a15ed78ac
    33b168. into 94aa6f0328834b0896a05cda6a6a9a0f, size=309.1m; total size for
    store is 2.1g
    2012-02-24 01:22:50,578 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000002391999,1329399349752.05a0d235efa8135618628e5124
    dc9507. into 0ca4a3b8221c42f992649431e7182a6d, size=329.6m; total size for
    store is 2.2g
    2012-02-24 01:25:03,055 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 5 file(s) in EVENT of
    TC,00000000000001221999,1329399349743.56d16361d17dce747fb0
    23ab15ded16b. into 47e9f68efb8a475dbce7db2c08b46f9c, size=1.0g; total size
    for store is 2.5g
    2012-02-24 01:25:38,876 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000001091999,1329399349742.5907e62062b4edc83033b4fda0
    fc105f. into d5148c6b4dbc4d3c8339f151b987e51a, size=326.4m; total size for
    store is 2.2g
    2012-02-24 01:26:10,242 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000005381999,1329399349772.15d2f2fb1ad338ab3b5e57077d
    874601. into bacf547a2b4f4627b630a18b88ec8ec6, size=306.5m; total size for
    store is 2.1g
    2012-02-24 01:27:47,690 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000010581999,1329399349800.8144106ed55b1571a720e46569
    af8ea6. into 827d75a8b3144ee3a7e53dae5fdecf3a, size=296.5m; total size for
    store is 2.1g
    2012-02-24 01:28:23,907 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000006681999,1329399349779.9e0dc98e734715e66702419dc2
    1e23ed. into 4a93423a056645d7a6f869f66c76fd0e, size=341.4m; total size for
    store is 2.1g
    2012-02-24 01:29:11,192 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000012661999,1329399349810.76715703e8bf5f3f960564834f
    d9417d. into 3ef0ae62a13a4ed98cd772eeb1e55fd0, size=438.1m; total size for
    store is 2.1g
    2012-02-24 01:29:49,930 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000011101999,1329399349802.a63a43695fdcc0be243cfe3ea6
    263981. into 2a542d20b2ed49b5964777c97f5f2a5b, size=355.2m; total size for
    store is 2.1g
    2012-02-24 01:33:08,747 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 5 file(s) in EVENT of
    TC,00000000000008761999,1329399349791.7c0b5c9e5c98e9e39ae7
    a3afcfaad4d9. into 93fb6ab4de7a4672aceed0e23cc9effc, size=1.2g; total size
    for store is 2.1g
    2012-02-24 01:35:21,194 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 5 file(s) in EVENT of
    TC,00000000000006811999,1329399349780.6051d2a9230460194d41
    f6249d087f6f. into 3adba15b5ae04bd1a18a18afeb7cf107, size=1.2g; total size
    for store is 2.1g
    2012-02-24 01:37:08,338 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000011621999,1329399349805.4d2bb3d00b0fa089163d6a0491dedadb.
    into 26ac704d826a495080e93f07b3c6e24d, size=395.6m; total size for store
    is
    2.6g
    2012-02-24 01:37:51,454 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000012011999,1329399349807.99d15c956a3897276d0fdeaa33611417.
    into 28aac65938bf493187f84e7019ca126d, size=393.8m; total size for store
    is
    2.1g
    2012-02-24 01:38:33,930 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000004861999,1329399349769.dce64ca59ca897a34aa6416de4f32dea.
    into 5dda42912eb64701b20639065a7612cb, size=393.6m; total size for store
    is
    2.1g
    2012-02-24 01:39:23,057 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000010841999,1329399349801.c7f801818067718327858c10b0f3ded2.
    into 39ce298723bc4f66a5bfe1f0729143d4, size=468.0m; total size for store
    is
    2.2g
    2012-02-24 01:40:16,369 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 4 file(s) in EVENT of
    TC,00000000000001611999,1329399349746.eb6f8b45363b6d3f254d4b6d3dab3617.
    into 7821cb65510840afaff12c1e008b28c4, size=491.1m; total size for store
    is
    2.1g
    2012-02-24 01:42:18,657 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 4 file(s) in EVENT of
    TC,00000000000002651999,1329399349754.5c75673b41370540e3c758e937181651.
    into a7a12651ff3d414bb722945157686c11, size=505.1m; total size for store
    is
    2.1g
    2012-02-24 01:43:10,404 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 4 file(s) in EVENT of
    TC,00000000000004211999,1329399349766.041d7f3c86c4d5657c6eeded7584fad8.
    into 7b8d8075a4d04873a50e517431935e03, size=482.2m; total size for store
    is
    2.1g
    2012-02-24 01:44:03,388 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 4 file(s) in EVENT of
    TC,00000000000011751999,1329399349805.21ac7408f58fa1b022b6a38b2f197915.
    into f2906f916c16442a89161fdf9551bc0c, size=507.9m; total size for store
    is
    2.1g
    2012-02-24 01:44:59,108 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 4 file(s) in EVENT of
    TC,00000000000010061999,1329399349797.27a2f37d5d6ac48c860cca5251be5731.
    into 60defcf94fce43488fffc619c509a9d3, size=524.2m; total size for store
    is
    2.1g
    2012-02-24 01:46:17,683 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 4 file(s) in EVENT of
    TC,00000000000011491999,1329399349804.c613c26eff42730acd1f
    ..
    ..
    ..
    ..


    Mikael.S


    On Fri, Feb 24, 2012 at 2:05 AM, Jean-Daniel Cryans
    wrote:
    Can you show us some logs that exhibit this behavior?

    Thx,

    J-D

    On Thu, Feb 23, 2012 at 2:58 PM, Mikael Sitruk <mikael.sitruk@gmail.com>
    wrote:
    Hi

    Before filling a jira, I would like to check this with you.
    It seems that the major compaction behavior changed between 0.90.x and 0.92
    (not taking about all the nice stuff you added there...)

    Effectively when i trigger a major compaction (via hbase shell or ui) when
    other compactions (minor) are running (because i had a lot of
    storefiles
    and a big compactionQueueSize), the major compaction is not executed. While
    in 0.90.X major_compact command was executed immediately after any current
    compaction.


    Mikael.S


    --
    Mikael.S
  • Mikael Sitruk at Feb 24, 2012 at 10:05 pm
    Hi Nicolas,

    Well
    1. I don't see such message, on the second hand I'm not using "debug"
    severity in the log (unless this is not a debug message)
    2. If major compaction are just entering the compaction queue it seems to
    me quite problematic; especially when you want to control the impact of
    compaction!
    I had already a big queue of compaction request (minor) due to an heavy
    load, so it waited at least 2 hours between the time I triggered the major
    compaction till it begun the cycle.
    Such behavior doesn't seem to be adequate for systems that have restricted
    window time frame for internal task, don't you think? (because if you are
    in the burst of minor compaction you will may miss your window)
    Should the major compaction request remove all the minor compaction queued
    and be the first?

    Mikael.S
    On Fri, Feb 24, 2012 at 6:36 PM, Nicolas Spiegelberg wrote:

    In 0.92, we can handle multiple compactions but we wait on major compact
    if minor compactions are ongoing for the same store. It should be
    enqueued and work after the minor compaction finishes, it's just not
    immediately enqueued. See HBASE-5330 for some discussion on this. There
    should be a log message in the regionserver to verify the the
    MAJOR_COMPACT request was received . The existing contract on major
    compaction requests are that they are asynchronous & the request has
    in-memory persistence.
    On 2/24/12 4:32 AM, "Mikael Sitruk" wrote:

    I triggered several time major_compaction between 22:30 and the time i
    wrote the original email 00:58
    From the log of one of the severs between those time no major compaction
    was running a major_compaction ended at 22:23 on the same server but this
    is because the number of file was high (after a load phase) and not
    manually triggered.
    On the system major_compaction are disabled.

    From the logs:
    1. 22:23:42 --> 22:23:48 - single major compaction runs
    2. 22:23:48 --> 23:44:27 - minor compaction runs
    3. 23:44:27 --> 23:44:32 - single major compaction runs
    5. 23:44:32 --> 01:13:49 - minor compaction are run
    5. 01:13:49 the cycle of major compaction began to run (heavily working
    only from 01:40:).
    -------------------------------
    *below a grep of compaction (ends, with major and minor to illustrate the
    above)*

    2012-02-23 22:23:48,272 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 1 file(s) in PI of
    TC,00000000000012921999,1329399349811.8dcc952ff57e87c6d82840e84eb870a6.
    into 7400a662b5924d3baf3ac520b1eb7ad9, size=49.0m; total size for store is
    49.0m
    2012-02-23 22:25:38,477 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000010451999,1329399349799.38739d7acb5f10198540bcfa0d4b5aef.
    into 77c43127e38846f78e7ada302d399732, size=540.8m; total size for store
    is
    2.1g
    2012-02-23 22:27:41,930 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000003951999,1329399349764.f55cfda6067ad1425ba68d20d82c945b.
    into f9c31ac78b3d4feb848b1e8d78975e34, size=532.0m; total size for store
    is
    2.1g
    2012-02-23 22:29:43,260 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000010971999,1329399349802.15720106584fe32f58287b6174970f31.
    into af2472f30a9f4c558323a3268f98183c, size=517.9m; total size for store
    is
    2.6g
    2012-02-23 22:30:19,476 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000010061999,1329399349797.27a2f37d5d6ac48c860cca5251be5731.
    into 3ac255c439ab4c518f4a938d542c9220, size=335.4m; total size for store
    is
    2.1g
    2012-02-23 22:31:15,051 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000008501999,1329399349789.fa0ac64d8ac9050f4acf2b0deb6e2dc7.
    into b8c733a27d02468a82f8a2c4d84db4ff, size=523.0m; total size for store
    is
    2.1g
    2012-02-23 22:32:33,430 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000004991999,1329399349770.46a2a16d484faacfcda36bc52bb218ba.
    into 5344911081424de69299128412d41573, size=538.0m; total size for store
    is
    2.2g
    2012-02-23 22:33:28,882 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000004081999,1329399349765.a5ca0a6a0a5561af5b6964e458615493.
    into d15c1e5b089d46beb2c4d01c796367a7, size=520.7m; total size for store
    is
    2.1g
    2012-02-23 22:34:46,714 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 3 file(s) in EVENT of
    TC,00000000000007591999,1329399349784.711b032969e1ae6cce8e16bb9b
    ...
    ..
    ..
    2012-02-23 23:44:32,215 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 1 file(s) in PI of
    TC,00000000000000051999,1329399349734.e7ba4a35ee8524fbf4e6f9f9ba845206.
    into 2c701bd613314198889d8b3d79b82992, size=40.8m; total size for store is
    40.8m
    2012-02-23 23:45:00,838 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000005251999,1329399349771.50df7d847b5e36be9982b47eb3234ad4.
    into 701d49ca010a41169ca495917b4ce3d4, size=1.3g; total size for store is
    2.1g
    2012-02-23 23:47:12,191 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000002261999,1329399349751.ffb8f2f3d9d5a23aee280359793557ae.
    into d2b8336a36b1451d9a7d38c6bf3e79ee, size=1.2g; total size for store is
    2.1g
    2012-02-23 23:49:28,440 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000011881999,1329399349806.81f2bb0de10974b617521b62b9b85f09.
    into 19208d50fe8543a682e3351c60801eb9, size=1.2g; total size for store is
    2.1g
    2012-02-23 23:52:12,523 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000000051999,1329399349734.e7ba4a35ee8524fbf4e6f9f9ba845206.
    into 639ffb3d8c434363aac0d007b6f78ce9, size=1.5g; total size for store is
    2.1g
    2012-02-23 23:56:14,058 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000005901999,1329399349775.48aa14a1feea4793ec68bfe076570257.
    into 3e2f293a33f14612b8fa1ad570799feb, size=1.6g; total size for store is
    2.1g
    2012-02-23 23:59:08,211 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000010451999,1329399349799.38739d7acb5f10198540bcfa0d4b5aef.
    into dd7f92dd786d4e3ea9cb733cb3a6d2f8, size=1.6g; total size for store is
    2.1g
    ...
    ...
    ...
    ...
    2012-02-24 01:17:15,611 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 6 file(s) in EVENT of
    TC,00000000000012401999,1329399349808.03a3d34ac2c0c7d09619
    8032a151120f. into eff7b5bda7bf4653a9f229c1b40e0e4f, size=1.3g; total size
    for store is 2.1g
    2012-02-24 01:17:51,057 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000000441999,1329399349737.02354040d2650db1d59de0e01b
    0edd1d. into ac1d5e5f1ab746229bad223f56534054, size=329.8m; total size for
    store is 2.2g
    2012-02-24 01:20:40,454 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 5 file(s) in EVENT of
    TC,00000000000012921999,1329399349811.8dcc952ff57e87c6d828
    40e84eb870a6. into ae82b7832eed4bafb9e64ed45574831c, size=978.9m; total
    size for store is 2.5g
    2012-02-24 01:21:13,635 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000007981999,1329399349786.7e26a6440421112a0a15ed78ac
    33b168. into 94aa6f0328834b0896a05cda6a6a9a0f, size=309.1m; total size for
    store is 2.1g
    2012-02-24 01:22:50,578 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000002391999,1329399349752.05a0d235efa8135618628e5124
    dc9507. into 0ca4a3b8221c42f992649431e7182a6d, size=329.6m; total size for
    store is 2.2g
    2012-02-24 01:25:03,055 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 5 file(s) in EVENT of
    TC,00000000000001221999,1329399349743.56d16361d17dce747fb0
    23ab15ded16b. into 47e9f68efb8a475dbce7db2c08b46f9c, size=1.0g; total size
    for store is 2.5g
    2012-02-24 01:25:38,876 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000001091999,1329399349742.5907e62062b4edc83033b4fda0
    fc105f. into d5148c6b4dbc4d3c8339f151b987e51a, size=326.4m; total size for
    store is 2.2g
    2012-02-24 01:26:10,242 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000005381999,1329399349772.15d2f2fb1ad338ab3b5e57077d
    874601. into bacf547a2b4f4627b630a18b88ec8ec6, size=306.5m; total size for
    store is 2.1g
    2012-02-24 01:27:47,690 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000010581999,1329399349800.8144106ed55b1571a720e46569
    af8ea6. into 827d75a8b3144ee3a7e53dae5fdecf3a, size=296.5m; total size for
    store is 2.1g
    2012-02-24 01:28:23,907 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000006681999,1329399349779.9e0dc98e734715e66702419dc2
    1e23ed. into 4a93423a056645d7a6f869f66c76fd0e, size=341.4m; total size for
    store is 2.1g
    2012-02-24 01:29:11,192 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000012661999,1329399349810.76715703e8bf5f3f960564834f
    d9417d. into 3ef0ae62a13a4ed98cd772eeb1e55fd0, size=438.1m; total size for
    store is 2.1g
    2012-02-24 01:29:49,930 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000011101999,1329399349802.a63a43695fdcc0be243cfe3ea6
    263981. into 2a542d20b2ed49b5964777c97f5f2a5b, size=355.2m; total size for
    store is 2.1g
    2012-02-24 01:33:08,747 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 5 file(s) in EVENT of
    TC,00000000000008761999,1329399349791.7c0b5c9e5c98e9e39ae7
    a3afcfaad4d9. into 93fb6ab4de7a4672aceed0e23cc9effc, size=1.2g; total size
    for store is 2.1g
    2012-02-24 01:35:21,194 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 5 file(s) in EVENT of
    TC,00000000000006811999,1329399349780.6051d2a9230460194d41
    f6249d087f6f. into 3adba15b5ae04bd1a18a18afeb7cf107, size=1.2g; total size
    for store is 2.1g
    2012-02-24 01:37:08,338 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000011621999,1329399349805.4d2bb3d00b0fa089163d6a0491dedadb.
    into 26ac704d826a495080e93f07b3c6e24d, size=395.6m; total size for store
    is
    2.6g
    2012-02-24 01:37:51,454 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000012011999,1329399349807.99d15c956a3897276d0fdeaa33611417.
    into 28aac65938bf493187f84e7019ca126d, size=393.8m; total size for store
    is
    2.1g
    2012-02-24 01:38:33,930 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000004861999,1329399349769.dce64ca59ca897a34aa6416de4f32dea.
    into 5dda42912eb64701b20639065a7612cb, size=393.6m; total size for store
    is
    2.1g
    2012-02-24 01:39:23,057 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed compaction of 4 file(s) in EVENT of
    TC,00000000000010841999,1329399349801.c7f801818067718327858c10b0f3ded2.
    into 39ce298723bc4f66a5bfe1f0729143d4, size=468.0m; total size for store
    is
    2.2g
    2012-02-24 01:40:16,369 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 4 file(s) in EVENT of
    TC,00000000000001611999,1329399349746.eb6f8b45363b6d3f254d4b6d3dab3617.
    into 7821cb65510840afaff12c1e008b28c4, size=491.1m; total size for store
    is
    2.1g
    2012-02-24 01:42:18,657 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 4 file(s) in EVENT of
    TC,00000000000002651999,1329399349754.5c75673b41370540e3c758e937181651.
    into a7a12651ff3d414bb722945157686c11, size=505.1m; total size for store
    is
    2.1g
    2012-02-24 01:43:10,404 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 4 file(s) in EVENT of
    TC,00000000000004211999,1329399349766.041d7f3c86c4d5657c6eeded7584fad8.
    into 7b8d8075a4d04873a50e517431935e03, size=482.2m; total size for store
    is
    2.1g
    2012-02-24 01:44:03,388 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 4 file(s) in EVENT of
    TC,00000000000011751999,1329399349805.21ac7408f58fa1b022b6a38b2f197915.
    into f2906f916c16442a89161fdf9551bc0c, size=507.9m; total size for store
    is
    2.1g
    2012-02-24 01:44:59,108 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 4 file(s) in EVENT of
    TC,00000000000010061999,1329399349797.27a2f37d5d6ac48c860cca5251be5731.
    into 60defcf94fce43488fffc619c509a9d3, size=524.2m; total size for store
    is
    2.1g
    2012-02-24 01:46:17,683 INFO org.apache.hadoop.hbase.regionserver.Store:
    Completed major compaction of 4 file(s) in EVENT of
    TC,00000000000011491999,1329399349804.c613c26eff42730acd1f
    ..
    ..
    ..
    ..


    Mikael.S


    On Fri, Feb 24, 2012 at 2:05 AM, Jean-Daniel Cryans
    wrote:
    Can you show us some logs that exhibit this behavior?

    Thx,

    J-D

    On Thu, Feb 23, 2012 at 2:58 PM, Mikael Sitruk <mikael.sitruk@gmail.com
    wrote:
    Hi

    Before filling a jira, I would like to check this with you.
    It seems that the major compaction behavior changed between 0.90.x and 0.92
    (not taking about all the nice stuff you added there...)

    Effectively when i trigger a major compaction (via hbase shell or ui) when
    other compactions (minor) are running (because i had a lot of
    storefiles
    and a big compactionQueueSize), the major compaction is not executed. While
    in 0.90.X major_compact command was executed immediately after any current
    compaction.


    Mikael.S


    --
    Mikael.S

    --
    Mikael.S

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categorieshbase, hadoop
postedFeb 23, '12 at 10:59p
activeFeb 24, '12 at 10:05p
posts5
users3
websitehbase.apache.org

People

Translate

site design / logo © 2022 Grokbase