Grokbase Groups HBase dev August 2009
FAQ
[Regression] Cannot save timestamp in the future
------------------------------------------------

Key: HBASE-1792
URL: https://issues.apache.org/jira/browse/HBASE-1792
Project: Hadoop HBase
Issue Type: Bug
Affects Versions: 0.20.0
Reporter: Jean-Daniel Cryans


0.20, compared to previous versions, doesn't let you save with a timestamp in the future and will set it to current time without telling you. This is really bad for users upgrading to 0.20 that were using those timestamps.

Example:

hbase(main):004:0> put 'testtable', 'r1', 'f1:c1', 'val', 5373965335336911168
0 row(s) in 0.0070 seconds
hbase(main):005:0> scan 'testtable'
ROW COLUMN+CELL
r1 column=f1:c1, timestamp=1251223892010, value=val
1 row(s) in 0.0380 seconds

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • stack (JIRA) at Aug 25, 2009 at 7:03 pm
    [ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747575#action_12747575 ]

    stack commented on HBASE-1792:
    ------------------------------

    Yeah, I suppose old days we'd leave the ts alone and only change it to current time if it was HConstants.LATEST_TIMESTAMP (IIRC).
    [Regression] Cannot save timestamp in the future
    ------------------------------------------------

    Key: HBASE-1792
    URL: https://issues.apache.org/jira/browse/HBASE-1792
    Project: Hadoop HBase
    Issue Type: Bug
    Affects Versions: 0.20.0
    Reporter: Jean-Daniel Cryans

    0.20, compared to previous versions, doesn't let you save with a timestamp in the future and will set it to current time without telling you. This is really bad for users upgrading to 0.20 that were using those timestamps.
    Example:
    hbase(main):004:0> put 'testtable', 'r1', 'f1:c1', 'val', 5373965335336911168
    0 row(s) in 0.0070 seconds
    hbase(main):005:0> scan 'testtable'
    ROW COLUMN+CELL
    r1 column=f1:c1, timestamp=1251223892010, value=val
    1 row(s) in 0.0380 seconds
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jonathan Gray (JIRA) at Aug 25, 2009 at 7:07 pm
    [ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747581#action_12747581 ]

    Jonathan Gray commented on HBASE-1792:
    --------------------------------------

    There was a number of long discussions related to this. Unfortunately, I forgot where we ended up.

    We tried to keep certain axioms, like "deletes only apply to older storefiles", but manually setting of stamps (future or past) breaks that. That means, if you modify stamps manually, your Gets can have indeterminate behavior.

    I'd be more than happy to remove the strictness here, and allow anything < Long.MAX_VALUE (if equal, rewrite as now()), with a warning in Put that manually setting of stamp can cause indeterminate behavior, especially in Gets.

    In any case, I'd like to see a unit test that verifies the behavior that we do have. There is already a bit of it in o.a.h.h.c.TestClient, but there should be more, perhaps another method or part of testVersions(). It should do insertions with past, present, future stamps, in all different orders, with lots of flushing in between (all issues with stamps / indeterminate behavior are related to multiple storefiles + memstore interaction).

    If we can create a new test, with comments explaining what the behavior is, and all prior tests still pass, then I'll be +1 on allowing future stamps.

    Stack, Erik, any recollection on why it was left this way?

    This should also not have an impact on META where determinate behavior of Gets is quite important.
    [Regression] Cannot save timestamp in the future
    ------------------------------------------------

    Key: HBASE-1792
    URL: https://issues.apache.org/jira/browse/HBASE-1792
    Project: Hadoop HBase
    Issue Type: Bug
    Affects Versions: 0.20.0
    Reporter: Jean-Daniel Cryans

    0.20, compared to previous versions, doesn't let you save with a timestamp in the future and will set it to current time without telling you. This is really bad for users upgrading to 0.20 that were using those timestamps.
    Example:
    hbase(main):004:0> put 'testtable', 'r1', 'f1:c1', 'val', 5373965335336911168
    0 row(s) in 0.0070 seconds
    hbase(main):005:0> scan 'testtable'
    ROW COLUMN+CELL
    r1 column=f1:c1, timestamp=1251223892010, value=val
    1 row(s) in 0.0380 seconds
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jean-Daniel Cryans (JIRA) at Aug 25, 2009 at 8:13 pm
    [ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747618#action_12747618 ]

    Jean-Daniel Cryans commented on HBASE-1792:
    -------------------------------------------

    Just so I don't forget it, the timestamp is replaced silently here in HRegion.updateKeys:

    {code}
    /**
    * Checks if any stamps are > now. If so, sets them to now.
    * <p>
    * This acts to be prevent users from inserting future stamps as well as
    * to replace LATEST_TIMESTAMP with now.
    * @param keys
    * @param now
    * @return <code>true</code> when updating the time stamp completed.
    */
    private boolean updateKeys(List<KeyValue> keys, byte [] now) {
    if(keys == null || keys.isEmpty()) {
    return false;
    }
    for(KeyValue key : keys) {
    key.updateLatestStamp(now);
    }
    return true;
    }
    {code}
    [Regression] Cannot save timestamp in the future
    ------------------------------------------------

    Key: HBASE-1792
    URL: https://issues.apache.org/jira/browse/HBASE-1792
    Project: Hadoop HBase
    Issue Type: Bug
    Affects Versions: 0.20.0
    Reporter: Jean-Daniel Cryans

    0.20, compared to previous versions, doesn't let you save with a timestamp in the future and will set it to current time without telling you. This is really bad for users upgrading to 0.20 that were using those timestamps.
    Example:
    hbase(main):004:0> put 'testtable', 'r1', 'f1:c1', 'val', 5373965335336911168
    0 row(s) in 0.0070 seconds
    hbase(main):005:0> scan 'testtable'
    ROW COLUMN+CELL
    r1 column=f1:c1, timestamp=1251223892010, value=val
    1 row(s) in 0.0380 seconds
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • ryan rawson (JIRA) at Aug 25, 2009 at 8:29 pm
    [ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747625#action_12747625 ]

    ryan rawson commented on HBASE-1792:
    ------------------------------------

    we should move back to Long.MAX_VALUE as the thing to replace to 'now' i think...

    The problem with deletes would be if the delete is set too far in the past or future, depending on the time range of the files that end up bracketing them. Solved with minor compaction, and works via scans.


    [Regression] Cannot save timestamp in the future
    ------------------------------------------------

    Key: HBASE-1792
    URL: https://issues.apache.org/jira/browse/HBASE-1792
    Project: Hadoop HBase
    Issue Type: Bug
    Affects Versions: 0.20.0
    Reporter: Jean-Daniel Cryans

    0.20, compared to previous versions, doesn't let you save with a timestamp in the future and will set it to current time without telling you. This is really bad for users upgrading to 0.20 that were using those timestamps.
    Example:
    hbase(main):004:0> put 'testtable', 'r1', 'f1:c1', 'val', 5373965335336911168
    0 row(s) in 0.0070 seconds
    hbase(main):005:0> scan 'testtable'
    ROW COLUMN+CELL
    r1 column=f1:c1, timestamp=1251223892010, value=val
    1 row(s) in 0.0380 seconds
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jean-Daniel Cryans (JIRA) at Aug 25, 2009 at 8:41 pm
    [ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jean-Daniel Cryans updated HBASE-1792:
    --------------------------------------

    Attachment: HBASE-1792.patch

    Currently testing this patch.
    [Regression] Cannot save timestamp in the future
    ------------------------------------------------

    Key: HBASE-1792
    URL: https://issues.apache.org/jira/browse/HBASE-1792
    Project: Hadoop HBase
    Issue Type: Bug
    Affects Versions: 0.20.0
    Reporter: Jean-Daniel Cryans
    Attachments: HBASE-1792.patch


    0.20, compared to previous versions, doesn't let you save with a timestamp in the future and will set it to current time without telling you. This is really bad for users upgrading to 0.20 that were using those timestamps.
    Example:
    hbase(main):004:0> put 'testtable', 'r1', 'f1:c1', 'val', 5373965335336911168
    0 row(s) in 0.0070 seconds
    hbase(main):005:0> scan 'testtable'
    ROW COLUMN+CELL
    r1 column=f1:c1, timestamp=1251223892010, value=val
    1 row(s) in 0.0380 seconds
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jonathan Gray (JIRA) at Aug 25, 2009 at 11:43 pm
    [ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747710#action_12747710 ]

    Jonathan Gray commented on HBASE-1792:
    --------------------------------------

    +1 for commit on branch and trunk after your testing, jd.
    [Regression] Cannot save timestamp in the future
    ------------------------------------------------

    Key: HBASE-1792
    URL: https://issues.apache.org/jira/browse/HBASE-1792
    Project: Hadoop HBase
    Issue Type: Bug
    Affects Versions: 0.20.0
    Reporter: Jean-Daniel Cryans
    Attachments: HBASE-1792.patch


    0.20, compared to previous versions, doesn't let you save with a timestamp in the future and will set it to current time without telling you. This is really bad for users upgrading to 0.20 that were using those timestamps.
    Example:
    hbase(main):004:0> put 'testtable', 'r1', 'f1:c1', 'val', 5373965335336911168
    0 row(s) in 0.0070 seconds
    hbase(main):005:0> scan 'testtable'
    ROW COLUMN+CELL
    r1 column=f1:c1, timestamp=1251223892010, value=val
    1 row(s) in 0.0380 seconds
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jean-Daniel Cryans (JIRA) at Aug 26, 2009 at 12:05 am
    [ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jean-Daniel Cryans resolved HBASE-1792.
    ---------------------------------------

    Resolution: Fixed
    Fix Version/s: 0.21.0
    0.20.0
    Assignee: Jean-Daniel Cryans
    Hadoop Flags: [Reviewed]

    Committed to branch and trunk, test passes and it fixes the issue.
    [Regression] Cannot save timestamp in the future
    ------------------------------------------------

    Key: HBASE-1792
    URL: https://issues.apache.org/jira/browse/HBASE-1792
    Project: Hadoop HBase
    Issue Type: Bug
    Affects Versions: 0.20.0
    Reporter: Jean-Daniel Cryans
    Assignee: Jean-Daniel Cryans
    Fix For: 0.20.0, 0.21.0

    Attachments: HBASE-1792.patch


    0.20, compared to previous versions, doesn't let you save with a timestamp in the future and will set it to current time without telling you. This is really bad for users upgrading to 0.20 that were using those timestamps.
    Example:
    hbase(main):004:0> put 'testtable', 'r1', 'f1:c1', 'val', 5373965335336911168
    0 row(s) in 0.0070 seconds
    hbase(main):005:0> scan 'testtable'
    ROW COLUMN+CELL
    r1 column=f1:c1, timestamp=1251223892010, value=val
    1 row(s) in 0.0380 seconds
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categorieshbase, hadoop
postedAug 25, '09 at 6:33p
activeAug 26, '09 at 12:05a
posts8
users1
websitehbase.apache.org

1 user in discussion

Jean-Daniel Cryans (JIRA): 8 posts

People

Translate

site design / logo © 2022 Grokbase