[HBase-dev] [jira] Created: (HBASE-1792) [Regression] Cannot save timestamp in the future

[ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747575#action_12747575 ]

stack commented on HBASE-1792:
------------------------------

Yeah, I suppose old days we'd leave the ts alone and only change it to current time if it was HConstants.LATEST_TIMESTAMP (IIRC).

[ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747581#action_12747581 ]

Jonathan Gray commented on HBASE-1792:
--------------------------------------

There was a number of long discussions related to this. Unfortunately, I forgot where we ended up.

We tried to keep certain axioms, like "deletes only apply to older storefiles", but manually setting of stamps (future or past) breaks that. That means, if you modify stamps manually, your Gets can have indeterminate behavior.

I'd be more than happy to remove the strictness here, and allow anything < Long.MAX_VALUE (if equal, rewrite as now()), with a warning in Put that manually setting of stamp can cause indeterminate behavior, especially in Gets.

In any case, I'd like to see a unit test that verifies the behavior that we do have. There is already a bit of it in o.a.h.h.c.TestClient, but there should be more, perhaps another method or part of testVersions(). It should do insertions with past, present, future stamps, in all different orders, with lots of flushing in between (all issues with stamps / indeterminate behavior are related to multiple storefiles + memstore interaction).

If we can create a new test, with comments explaining what the behavior is, and all prior tests still pass, then I'll be +1 on allowing future stamps.

Stack, Erik, any recollection on why it was left this way?

This should also not have an impact on META where determinate behavior of Gets is quite important.

[ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747618#action_12747618 ]

Jean-Daniel Cryans commented on HBASE-1792:
-------------------------------------------

Just so I don't forget it, the timestamp is replaced silently here in HRegion.updateKeys:

{code}
/**
* Checks if any stamps are > now. If so, sets them to now.
* <p>
* This acts to be prevent users from inserting future stamps as well as
* to replace LATEST_TIMESTAMP with now.
* @param keys
* @param now
* @return <code>true</code> when updating the time stamp completed.
*/
private boolean updateKeys(List<KeyValue> keys, byte [] now) {
if(keys == null || keys.isEmpty()) {
return false;
}
for(KeyValue key : keys) {
key.updateLatestStamp(now);
}
return true;
}
{code}

[ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747625#action_12747625 ]

ryan rawson commented on HBASE-1792:
------------------------------------

we should move back to Long.MAX_VALUE as the thing to replace to 'now' i think...

The problem with deletes would be if the delete is set too far in the past or future, depending on the time range of the files that end up bracketing them. Solved with minor compaction, and works via scans.

[ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jean-Daniel Cryans updated HBASE-1792:
--------------------------------------

Attachment: HBASE-1792.patch

Currently testing this patch.

[ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747710#action_12747710 ]

Jonathan Gray commented on HBASE-1792:
--------------------------------------

+1 for commit on branch and trunk after your testing, jd.

[ https://issues.apache.org/jira/browse/HBASE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jean-Daniel Cryans resolved HBASE-1792.
---------------------------------------

Resolution: Fixed
Fix Version/s: 0.21.0
0.20.0
Assignee: Jean-Daniel Cryans
Hadoop Flags: [Reviewed]

Committed to branch and trunk, test passes and it fixes the issue.