FAQ
Need to handle access token expiration when re-establishing the pipeline for dfs write
--------------------------------------------------------------------------------------

Key: HADOOP-6048
URL: https://issues.apache.org/jira/browse/HADOOP-6048
Project: Hadoop Core
Issue Type: Bug
Components: dfs, security
Reporter: Kan Zhang
Assignee: Kan Zhang


Original access token may have expired when re-establishing the pipeline within processDatanodeError().

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Kan Zhang (JIRA) at Jun 16, 2009 at 5:02 am
    [ https://issues.apache.org/jira/browse/HADOOP-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Kan Zhang updated HADOOP-6048:
    ------------------------------

    Attachment: 6048-09.patch

    this patch does the following.
    1. when the primary datanode recovers a block, it sends back a newly generated access token along a new generation stamp if any.
    2. add a namenode method so that a DFSClient can call to get a new access token when access token becomes expired during datanode error recovery.
    3. To differentiate access token error from other errors, datanode now needs to send back a status code together with firstBadLink during pipeline setup for dfs write (only for DFSClients, not for other datanodes).
    Need to handle access token expiration when re-establishing the pipeline for dfs write
    --------------------------------------------------------------------------------------

    Key: HADOOP-6048
    URL: https://issues.apache.org/jira/browse/HADOOP-6048
    Project: Hadoop Core
    Issue Type: Bug
    Components: dfs, security
    Reporter: Kan Zhang
    Assignee: Kan Zhang
    Attachments: 6048-09.patch


    Original access token may have expired when re-establishing the pipeline within processDatanodeError().
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Kan Zhang (JIRA) at Jun 16, 2009 at 5:02 am
    [ https://issues.apache.org/jira/browse/HADOOP-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Kan Zhang updated HADOOP-6048:
    ------------------------------

    Status: Patch Available (was: Open)
    Need to handle access token expiration when re-establishing the pipeline for dfs write
    --------------------------------------------------------------------------------------

    Key: HADOOP-6048
    URL: https://issues.apache.org/jira/browse/HADOOP-6048
    Project: Hadoop Core
    Issue Type: Bug
    Components: dfs, security
    Reporter: Kan Zhang
    Assignee: Kan Zhang
    Attachments: 6048-09.patch


    Original access token may have expired when re-establishing the pipeline within processDatanodeError().
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Kan Zhang (JIRA) at Jun 16, 2009 at 5:19 am
    [ https://issues.apache.org/jira/browse/HADOOP-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Kan Zhang updated HADOOP-6048:
    ------------------------------

    Attachment: 6048-11.patch

    updated ClientProtocol version.
    Need to handle access token expiration when re-establishing the pipeline for dfs write
    --------------------------------------------------------------------------------------

    Key: HADOOP-6048
    URL: https://issues.apache.org/jira/browse/HADOOP-6048
    Project: Hadoop Core
    Issue Type: Bug
    Components: dfs, security
    Reporter: Kan Zhang
    Assignee: Kan Zhang
    Attachments: 6048-09.patch, 6048-11.patch


    Original access token may have expired when re-establishing the pipeline within processDatanodeError().
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Kan Zhang (JIRA) at Jun 17, 2009 at 1:24 am
    [ https://issues.apache.org/jira/browse/HADOOP-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Kan Zhang updated HADOOP-6048:
    ------------------------------

    Attachment: 6048-12.patch

    attaching a new patch, which relies on block recovery to always return a newly generated access token, so that there is no need for adding a new method to ClientProtocol to get access token.
    Need to handle access token expiration when re-establishing the pipeline for dfs write
    --------------------------------------------------------------------------------------

    Key: HADOOP-6048
    URL: https://issues.apache.org/jira/browse/HADOOP-6048
    Project: Hadoop Core
    Issue Type: Bug
    Components: dfs, security
    Reporter: Kan Zhang
    Assignee: Kan Zhang
    Attachments: 6048-09.patch, 6048-11.patch, 6048-12.patch


    Original access token may have expired when re-establishing the pipeline within processDatanodeError().
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Kan Zhang (JIRA) at Jun 17, 2009 at 1:46 am
    [ https://issues.apache.org/jira/browse/HADOOP-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Kan Zhang updated HADOOP-6048:
    ------------------------------

    Attachment: 6048-14.patch

    minor updates
    Need to handle access token expiration when re-establishing the pipeline for dfs write
    --------------------------------------------------------------------------------------

    Key: HADOOP-6048
    URL: https://issues.apache.org/jira/browse/HADOOP-6048
    Project: Hadoop Core
    Issue Type: Bug
    Components: dfs, security
    Reporter: Kan Zhang
    Assignee: Kan Zhang
    Attachments: 6048-09.patch, 6048-11.patch, 6048-12.patch, 6048-14.patch


    Original access token may have expired when re-establishing the pipeline within processDatanodeError().
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Kan Zhang (JIRA) at Jun 17, 2009 at 4:42 am
    [ https://issues.apache.org/jira/browse/HADOOP-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12720494#action_12720494 ]

    Kan Zhang commented on HADOOP-6048:
    -----------------------------------

    To be clear to the reader, the last patch removed null checking on the return value from recoverBlock() since it always returns a LocatedBlock object or throw an exception.
    Need to handle access token expiration when re-establishing the pipeline for dfs write
    --------------------------------------------------------------------------------------

    Key: HADOOP-6048
    URL: https://issues.apache.org/jira/browse/HADOOP-6048
    Project: Hadoop Core
    Issue Type: Bug
    Components: dfs, security
    Reporter: Kan Zhang
    Assignee: Kan Zhang
    Attachments: 6048-09.patch, 6048-11.patch, 6048-12.patch, 6048-14.patch


    Original access token may have expired when re-establishing the pipeline within processDatanodeError().
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Kan Zhang (JIRA) at Jun 17, 2009 at 5:28 am
    [ https://issues.apache.org/jira/browse/HADOOP-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Kan Zhang updated HADOOP-6048:
    ------------------------------

    Attachment: 6048-15.patch

    one cosmetic change.
    Need to handle access token expiration when re-establishing the pipeline for dfs write
    --------------------------------------------------------------------------------------

    Key: HADOOP-6048
    URL: https://issues.apache.org/jira/browse/HADOOP-6048
    Project: Hadoop Core
    Issue Type: Bug
    Components: dfs, security
    Reporter: Kan Zhang
    Assignee: Kan Zhang
    Attachments: 6048-09.patch, 6048-11.patch, 6048-12.patch, 6048-14.patch, 6048-15.patch


    Original access token may have expired when re-establishing the pipeline within processDatanodeError().
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Kan Zhang (JIRA) at Jun 17, 2009 at 5:34 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Kan Zhang updated HADOOP-6048:
    ------------------------------

    Attachment: 6048-16.patch

    attached a new patch. removed the testing for InvalidAccessTokenException in processDatanodeError() since it can never happen. processDatanodeError() is called in 2 places. 1) when an error happens while writing. since the pipeline has already been set up in this case, the error can't be access token error. 2) when initializing for append, in which case, there is no previous error yet.
    Need to handle access token expiration when re-establishing the pipeline for dfs write
    --------------------------------------------------------------------------------------

    Key: HADOOP-6048
    URL: https://issues.apache.org/jira/browse/HADOOP-6048
    Project: Hadoop Core
    Issue Type: Bug
    Components: dfs, security
    Reporter: Kan Zhang
    Assignee: Kan Zhang
    Attachments: 6048-09.patch, 6048-11.patch, 6048-12.patch, 6048-14.patch, 6048-15.patch, 6048-16.patch


    Original access token may have expired when re-establishing the pipeline within processDatanodeError().
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Kan Zhang (JIRA) at Jun 18, 2009 at 5:26 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Kan Zhang updated HADOOP-6048:
    ------------------------------

    Status: Patch Available (was: Open)
    Need to handle access token expiration when re-establishing the pipeline for dfs write
    --------------------------------------------------------------------------------------

    Key: HADOOP-6048
    URL: https://issues.apache.org/jira/browse/HADOOP-6048
    Project: Hadoop Core
    Issue Type: Bug
    Components: dfs, security
    Reporter: Kan Zhang
    Assignee: Kan Zhang
    Attachments: 6048-09.patch, 6048-11.patch, 6048-12.patch, 6048-14.patch, 6048-15.patch, 6048-16.patch


    Original access token may have expired when re-establishing the pipeline within processDatanodeError().
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Kan Zhang (JIRA) at Jun 18, 2009 at 5:26 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Kan Zhang updated HADOOP-6048:
    ------------------------------

    Status: Open (was: Patch Available)
    Need to handle access token expiration when re-establishing the pipeline for dfs write
    --------------------------------------------------------------------------------------

    Key: HADOOP-6048
    URL: https://issues.apache.org/jira/browse/HADOOP-6048
    Project: Hadoop Core
    Issue Type: Bug
    Components: dfs, security
    Reporter: Kan Zhang
    Assignee: Kan Zhang
    Attachments: 6048-09.patch, 6048-11.patch, 6048-12.patch, 6048-14.patch, 6048-15.patch, 6048-16.patch


    Original access token may have expired when re-establishing the pipeline within processDatanodeError().
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Kan Zhang (JIRA) at Jun 18, 2009 at 10:54 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Kan Zhang updated HADOOP-6048:
    ------------------------------

    Attachment: 6048-17.patch
    Need to handle access token expiration when re-establishing the pipeline for dfs write
    --------------------------------------------------------------------------------------

    Key: HADOOP-6048
    URL: https://issues.apache.org/jira/browse/HADOOP-6048
    Project: Hadoop Core
    Issue Type: Bug
    Components: dfs, security
    Reporter: Kan Zhang
    Assignee: Kan Zhang
    Attachments: 6048-09.patch, 6048-11.patch, 6048-12.patch, 6048-14.patch, 6048-15.patch, 6048-16.patch, 6048-17.patch


    Original access token may have expired when re-establishing the pipeline within processDatanodeError().
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Kan Zhang (JIRA) at Jun 19, 2009 at 1:17 am
    [ https://issues.apache.org/jira/browse/HADOOP-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Kan Zhang updated HADOOP-6048:
    ------------------------------

    Attachment: 6048-18.patch
    Need to handle access token expiration when re-establishing the pipeline for dfs write
    --------------------------------------------------------------------------------------

    Key: HADOOP-6048
    URL: https://issues.apache.org/jira/browse/HADOOP-6048
    Project: Hadoop Core
    Issue Type: Bug
    Components: dfs, security
    Reporter: Kan Zhang
    Assignee: Kan Zhang
    Attachments: 6048-09.patch, 6048-11.patch, 6048-12.patch, 6048-14.patch, 6048-15.patch, 6048-16.patch, 6048-17.patch, 6048-18.patch


    Original access token may have expired when re-establishing the pipeline within processDatanodeError().
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Kan Zhang (JIRA) at Jun 19, 2009 at 1:29 am
    [ https://issues.apache.org/jira/browse/HADOOP-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12721606#action_12721606 ]

    Kan Zhang commented on HADOOP-6048:
    -----------------------------------

    attached a new patch, which bumps up the ClientDatanodeProtocol version to signal the fact that now recoverBlock() never returns null and always returns a newly generated access token as part of the return value. Also, reverted the changes made in previous patches that changed the way how lastException is set by sync() and createBlockOutputStream().
    Need to handle access token expiration when re-establishing the pipeline for dfs write
    --------------------------------------------------------------------------------------

    Key: HADOOP-6048
    URL: https://issues.apache.org/jira/browse/HADOOP-6048
    Project: Hadoop Core
    Issue Type: Bug
    Components: dfs, security
    Reporter: Kan Zhang
    Assignee: Kan Zhang
    Attachments: 6048-09.patch, 6048-11.patch, 6048-12.patch, 6048-14.patch, 6048-15.patch, 6048-16.patch, 6048-17.patch, 6048-18.patch


    Original access token may have expired when re-establishing the pipeline within processDatanodeError().
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Jun 19, 2009 at 10:17 am
    [ https://issues.apache.org/jira/browse/HADOOP-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12721724#action_12721724 ]

    Hadoop QA commented on HADOOP-6048:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12411172/6048-18.patch
    against trunk revision 786278.

    +1 @author. The patch does not contain any @author tags.

    +1 tests included. The patch appears to include 3 new or modified tests.

    +1 javadoc. The javadoc tool did not generate any warning messages.

    +1 javac. The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs. The patch does not introduce any new Findbugs warnings.

    +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

    +1 release audit. The applied patch does not increase the total number of release audit warnings.

    +1 core tests. The patch passed core unit tests.

    -1 contrib tests. The patch failed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/533/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/533/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/533/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/533/console

    This message is automatically generated.
    Need to handle access token expiration when re-establishing the pipeline for dfs write
    --------------------------------------------------------------------------------------

    Key: HADOOP-6048
    URL: https://issues.apache.org/jira/browse/HADOOP-6048
    Project: Hadoop Core
    Issue Type: Bug
    Components: dfs, security
    Reporter: Kan Zhang
    Assignee: Kan Zhang
    Attachments: 6048-09.patch, 6048-11.patch, 6048-12.patch, 6048-14.patch, 6048-15.patch, 6048-16.patch, 6048-17.patch, 6048-18.patch


    Original access token may have expired when re-establishing the pipeline within processDatanodeError().
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedJun 16, '09 at 4:26a
activeJun 19, '09 at 10:17a
posts15
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Hadoop QA (JIRA): 15 posts

People

Translate

site design / logo © 2022 Grokbase