FAQ
JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
-----------------------------------------------------------------------------------------------------

Key: HADOOP-3780
URL: https://issues.apache.org/jira/browse/HADOOP-3780
Project: Hadoop Core
Issue Type: Improvement
Components: mapred
Reporter: Amar Kamat


This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Amar Kamat (JIRA) at Jul 17, 2008 at 12:14 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12614307#action_12614307 ]

    Amar Kamat commented on HADOOP-3780:
    ------------------------------------

    The problem that we are facing is as follows :
    SYM||Stands for||Description|| Used for||
    IC | Initial contact | whether the TT is connected to the JT or not, TT's point of view | Re-init/Sync the TT|
    SB | Seen before | whether there are some previous status entries | Mark a TT as lost|
    HBE | Heartbeat entry | whether the TT is connected/registered, JT's point of view | Re-init/Sync the TT|
    JTR| JT restarted | Whether the JT has restarted | Re-init/Sync the TT|
    Rules :
    IC||HBE||SB||JTR||Action||
    false|false|-|true|SYNC|
    false|false|-|false|Re-init|
    false|true|-|-|Re-send prev response|
    true|-|true|-|Mark lost (kill tasks)|
    false|-|false|-|make SB false i.e clear previous status entries|

    {noformat}

    0) JT restarts and hence HBE for all TT's will be false.
    1) TT connects to the restarted JT with IC=false
    2) JT sends a SYNC
    3) TT uploads the task statuses
    4) JT (as a part of heartbeat) tries to update the task states/status
    5) If (4) is successful : JT makes an HBE=true for this TT
    6) If (4) fails : the JT has made some changes in the task states but HBE=false.
    Consider task t being marked as SUCCEEDED before the SYNC fails.
    7) TT comes back with IC = false
    8) IC == false && HBE == false && JTR == true .... JT sends a SYNC again
    9) TT responds back with IC = true and all updates
    10) JT tries (4) again. Since IC == true and SB == true, JT consider this TT as lost.
    11) This causes the task t to be marked as KILLED
    12) In the same method the status updates are applied and hence t will be marked as SUCCEEDED
    13) Now we have task completion events with a same task marked as KILLED and SUCCEEDED.
    14) Since task t is marked as SUCCEEDED later, the JT assumes that the TIP is completed while the reducers keep on ignoring the task t's output.
    15) Job stucks
    {noformat}

    This problem will not occur if {{(4)}} succeeds without any problem i.e every {{SYNC}} should make HBE = true. {{4}} can only fail if the tracker is not resolved. Hence inline resolution solves the problem.

    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat

    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Devaraj Das (JIRA) at Jul 17, 2008 at 1:06 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12614321#action_12614321 ]

    Devaraj Das commented on HADOOP-3780:
    -------------------------------------

    +1 for the synchronous resolution
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat

    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amar Kamat (JIRA) at Jul 17, 2008 at 1:24 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12614307#action_12614307 ]

    amar_kamat edited comment on HADOOP-3780 at 7/17/08 6:21 AM:
    -------------------------------------------------------------

    The reason why this issue is important for HADOOP-3245 is as follows :

    _Summary_ :
    In HADOOP-3245 we are adding a new operation called _SYNC_ operation. This directs the task tracker to upload its local state to the jobtracker. The whole design expects the _SYNC_ operation to complete in one go. Partial updates can cause the JobTracker to be in an inconsistent state and might cause the job to get stuck. As of now, the only thing that can cause the _SYNC_ operation to fail is an update from an unresolved tracker. Under such conditions the JT is partially updated, which breaks HADOOP-3245.

    _Info:_
    SYM||Stands for||Description|| Used for||
    IC | Initial contact | whether the TT is connected to the JT or not, TT's point of view | Re-init/Sync the TT|
    SB | Seen before | whether there are some previous status entries | Mark a TT as lost|
    HBE | Heartbeat entry | whether the TT is connected/registered, JT's point of view | Re-init/Sync the TT|
    JTR| JT restarted | Whether the JT has restarted | Re-init/Sync the TT|
    _Rules :_
    IC||HBE||SB||JTR||Action||
    false|false|-|true|SYNC|
    false|false|-|false|Re-init|
    false|true|-|-|Re-send prev response|
    true|-|true|-|Mark lost (kill tasks)|
    false|-|false|-|make SB false i.e clear previous status entries|
    _Description :_
    {noformat}

    0) JT restarts and hence HBE for all TT's will be false.
    1) TT connects to the restarted JT with IC=false.
    2) JT sends a SYNC operation to the TT.
    3) TT uploads the task statuses with IC = true.
    4) JT (as a part of heartbeat) tries to update the task states/status.
    5) If (4) is successful : JT makes an HBE=true for this TT.
    6) If (4) fails : the JT has made some changes in the task states but HBE=false.
    Consider task t being marked as SUCCEEDED before the SYNC fails.
    7) TT comes back with IC = false.
    8) IC == false && HBE == false && JTR == true .... JT sends a SYNC again.
    9) (3) happens again.
    10) (4) happens again. Since IC == true and SB == true, JT consider this TT as lost.
    11) This causes the task t to be marked as KILLED.
    12) In the same method the status updates are applied and hence t will be marked as SUCCEEDED.
    13) Now we have task completion events with a same task marked as KILLED and SUCCEEDED.
    14) Since task t is marked as SUCCEEDED later, the JT assumes that the TIP is completed while the reducers keep
    on ignoring the task t's output.
    15) Job is stuck.
    {noformat}

    This problem will not occur if {{(4)}} succeeds without any problem i.e every {{SYNC}} should make HBE = true. {{4}} can only fail if the tracker is not resolved. Hence inline resolution solves the problem.


    was (Author: amar_kamat):
    The problem that we are facing is as follows :
    SYM||Stands for||Description|| Used for||
    IC | Initial contact | whether the TT is connected to the JT or not, TT's point of view | Re-init/Sync the TT|
    SB | Seen before | whether there are some previous status entries | Mark a TT as lost|
    HBE | Heartbeat entry | whether the TT is connected/registered, JT's point of view | Re-init/Sync the TT|
    JTR| JT restarted | Whether the JT has restarted | Re-init/Sync the TT|
    Rules :
    IC||HBE||SB||JTR||Action||
    false|false|-|true|SYNC|
    false|false|-|false|Re-init|
    false|true|-|-|Re-send prev response|
    true|-|true|-|Mark lost (kill tasks)|
    false|-|false|-|make SB false i.e clear previous status entries|

    {noformat}

    0) JT restarts and hence HBE for all TT's will be false.
    1) TT connects to the restarted JT with IC=false
    2) JT sends a SYNC
    3) TT uploads the task statuses
    4) JT (as a part of heartbeat) tries to update the task states/status
    5) If (4) is successful : JT makes an HBE=true for this TT
    6) If (4) fails : the JT has made some changes in the task states but HBE=false.
    Consider task t being marked as SUCCEEDED before the SYNC fails.
    7) TT comes back with IC = false
    8) IC == false && HBE == false && JTR == true .... JT sends a SYNC again
    9) TT responds back with IC = true and all updates
    10) JT tries (4) again. Since IC == true and SB == true, JT consider this TT as lost.
    11) This causes the task t to be marked as KILLED
    12) In the same method the status updates are applied and hence t will be marked as SUCCEEDED
    13) Now we have task completion events with a same task marked as KILLED and SUCCEEDED.
    14) Since task t is marked as SUCCEEDED later, the JT assumes that the TIP is completed while the reducers keep on ignoring the task t's output.
    15) Job stucks
    {noformat}

    This problem will not occur if {{(4)}} succeeds without any problem i.e every {{SYNC}} should make HBE = true. {{4}} can only fail if the tracker is not resolved. Hence inline resolution solves the problem.

    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat

    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amar Kamat (JIRA) at Jul 17, 2008 at 1:25 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Amar Kamat updated HADOOP-3780:
    -------------------------------

    Attachment: HADOOP-3780-v1.patch

    Here is a patch the tries to get the resolution inline. Testing in progress.
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Attachments: HADOOP-3780-v1.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amar Kamat (JIRA) at Jul 21, 2008 at 5:23 am
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Amar Kamat updated HADOOP-3780:
    -------------------------------

    Attachment: HADOOP-3245-v4.2.patch

    Updated the patch to trunk.
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Attachments: HADOOP-3245-v4.2.patch, HADOOP-3780-v1.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amar Kamat (JIRA) at Jul 21, 2008 at 5:26 am
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Amar Kamat updated HADOOP-3780:
    -------------------------------

    Attachment: (was: HADOOP-3245-v4.2.patch)
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Attachments: HADOOP-3780-v1.1.patch, HADOOP-3780-v1.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amar Kamat (JIRA) at Jul 21, 2008 at 5:26 am
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Amar Kamat updated HADOOP-3780:
    -------------------------------

    Assignee: Amar Kamat
    Status: Patch Available (was: Open)
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Attachments: HADOOP-3780-v1.1.patch, HADOOP-3780-v1.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amar Kamat (JIRA) at Jul 21, 2008 at 5:26 am
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Amar Kamat updated HADOOP-3780:
    -------------------------------

    Attachment: HADOOP-3780-v1.1.patch
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Attachments: HADOOP-3780-v1.1.patch, HADOOP-3780-v1.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amar Kamat (JIRA) at Jul 21, 2008 at 2:37 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Amar Kamat updated HADOOP-3780:
    -------------------------------

    Status: Patch Available (was: Open)

    Updated ....
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Attachments: HADOOP-3780-v1.1.patch, HADOOP-3780-v1.2.patch, HADOOP-3780-v1.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amar Kamat (JIRA) at Jul 21, 2008 at 2:38 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Amar Kamat updated HADOOP-3780:
    -------------------------------

    Attachment: HADOOP-3780-v1.2.patch

    Modified the test case to reflect the changes.
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Attachments: HADOOP-3780-v1.1.patch, HADOOP-3780-v1.2.patch, HADOOP-3780-v1.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amar Kamat (JIRA) at Jul 21, 2008 at 2:38 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Amar Kamat updated HADOOP-3780:
    -------------------------------

    Status: Open (was: Patch Available)
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Attachments: HADOOP-3780-v1.1.patch, HADOOP-3780-v1.2.patch, HADOOP-3780-v1.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Jul 21, 2008 at 6:22 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615348#action_12615348 ]

    Hadoop QA commented on HADOOP-3780:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12386527/HADOOP-3780-v1.2.patch
    against trunk revision 678196.

    +1 @author. The patch does not contain any @author tags.

    +1 tests included. The patch appears to include 3 new or modified tests.

    +1 javadoc. The javadoc tool did not generate any warning messages.

    +1 javac. The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs. The patch does not introduce any new Findbugs warnings.

    +1 release audit. The applied patch does not increase the total number of release audit warnings.

    -1 core tests. The patch failed core unit tests.

    +1 contrib tests. The patch passed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2915/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2915/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2915/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2915/console

    This message is automatically generated.
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Attachments: HADOOP-3780-v1.1.patch, HADOOP-3780-v1.2.patch, HADOOP-3780-v1.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Owen O'Malley (JIRA) at Aug 12, 2008 at 9:48 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Owen O'Malley updated HADOOP-3780:
    ----------------------------------

    Resolution: Fixed
    Hadoop Flags: [Reviewed]
    Status: Resolved (was: Patch Available)

    I just committed this. Thanks, Amar!
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Attachments: HADOOP-3780-v1.1.patch, HADOOP-3780-v1.2.patch, HADOOP-3780-v1.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Owen O'Malley (JIRA) at Aug 12, 2008 at 10:00 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Owen O'Malley updated HADOOP-3780:
    ----------------------------------

    Fix Version/s: 0.19.0
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.19.0

    Attachments: HADOOP-3780-v1.1.patch, HADOOP-3780-v1.2.patch, HADOOP-3780-v1.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hudson (JIRA) at Aug 22, 2008 at 12:42 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12624800#action_12624800 ]

    Hudson commented on HADOOP-3780:
    --------------------------------

    Integrated in Hadoop-trunk #581 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/581/])
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.19.0

    Attachments: HADOOP-3780-v1.1.patch, HADOOP-3780-v1.2.patch, HADOOP-3780-v1.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Steve Loughran (JIRA) at Aug 22, 2008 at 1:43 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12624825#action_12624825 ]

    Steve Loughran commented on HADOOP-3780:
    ----------------------------------------

    This seems to render the member variable numResolved unimportant, and the method moot. The numResolved count is now always zero, so getNumResolved() == 0, so breaking any tests that used this to wait for the cluster to come up.

    1. How can I count the #of task trackers under a job tracker?
    2. Can this number be passed to getNumResolved() for BC, or can that be deleted
    3. numResolved should be deleted; anyone that is using it needs to know their code has broken.
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.19.0

    Attachments: HADOOP-3780-v1.1.patch, HADOOP-3780-v1.2.patch, HADOOP-3780-v1.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Steve Loughran (JIRA) at Aug 22, 2008 at 1:55 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12624827#action_12624827 ]

    Steve Loughran commented on HADOOP-3780:
    ----------------------------------------

    patching TaskTracker.getNumResolvedTaskTrackers() to return taskTrackers.size() appears to work; there's no need to make this synchronized.

    public int getNumResolvedTaskTrackers() {
    return taskTrackers.size();
    }

    Is this the right thing to do? Should the method name stay the same?
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.19.0

    Attachments: HADOOP-3780-v1.1.patch, HADOOP-3780-v1.2.patch, HADOOP-3780-v1.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Nigel Daley (JIRA) at Dec 2, 2008 at 7:55 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12652480#action_12652480 ]

    Nigel Daley commented on HADOOP-3780:
    -------------------------------------

    Should this be fixed in 0.18.3 too?
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.19.0

    Attachments: HADOOP-3780-v1.1.patch, HADOOP-3780-v1.2.patch, HADOOP-3780-v1.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amareshwari Sriramadasu (JIRA) at Dec 3, 2008 at 3:33 am
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12652666#action_12652666 ]

    Amareshwari Sriramadasu commented on HADOOP-3780:
    -------------------------------------------------

    bq. Should this be fixed in 0.18.3 too?

    +1
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.19.0

    Attachments: HADOOP-3780-v1.1.patch, HADOOP-3780-v1.2.patch, HADOOP-3780-v1.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Ravi Gummadi (JIRA) at Dec 26, 2008 at 4:41 am
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Ravi Gummadi updated HADOOP-3780:
    ---------------------------------

    Attachment: HADOOP18-3780.patch

    Attached the patch for branch 18.

    [exec] -1 overall.
    [exec]
    [exec] +1 @author. The patch does not contain any @author tags.
    [exec]
    [exec] +1 tests included. The patch appears to include 3 new or modified tests.
    [exec]
    [exec] -1 javadoc. The javadoc tool appears to have generated 1 warning messages.
    [exec]
    [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
    [exec]
    [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.



    The javadoc warning is not related to this patch.

    Unit tests also passed on my machine in branch 18.
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.19.0

    Attachments: HADOOP-3780-v1.1.patch, HADOOP-3780-v1.2.patch, HADOOP-3780-v1.patch, HADOOP18-3780.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amar Kamat (JIRA) at Dec 30, 2008 at 10:15 am
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12659784#action_12659784 ]

    Amar Kamat commented on HADOOP-3780:
    ------------------------------------

    +1.
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.19.0

    Attachments: HADOOP-3780-v1.1.patch, HADOOP-3780-v1.2.patch, HADOOP-3780-v1.patch, HADOOP18-3780.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Devaraj Das (JIRA) at Dec 30, 2008 at 12:01 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12659800#action_12659800 ]

    Devaraj Das commented on HADOOP-3780:
    -------------------------------------

    I committed this patch to the 0.18 branch. Thanks Ravi!
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.19.0

    Attachments: HADOOP-3780-v1.1.patch, HADOOP-3780-v1.2.patch, HADOOP-3780-v1.patch, HADOOP18-3780.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Devaraj Das (JIRA) at Dec 30, 2008 at 12:05 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Devaraj Das updated HADOOP-3780:
    --------------------------------

    Fix Version/s: 0.18.3
    JobTracker should synchronously resolve the tasktracker's network location when the tracker registers
    -----------------------------------------------------------------------------------------------------

    Key: HADOOP-3780
    URL: https://issues.apache.org/jira/browse/HADOOP-3780
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.18.3, 0.19.0

    Attachments: HADOOP-3780-v1.1.patch, HADOOP-3780-v1.2.patch, HADOOP-3780-v1.patch, HADOOP18-3780.patch


    This issue is inspired by HADOOP-3620. In JobTracker, the network address of tracker gets resolved asynchronously. Now it can be done inline i.e while the trackers register. This is of great help for HADOOP-3245 where this enhancement makes the design simpler.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedJul 17, '08 at 10:06a
activeDec 30, '08 at 12:05p
posts24
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Devaraj Das (JIRA): 24 posts

People

Translate

site design / logo © 2022 Grokbase