FAQ
tip and task names should reflect the job name
----------------------------------------------

Key: HADOOP-150
URL: http://issues.apache.org/jira/browse/HADOOP-150
Project: Hadoop
Type: Improvement

Components: mapred
Versions: 0.1.1
Reporter: Owen O'Malley
Assigned to: Owen O'Malley


The tip and task names should be related to the job id. I'd propose:

job name: job_<random 32bit/base36>
tip: tip_<job id>_[mr]_<fragment #>
task: task_<tip id>_<attempt #>

so examples would be:
job_abc123
tip_abc123_m_00034
task_abc123_m_00034_1



--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira

Search Discussions

  • Doug Cutting (JIRA) at Apr 19, 2006 at 6:52 pm
    [ http://issues.apache.org/jira/browse/HADOOP-150?page=comments#action_12375183 ]

    Doug Cutting commented on HADOOP-150:
    -------------------------------------

    +1

    Does the job name need to be random, even? Could we allocate these sequentially too?

    Also, we don't even need to name the taskTracker's with a random number. We can use hostname:port, where the port is one used by one of the tasktracker's daemons (map output or umbilical). That would make these more meaningful too.

    Another random note: the umbilical and map output RPC servers could actually be run on the same port. The way RPC is currently implemented, all public methods on the object served are available to be called remotely. This is perhaps unsafe. Perhaps RPC should be changed so that a server names the interface whose methods calls should be restricted to...
    tip and task names should reflect the job name
    ----------------------------------------------

    Key: HADOOP-150
    URL: http://issues.apache.org/jira/browse/HADOOP-150
    Project: Hadoop
    Type: Improvement
    Components: mapred
    Versions: 0.1.1
    Reporter: Owen O'Malley
    Assignee: Owen O'Malley
    The tip and task names should be related to the job id. I'd propose:
    job name: job_<random 32bit/base36>
    tip: tip_<job id>_[mr]_<fragment #>
    task: task_<tip id>_<attempt #>
    so examples would be:
    job_abc123
    tip_abc123_m_00034
    task_abc123_m_00034_1
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators:
    http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see:
    http://www.atlassian.com/software/jira
  • Owen O'Malley (JIRA) at Apr 21, 2006 at 7:26 pm
    [ http://issues.apache.org/jira/browse/HADOOP-150?page=all ]

    Owen O'Malley updated HADOOP-150:
    ---------------------------------

    Attachment: task_id.patch

    Here is a patch that changes the names to match the given pattern.
    tip and task names should reflect the job name
    ----------------------------------------------

    Key: HADOOP-150
    URL: http://issues.apache.org/jira/browse/HADOOP-150
    Project: Hadoop
    Type: Improvement
    Components: mapred
    Versions: 0.1.1
    Reporter: Owen O'Malley
    Assignee: Owen O'Malley
    Attachments: task_id.patch

    The tip and task names should be related to the job id. I'd propose:
    job name: job_<random 32bit/base36>
    tip: tip_<job id>_[mr]_<fragment #>
    task: task_<tip id>_<attempt #>
    so examples would be:
    job_abc123
    tip_abc123_m_00034
    task_abc123_m_00034_1
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators:
    http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see:
    http://www.atlassian.com/software/jira
  • Owen O'Malley (JIRA) at Apr 21, 2006 at 8:44 pm
    [ http://issues.apache.org/jira/browse/HADOOP-150?page=all ]

    Owen O'Malley updated HADOOP-150:
    ---------------------------------

    Attachment: task-id-2.patch

    Ok, I forgot the "task_" in the prefix and I changed the job ids from random to sequential within a job tracker.
    tip and task names should reflect the job name
    ----------------------------------------------

    Key: HADOOP-150
    URL: http://issues.apache.org/jira/browse/HADOOP-150
    Project: Hadoop
    Type: Improvement
    Components: mapred
    Versions: 0.1.1
    Reporter: Owen O'Malley
    Assignee: Owen O'Malley
    Attachments: task-id-2.patch, task_id.patch

    The tip and task names should be related to the job id. I'd propose:
    job name: job_<random 32bit/base36>
    tip: tip_<job id>_[mr]_<fragment #>
    task: task_<tip id>_<attempt #>
    so examples would be:
    job_abc123
    tip_abc123_m_00034
    task_abc123_m_00034_1
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators:
    http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see:
    http://www.atlassian.com/software/jira
  • Owen O'Malley (JIRA) at Apr 21, 2006 at 9:19 pm
    [ http://issues.apache.org/jira/browse/HADOOP-150?page=all ]

    Owen O'Malley updated HADOOP-150:
    ---------------------------------

    Attachment: (was: task-id-2.patch)
    tip and task names should reflect the job name
    ----------------------------------------------

    Key: HADOOP-150
    URL: http://issues.apache.org/jira/browse/HADOOP-150
    Project: Hadoop
    Type: Improvement
    Components: mapred
    Versions: 0.1.1
    Reporter: Owen O'Malley
    Assignee: Owen O'Malley
    Attachments: task-id-3.patch, task_id.patch

    The tip and task names should be related to the job id. I'd propose:
    job name: job_<random 32bit/base36>
    tip: tip_<job id>_[mr]_<fragment #>
    task: task_<tip id>_<attempt #>
    so examples would be:
    job_abc123
    tip_abc123_m_00034
    task_abc123_m_00034_1
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators:
    http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see:
    http://www.atlassian.com/software/jira
  • Owen O'Malley (JIRA) at Apr 21, 2006 at 9:19 pm
    [ http://issues.apache.org/jira/browse/HADOOP-150?page=all ]

    Owen O'Malley updated HADOOP-150:
    ---------------------------------

    Attachment: task-id-3.patch

    A minor correction...
    tip and task names should reflect the job name
    ----------------------------------------------

    Key: HADOOP-150
    URL: http://issues.apache.org/jira/browse/HADOOP-150
    Project: Hadoop
    Type: Improvement
    Components: mapred
    Versions: 0.1.1
    Reporter: Owen O'Malley
    Assignee: Owen O'Malley
    Attachments: task-id-3.patch, task_id.patch

    The tip and task names should be related to the job id. I'd propose:
    job name: job_<random 32bit/base36>
    tip: tip_<job id>_[mr]_<fragment #>
    task: task_<tip id>_<attempt #>
    so examples would be:
    job_abc123
    tip_abc123_m_00034
    task_abc123_m_00034_1
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators:
    http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see:
    http://www.atlassian.com/software/jira
  • Doug Cutting (JIRA) at Apr 24, 2006 at 6:03 pm
    [ http://issues.apache.org/jira/browse/HADOOP-150?page=all ]

    Doug Cutting resolved HADOOP-150:
    ---------------------------------

    Fix Version: 0.2
    Resolution: Fixed

    I just committed this. Thanks, Owen!
    tip and task names should reflect the job name
    ----------------------------------------------

    Key: HADOOP-150
    URL: http://issues.apache.org/jira/browse/HADOOP-150
    Project: Hadoop
    Type: Improvement
    Components: mapred
    Versions: 0.1.1
    Reporter: Owen O'Malley
    Assignee: Owen O'Malley
    Fix For: 0.2
    Attachments: task-id-3.patch, task_id.patch

    The tip and task names should be related to the job id. I'd propose:
    job name: job_<random 32bit/base36>
    tip: tip_<job id>_[mr]_<fragment #>
    task: task_<tip id>_<attempt #>
    so examples would be:
    job_abc123
    tip_abc123_m_00034
    task_abc123_m_00034_1
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators:
    http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see:
    http://www.atlassian.com/software/jira

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedApr 19, '06 at 6:36p
activeApr 24, '06 at 6:03p
posts7
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Doug Cutting (JIRA): 7 posts

People

Translate

site design / logo © 2022 Grokbase