FAQ
HADOOP-2838 doesnt work as expected
-----------------------------------

Key: HADOOP-5981
URL: https://issues.apache.org/jira/browse/HADOOP-5981
Project: Hadoop Core
Issue Type: Bug
Components: mapred
Reporter: Amar Kamat
Assignee: Amar Kamat
Fix For: 0.21.0


The substitution feature i.e X=$X:/tmp doesnt work as expected.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Amar Kamat (JIRA) at Jun 5, 2009 at 12:24 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Amar Kamat updated HADOOP-5981:
    -------------------------------

    Attachment: HADOOP-5981-v1.1.patch
    HADOOP-5981-v1.1-branch-20.patch

    Attaching a patch [HADOOP-5981-v1.1.patch] that fixes the issue. The issue got masked because of the improper merge of the two testcases. Tested this patch to see if it works. The testcase now passes. Running test-patch now.

    Note that [HADOOP-5981-v1.1-branch-20.patch] is an example patch not to be committed to branch 20.
    HADOOP-2838 doesnt work as expected
    -----------------------------------

    Key: HADOOP-5981
    URL: https://issues.apache.org/jira/browse/HADOOP-5981
    Project: Hadoop Core
    Issue Type: Bug
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.21.0

    Attachments: HADOOP-5981-v1.1-branch-20.patch, HADOOP-5981-v1.1.patch


    The substitution feature i.e X=$X:/tmp doesnt work as expected.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amar Kamat (JIRA) at Jun 5, 2009 at 12:44 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716602#action_12716602 ]

    Amar Kamat commented on HADOOP-5981:
    ------------------------------------

    Result of test-patch
    [exec] +1 overall.
    [exec]
    [exec] +1 @author. The patch does not contain any @author tags.
    [exec]
    [exec] +1 tests included. The patch appears to include 3 new or modified tests.
    [exec]
    [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
    [exec]
    [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
    [exec]
    [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
    [exec]
    [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
    [exec]
    [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
    HADOOP-2838 doesnt work as expected
    -----------------------------------

    Key: HADOOP-5981
    URL: https://issues.apache.org/jira/browse/HADOOP-5981
    Project: Hadoop Core
    Issue Type: Bug
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.21.0

    Attachments: HADOOP-5981-v1.1-branch-20.patch, HADOOP-5981-v1.1.patch


    The substitution feature i.e X=$X:/tmp doesnt work as expected.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Sreekanth Ramakrishnan (JIRA) at Jun 5, 2009 at 1:44 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716614#action_12716614 ]

    Sreekanth Ramakrishnan commented on HADOOP-5981:
    ------------------------------------------------

    The changes look fine to me.
    HADOOP-2838 doesnt work as expected
    -----------------------------------

    Key: HADOOP-5981
    URL: https://issues.apache.org/jira/browse/HADOOP-5981
    Project: Hadoop Core
    Issue Type: Bug
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.21.0

    Attachments: HADOOP-5981-v1.1-branch-20.patch, HADOOP-5981-v1.1.patch


    The substitution feature i.e X=$X:/tmp doesnt work as expected.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amar Kamat (JIRA) at Jun 5, 2009 at 3:56 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716654#action_12716654 ]

    Amar Kamat commented on HADOOP-5981:
    ------------------------------------

    Ant tests passed on my box.
    HADOOP-2838 doesnt work as expected
    -----------------------------------

    Key: HADOOP-5981
    URL: https://issues.apache.org/jira/browse/HADOOP-5981
    Project: Hadoop Core
    Issue Type: Bug
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.21.0

    Attachments: HADOOP-5981-v1.1-branch-20.patch, HADOOP-5981-v1.1.patch


    The substitution feature i.e X=$X:/tmp doesnt work as expected.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hemanth Yamijala (JIRA) at Jun 8, 2009 at 10:32 am
    [ https://issues.apache.org/jira/browse/HADOOP-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717225#action_12717225 ]

    Hemanth Yamijala commented on HADOOP-5981:
    ------------------------------------------

    Hmm. I don't fully understand this patch.

    So, what is the requirement with respect to variable substitution in mapred.child.env ? Currently, it looks like unless the key has LD_LIBRARY_PATH, it is not going to do anything useful.

    For e.g. if I have mapred.child.env with the value as "key=$value", where 'value' is in the environment of the Tasktracker, since 'key' is not added to the 'env' variable in the taskrunner method, it will not work, right ? I thought the use case would be that if any thing in the value should be picked from the tasktracker's environment, it should be replaced. Isn't this the use case ?
    HADOOP-2838 doesnt work as expected
    -----------------------------------

    Key: HADOOP-5981
    URL: https://issues.apache.org/jira/browse/HADOOP-5981
    Project: Hadoop Core
    Issue Type: Bug
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.21.0

    Attachments: HADOOP-5981-v1.1-branch-20.patch, HADOOP-5981-v1.1.patch


    The substitution feature i.e X=$X:/tmp doesnt work as expected.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amar Kamat (JIRA) at Jun 8, 2009 at 2:40 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717272#action_12717272 ]

    Amar Kamat commented on HADOOP-5981:
    ------------------------------------

    Hemanth,
    This is just a bugfix to do with HADOOP-2838. Currently we can pass
    # x=:/tmp [sets x to :/tmp]
    # x=$x:/tmp [appends :/tmp to x]

    We can support x=$y:/tmp but I didnt have any usecase for that. Hemanth, if you think thats a valid usecase then plz file a jira. Was this your concern?
    HADOOP-2838 doesnt work as expected
    -----------------------------------

    Key: HADOOP-5981
    URL: https://issues.apache.org/jira/browse/HADOOP-5981
    Project: Hadoop Core
    Issue Type: Bug
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.21.0

    Attachments: HADOOP-5981-v1.1-branch-20.patch, HADOOP-5981-v1.1.patch


    The substitution feature i.e X=$X:/tmp doesnt work as expected.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amar Kamat (JIRA) at Jun 9, 2009 at 12:10 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Amar Kamat updated HADOOP-5981:
    -------------------------------

    Attachment: HADOOP-5981-v2.0.patch

    Attaching a new patch that
    # inherits the tastracker's env variable value for substitution
    # fixes the substitution code
    # fixes the testcase
    # adds meaningful diagnostic information upon faulty env

    Result of test-patch
    [exec] +1 overall.
    [exec]
    [exec] +1 @author. The patch does not contain any @author tags.
    [exec]
    [exec] +1 tests included. The patch appears to include 3 new or modified tests.
    [exec]
    [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
    [exec]
    [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
    [exec]
    [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
    [exec]
    [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
    [exec]
    [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.

    Tested this patch on a single node cluster and works as expected. Here is what I did
    No||Param||Output||
    1|A|Error seen on webui and job fails|
    2|A=B|Child env has a parameter A set to B|
    2|A=$A:B|Child env has a parameter A set to :B|
    2|LD_LIBRARY_PATH=$LD_LIBRARY_PATH:B|Child env has B appended to the modified LD (work dir)|
    2|PATH=$PATH:B|Child env has PATH inherited from tt and appended B|
    Running ant test now
    HADOOP-2838 doesnt work as expected
    -----------------------------------

    Key: HADOOP-5981
    URL: https://issues.apache.org/jira/browse/HADOOP-5981
    Project: Hadoop Core
    Issue Type: Bug
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.21.0

    Attachments: HADOOP-5981-v1.1-branch-20.patch, HADOOP-5981-v1.1.patch, HADOOP-5981-v2.0.patch


    The substitution feature i.e X=$X:/tmp doesnt work as expected.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Sreekanth Ramakrishnan (JIRA) at Jun 9, 2009 at 1:42 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717672#action_12717672 ]

    Sreekanth Ramakrishnan commented on HADOOP-5981:
    ------------------------------------------------

    Changes look fine with me, with respect to the blocked issue HADOOP-5980, verified the fix with respect to {{LinuxTaskController}}
    HADOOP-2838 doesnt work as expected
    -----------------------------------

    Key: HADOOP-5981
    URL: https://issues.apache.org/jira/browse/HADOOP-5981
    Project: Hadoop Core
    Issue Type: Bug
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.21.0

    Attachments: HADOOP-5981-v1.1-branch-20.patch, HADOOP-5981-v1.1.patch, HADOOP-5981-v2.0.patch


    The substitution feature i.e X=$X:/tmp doesnt work as expected.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amar Kamat (JIRA) at Jun 10, 2009 at 9:54 am
    [ https://issues.apache.org/jira/browse/HADOOP-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718012#action_12718012 ]

    Amar Kamat commented on HADOOP-5981:
    ------------------------------------

    Ant tests passed on my box.
    HADOOP-2838 doesnt work as expected
    -----------------------------------

    Key: HADOOP-5981
    URL: https://issues.apache.org/jira/browse/HADOOP-5981
    Project: Hadoop Core
    Issue Type: Bug
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.21.0

    Attachments: HADOOP-5981-v1.1-branch-20.patch, HADOOP-5981-v1.1.patch, HADOOP-5981-v2.0.patch


    The substitution feature i.e X=$X:/tmp doesnt work as expected.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Sharad Agarwal (JIRA) at Jun 11, 2009 at 7:47 am
    [ https://issues.apache.org/jira/browse/HADOOP-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Sharad Agarwal resolved HADOOP-5981.
    ------------------------------------

    Resolution: Fixed
    Hadoop Flags: [Reviewed]

    I just committed this. Thanks Amar!
    HADOOP-2838 doesnt work as expected
    -----------------------------------

    Key: HADOOP-5981
    URL: https://issues.apache.org/jira/browse/HADOOP-5981
    Project: Hadoop Core
    Issue Type: Bug
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.21.0

    Attachments: HADOOP-5981-v1.1-branch-20.patch, HADOOP-5981-v1.1.patch, HADOOP-5981-v2.0.patch


    The substitution feature i.e X=$X:/tmp doesnt work as expected.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amar Kamat (JIRA) at Jun 12, 2009 at 4:46 am
    [ https://issues.apache.org/jira/browse/HADOOP-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Amar Kamat updated HADOOP-5981:
    -------------------------------

    Attachment: hadoop5981-branch-20-example.patch

    Attaching an example patch for 20 branch (not to be committed).
    HADOOP-2838 doesnt work as expected
    -----------------------------------

    Key: HADOOP-5981
    URL: https://issues.apache.org/jira/browse/HADOOP-5981
    Project: Hadoop Core
    Issue Type: Bug
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.21.0

    Attachments: HADOOP-5981-v1.1-branch-20.patch, HADOOP-5981-v1.1.patch, HADOOP-5981-v2.0.patch, hadoop5981-branch-20-example.patch


    The substitution feature i.e X=$X:/tmp doesnt work as expected.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amar Kamat (JIRA) at Jun 12, 2009 at 10:12 am
    [ https://issues.apache.org/jira/browse/HADOOP-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Amar Kamat updated HADOOP-5981:
    -------------------------------

    Release Note:
    This issue completes the feature mentioned in HADOOP-2838. HADOOP-2838 provided a way to set env variables in child process. This issue provides a way to inherit tt's env variables and append or reset it. So now
    X=$X:y will inherit X (if there) and append y to it.
    HADOOP-2838 doesnt work as expected
    -----------------------------------

    Key: HADOOP-5981
    URL: https://issues.apache.org/jira/browse/HADOOP-5981
    Project: Hadoop Core
    Issue Type: Bug
    Components: mapred
    Reporter: Amar Kamat
    Assignee: Amar Kamat
    Fix For: 0.21.0

    Attachments: HADOOP-5981-v1.1-branch-20.patch, HADOOP-5981-v1.1.patch, HADOOP-5981-v2.0.patch, hadoop5981-branch-20-example.patch


    The substitution feature i.e X=$X:/tmp doesnt work as expected.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedJun 5, '09 at 11:38a
activeJun 12, '09 at 10:12a
posts13
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Amar Kamat (JIRA): 13 posts

People

Translate

site design / logo © 2022 Grokbase