FAQ
Hi,

Assuming my program implements the ToolRunner, my question is where does the "run" function execute? ie. which daemon (DataNode/TT) ? or is it on the local machine where it is run?

Thank you,
Maha

Search Discussions

  • Michael Segel at Mar 2, 2011 at 8:38 pm
    Run is local to your edge machine where you launched your job.
    It then connects to the cluster / job tracker ...

    HTH

    -Mike
    From: maha@umail.ucsb.edu
    Subject: ToolRunner run function
    Date: Wed, 2 Mar 2011 12:10:05 -0800
    To: common-user@hadoop.apache.org

    Hi,

    Assuming my program implements the ToolRunner, my question is where does the "run" function execute? ie. which daemon (DataNode/TT) ? or is it on the local machine where it is run?

    Thank you,
    Maha
  • Maha at Mar 2, 2011 at 9:05 pm
    Thanks Mike :)

    I was also wondering what if:

    hdfs.CopyToLocal( src-file, dst-file) ; // is executed on node N

    and there exists a copy of src-file from the replication process in that same node(N) local file system ?

    Will hdfs recognize that there is already a copy in there and hence just move that copy to dst-file path ?
    OR
    Will hdfs go ahead with the copy and hence node N will have two copies of the src-file? (ie. one on HDFS namespace and another in the local file system)


    Thanks,

    Maha
    On Mar 2, 2011, at 12:38 PM, Michael Segel wrote:



    Run is local to your edge machine where you launched your job.
    It then connects to the cluster / job tracker ...

    HTH

    -Mike
    From: maha@umail.ucsb.edu
    Subject: ToolRunner run function
    Date: Wed, 2 Mar 2011 12:10:05 -0800
    To: common-user@hadoop.apache.org

    Hi,

    Assuming my program implements the ToolRunner, my question is where does the "run" function execute? ie. which daemon (DataNode/TT) ? or is it on the local machine where it is run?

    Thank you,
    Maha
  • Maha at Mar 2, 2011 at 9:12 pm
    On a pseudo distributed mode, it actually just "move" the copy and not reproduce it :)
    Thanks anyways,

    Maha
    On Mar 2, 2011, at 1:04 PM, maha wrote:

    Thanks Mike :)

    I was also wondering what if:

    hdfs.CopyToLocal( src-file, dst-file) ; // is executed on node N

    and there exists a copy of src-file from the replication process in that same node(N) local file system ?

    Will hdfs recognize that there is already a copy in there and hence just move that copy to dst-file path ?
    OR
    Will hdfs go ahead with the copy and hence node N will have two copies of the src-file? (ie. one on HDFS namespace and another in the local file system)


    Thanks,

    Maha
    On Mar 2, 2011, at 12:38 PM, Michael Segel wrote:



    Run is local to your edge machine where you launched your job.
    It then connects to the cluster / job tracker ...

    HTH

    -Mike
    From: maha@umail.ucsb.edu
    Subject: ToolRunner run function
    Date: Wed, 2 Mar 2011 12:10:05 -0800
    To: common-user@hadoop.apache.org

    Hi,

    Assuming my program implements the ToolRunner, my question is where does the "run" function execute? ie. which daemon (DataNode/TT) ? or is it on the local machine where it is run?

    Thank you,
    Maha

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedMar 2, '11 at 8:10p
activeMar 2, '11 at 9:12p
posts4
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Maha: 3 posts Michael Segel: 1 post

People

Translate

site design / logo © 2022 Grokbase