FAQ
Hi all,

I want to write a file into HDFS from outside the Hadoop cluser.
The hadoop runs on nodeA, nodeB, nodeC, ...
but my program runs on another node on the same local network (but not
in the hadoop cluser),
so, how to write a file into the HDFS from this node?

Could anyone help?
Thanks!

Search Discussions

  • Ayon Sinha at Jul 25, 2011 at 6:00 am
    You need to have the hdfs classes on your client machine, which will provide you with the HDFS API to read/write from the cluster. Your client machine does not have to be a part of the cluster.

    -Ayon
    See My Photos on Flickr
    Also check out my Blog for answers to commonly asked questions.



    ________________________________
    From: Yang Xiaoliang <yangxiaoliang2006@gmail.com>
    To: hdfs-user@hadoop.apache.org
    Sent: Sunday, July 24, 2011 9:18 PM
    Subject: access HDFS file from outside the Hadoop Cluster

    Hi all,

    I want to write a file into HDFS from outside the Hadoop cluser.
    The hadoop runs on nodeA, nodeB, nodeC, ...
    but my program runs on another node on the same local network (but not
    in the hadoop cluser),
    so, how to write a file into the HDFS from this node?

    Could anyone help?
    Thanks!
  • Uma Mahesh at Jul 25, 2011 at 9:18 am
    Hi ,

    If you want to write the files from out side of the cluster, you can put
    the hadoop-common*.jar and hadoop-hdfs*.jar into your classpath.

    There will be one class FileSystem.java in that libraries, this class will
    have the fileSystem APIs. (For example copyToLocal, copyFromLocal, rename ,
    delete..etc)

    using this apis, you can do.

    FileSystem fs=new DistributedFileSystem();
    fs.initialize( <nameNode URI {ex: hdfs//10.18.52.61:9000/} >, <configObj
    {new Configuration()}>);
    fs.copyFromLocal(.....);

    This should work for you.


    Regards,
    Uma

    -----Original Message-----
    From: Yang Xiaoliang
    Sent: Monday, July 25, 2011 9:48 AM
    To: hdfs-user@hadoop.apache.org
    Subject: access HDFS file from outside the Hadoop Cluster

    Hi all,

    I want to write a file into HDFS from outside the Hadoop cluser.
    The hadoop runs on nodeA, nodeB, nodeC, ...
    but my program runs on another node on the same local network (but not in
    the hadoop cluser), so, how to write a file into the HDFS from this node?

    Could anyone help?
    Thanks!

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouphdfs-user @
categorieshadoop
postedJul 25, '11 at 4:19a
activeJul 25, '11 at 9:18a
posts3
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase