FAQ
We have a non-HDFS cluster (it's not a S3 cluster either). We would like to
use Hadoop distcp to copy some of the files to a Hadoop cluster. Is it
currently supported?

Thanks.
--
View this message in context: http://old.nabble.com/Can-distcp-handle-the-copy-from-a-non-HDFS-cluster-to-a-HDFS-cluster--tp29967710p29967710.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.

Search Discussions

  • Aniket ray at Oct 20, 2010 at 4:29 am
    Hi,

    Did you get any way to copy from a non-HDFS linux cluster to another HDFS
    cluster?

    Thanks,
    Aniket Ray
    On Fri, Oct 15, 2010 at 5:39 AM, FreeFoodEveryday wrote:


    We have a non-HDFS cluster (it's not a S3 cluster either). We would like to
    use Hadoop distcp to copy some of the files to a Hadoop cluster. Is it
    currently supported?

    Thanks.
    --
    View this message in context:
    http://old.nabble.com/Can-distcp-handle-the-copy-from-a-non-HDFS-cluster-to-a-HDFS-cluster--tp29967710p29967710.html
    Sent from the Hadoop core-user mailing list archive at Nabble.com.
  • Harsh J at Oct 20, 2010 at 12:18 pm
    If Hadoop can work with it, then sure. For example, the following
    works since the uris are acceptable by Hadoop:

    hadoop distcp hdfs://master:9000/user/hadoop/somefile.txt
    file:///home/hadoop/abc.def

    However, doing the above won't ensure it comes back to your machine,
    instead abc.def would be created on the node the map task ran on,
    cause that's how file:// is resolved. Just a heads up if you're gonna
    use that on a non NFS thingy.
    On Fri, Oct 15, 2010 at 5:39 AM, FreeFoodEveryday wrote:

    We have a non-HDFS cluster (it's not a S3 cluster either). We would like to
    use Hadoop distcp to copy some of the files to a Hadoop cluster. Is it
    currently supported?

    Thanks.
    --
    View this message in context: http://old.nabble.com/Can-distcp-handle-the-copy-from-a-non-HDFS-cluster-to-a-HDFS-cluster--tp29967710p29967710.html
    Sent from the Hadoop core-user mailing list archive at Nabble.com.


    --
    Harsh J
    www.harshj.com

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedOct 15, '10 at 12:09a
activeOct 20, '10 at 12:18p
posts3
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase