Grokbase › Groups › Hadoop › common-user › August 2009
FAQ
Is there any way to set the replication to a specific number while
FTP-ing to the HDFS?
I have dfs.replication set to 2 on the nodes, but when I transfer
something via FTP to HDFS it replicates 3 times.

Thanks.
--

-Turner Kunkel

Search Discussions

  • Brian Bockelman at Aug 25, 2009 at 4:32 pm ⇧
    Hey Turner,

    Replication level is requested by the client. You need to set the
    replication factor on the hadoop configuration of the client, not the
    nodes.

    Brian
    On Aug 25, 2009, at 11:28 AM, Turner Kunkel wrote:

    Is there any way to set the replication to a specific number while
    FTP-ing to the HDFS?
    I have dfs.replication set to 2 on the nodes, but when I transfer
    something via FTP to HDFS it replicates 3 times.

    Thanks.
    --

    -Turner Kunkel
  • Thkunkel at Aug 25, 2009 at 5:00 pm ⇧
    Gotcha, thanks.

    Any idea how to do that on a Windows machine? My Hadoop cluster is on 3
    Ubuntu nodes, but I'm transferring data from a bunch of Windows servers. Or
    can I configure replication in the FTP client itself?

    -Turner


    Brian Bockelman wrote:
    Hey Turner,

    Replication level is requested by the client. You need to set the
    replication factor on the hadoop configuration of the client, not the
    nodes.

    Brian
    On Aug 25, 2009, at 11:28 AM, Turner Kunkel wrote:

    Is there any way to set the replication to a specific number while
    FTP-ing to the HDFS?
    I have dfs.replication set to 2 on the nodes, but when I transfer
    something via FTP to HDFS it replicates 3 times.

    Thanks.
    --

    -Turner Kunkel

    --
    View this message in context: http://www.nabble.com/Replication-happens-too-many-time-when-FTP-to-HDFS-tp25137745p25138298.html
    Sent from the Hadoop core-user mailing list archive at Nabble.com.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedAug 25, '09 at 4:28p
activeAug 25, '09 at 5:00p
posts3
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Thkunkel: 2 posts Brian Bockelman: 1 post

Content

People

Support

Translate

site design / logo © 2022 Grokbase