FAQ
Hello,

HDFS supports http read-only access to filesystem. Is it possible to use wget to download file using some url like
http://<namenode>:<web<http://%3cnamenode%3e:%3cweb> gui port>/..

Thanks
Jaydeep


DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.

Search Discussions

  • Harsh J at Jun 15, 2010 at 4:08 pm
    Sure you can. A http download option is also provided in the DataNode
    web-interface (def-port:50075).

    Use the "streamFile" feature of the same. An example follows.

    If I have a file called 'results' lying as /user/hadoop/results, I'll do:
    wget http://hostname.top.dom:50075/streamFile?filename=/user/hadoop/results
    -O results

    This will get me the file data in the wget-local file 'results'

    On Tue, Jun 15, 2010 at 7:12 PM, Jaydeep Ayachit
    wrote:
    Hello,

    HDFS supports http read-only access to filesystem. Is it possible to use wget to download file using some url like
    http://<namenode>:<web<http://%3cnamenode%3e:%3cweb> gui port>/..

    Thanks
    Jaydeep


    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.


    --
    Harsh J
    www.harshj.com
  • Jaydeep Ayachit at Jun 15, 2010 at 4:30 pm
    Thanks, data node may not be known. Is it possible to direct url to namenode and namenode handling streaming by fetching data from various data nodes?

    Regards
    Jaydeep

    -----Original Message-----
    From: Harsh J
    Sent: Tuesday, June 15, 2010 9:38 PM
    To: common-user@hadoop.apache.org
    Subject: Re: Using wget to download file from HDFS

    Sure you can. A http download option is also provided in the DataNode
    web-interface (def-port:50075).

    Use the "streamFile" feature of the same. An example follows.

    If I have a file called 'results' lying as /user/hadoop/results, I'll do:
    wget http://hostname.top.dom:50075/streamFile?filename=/user/hadoop/results
    -O results

    This will get me the file data in the wget-local file 'results'

    On Tue, Jun 15, 2010 at 7:12 PM, Jaydeep Ayachit
    wrote:
    Hello,

    HDFS supports http read-only access to filesystem. Is it possible to use wget to download file using some url like
    http://<namenode>:<web<http://%3cnamenode%3e:%3cweb> gui port>/..

    Thanks
    Jaydeep


    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.


    --
    Harsh J
    www.harshj.com

    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
  • Edward Capriolo at Jun 15, 2010 at 5:13 pm

    On Tue, Jun 15, 2010 at 12:30 PM, Jaydeep Ayachit wrote:

    Thanks, data node may not be known. Is it possible to direct url to
    namenode and namenode handling streaming by fetching data from various data
    nodes?

    Regards
    Jaydeep

    -----Original Message-----
    From: Harsh J
    Sent: Tuesday, June 15, 2010 9:38 PM
    To: common-user@hadoop.apache.org
    Subject: Re: Using wget to download file from HDFS

    Sure you can. A http download option is also provided in the DataNode
    web-interface (def-port:50075).

    Use the "streamFile" feature of the same. An example follows.

    If I have a file called 'results' lying as /user/hadoop/results, I'll do:
    wget
    http://hostname.top.dom:50075/streamFile?filename=/user/hadoop/results
    -O<http://hostname.top.dom:50075/streamFile?filename=/user/hadoop/results%0A-O>results

    This will get me the file data in the wget-local file 'results'

    On Tue, Jun 15, 2010 at 7:12 PM, Jaydeep Ayachit
    wrote:
    Hello,

    HDFS supports http read-only access to filesystem. Is it possible to use
    wget to download file using some url like
    http://<namenode>:<web<http://%3cnamenode%3e:%3cweb> gui port>/..

    Thanks
    Jaydeep


    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information which is
    the property of Persistent Systems Ltd. It is intended only for the use of
    the individual or entity to which it is addressed. If you are not the
    intended recipient, you are not authorized to read, retain, copy, print,
    distribute or use this message. If you have received this communication in
    error, please notify the sender and delete all copies of this message.
    Persistent Systems Ltd. does not accept any liability for virus infected
    mails.


    --
    Harsh J
    www.harshj.com

    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information which is
    the property of Persistent Systems Ltd. It is intended only for the use of
    the individual or entity to which it is addressed. If you are not the
    intended recipient, you are not authorized to read, retain, copy, print,
    distribute or use this message. If you have received this communication in
    error, please notify the sender and delete all copies of this message.
    Persistent Systems Ltd. does not accept any liability for virus infected
    mails.

    To accomplish something like this: You have to use the name node web
    interface and extract the names of datanodes from the html, then follow the
    above process.

    :::Edward reaches in his bag of tricks:::

    Or you can kick up a webserver with tomcat to serve HDFS.

    http://www.edwardcapriolo.com/wiki/en/Tomcat_Hadoop
  • Owen O'Malley at Jun 15, 2010 at 6:15 pm

    On Jun 15, 2010, at 9:30 AM, Jaydeep Ayachit wrote:

    Thanks, data node may not be known. Is it possible to direct url to
    namenode and namenode handling streaming by fetching data from
    various data nodes?
    If you access the servlet on the NameNode, it will automatically
    redirect you to a data node that has some of the data on it. You
    certainly should not pick a random data node yourself.

    Also note that in yahoo 0.20.104 or 0.22, you'll need a Kerberos
    ticket or delegation token to use the servlet.

    -- Owen

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedJun 15, '10 at 1:43p
activeJun 15, '10 at 6:15p
posts5
users4
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase