Hi,

I want to read a file that has 100MB of size and it is in the HDFS. How
should I do it? Is it with IOUtils.readFully?

Can anyone give me an example?



--
Thanks,




--
Thanks,

Search Discussions

  • Uma Maheswara Rao G at Dec 16, 2011 at 4:52 pm
    Yes you can use utility methods from IOUtils

    ex:
    FileOutputStream fo = new FileOutputStream (file);
    IOUtils.copyBytes(fs.open(fileName), fo, 1024, true);
    here fs is DFS stream.

    other option is, you can make use of FileSystem apis.

    EX:
    FileSystem fs=new DistributedFileSystem();
    fs.initialize(new URI("namenode_uri"), conf);
    fs.copyToLocalFile(new Path(SRC_PATH),new Path(DST_PATH));

    here you can give source path as DFS path and dst path local fileststem path.

    hope it helps.

    Regards,
    Uma
    ________________________________
    From: Pedro Costa [psdc1978@gmail.com]
    Sent: Friday, December 16, 2011 9:27 PM
    To: mapreduce-user
    Subject: Fwd: how read a file in HDFS?



    Hi,

    I want to read a file that has 100MB of size and it is in the HDFS. How should I do it? Is it with IOUtils.readFully?

    Can anyone give me an example?



    --
    Thanks,




    --
    Thanks,

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmapreduce-user @
categorieshadoop
postedDec 16, '11 at 3:58p
activeDec 16, '11 at 4:52p
posts2
users2
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase