FAQ
Gentles,

I want to use the CombineFileInputFormat of Hadoop 0.20.0 / 0.20.2 such
that it processes 1 file per record and also doesn't compromise on data -
locality (which it normally takes care of).

It is mentioned in Tom White's Hadoop Definitive Guide but he has not shown
how to do it. Instead, he moves on to Sequence Files.

I am pretty confused on what is the meaning of processed variable in a
record reader. Any code example would be of tremendous help.

Thanks in advance..
Cheers!
Manoj.

Search Discussions

  • Harsh J at Jul 12, 2012 at 3:55 pm
    Hey Manoj,

    I find the asker name here quite strange, although it is the same
    question, ha: http://stackoverflow.com/questions/10380200/how-to-use-combinefileinputformat-in-hadoop

    Anyhow, here's one example:
    http://blog.yetitrails.com/2011/04/dealing-with-lots-of-small-files-in.html
    On Thu, Jul 12, 2012 at 8:33 PM, Manoj Babu wrote:
    Gentles,

    I want to use the CombineFileInputFormat of Hadoop 0.20.0 / 0.20.2 such that
    it processes 1 file per record and also doesn't compromise on data -
    locality (which it normally takes care of).

    It is mentioned in Tom White's Hadoop Definitive Guide but he has not shown
    how to do it. Instead, he moves on to Sequence Files.

    I am pretty confused on what is the meaning of processed variable in a
    record reader. Any code example would be of tremendous help.

    Thanks in advance..

    Cheers!
    Manoj.


    --
    Harsh J
  • Manoj Babu at Jul 12, 2012 at 4:10 pm
    Ya Harsh its been posted 2 months back no response, if you google for
    CombineFileInputFormat sure you will see it.

    Mapredue user group rocks!
    Thanks for the response.
    On 12 Jul 2012 21:25, "Harsh J" wrote:

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmapreduce-user @
categorieshadoop
postedJul 12, '12 at 3:04p
activeJul 12, '12 at 4:10p
posts3
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Manoj Babu: 2 posts Harsh J: 1 post

People

Translate

site design / logo © 2021 Grokbase