I have an array computed in map stage and want its ouptut recorded or sent to file. Reducer can be naive and just send the output to file. Need to get a text output written to file for this array.

Search Discussions

  • Praveen Sripati at Dec 4, 2011 at 3:34 pm
    Matt,

    You can extend ArrayWritable. Also use TextOutputFormat as the output
    format.

    In the TextOutputFormat key.toString() and value.toString() are called, so
    override toString() in the subclass of ArrayWritable to get the desired
    output format for the array. If toString() is not overridden then class
    name and hex of the hash code is written to the output.

    Regards,
    Praveen
    On Sun, Dec 4, 2011 at 6:56 PM, Matt Denn wrote:

    I have an array computed in map stage and want its ouptut recorded or sent
    to file. Reducer can be naive and just send the output to file. Need to get
    a text output written to file for this array.
  • Harsh J at Dec 4, 2011 at 5:15 pm
    Also, you don't need to run a reducer if you do not want one. Set your number of reduce tasks to 0 before launching your job.
    On 04-Dec-2011, at 9:03 PM, Praveen Sripati wrote:


    Matt,

    You can extend ArrayWritable. Also use TextOutputFormat as the output format.

    In the TextOutputFormat key.toString() and value.toString() are called, so override toString() in the subclass of ArrayWritable to get the desired output format for the array. If toString() is not overridden then class name and hex of the hash code is written to the output.

    Regards,
    Praveen

    On Sun, Dec 4, 2011 at 6:56 PM, Matt Denn wrote:
    I have an array computed in map stage and want its ouptut recorded or sent to file. Reducer can be naive and just send the output to file. Need to get a text output written to file for this array.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmapreduce-user @
categorieshadoop
postedDec 4, '11 at 1:28p
activeDec 4, '11 at 5:15p
posts3
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase