Grokbase › Groups › Hadoop › mapreduce-user › March 2011
FAQ
Hi,
I can't seem to be able to find either jira or implementation of
MultipleOutputFormat in new api in either 0.21 or 0.22 branches.
Are there any plans to port that to new api as well?

thanks in advance.
-Dmitriy

Search Discussions

  • Alejandro Abdelnur at Mar 30, 2011 at 3:57 am ⇧
    Dmitriy,

    Have you check the MultipleOutputs instead? It provides similar
    functionality.

    Alejandro
    On Wed, Mar 30, 2011 at 11:39 AM, Dmitriy Lyubimov wrote:

    Hi,
    I can't seem to be able to find either jira or implementation of
    MultipleOutputFormat in new api in either 0.21 or 0.22 branches.
    Are there any plans to port that to new api as well?

    thanks in advance.
    -Dmitriy
  • Dmitriy Lyubimov at Mar 30, 2011 at 4:02 am ⇧
    yes.. but in my old code the file names are created on the fly (it
    basically creates partitions based on a time field). I dont think
    MultipleOutputs is not suitable to create partitions on the fly.
    On Tue, Mar 29, 2011 at 8:56 PM, Alejandro Abdelnur wrote:
    Dmitriy,
    Have you check the MultipleOutputs instead? It provides similar
    functionality.
    Alejandro
    On Wed, Mar 30, 2011 at 11:39 AM, Dmitriy Lyubimov wrote:

    Hi,
    I can't seem to be able to find either jira or implementation of
    MultipleOutputFormat in new api in either 0.21 or 0.22 branches.
    Are there any plans to port that to new api as well?

    thanks in advance.
    -Dmitriy
  • Alejandro Abdelnur at Mar 30, 2011 at 4:10 am ⇧
    You should be able to create partitions on the fly.

    Check the last example in the javadocs:

    http://hadoop.apache.org/mapreduce/docs/r0.21.0/api/org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.html

    ...

    mos.write(key, new Text("value"), generateFileName(key, new Text("value")));

    Hope this helps.

    Alejandro
    On Wed, Mar 30, 2011 at 12:02 PM, Dmitriy Lyubimov wrote:

    yes.. but in my old code the file names are created on the fly (it
    basically creates partitions based on a time field). I dont think
    MultipleOutputs is not suitable to create partitions on the fly.
    On Tue, Mar 29, 2011 at 8:56 PM, Alejandro Abdelnur wrote:
    Dmitriy,
    Have you check the MultipleOutputs instead? It provides similar
    functionality.
    Alejandro

    On Wed, Mar 30, 2011 at 11:39 AM, Dmitriy Lyubimov <dlieu.7@gmail.com>
    wrote:
    Hi,
    I can't seem to be able to find either jira or implementation of
    MultipleOutputFormat in new api in either 0.21 or 0.22 branches.
    Are there any plans to port that to new api as well?

    thanks in advance.
    -Dmitriy
  • Dmitriy Lyubimov at Mar 30, 2011 at 4:13 am ⇧
    Oh. ok, let me look at it. thanks a bunch.
    On Tue, Mar 29, 2011 at 9:09 PM, Alejandro Abdelnur wrote:
    You should be able to create partitions on the fly.
    Check the last example in the javadocs:

    http://hadoop.apache.org/mapreduce/docs/r0.21.0/api/org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.html

    ...

    mos.write(key, new Text("value"), generateFileName(key, new
    Text("value")));

    Hope this helps.
    Alejandro
    On Wed, Mar 30, 2011 at 12:02 PM, Dmitriy Lyubimov wrote:

    yes.. but in my old code the file names are created on the fly (it
    basically creates partitions based on a time field). I dont think
    MultipleOutputs is not suitable to create partitions on the fly.

    On Tue, Mar 29, 2011 at 8:56 PM, Alejandro Abdelnur <tucu@cloudera.com>
    wrote:
    Dmitriy,
    Have you check the MultipleOutputs instead? It provides similar
    functionality.
    Alejandro

    On Wed, Mar 30, 2011 at 11:39 AM, Dmitriy Lyubimov <dlieu.7@gmail.com>
    wrote:
    Hi,
    I can't seem to be able to find either jira or implementation of
    MultipleOutputFormat in new api in either 0.21 or 0.22 branches.
    Are there any plans to port that to new api as well?

    thanks in advance.
    -Dmitriy

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmapreduce-user @
categorieshadoop
postedMar 30, '11 at 3:40a
activeMar 30, '11 at 4:13a
posts5
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Dmitriy Lyubimov: 3 posts Alejandro Abdelnur: 2 posts

Content

People

Support

Translate

site design / logo © 2022 Grokbase