FAQ
Hadoop 0.21 using the new API. All working.
Then I try to use MultipleOutputs in my reducer:

private MultipleOutputs<Text, Text> mos;

protected void setup(Context context) throws IOException,
InterruptedException {
mos = new MultipleOutputs<Text, Text>(context);
}

protected String generateFileName(Text k, Text v) {
return "mos";
}

public void reduce(Text key, Iterable<Text> values, Context
context) throws IOException, InterruptedException {
mos.write("test", key, new Text(""), "test");
mos.write("test", key, new Text(""));
}

And start it up

Job job = Job.getInstance(cluster);
job.setJarByClass(Test.class);
job.setInputFormatClass(MyInputFormat.class);
job.setMapperClass(FanoutMapper.class);
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(Text.class);
job.setCombinerClass(FanoutReducer.class);
job.setReducerClass(FanoutReducer.class);
job.setOutputFormatClass(SequenceFileOutputFormat.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(Text.class);
SequenceFileOutputFormat.setOutputPath(job, new Path(outputPath +
"/output"));
MultipleOutputs.addNamedOutput(job, "test",
SequenceFileOutputFormat.class, Text.class, Text.class);

job.waitForCompletion(true);

But suddenly I get TONS of

jvm.JvmMetrics: Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized

and it feels much slower. (Maybe just because of the logging?)

What's going on?

Also I am wondering how I can specify the path for the output.
I want to use just the "test" output for another job.

cheers
--
Torsten

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmapreduce-user @
categorieshadoop
postedJun 10, '10 at 1:23p
activeJun 10, '10 at 1:23p
posts1
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Torsten Curdt: 1 post

People

Translate

site design / logo © 2022 Grokbase