[ https://issues.apache.org/jira/browse/HADOOP-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12573306#action_12573306 ]

Runping Qi commented on HADOOP-2906:

I think the extra javac warning is due to the
@SuppressWarnings("unchecked") directive in the following code

public void write(WritableComparable key, Writable value) throws IOException {

// get the file name based on the key
String keyBasedPath = generateFileNameForKey(key, myName);

// get the file name based on the input file name
String finalPath = getInputFileBasedOutputFileName(myJob, keyBasedPath);

// get the actual key
WritableComparable actualKey = generateActualKey(key);

RecordWriter rw = this.recordWriters.get(finalPath);
if (rw == null) {
// if we don't have the record writer yet for the final path, create one
// and add it to the cache
rw = getRecordWriter_inner(myFS, myJob, finalPath, myProgressable);
this.recordWriters.put(finalPath, rw);
rw.write(actualKey, value);
Since javac warns about
rw.write(actualKey, value)
The reason for that is rw is RecordWriter type, not the parameterized one.
The reason for that is that rw may be a record writer generated by SequenceFileOutputFormat
which does not generate object of parameterized RecordWriter. Tried a few ways to get rid of the warning, but all failed.

output format classes that can write to different files depending on keys and/or config variable

Key: HADOOP-2906
URL: https://issues.apache.org/jira/browse/HADOOP-2906
Project: Hadoop Core
Issue Type: New Feature
Components: mapred
Reporter: Runping Qi
Assignee: Runping Qi
Attachments: patch.2096.txt

I've a few apps that require to write out data into different files/directories depending on keys and/or configuration variables.
I've implemented such classes for those apps. I noticed that many other users have similar need from time to time.
So I think it may be a good idea to contribute to Hadoop mapred.lib package so that other users can benefit from it.
This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

Search Discussions

Discussion Posts


Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 7 of 35 | next ›
Discussion Overview
groupcommon-dev @
postedFeb 26, '08 at 11:31p
activeMar 6, '08 at 12:30p

1 user in discussion

Hudson (JIRA): 35 posts



site design / logo © 2022 Grokbase