[
https://issues.apache.org/jira/browse/HADOOP-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12573306#action_12573306 ]
Runping Qi commented on HADOOP-2906:
------------------------------------
I think the extra javac warning is due to the
@SuppressWarnings("unchecked") directive in the following code
{code}
@SuppressWarnings("unchecked")
public void write(WritableComparable key, Writable value) throws IOException {
// get the file name based on the key
String keyBasedPath = generateFileNameForKey(key, myName);
// get the file name based on the input file name
String finalPath = getInputFileBasedOutputFileName(myJob, keyBasedPath);
// get the actual key
WritableComparable actualKey = generateActualKey(key);
RecordWriter rw = this.recordWriters.get(finalPath);
if (rw == null) {
// if we don't have the record writer yet for the final path, create one
// and add it to the cache
rw = getRecordWriter_inner(myFS, myJob, finalPath, myProgressable);
this.recordWriters.put(finalPath, rw);
}
rw.write(actualKey, value);
};
{code}
Since javac warns about
rw.write(actualKey, value)
The reason for that is rw is RecordWriter type, not the parameterized one.
The reason for that is that rw may be a record writer generated by SequenceFileOutputFormat
which does not generate object of parameterized RecordWriter. Tried a few ways to get rid of the warning, but all failed.
output format classes that can write to different files depending on keys and/or config variable
-------------------------------------------------------------------------------------------------
Key: HADOOP-2906
URL:
https://issues.apache.org/jira/browse/HADOOP-2906Project: Hadoop Core
Issue Type: New Feature
Components: mapred
Reporter: Runping Qi
Assignee: Runping Qi
Attachments: patch.2096.txt
I've a few apps that require to write out data into different files/directories depending on keys and/or configuration variables.
I've implemented such classes for those apps. I noticed that many other users have similar need from time to time.
So I think it may be a good idea to contribute to Hadoop mapred.lib package so that other users can benefit from it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
