FAQ
[ https://issues.apache.org/jira/browse/HADOOP-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12573306#action_12573306 ]

Runping Qi commented on HADOOP-2906:
------------------------------------


I think the extra javac warning is due to the
@SuppressWarnings("unchecked") directive in the following code

{code}
@SuppressWarnings("unchecked")
public void write(WritableComparable key, Writable value) throws IOException {

// get the file name based on the key
String keyBasedPath = generateFileNameForKey(key, myName);

// get the file name based on the input file name
String finalPath = getInputFileBasedOutputFileName(myJob, keyBasedPath);

// get the actual key
WritableComparable actualKey = generateActualKey(key);

RecordWriter rw = this.recordWriters.get(finalPath);
if (rw == null) {
// if we don't have the record writer yet for the final path, create one
// and add it to the cache
rw = getRecordWriter_inner(myFS, myJob, finalPath, myProgressable);
this.recordWriters.put(finalPath, rw);
}
rw.write(actualKey, value);
};
{code}
Since javac warns about
rw.write(actualKey, value)
The reason for that is rw is RecordWriter type, not the parameterized one.
The reason for that is that rw may be a record writer generated by SequenceFileOutputFormat
which does not generate object of parameterized RecordWriter. Tried a few ways to get rid of the warning, but all failed.

output format classes that can write to different files depending on keys and/or config variable
-------------------------------------------------------------------------------------------------

Key: HADOOP-2906
URL: https://issues.apache.org/jira/browse/HADOOP-2906
Project: Hadoop Core
Issue Type: New Feature
Components: mapred
Reporter: Runping Qi
Assignee: Runping Qi
Attachments: patch.2096.txt


I've a few apps that require to write out data into different files/directories depending on keys and/or configuration variables.
I've implemented such classes for those apps. I noticed that many other users have similar need from time to time.
So I think it may be a good idea to contribute to Hadoop mapred.lib package so that other users can benefit from it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 7 of 35 | next ›
Discussion Overview
groupcommon-dev @
categorieshadoop
postedFeb 26, '08 at 11:31p
activeMar 6, '08 at 12:30p
posts35
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Hudson (JIRA): 35 posts

People

Translate

site design / logo © 2022 Grokbase