On Nov 8, 2007, at 5:14 PM, Milind A Bhandarkar wrote:
Does pipes deserializes and serializes data for the identity
mappers or just "passes it through" ? (Streaming converts input to
text, afaik)
Does pipes deserializes and serializes data for the identity
mappers or just "passes it through" ? (Streaming converts input to
text, afaik)
program. The C++ program gets them as C++ strings, which are
effectively byte arrays. Pipes does not do the conversion to Java
strings that streaming does. Therefore, pipes can support arbitrary
Writable objects. Hopefully in the future, we can change the map/
reduce api to provide access to the raw bytes in the mapper and
reducer as an option. In that case, pipes would not need to serialize
at all.
-- Owen