I got the following error when I tried to do gzip compression on map output, using hadoop-0.20.1.
settings in mapred-site.xml-->
I read the src that Writer in IFile takes care of map output compression. So, it seems to me that I didn't have gzip native library built or didn't have correct settings. There is no "built" folder in HADOOP_HOME and no "native" in "lib" folder in HADOOP_HOME. I checked that I have gzip and zlib installed. So, next is to build hadoop native library on top of these. How to do that? Is it a simple matter of pointing some variable to gzip or zlib libs or should I use "build.xml" in hadoop to build some target, what target should I build?
Grokbase › Groups › Hadoop › common-user › March 2010