So I've spent quite a bit of time on trying to get lzo working with
Right now I'm trying to use cascalog-lzo, and I run into this error.
ERROR lzo.GPLNativeCodeLoader: Could not load native gpl library
java.lang.UnsatisfiedLinkError: no gplcompression in java.library.path
I first tried compiling the hadoop-gpl-compression, and placing the
binaries and jar in HADOOP_HOME/lib/native/arch/ and HADOOP_HOME/lib/
respectively. Then I tried the same thing with twitter's hadoop-lzo
thinking maybe that would work instead? Neither worked.
I believe I need to somehow load the gplcompression jar inside the cascalog
project, but I thought cascalog-lzo was suppose to handle this for me?
I'm on arch-linux-64bit
My project just copies a txt file to an lzo compressed text file
[cascalog.more-taps :only (hfs-delimited)]
(defn -main [in out & args]
(?<- (hfs-lzo-textline out)
((hfs-delimited in :skip-header? true) ?doc ?line)))
Is there something special I need to do with hadoop, to allow lzo
compression, or is there something special I need to do with my project? or
Any help is appreciated! Thank you
PS. Unfortunately the latest version of cascalog-lzo fails in lein deps
because the dependency "com.google:protobuf-java:jar:2.3.0" should be