FAQ
Hi,
I have installed lzo-2.03 to my Linux box.
But still my code for writing a SequenceFile using LZOcodec returns the following error:

util.NativeCodeLoader: Loaded the native-hadoop library
java.lang.UnsatisfiedLinkError: Cannot load liblzo2.so!

What needs to be done to make this work?

Thanks a lot,
-Songting

Search Discussions

  • Arun C Murthy at Oct 10, 2008 at 1:38 am

    On Oct 9, 2008, at 5:58 PM, Songting Chen wrote:

    Hi,
    I have installed lzo-2.03 to my Linux box.
    But still my code for writing a SequenceFile using LZOcodec returns
    the following error:

    util.NativeCodeLoader: Loaded the native-hadoop library
    java.lang.UnsatisfiedLinkError: Cannot load liblzo2.so!

    What needs to be done to make this work?
    http://hadoop.apache.org/core/docs/current/native_libraries.html

    Arun
  • Songting Chen at Oct 10, 2008 at 1:48 am
    Thanks, Arun.

    Does that mean I have to rebuild the native library?

    Also, the LZO installation puts liblzo2.a and liblzo2.la under /usr/local/lib. There is no liblzo2.so there. Do I need to rename them to liblzo2.so somehow?


    --- On Thu, 10/9/08, Arun C Murthy wrote:
    From: Arun C Murthy <acm@yahoo-inc.com>
    Subject: Re: How to make LZO work?
    To: core-user@hadoop.apache.org
    Date: Thursday, October 9, 2008, 6:35 PM
    On Oct 9, 2008, at 5:58 PM, Songting Chen wrote:

    Hi,
    I have installed lzo-2.03 to my Linux box.
    But still my code for writing a SequenceFile using
    LZOcodec returns
    the following error:

    util.NativeCodeLoader: Loaded the native-hadoop library
    java.lang.UnsatisfiedLinkError: Cannot load
    liblzo2.so!
    What needs to be done to make this work?
    http://hadoop.apache.org/core/docs/current/native_libraries.html

    Arun
  • Arun C Murthy at Oct 10, 2008 at 7:43 am

    On Oct 9, 2008, at 6:46 PM, Songting Chen wrote:

    Thanks, Arun.

    Does that mean I have to rebuild the native library?
    No, hadoop releases come bundled with pre-built 32/64 bit libhadoop.so
    for Linux...

    Arun
    Also, the LZO installation puts liblzo2.a and liblzo2.la under /usr/
    local/lib. There is no liblzo2.so there. Do I need to rename them to
    liblzo2.so somehow?


    --- On Thu, 10/9/08, Arun C Murthy wrote:
    From: Arun C Murthy <acm@yahoo-inc.com>
    Subject: Re: How to make LZO work?
    To: core-user@hadoop.apache.org
    Date: Thursday, October 9, 2008, 6:35 PM
    On Oct 9, 2008, at 5:58 PM, Songting Chen wrote:

    Hi,
    I have installed lzo-2.03 to my Linux box.
    But still my code for writing a SequenceFile using
    LZOcodec returns
    the following error:

    util.NativeCodeLoader: Loaded the native-hadoop library
    java.lang.UnsatisfiedLinkError: Cannot load
    liblzo2.so!
    What needs to be done to make this work?
    http://hadoop.apache.org/core/docs/current/native_libraries.html

    Arun
  • Allen Wittenauer at Oct 10, 2008 at 2:48 pm

    On 10/9/08 6:46 PM, "Songting Chen" wrote:
    Does that mean I have to rebuild the native library?

    Also, the LZO installation puts liblzo2.a and liblzo2.la under /usr/local/lib.
    There is no liblzo2.so there. Do I need to rename them to liblzo2.so somehow?

    You need to compile and install lzo2 as a shared library. IIRC, this is
    not the default.


    Also, the shared version (.so) will need to be part of your link path
    (LD_LIBRARY_PATH env var, /etc/ld.so.conf on Linux, runtime option (usually
    -R) to ld, ...) when you fire up the JVM so that Java can locate it when it
    needs it.
  • Songting Chen at Oct 10, 2008 at 6:14 pm
    Thanks, Allen.

    I checked the INSTALL doc in lzo-2.03 package.

    It's said to use './configure --enable-shared' command to build shared library. I followed that instruction and recompile / install the package.

    But after that, still only .a, .la appear in /usr/local/lib.

    -rw-r--r-- 1 root root 855742 Oct 10 10:50 liblzo2.a
    -rwxr-xr-x 1 root root 740 Oct 10 10:50 liblzo2.la

    How do I generate the liblzo2.so file?

    --- On Fri, 10/10/08, Allen Wittenauer wrote:
    From: Allen Wittenauer <aw@yahoo-inc.com>
    Subject: Re: How to make LZO work?
    To: core-user@hadoop.apache.org
    Date: Friday, October 10, 2008, 7:44 AM
    On 10/9/08 6:46 PM, "Songting Chen"
    wrote:
    Does that mean I have to rebuild the native library?

    Also, the LZO installation puts liblzo2.a and
    liblzo2.la under /usr/local/lib.
    There is no liblzo2.so there. Do I need to rename them
    to liblzo2.so somehow?


    You need to compile and install lzo2 as a shared
    library. IIRC, this is
    not the default.


    Also, the shared version (.so) will need to be part of
    your link path
    (LD_LIBRARY_PATH env var, /etc/ld.so.conf on Linux, runtime
    option (usually
    -R) to ld, ...) when you fire up the JVM so that Java can
    locate it when it
    needs it.
  • Songting Chen at Oct 10, 2008 at 7:46 pm
    I switched to lzo-2.02 package. This time liblzo2.so was built.

    Now everything worked.

    Thanks,
    -Songting


    --- On Fri, 10/10/08, Songting Chen wrote:
    From: Songting Chen <ken_cst1998@yahoo.com>
    Subject: Re: How to make LZO work?
    To: core-user@hadoop.apache.org
    Date: Friday, October 10, 2008, 11:14 AM
    Thanks, Allen.

    I checked the INSTALL doc in lzo-2.03 package.

    It's said to use './configure --enable-shared'
    command to build shared library. I followed that instruction
    and recompile / install the package.

    But after that, still only .a, .la appear in
    /usr/local/lib.

    -rw-r--r-- 1 root root 855742 Oct 10 10:50 liblzo2.a
    -rwxr-xr-x 1 root root 740 Oct 10 10:50 liblzo2.la

    How do I generate the liblzo2.so file?

    --- On Fri, 10/10/08, Allen Wittenauer
    wrote:
    From: Allen Wittenauer <aw@yahoo-inc.com>
    Subject: Re: How to make LZO work?
    To: core-user@hadoop.apache.org
    Date: Friday, October 10, 2008, 7:44 AM
    On 10/9/08 6:46 PM, "Songting Chen"
    wrote:
    Does that mean I have to rebuild the native
    library?
    Also, the LZO installation puts liblzo2.a and
    liblzo2.la under /usr/local/lib.
    There is no liblzo2.so there. Do I need to rename
    them
    to liblzo2.so somehow?


    You need to compile and install lzo2 as a shared
    library. IIRC, this is
    not the default.


    Also, the shared version (.so) will need to be part of
    your link path
    (LD_LIBRARY_PATH env var, /etc/ld.so.conf on Linux, runtime
    option (usually
    -R) to ld, ...) when you fire up the JVM so that Java can
    locate it when it
    needs it.
  • Arun C Murthy at Oct 10, 2008 at 8:06 am

    On Oct 10, 2008, at 12:51 AM, Songting Chen wrote:

    Thanks Arun.

    Still sort of confused.

    If I don't need to rebuild the library, after lzo is installed, it's
    still not working.
    Could you please elaborate? What is the error/exception?

    Arun
    My code:

    writer = SequenceFile.createWriter(fileSys, jobConf,
    file, LongWritable.class, BytesWritable.class,
    SequenceFile.CompressionType.BLOCK, new LzoCodec());

    Rebuilding the library gave some weird error too.


    --- On Fri, 10/10/08, Arun C Murthy wrote:
    From: Arun C Murthy <acm@yahoo-inc.com>
    Subject: Re: How to make LZO work?
    To: core-user@hadoop.apache.org
    Date: Friday, October 10, 2008, 12:40 AM
    On Oct 9, 2008, at 6:46 PM, Songting Chen wrote:

    Thanks, Arun.

    Does that mean I have to rebuild the native library?
    No, hadoop releases come bundled with pre-built 32/64 bit
    libhadoop.so
    for Linux...

    Arun
    Also, the LZO installation puts liblzo2.a and
    liblzo2.la under /usr/
    local/lib. There is no liblzo2.so there. Do I need to
    rename them to
    liblzo2.so somehow?


    --- On Thu, 10/9/08, Arun C Murthy
    wrote:
    From: Arun C Murthy <acm@yahoo-inc.com>
    Subject: Re: How to make LZO work?
    To: core-user@hadoop.apache.org
    Date: Thursday, October 9, 2008, 6:35 PM
    On Oct 9, 2008, at 5:58 PM, Songting Chen wrote:

    Hi,
    I have installed lzo-2.03 to my Linux box.
    But still my code for writing a SequenceFile
    using
    LZOcodec returns
    the following error:

    util.NativeCodeLoader: Loaded the
    native-hadoop
    library
    java.lang.UnsatisfiedLinkError: Cannot load
    liblzo2.so!
    What needs to be done to make this work?
    http://hadoop.apache.org/core/docs/current/native_libraries.html
    Arun

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedOct 10, '08 at 12:59a
activeOct 10, '08 at 7:46p
posts8
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase