FAQ
Remove explicit dynamic loading of libz in native code
------------------------------------------------------

Key: HADOOP-6069
URL: https://issues.apache.org/jira/browse/HADOOP-6069
Project: Hadoop Core
Issue Type: Improvement
Reporter: Todd Lipcon
Assignee: Todd Lipcon


The native zlib code currently uses dlopen/dlsym to dynamically load libz. This used to make sense when there was an lzo option (so you could load libhadoop for lzo purposes without requiring libz as well). Now that libhadoop only has zlib as an option, it makes sense to just add it as an ld flag and let it be automatically loaded as a shlib dependency. I also doubt that there are any distros where libz isn't required by the base system.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Todd Lipcon (JIRA) at Jun 17, 2009 at 11:28 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Todd Lipcon updated HADOOP-6069:
    --------------------------------

    Attachment: hadoop-6069.txt
    hadoop-6069-guts.txt

    This patch gets rid of the dynamic loading of zlib from the native code. I've attached two versions, one which includes just the changes to the autoconf source files (-guts), and another which has the full patch to be applied to trunk.

    This also cleans up the autotools setup a bit:
    - There's no need to check HAVE_FOO_H cpp conditionals since we can just make configure error out at that point if they aren't found
    - AC_HEADER_STDC and AC_C_CONST are considered obsolescent since any modern system (where one might compile Hadoop) will pass these checks
    - AC_CHECK_FUNCS([memset]) was unnecessary since HAVE_MEMSET was never checked

    Remove explicit dynamic loading of libz in native code
    ------------------------------------------------------

    Key: HADOOP-6069
    URL: https://issues.apache.org/jira/browse/HADOOP-6069
    Project: Hadoop Core
    Issue Type: Improvement
    Reporter: Todd Lipcon
    Assignee: Todd Lipcon
    Attachments: hadoop-6069-guts.txt, hadoop-6069.txt


    The native zlib code currently uses dlopen/dlsym to dynamically load libz. This used to make sense when there was an lzo option (so you could load libhadoop for lzo purposes without requiring libz as well). Now that libhadoop only has zlib as an option, it makes sense to just add it as an ld flag and let it be automatically loaded as a shlib dependency. I also doubt that there are any distros where libz isn't required by the base system.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Todd Lipcon (JIRA) at Jun 18, 2009 at 3:38 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Todd Lipcon updated HADOOP-6069:
    --------------------------------

    Status: Patch Available (was: Open)
    Remove explicit dynamic loading of libz in native code
    ------------------------------------------------------

    Key: HADOOP-6069
    URL: https://issues.apache.org/jira/browse/HADOOP-6069
    Project: Hadoop Core
    Issue Type: Improvement
    Reporter: Todd Lipcon
    Assignee: Todd Lipcon
    Attachments: hadoop-6069-guts.txt, hadoop-6069.txt


    The native zlib code currently uses dlopen/dlsym to dynamically load libz. This used to make sense when there was an lzo option (so you could load libhadoop for lzo purposes without requiring libz as well). Now that libhadoop only has zlib as an option, it makes sense to just add it as an ld flag and let it be automatically loaded as a shlib dependency. I also doubt that there are any distros where libz isn't required by the base system.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Jun 19, 2009 at 5:33 am
    [ https://issues.apache.org/jira/browse/HADOOP-6069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12721657#action_12721657 ]

    Hadoop QA commented on HADOOP-6069:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12411009/hadoop-6069.txt
    against trunk revision 786278.

    +1 @author. The patch does not contain any @author tags.

    +1 tests included. The patch appears to include 1 new or modified tests.

    +1 javadoc. The javadoc tool did not generate any warning messages.

    +1 javac. The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs. The patch does not introduce any new Findbugs warnings.

    +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

    +1 release audit. The applied patch does not increase the total number of release audit warnings.

    -1 core tests. The patch failed core unit tests.

    -1 contrib tests. The patch failed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/532/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/532/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/532/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/532/console

    This message is automatically generated.
    Remove explicit dynamic loading of libz in native code
    ------------------------------------------------------

    Key: HADOOP-6069
    URL: https://issues.apache.org/jira/browse/HADOOP-6069
    Project: Hadoop Core
    Issue Type: Improvement
    Reporter: Todd Lipcon
    Assignee: Todd Lipcon
    Attachments: hadoop-6069-guts.txt, hadoop-6069.txt


    The native zlib code currently uses dlopen/dlsym to dynamically load libz. This used to make sense when there was an lzo option (so you could load libhadoop for lzo purposes without requiring libz as well). Now that libhadoop only has zlib as an option, it makes sense to just add it as an ld flag and let it be automatically loaded as a shlib dependency. I also doubt that there are any distros where libz isn't required by the base system.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Todd Lipcon (JIRA) at Jun 19, 2009 at 5:37 am
    [ https://issues.apache.org/jira/browse/HADOOP-6069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12721658#action_12721658 ]

    Todd Lipcon commented on HADOOP-6069:
    -------------------------------------

    Failing tests are unrelated. This should be tested with the existing TestCodec - not 100% sure whether Hudson runs that by default, but it passes on my machine.
    Remove explicit dynamic loading of libz in native code
    ------------------------------------------------------

    Key: HADOOP-6069
    URL: https://issues.apache.org/jira/browse/HADOOP-6069
    Project: Hadoop Core
    Issue Type: Improvement
    Reporter: Todd Lipcon
    Assignee: Todd Lipcon
    Attachments: hadoop-6069-guts.txt, hadoop-6069.txt


    The native zlib code currently uses dlopen/dlsym to dynamically load libz. This used to make sense when there was an lzo option (so you could load libhadoop for lzo purposes without requiring libz as well). Now that libhadoop only has zlib as an option, it makes sense to just add it as an ld flag and let it be automatically loaded as a shlib dependency. I also doubt that there are any distros where libz isn't required by the base system.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedJun 17, '09 at 4:58p
activeJun 19, '09 at 5:37a
posts5
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Todd Lipcon (JIRA): 5 posts

People

Translate

site design / logo © 2022 Grokbase