FAQ
proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
----------------------------------------------------------------------------------------

Key: HADOOP-5851
URL: https://issues.apache.org/jira/browse/HADOOP-5851
Project: Hadoop Core
Issue Type: New Feature
Components: contrib/hdfsproxy
Reporter: zhiyong zhang
Assignee: zhiyong zhang
Priority: Critical


It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.

when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • zhiyong zhang (JIRA) at May 29, 2009 at 4:06 am
    [ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    zhiyong zhang updated HADOOP-5851:
    ----------------------------------

    Attachment: HADOOP-5851.patch

    a) add another filter code named LdapIpDirFilter.java, will be installed as a deployment descriptor (in web.xml).

    b) program flowchart.
    1. IP = HttpServletRequest.getRemoteAddr()
    2. Ldap.search(uniqueMember:cn=IP) to find role.
    2.a. if nothing found, 403 return
    3. role A was found, from role A, extract userId and HDFS root paths allowed, say PLIST.
    4. log userId.
    5. get PATH = HttpServletRequest.getPathInfo()
    6. check if PATH or its parents is in PLIST
    6.a. if not, 403 return.
    7. access allowed, filter pass

    c) use a Dummy LDAP Object server to do the unit test.
    proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
    ----------------------------------------------------------------------------------------

    Key: HADOOP-5851
    URL: https://issues.apache.org/jira/browse/HADOOP-5851
    Project: Hadoop Core
    Issue Type: New Feature
    Components: contrib/hdfsproxy
    Reporter: zhiyong zhang
    Assignee: zhiyong zhang
    Priority: Critical
    Attachments: HADOOP-5851.patch


    It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
    when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • zhiyong zhang (JIRA) at Jun 2, 2009 at 9:15 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    zhiyong zhang updated HADOOP-5851:
    ----------------------------------

    Attachment: HADOOP-5851.patch

    use hostname.split("[-\\.]") to extract servlet context path (war file name)
    proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
    ----------------------------------------------------------------------------------------

    Key: HADOOP-5851
    URL: https://issues.apache.org/jira/browse/HADOOP-5851
    Project: Hadoop Core
    Issue Type: New Feature
    Components: contrib/hdfsproxy
    Reporter: zhiyong zhang
    Assignee: zhiyong zhang
    Priority: Critical
    Attachments: HADOOP-5851.patch, HADOOP-5851.patch


    It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
    when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • zhiyong zhang (JIRA) at Jun 4, 2009 at 4:51 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    zhiyong zhang updated HADOOP-5851:
    ----------------------------------

    Attachment: HADOOP-5851.patch

    in client side (HsftpFileSystem.java), use a DummyTrustStoreManager to accept all server certificates, in case
    ssl-client.xml is not defined.

    like curl's -k option.

    use this client, user will not need to worry about importing server CA to truststore.


    proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
    ----------------------------------------------------------------------------------------

    Key: HADOOP-5851
    URL: https://issues.apache.org/jira/browse/HADOOP-5851
    Project: Hadoop Core
    Issue Type: New Feature
    Components: contrib/hdfsproxy
    Reporter: zhiyong zhang
    Assignee: zhiyong zhang
    Priority: Critical
    Attachments: HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch


    It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
    when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Kan Zhang (JIRA) at Jun 4, 2009 at 8:53 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716397#action_12716397 ]

    Kan Zhang commented on HADOOP-5851:
    -----------------------------------

    @zhiyong, why you want to do that? If SSL authentication is not needed, one can use HFTP instead of HSFTP.
    proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
    ----------------------------------------------------------------------------------------

    Key: HADOOP-5851
    URL: https://issues.apache.org/jira/browse/HADOOP-5851
    Project: Hadoop Core
    Issue Type: New Feature
    Components: contrib/hdfsproxy
    Reporter: zhiyong zhang
    Assignee: zhiyong zhang
    Priority: Critical
    Attachments: HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch


    It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
    when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Kan Zhang (JIRA) at Jun 4, 2009 at 8:57 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716401#action_12716401 ]

    Kan Zhang commented on HADOOP-5851:
    -----------------------------------

    I can see that for encryption purpose you may still want to use SSL. In that case, I suggest you make it an option instead of the default behavior. The user has to explicitly specify this option to enable it.
    proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
    ----------------------------------------------------------------------------------------

    Key: HADOOP-5851
    URL: https://issues.apache.org/jira/browse/HADOOP-5851
    Project: Hadoop Core
    Issue Type: New Feature
    Components: contrib/hdfsproxy
    Reporter: zhiyong zhang
    Assignee: zhiyong zhang
    Priority: Critical
    Attachments: HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch


    It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
    when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • zhiyong zhang (JIRA) at Jun 8, 2009 at 6:52 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    zhiyong zhang updated HADOOP-5851:
    ----------------------------------

    Attachment: HADOOP-5851.patch

    1.) As suggested by Kan, trust all server certs should not be the default setting.

    Add
    <property>
    <name>ssl.client.do.not.authenticate.server</name>
    <value>false</value>
    <description> if true, trust all server certificates
    </description>
    </property>
    in ssl-client.xml configuration. By default, still need to validate server certificates.

    2.) Did a code walk-through with Rob. W. Also discussed with Kan, the should get the group information from LDAP server instead of using proxyUgiManager. changed this part so that proxy will pass HDFS userId and group info to source cluster.

    3.) Merged all configuration files into one configuration file (hdfsproxy-default.xml), this saves some effort to manage the configuration files and war files.
    The configuration in hdfsproxy-default.xml should contain
    <name>fs.default.name</name>
    <name>dfs.block.size</name>
    <name>io.file.buffer.size</name>
    in addition to ldap-based properties.




    proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
    ----------------------------------------------------------------------------------------

    Key: HADOOP-5851
    URL: https://issues.apache.org/jira/browse/HADOOP-5851
    Project: Hadoop Core
    Issue Type: New Feature
    Components: contrib/hdfsproxy
    Reporter: zhiyong zhang
    Assignee: zhiyong zhang
    Priority: Critical
    Attachments: HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch


    It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
    when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 23, 2009 at 11:25 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12723360#action_12723360 ]

    Chris Douglas commented on HADOOP-5851:
    ---------------------------------------

    The patch needs to be regenerated for the code split.

    The contents look fine, though I'd like to see the TrustManager path completely replace the System.setProperty idiom now that it is understood. The only other minor nit would be here:
    {noformat}
    + if (LOG.isDebugEnabled())
    + e.printStackTrace();
    {noformat}
    Using {{LOG.debug("Useful message", e)}} is more standard.
    proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
    ----------------------------------------------------------------------------------------

    Key: HADOOP-5851
    URL: https://issues.apache.org/jira/browse/HADOOP-5851
    Project: Hadoop Common
    Issue Type: New Feature
    Components: contrib/hdfsproxy
    Reporter: zhiyong zhang
    Assignee: zhiyong zhang
    Priority: Critical
    Attachments: HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch


    It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
    when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Philip Zeyliger (JIRA) at Jun 23, 2009 at 11:39 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12723363#action_12723363 ]

    Philip Zeyliger commented on HADOOP-5851:
    -----------------------------------------

    I'm traveling overseas and will return on June 29th. For urgent
    matters, please contact Amr Awadallah.

    proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
    ----------------------------------------------------------------------------------------

    Key: HADOOP-5851
    URL: https://issues.apache.org/jira/browse/HADOOP-5851
    Project: Hadoop Common
    Issue Type: New Feature
    Components: contrib/hdfsproxy
    Reporter: zhiyong zhang
    Assignee: zhiyong zhang
    Priority: Critical
    Attachments: HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch


    It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
    when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • zhiyong zhang (JIRA) at Jun 24, 2009 at 9:57 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    zhiyong zhang updated HADOOP-5851:
    ----------------------------------

    Attachment: HADOOP-5851.patch

    1. removed System.setProperty. Use Keymanagers and TrustManagers instead.

    2. re-patched in new hdfs trunk after project split.

    3. replaced the printStackTrace with normal LOG.debug(e.toString()).
    proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
    ----------------------------------------------------------------------------------------

    Key: HADOOP-5851
    URL: https://issues.apache.org/jira/browse/HADOOP-5851
    Project: Hadoop Common
    Issue Type: New Feature
    Components: contrib/hdfsproxy
    Reporter: zhiyong zhang
    Assignee: zhiyong zhang
    Priority: Critical
    Attachments: HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch


    It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
    when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • zhiyong zhang (JIRA) at Jun 24, 2009 at 11:46 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    zhiyong zhang updated HADOOP-5851:
    ----------------------------------

    Attachment: HADOOP-5851.patch

    corrected several typos and changed several places for code optimization.
    proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
    ----------------------------------------------------------------------------------------

    Key: HADOOP-5851
    URL: https://issues.apache.org/jira/browse/HADOOP-5851
    Project: Hadoop Common
    Issue Type: New Feature
    Components: contrib/hdfsproxy
    Reporter: zhiyong zhang
    Assignee: zhiyong zhang
    Priority: Critical
    Attachments: HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch


    It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
    when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • zhiyong zhang (JIRA) at Jun 25, 2009 at 1:33 am
    [ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    zhiyong zhang updated HADOOP-5851:
    ----------------------------------

    Attachment: HADOOP-5851.patch

    remove find-bug warnings.
    proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
    ----------------------------------------------------------------------------------------

    Key: HADOOP-5851
    URL: https://issues.apache.org/jira/browse/HADOOP-5851
    Project: Hadoop Common
    Issue Type: New Feature
    Components: contrib/hdfsproxy
    Reporter: zhiyong zhang
    Assignee: zhiyong zhang
    Priority: Critical
    Attachments: HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch


    It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
    when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedMay 15, '09 at 6:54p
activeJun 25, '09 at 1:33a
posts12
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

zhiyong zhang (JIRA): 12 posts

People

Translate

site design / logo © 2022 Grokbase