FAQ
Refactor FailoverController/HAAdmin code to add an abstract class for "target" services
---------------------------------------------------------------------------------------

Key: HADOOP-8193
URL: https://issues.apache.org/jira/browse/HADOOP-8193
Project: Hadoop Common
Issue Type: Improvement
Components: ha
Affects Versions: 0.24.0, 0.23.3
Reporter: Todd Lipcon
Assignee: Todd Lipcon


In working at HADOOP-8077, HDFS-3084, and HDFS-3072, I ran into various difficulties which are an artifact of the current design. A few of these:
- the service name is "resolved" from the logical name (eg ns1.nn1) to an IP address at the outer layer of DFSHAAdmin
-- this means it's difficult to provide the logical name "ns1.nn1" to fence scripts (HDFS-3084)
-- this means it's difficult to configure fencing method per-namespace (since the FailoverController doesn't know what the namespace is) (HADOOP-8077)
- the configuration for HA HDFS is weirdly split between core-site and hdfs-site, even though most users see this as an HDFS feature. For example, users expect to configure NN fencing configurations in hdfs-site, and expect the keys to have a dfs.* prefix
- proxies are constructed at the outer layer of the admin commands. This means it's impossible for the inner layers (eg FailoverController.failover) to re-construct proxies with different timeouts (HDFS-3072)

The proposed refactor is to add a new interface (tentatively named HAServiceTarget) which refers to target for one of the admin commands. An instance of this class is responsible for creating proxies, creating fencers, mapping back to a logical name, etc. The HDFS implementation of this class can then provide different results based on the particular nameservice, can use HDFS-specific configuration prefixes, etc. Using this class as the argument for fencing methods also makes the API more evolvable in the future, since we can add new getters to HAServiceTarget (whereas the current InetSocketAddress is quite limiting)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Search Discussions

  • Todd Lipcon (Resolved) (JIRA) at Mar 24, 2012 at 11:50 pm
    [ https://issues.apache.org/jira/browse/HADOOP-8193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Todd Lipcon resolved HADOOP-8193.
    ---------------------------------

    Resolution: Fixed
    Fix Version/s: 0.24.0
    0.23.3
    Hadoop Flags: Reviewed

    Committed to 23 and trunk. Also double-checked findbugs before committing. Thanks for reviews.
    Refactor FailoverController/HAAdmin code to add an abstract class for "target" services
    ---------------------------------------------------------------------------------------

    Key: HADOOP-8193
    URL: https://issues.apache.org/jira/browse/HADOOP-8193
    Project: Hadoop Common
    Issue Type: Improvement
    Components: ha
    Affects Versions: 0.23.3, 0.24.0
    Reporter: Todd Lipcon
    Assignee: Todd Lipcon
    Fix For: 0.23.3, 0.24.0

    Attachments: hadoop-8193.txt, hadoop-8193.txt, hadoop-8193.txt


    In working at HADOOP-8077, HDFS-3084, and HDFS-3072, I ran into various difficulties which are an artifact of the current design. A few of these:
    - the service name is "resolved" from the logical name (eg ns1.nn1) to an IP address at the outer layer of DFSHAAdmin
    -- this means it's difficult to provide the logical name "ns1.nn1" to fence scripts (HDFS-3084)
    -- this means it's difficult to configure fencing method per-namespace (since the FailoverController doesn't know what the namespace is) (HADOOP-8077)
    - the configuration for HA HDFS is weirdly split between core-site and hdfs-site, even though most users see this as an HDFS feature. For example, users expect to configure NN fencing configurations in hdfs-site, and expect the keys to have a dfs.* prefix
    - proxies are constructed at the outer layer of the admin commands. This means it's impossible for the inner layers (eg FailoverController.failover) to re-construct proxies with different timeouts (HDFS-3072)
    The proposed refactor is to add a new interface (tentatively named HAServiceTarget) which refers to target for one of the admin commands. An instance of this class is responsible for creating proxies, creating fencers, mapping back to a logical name, etc. The HDFS implementation of this class can then provide different results based on the particular nameservice, can use HDFS-specific configuration prefixes, etc. Using this class as the argument for fencing methods also makes the API more evolvable in the future, since we can add new getters to HAServiceTarget (whereas the current InetSocketAddress is quite limiting)
    --
    This message is automatically generated by JIRA.
    If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
    For more information on JIRA, see: http://www.atlassian.com/software/jira
  • Todd Lipcon (Reopened) (JIRA) at Mar 25, 2012 at 9:34 pm
    [ https://issues.apache.org/jira/browse/HADOOP-8193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Todd Lipcon reopened HADOOP-8193:
    ---------------------------------

    Refactor FailoverController/HAAdmin code to add an abstract class for "target" services
    ---------------------------------------------------------------------------------------

    Key: HADOOP-8193
    URL: https://issues.apache.org/jira/browse/HADOOP-8193
    Project: Hadoop Common
    Issue Type: Improvement
    Components: ha
    Affects Versions: 0.23.3, 0.24.0
    Reporter: Todd Lipcon
    Assignee: Todd Lipcon
    Fix For: 0.23.3, 0.24.0

    Attachments: hadoop-8193.txt, hadoop-8193.txt, hadoop-8193.txt


    In working at HADOOP-8077, HDFS-3084, and HDFS-3072, I ran into various difficulties which are an artifact of the current design. A few of these:
    - the service name is "resolved" from the logical name (eg ns1.nn1) to an IP address at the outer layer of DFSHAAdmin
    -- this means it's difficult to provide the logical name "ns1.nn1" to fence scripts (HDFS-3084)
    -- this means it's difficult to configure fencing method per-namespace (since the FailoverController doesn't know what the namespace is) (HADOOP-8077)
    - the configuration for HA HDFS is weirdly split between core-site and hdfs-site, even though most users see this as an HDFS feature. For example, users expect to configure NN fencing configurations in hdfs-site, and expect the keys to have a dfs.* prefix
    - proxies are constructed at the outer layer of the admin commands. This means it's impossible for the inner layers (eg FailoverController.failover) to re-construct proxies with different timeouts (HDFS-3072)
    The proposed refactor is to add a new interface (tentatively named HAServiceTarget) which refers to target for one of the admin commands. An instance of this class is responsible for creating proxies, creating fencers, mapping back to a logical name, etc. The HDFS implementation of this class can then provide different results based on the particular nameservice, can use HDFS-specific configuration prefixes, etc. Using this class as the argument for fencing methods also makes the API more evolvable in the future, since we can add new getters to HAServiceTarget (whereas the current InetSocketAddress is quite limiting)
    --
    This message is automatically generated by JIRA.
    If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
    For more information on JIRA, see: http://www.atlassian.com/software/jira
  • Todd Lipcon (Resolved) (JIRA) at Mar 26, 2012 at 1:56 am
    [ https://issues.apache.org/jira/browse/HADOOP-8193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Todd Lipcon resolved HADOOP-8193.
    ---------------------------------

    Resolution: Fixed

    Re-committed with the delta after running the full common and HDFS test suites (only got known failures due to unrelated issues)
    Refactor FailoverController/HAAdmin code to add an abstract class for "target" services
    ---------------------------------------------------------------------------------------

    Key: HADOOP-8193
    URL: https://issues.apache.org/jira/browse/HADOOP-8193
    Project: Hadoop Common
    Issue Type: Improvement
    Components: ha
    Affects Versions: 0.23.3, 0.24.0
    Reporter: Todd Lipcon
    Assignee: Todd Lipcon
    Fix For: 0.23.3, 0.24.0

    Attachments: hadoop-8193.txt, hadoop-8193.txt, hadoop-8193.txt, hdfs-8193-fix-npe.txt


    In working at HADOOP-8077, HDFS-3084, and HDFS-3072, I ran into various difficulties which are an artifact of the current design. A few of these:
    - the service name is "resolved" from the logical name (eg ns1.nn1) to an IP address at the outer layer of DFSHAAdmin
    -- this means it's difficult to provide the logical name "ns1.nn1" to fence scripts (HDFS-3084)
    -- this means it's difficult to configure fencing method per-namespace (since the FailoverController doesn't know what the namespace is) (HADOOP-8077)
    - the configuration for HA HDFS is weirdly split between core-site and hdfs-site, even though most users see this as an HDFS feature. For example, users expect to configure NN fencing configurations in hdfs-site, and expect the keys to have a dfs.* prefix
    - proxies are constructed at the outer layer of the admin commands. This means it's impossible for the inner layers (eg FailoverController.failover) to re-construct proxies with different timeouts (HDFS-3072)
    The proposed refactor is to add a new interface (tentatively named HAServiceTarget) which refers to target for one of the admin commands. An instance of this class is responsible for creating proxies, creating fencers, mapping back to a logical name, etc. The HDFS implementation of this class can then provide different results based on the particular nameservice, can use HDFS-specific configuration prefixes, etc. Using this class as the argument for fencing methods also makes the API more evolvable in the future, since we can add new getters to HAServiceTarget (whereas the current InetSocketAddress is quite limiting)
    --
    This message is automatically generated by JIRA.
    If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
    For more information on JIRA, see: http://www.atlassian.com/software/jira

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedMar 20, '12 at 9:58p
activeMar 26, '12 at 1:56a
posts4
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Todd Lipcon (Resolved) (JIRA): 4 posts

People

Translate

site design / logo © 2022 Grokbase