FAQ
Ensure there is always at least one server.X line in ZooKeeper conf/zoo.cfg
---------------------------------------------------------------------------

Key: HBASE-1204
URL: https://issues.apache.org/jira/browse/HBASE-1204
Project: Hadoop HBase
Issue Type: Improvement
Reporter: Nitay Joffe


This is needed so that we can get rid of the "hbase.master" property from the configuration XML.
From HBASE-1145:
There is a bit of an issue regarding getting rid of the "hbase.master" property in hbase-site.xml. Currently, anybody in HBase needing ZooKeeper parses the server.X lines out of conf/zoo.cfg to know how to connect to ZooKeeper. If there are no server.X lines, it means we are running a single instance ZooKeeper on the master server. To find that server, we use the "hbase.master" property in hbase-site.xml.

After discussing it with Andrew Purtell and Michael Stack, we came up with the following:

We have three cases:
1) "local" mode. All servers/clients are on localhost.
2) Distributed operation with single instance ZooKeeper running on master host.
3) Distributed operation with distributed ZooKeeper running.

To fix these cases (1 and 2) and allow us to remove the "hbase.master" property from hbase-site.xml, we need to make sure there is _always_ a server.X line with the master's address in the zoo.cfg. Even when you're running a single instance ZooKeeper on the master server, there should be a single server.X line with the master's address.

To avoid adding a lot of configuration headache on the user, we want to have the default conf/zoo.cfg have a sensible default for the single server master hostname case. We suggest shipping a new ZooKeeper jar with ZOOKEEPER-279 patched in, and using the variable substitution to inject the master hostname into zoo.cfg. For example, our new zoo.cfg would look something like:


{code}
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=${hbase.tmp.dir}/zookeeper
# the port at which the clients will connect
clientPort=2181

server.0=${master.address.hostname}:2888:3888
{code}


The "master.address.hostname" is a new property being added to the hbase-site.xml during this period while we transition users to ZooKeeper. If the property is not set, we will use gethostname() to fill in the variable and produce copious warning messages that it may not work and the user should either fill in that variable in hbase-site.xml, or edit zoo.cfg by hand.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Nitay Joffe (JIRA) at Feb 19, 2009 at 7:58 pm
    [ https://issues.apache.org/jira/browse/HBASE-1204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Nitay Joffe reassigned HBASE-1204:
    ----------------------------------

    Assignee: Nitay Joffe
    Ensure there is always at least one server.X line in ZooKeeper conf/zoo.cfg
    ---------------------------------------------------------------------------

    Key: HBASE-1204
    URL: https://issues.apache.org/jira/browse/HBASE-1204
    Project: Hadoop HBase
    Issue Type: Improvement
    Reporter: Nitay Joffe
    Assignee: Nitay Joffe

    This is needed so that we can get rid of the "hbase.master" property from the configuration XML.
    From HBASE-1145:
    There is a bit of an issue regarding getting rid of the "hbase.master" property in hbase-site.xml. Currently, anybody in HBase needing ZooKeeper parses the server.X lines out of conf/zoo.cfg to know how to connect to ZooKeeper. If there are no server.X lines, it means we are running a single instance ZooKeeper on the master server. To find that server, we use the "hbase.master" property in hbase-site.xml.
    After discussing it with Andrew Purtell and Michael Stack, we came up with the following:
    We have three cases:
    1) "local" mode. All servers/clients are on localhost.
    2) Distributed operation with single instance ZooKeeper running on master host.
    3) Distributed operation with distributed ZooKeeper running.
    To fix these cases (1 and 2) and allow us to remove the "hbase.master" property from hbase-site.xml, we need to make sure there is _always_ a server.X line with the master's address in the zoo.cfg. Even when you're running a single instance ZooKeeper on the master server, there should be a single server.X line with the master's address.
    To avoid adding a lot of configuration headache on the user, we want to have the default conf/zoo.cfg have a sensible default for the single server master hostname case. We suggest shipping a new ZooKeeper jar with ZOOKEEPER-279 patched in, and using the variable substitution to inject the master hostname into zoo.cfg. For example, our new zoo.cfg would look something like:
    {code}
    # The number of milliseconds of each tick
    tickTime=2000
    # The number of ticks that the initial
    # synchronization phase can take
    initLimit=10
    # The number of ticks that can pass between
    # sending a request and getting an acknowledgement
    syncLimit=5
    # the directory where the snapshot is stored.
    dataDir=${hbase.tmp.dir}/zookeeper
    # the port at which the clients will connect
    clientPort=2181
    server.0=${master.address.hostname}:2888:3888
    {code}
    The "master.address.hostname" is a new property being added to the hbase-site.xml during this period while we transition users to ZooKeeper. If the property is not set, we will use gethostname() to fill in the variable and produce copious warning messages that it may not work and the user should either fill in that variable in hbase-site.xml, or edit zoo.cfg by hand.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Nitay Joffe (JIRA) at Mar 6, 2009 at 2:06 am
    [ https://issues.apache.org/jira/browse/HBASE-1204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Nitay Joffe updated HBASE-1204:
    -------------------------------

    Attachment: hbase-1204.patch

    This patch depends on updated ZooKeeper jar from HBASE-1241
    Ensure there is always at least one server.X line in ZooKeeper conf/zoo.cfg
    ---------------------------------------------------------------------------

    Key: HBASE-1204
    URL: https://issues.apache.org/jira/browse/HBASE-1204
    Project: Hadoop HBase
    Issue Type: Improvement
    Reporter: Nitay Joffe
    Assignee: Nitay Joffe
    Attachments: hbase-1204.patch


    This is needed so that we can get rid of the "hbase.master" property from the configuration XML.
    From HBASE-1145:
    There is a bit of an issue regarding getting rid of the "hbase.master" property in hbase-site.xml. Currently, anybody in HBase needing ZooKeeper parses the server.X lines out of conf/zoo.cfg to know how to connect to ZooKeeper. If there are no server.X lines, it means we are running a single instance ZooKeeper on the master server. To find that server, we use the "hbase.master" property in hbase-site.xml.
    After discussing it with Andrew Purtell and Michael Stack, we came up with the following:
    We have three cases:
    1) "local" mode. All servers/clients are on localhost.
    2) Distributed operation with single instance ZooKeeper running on master host.
    3) Distributed operation with distributed ZooKeeper running.
    To fix these cases (1 and 2) and allow us to remove the "hbase.master" property from hbase-site.xml, we need to make sure there is _always_ a server.X line with the master's address in the zoo.cfg. Even when you're running a single instance ZooKeeper on the master server, there should be a single server.X line with the master's address.
    To avoid adding a lot of configuration headache on the user, we want to have the default conf/zoo.cfg have a sensible default for the single server master hostname case. We suggest shipping a new ZooKeeper jar with ZOOKEEPER-279 patched in, and using the variable substitution to inject the master hostname into zoo.cfg. For example, our new zoo.cfg would look something like:
    {code}
    # The number of milliseconds of each tick
    tickTime=2000
    # The number of ticks that the initial
    # synchronization phase can take
    initLimit=10
    # The number of ticks that can pass between
    # sending a request and getting an acknowledgement
    syncLimit=5
    # the directory where the snapshot is stored.
    dataDir=${hbase.tmp.dir}/zookeeper
    # the port at which the clients will connect
    clientPort=2181
    server.0=${master.address.hostname}:2888:3888
    {code}
    The "master.address.hostname" is a new property being added to the hbase-site.xml during this period while we transition users to ZooKeeper. If the property is not set, we will use gethostname() to fill in the variable and produce copious warning messages that it may not work and the user should either fill in that variable in hbase-site.xml, or edit zoo.cfg by hand.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Mar 18, 2009 at 8:54 pm
    [ https://issues.apache.org/jira/browse/HBASE-1204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12683162#action_12683162 ]

    stack commented on HBASE-1204:
    ------------------------------

    Nitay, this should be close, right?
    Ensure there is always at least one server.X line in ZooKeeper conf/zoo.cfg
    ---------------------------------------------------------------------------

    Key: HBASE-1204
    URL: https://issues.apache.org/jira/browse/HBASE-1204
    Project: Hadoop HBase
    Issue Type: Improvement
    Reporter: Nitay Joffe
    Assignee: Nitay Joffe
    Attachments: hbase-1204.patch


    This is needed so that we can get rid of the "hbase.master" property from the configuration XML.
    From HBASE-1145:
    There is a bit of an issue regarding getting rid of the "hbase.master" property in hbase-site.xml. Currently, anybody in HBase needing ZooKeeper parses the server.X lines out of conf/zoo.cfg to know how to connect to ZooKeeper. If there are no server.X lines, it means we are running a single instance ZooKeeper on the master server. To find that server, we use the "hbase.master" property in hbase-site.xml.
    After discussing it with Andrew Purtell and Michael Stack, we came up with the following:
    We have three cases:
    1) "local" mode. All servers/clients are on localhost.
    2) Distributed operation with single instance ZooKeeper running on master host.
    3) Distributed operation with distributed ZooKeeper running.
    To fix these cases (1 and 2) and allow us to remove the "hbase.master" property from hbase-site.xml, we need to make sure there is _always_ a server.X line with the master's address in the zoo.cfg. Even when you're running a single instance ZooKeeper on the master server, there should be a single server.X line with the master's address.
    To avoid adding a lot of configuration headache on the user, we want to have the default conf/zoo.cfg have a sensible default for the single server master hostname case. We suggest shipping a new ZooKeeper jar with ZOOKEEPER-279 patched in, and using the variable substitution to inject the master hostname into zoo.cfg. For example, our new zoo.cfg would look something like:
    {code}
    # The number of milliseconds of each tick
    tickTime=2000
    # The number of ticks that the initial
    # synchronization phase can take
    initLimit=10
    # The number of ticks that can pass between
    # sending a request and getting an acknowledgement
    syncLimit=5
    # the directory where the snapshot is stored.
    dataDir=${hbase.tmp.dir}/zookeeper
    # the port at which the clients will connect
    clientPort=2181
    server.0=${master.address.hostname}:2888:3888
    {code}
    The "master.address.hostname" is a new property being added to the hbase-site.xml during this period while we transition users to ZooKeeper. If the property is not set, we will use gethostname() to fill in the variable and produce copious warning messages that it may not work and the user should either fill in that variable in hbase-site.xml, or edit zoo.cfg by hand.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Nitay Joffe (JIRA) at Mar 18, 2009 at 8:56 pm
    [ https://issues.apache.org/jira/browse/HBASE-1204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Work on HBASE-1204 started by Nitay Joffe.
    Ensure there is always at least one server.X line in ZooKeeper conf/zoo.cfg
    ---------------------------------------------------------------------------

    Key: HBASE-1204
    URL: https://issues.apache.org/jira/browse/HBASE-1204
    Project: Hadoop HBase
    Issue Type: Improvement
    Reporter: Nitay Joffe
    Assignee: Nitay Joffe
    Attachments: hbase-1204.patch


    This is needed so that we can get rid of the "hbase.master" property from the configuration XML.
    From HBASE-1145:
    There is a bit of an issue regarding getting rid of the "hbase.master" property in hbase-site.xml. Currently, anybody in HBase needing ZooKeeper parses the server.X lines out of conf/zoo.cfg to know how to connect to ZooKeeper. If there are no server.X lines, it means we are running a single instance ZooKeeper on the master server. To find that server, we use the "hbase.master" property in hbase-site.xml.
    After discussing it with Andrew Purtell and Michael Stack, we came up with the following:
    We have three cases:
    1) "local" mode. All servers/clients are on localhost.
    2) Distributed operation with single instance ZooKeeper running on master host.
    3) Distributed operation with distributed ZooKeeper running.
    To fix these cases (1 and 2) and allow us to remove the "hbase.master" property from hbase-site.xml, we need to make sure there is _always_ a server.X line with the master's address in the zoo.cfg. Even when you're running a single instance ZooKeeper on the master server, there should be a single server.X line with the master's address.
    To avoid adding a lot of configuration headache on the user, we want to have the default conf/zoo.cfg have a sensible default for the single server master hostname case. We suggest shipping a new ZooKeeper jar with ZOOKEEPER-279 patched in, and using the variable substitution to inject the master hostname into zoo.cfg. For example, our new zoo.cfg would look something like:
    {code}
    # The number of milliseconds of each tick
    tickTime=2000
    # The number of ticks that the initial
    # synchronization phase can take
    initLimit=10
    # The number of ticks that can pass between
    # sending a request and getting an acknowledgement
    syncLimit=5
    # the directory where the snapshot is stored.
    dataDir=${hbase.tmp.dir}/zookeeper
    # the port at which the clients will connect
    clientPort=2181
    server.0=${master.address.hostname}:2888:3888
    {code}
    The "master.address.hostname" is a new property being added to the hbase-site.xml during this period while we transition users to ZooKeeper. If the property is not set, we will use gethostname() to fill in the variable and produce copious warning messages that it may not work and the user should either fill in that variable in hbase-site.xml, or edit zoo.cfg by hand.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Nitay Joffe (JIRA) at Mar 18, 2009 at 8:56 pm
    [ https://issues.apache.org/jira/browse/HBASE-1204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Nitay Joffe resolved HBASE-1204.
    --------------------------------

    Resolution: Duplicate

    Yes, thanks.
    Ensure there is always at least one server.X line in ZooKeeper conf/zoo.cfg
    ---------------------------------------------------------------------------

    Key: HBASE-1204
    URL: https://issues.apache.org/jira/browse/HBASE-1204
    Project: Hadoop HBase
    Issue Type: Improvement
    Reporter: Nitay Joffe
    Assignee: Nitay Joffe
    Attachments: hbase-1204.patch


    This is needed so that we can get rid of the "hbase.master" property from the configuration XML.
    From HBASE-1145:
    There is a bit of an issue regarding getting rid of the "hbase.master" property in hbase-site.xml. Currently, anybody in HBase needing ZooKeeper parses the server.X lines out of conf/zoo.cfg to know how to connect to ZooKeeper. If there are no server.X lines, it means we are running a single instance ZooKeeper on the master server. To find that server, we use the "hbase.master" property in hbase-site.xml.
    After discussing it with Andrew Purtell and Michael Stack, we came up with the following:
    We have three cases:
    1) "local" mode. All servers/clients are on localhost.
    2) Distributed operation with single instance ZooKeeper running on master host.
    3) Distributed operation with distributed ZooKeeper running.
    To fix these cases (1 and 2) and allow us to remove the "hbase.master" property from hbase-site.xml, we need to make sure there is _always_ a server.X line with the master's address in the zoo.cfg. Even when you're running a single instance ZooKeeper on the master server, there should be a single server.X line with the master's address.
    To avoid adding a lot of configuration headache on the user, we want to have the default conf/zoo.cfg have a sensible default for the single server master hostname case. We suggest shipping a new ZooKeeper jar with ZOOKEEPER-279 patched in, and using the variable substitution to inject the master hostname into zoo.cfg. For example, our new zoo.cfg would look something like:
    {code}
    # The number of milliseconds of each tick
    tickTime=2000
    # The number of ticks that the initial
    # synchronization phase can take
    initLimit=10
    # The number of ticks that can pass between
    # sending a request and getting an acknowledgement
    syncLimit=5
    # the directory where the snapshot is stored.
    dataDir=${hbase.tmp.dir}/zookeeper
    # the port at which the clients will connect
    clientPort=2181
    server.0=${master.address.hostname}:2888:3888
    {code}
    The "master.address.hostname" is a new property being added to the hbase-site.xml during this period while we transition users to ZooKeeper. If the property is not set, we will use gethostname() to fill in the variable and produce copious warning messages that it may not work and the user should either fill in that variable in hbase-site.xml, or edit zoo.cfg by hand.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categorieshbase, hadoop
postedFeb 19, '09 at 12:31a
activeMar 18, '09 at 8:56p
posts6
users1
websitehbase.apache.org

1 user in discussion

Nitay Joffe (JIRA): 6 posts

People

Translate

site design / logo © 2022 Grokbase