FAQ
Hi,

I added some properties in HS2 hive-site.xml security valve, and restart
HS2, but the auto-generated hive-site.xml is still not changed.
What am I missing?

Thanks,
Xiong

To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

Search Discussions

  • Harsh J at Sep 8, 2013 at 4:05 am
    What is the exact name of the safety valve you modified and what
    generated file are you checking? For HS2, which is a service daemon,
    the right file to check would be at CM -> Hive -> Instances -> Hive
    Server 2 -> Process -> Show Config and Environment Files ->
    hive-site.xml.
    On Sun, Sep 8, 2013 at 8:16 AM, xiong xu wrote:
    Hi,

    I added some properties in HS2 hive-site.xml security valve, and restart
    HS2, but the auto-generated hive-site.xml is still not changed.
    What am I missing?

    Thanks,
    Xiong

    To unsubscribe from this group and stop receiving emails from it, send an
    email to scm-users+unsubscribe@cloudera.org.


    --
    Harsh J

    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Xiong xu at Sep 8, 2013 at 8:18 am
    Hi Harsh,

    The property name I added in safety valve are :
    hive.server2.authentication, hive.aux.jars.path
    And I did checked the hive-site.xml you mentioned.

    Xiong

    在 2013年9月8日星期日UTC+8下午12时04分43秒,Harsh J写道:
    What is the exact name of the safety valve you modified and what
    generated file are you checking? For HS2, which is a service daemon,
    the right file to check would be at CM -> Hive -> Instances -> Hive
    Server 2 -> Process -> Show Config and Environment Files ->
    hive-site.xml.

    On Sun, Sep 8, 2013 at 8:16 AM, xiong xu <xu.xi...@gmail.com <javascript:>>
    wrote:
    Hi,

    I added some properties in HS2 hive-site.xml security valve, and restart
    HS2, but the auto-generated hive-site.xml is still not changed.
    What am I missing?

    Thanks,
    Xiong

    To unsubscribe from this group and stop receiving emails from it, send an
    email to scm-users+...@cloudera.org <javascript:>.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Xiong xu at Sep 8, 2013 at 9:25 am
    Here is what I put in the safety valve:
    <property>
       <name>hive.server2.thrift.max.worker.threads</name>
       <value>30</value>
       <description>Maximum number of Thrift worker threads</description>
    </property>
    <property>
         <name>hive.support.concurrency</name>
         <value>false</value>
    </property>
    <property>
      <name>hive.server2.authentication</name>
      <value>NOSASL</value>
    </property>
    <property>
      <name>hive.aux.jars.path</name>
      <value>file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/Rank.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/DateFunction.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/format.jar</value>
    </property>

    And here is the generated
    file(/cmf/process/3050/config?filename=hive-site.xml) contains:

    <?xml version="1.0" encoding="UTF-8"?>

    <!--Autogenerated by Cloudera CM on 2013-09-07T16:30:37.546Z-->
    <configuration>
       <property>
         <name>hive.metastore.local</name>
         <value>false</value>
       </property>
       <property>
         <name>hive.metastore.uris</name>
         <value>thrift://btirsvr69.domain.test:9083</value>
       </property>
       <property>
         <name>hive.metastore.warehouse.dir</name>
         <value>/user/hive/warehouse</value>
       </property>
       <property>
         <name>hive.warehouse.subdir.inherit.perms</name>
         <value>true</value>
       </property>
       <property>
         <name>mapred.reduce.tasks</name>
         <value>-1</value>
       </property>
       <property>
         <name>hive.exec.reducers.bytes.per.reducer</name>
         <value>1073741824</value>
       </property>
       <property>
         <name>hive.exec.reducers.max</name>
         <value>999</value>
       </property>
       <property>
         <name>hive.metastore.execute.setugi</name>
         <value>true</value>
       </property>
       <!--'hive.support.concurrency', originally set to 'true' (non-final), is overridden below by a safety valve-->
       <property>
         <name>hive.zookeeper.quorum</name>
         <value>btirsvr70.domain.test,btirsvr69.domain.test,btirsvr74.domain.test</value>
       </property>
       <property>
         <name>hive.zookeeper.client.port</name>
         <value>2181</value>
       </property>
       <property>
         <name>hive.server2.thrift.port</name>
         <value>10000</value>
       </property>
       <property>
         <name>hive.server2.enable.impersonation</name>
         <value>true</value>
       </property>
       <property>
         <name>fs.hdfs.impl.disable.cache</name>
         <value>true</value>
       </property>
       <property>
         <name>hive.support.concurrency</name>
         <value>false</value>
       </property>
       <property>
         <name>hive.server2.thrift.max.worker.threads</name>
         <value>30</value>
       </property>
    </configuration>


    在 2013年9月8日星期日UTC+8下午12时04分43秒,Harsh J写道:
    What is the exact name of the safety valve you modified and what
    generated file are you checking? For HS2, which is a service daemon,
    the right file to check would be at CM -> Hive -> Instances -> Hive
    Server 2 -> Process -> Show Config and Environment Files ->
    hive-site.xml.

    On Sun, Sep 8, 2013 at 8:16 AM, xiong xu <xu.xi...@gmail.com <javascript:>>
    wrote:
    Hi,

    I added some properties in HS2 hive-site.xml security valve, and restart
    HS2, but the auto-generated hive-site.xml is still not changed.
    What am I missing?

    Thanks,
    Xiong

    To unsubscribe from this group and stop receiving emails from it, send an
    email to scm-users+...@cloudera.org <javascript:>.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Harsh J at Sep 8, 2013 at 11:09 am
    Hi,

    There are multiple service-specific safety valves. What is the exact
    name of the safety valve you modified, and did you restart the HS2
    process?
    On Sun, Sep 8, 2013 at 2:55 PM, xiong xu wrote:
    Here is what I put in the safety valve:
    <property>
    <name>hive.server2.thrift.max.worker.threads</name>
    <value>30</value>
    <description>Maximum number of Thrift worker threads</description>
    </property>
    <property>
    <name>hive.support.concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.authentication</name>
    <value>NOSASL</value>
    </property>
    <property>
    <name>hive.aux.jars.path</name>

    <value>file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/Rank.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/DateFunction.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/format.jar</value>
    </property>

    And here is the generated
    file(/cmf/process/3050/config?filename=hive-site.xml) contains:

    <?xml version="1.0" encoding="UTF-8"?>

    <!--Autogenerated by Cloudera CM on 2013-09-07T16:30:37.546Z-->
    <configuration>
    <property>
    <name>hive.metastore.local</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.metastore.uris</name>
    <value>thrift://btirsvr69.domain.test:9083</value>
    </property>
    <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
    </property>
    <property>
    <name>hive.warehouse.subdir.inherit.perms</name>
    <value>true</value>
    </property>
    <property>
    <name>mapred.reduce.tasks</name>
    <value>-1</value>
    </property>
    <property>
    <name>hive.exec.reducers.bytes.per.reducer</name>
    <value>1073741824</value>
    </property>
    <property>
    <name>hive.exec.reducers.max</name>
    <value>999</value>
    </property>
    <property>
    <name>hive.metastore.execute.setugi</name>
    <value>true</value>
    </property>
    <!--'hive.support.concurrency', originally set to 'true' (non-final), is
    overridden below by a safety valve-->
    <property>
    <name>hive.zookeeper.quorum</name>

    <value>btirsvr70.domain.test,btirsvr69.domain.test,btirsvr74.domain.test</value>
    </property>
    <property>
    <name>hive.zookeeper.client.port</name>
    <value>2181</value>
    </property>
    <property>
    <name>hive.server2.thrift.port</name>
    <value>10000</value>
    </property>
    <property>
    <name>hive.server2.enable.impersonation</name>
    <value>true</value>
    </property>
    <property>
    <name>fs.hdfs.impl.disable.cache</name>
    <value>true</value>
    </property>
    <property>
    <name>hive.support.concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.thrift.max.worker.threads</name>
    <value>30</value>
    </property>
    </configuration>


    在 2013年9月8日星期日UTC+8下午12时04分43秒,Harsh J写道:
    What is the exact name of the safety valve you modified and what
    generated file are you checking? For HS2, which is a service daemon,
    the right file to check would be at CM -> Hive -> Instances -> Hive
    Server 2 -> Process -> Show Config and Environment Files ->
    hive-site.xml.
    On Sun, Sep 8, 2013 at 8:16 AM, xiong xu wrote:
    Hi,

    I added some properties in HS2 hive-site.xml security valve, and restart
    HS2, but the auto-generated hive-site.xml is still not changed.
    What am I missing?

    Thanks,
    Xiong

    To unsubscribe from this group and stop receiving emails from it, send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it, send an
    email to scm-users+unsubscribe@cloudera.org.


    --
    Harsh J

    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Xiong xu at Sep 8, 2013 at 1:01 pm
    I lookup this name from the
    html(http://.../cmf/services/135/config?groupParent=&groupKey=config.hive.group_hive1-HIVESERVER2-BASE)
    source:
      name="config.group_hive1-HIVESERVER2-BASE.hive_hs2_config_safety_valve"
    And, yes, I did restart HS2.

    在 2013年9月8日星期日UTC+8下午7时08分51秒,Harsh J写道:
    Hi,

    There are multiple service-specific safety valves. What is the exact
    name of the safety valve you modified, and did you restart the HS2
    process?

    On Sun, Sep 8, 2013 at 2:55 PM, xiong xu <xu.xi...@gmail.com <javascript:>>
    wrote:
    Here is what I put in the safety valve:
    <property>
    <name>hive.server2.thrift.max.worker.threads</name>
    <value>30</value>
    <description>Maximum number of Thrift worker threads</description>
    </property>
    <property>
    <name>hive.support.concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.authentication</name>
    <value>NOSASL</value>
    </property>
    <property>
    <name>hive.aux.jars.path</name>

    <value>file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/Rank.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/DateFunction.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/format.jar</value>
    </property>

    And here is the generated
    file(/cmf/process/3050/config?filename=hive-site.xml) contains:

    <?xml version="1.0" encoding="UTF-8"?>

    <!--Autogenerated by Cloudera CM on 2013-09-07T16:30:37.546Z-->
    <configuration>
    <property>
    <name>hive.metastore.local</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.metastore.uris</name>
    <value>thrift://btirsvr69.domain.test:9083</value>
    </property>
    <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
    </property>
    <property>
    <name>hive.warehouse.subdir.inherit.perms</name>
    <value>true</value>
    </property>
    <property>
    <name>mapred.reduce.tasks</name>
    <value>-1</value>
    </property>
    <property>
    <name>hive.exec.reducers.bytes.per.reducer</name>
    <value>1073741824</value>
    </property>
    <property>
    <name>hive.exec.reducers.max</name>
    <value>999</value>
    </property>
    <property>
    <name>hive.metastore.execute.setugi</name>
    <value>true</value>
    </property>
    <!--'hive.support.concurrency', originally set to 'true' (non-final), is
    overridden below by a safety valve-->
    <property>
    <name>hive.zookeeper.quorum</name>

    <value>btirsvr70.domain.test,btirsvr69.domain.test,btirsvr74.domain.test</value>
    </property>
    <property>
    <name>hive.zookeeper.client.port</name>
    <value>2181</value>
    </property>
    <property>
    <name>hive.server2.thrift.port</name>
    <value>10000</value>
    </property>
    <property>
    <name>hive.server2.enable.impersonation</name>
    <value>true</value>
    </property>
    <property>
    <name>fs.hdfs.impl.disable.cache</name>
    <value>true</value>
    </property>
    <property>
    <name>hive.support.concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.thrift.max.worker.threads</name>
    <value>30</value>
    </property>
    </configuration>


    在 2013年9月8日星期日UTC+8下午12时04分43秒,Harsh J写道:
    What is the exact name of the safety valve you modified and what
    generated file are you checking? For HS2, which is a service daemon,
    the right file to check would be at CM -> Hive -> Instances -> Hive
    Server 2 -> Process -> Show Config and Environment Files ->
    hive-site.xml.
    On Sun, Sep 8, 2013 at 8:16 AM, xiong xu wrote:
    Hi,

    I added some properties in HS2 hive-site.xml security valve, and
    restart
    HS2, but the auto-generated hive-site.xml is still not changed.
    What am I missing?

    Thanks,
    Xiong

    To unsubscribe from this group and stop receiving emails from it,
    send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it, send an
    email to scm-users+...@cloudera.org <javascript:>.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Darren Lo at Sep 8, 2013 at 9:36 pm
    Hi Xiong,

    It seems like some of your safety valve is taking effect (concurrency gets
    overridden to false), while others are not (authentication and aux jars).
    You already said you restarted HS2, which would otherwise have been my
    guess as to the problem. Might be worth restarting again just to be sure.
    Do you have other safety valves set for Hive that could be interfering?

    What CM version are you using?

    Thanks,
    Darren

    On Sun, Sep 8, 2013 at 6:01 AM, xiong xu wrote:

    I lookup this name from the html(http://.../cmf/services/135/config?groupParent=&groupKey=config.hive.group_hive1-HIVESERVER2-BASE)
    source:
    name="config.group_hive1-HIVESERVER2-BASE.hive_hs2_config_safety_valve"
    And, yes, I did restart HS2.

    在 2013年9月8日星期日UTC+8下午7时08分51秒,Harsh J写道:
    Hi,

    There are multiple service-specific safety valves. What is the exact
    name of the safety valve you modified, and did you restart the HS2
    process?
    On Sun, Sep 8, 2013 at 2:55 PM, xiong xu wrote:
    Here is what I put in the safety valve:
    <property>
    <name>hive.server2.thrift.max.**worker.threads</name>
    <value>30</value>
    <description>Maximum number of Thrift worker threads</description>
    </property>
    <property>
    <name>hive.support.**concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.**authentication</name>
    <value>NOSASL</value>
    </property>
    <property>
    <name>hive.aux.jars.path</**name>

    <value>file:///opt/cloudera/**parcels/CDH-4.2.1-1.cdh4.2.1.**
    p0.5/lib/hive/lib/Rank.jar,**file:///opt/cloudera/parcels/**
    CDH-4.2.1-1.cdh4.2.1.p0.5/lib/**hive/lib/DateFunction.jar,**
    file:///opt/cloudera/parcels/**CDH-4.2.1-1.cdh4.2.1.p0.5/lib/**hive/lib/format.jar</value>
    </property>

    And here is the generated
    file(/cmf/process/3050/config?**filename=hive-site.xml) contains:

    <?xml version="1.0" encoding="UTF-8"?>

    <!--Autogenerated by Cloudera CM on 2013-09-07T16:30:37.546Z-->
    <configuration>
    <property>
    <name>hive.metastore.local</**name>
    <value>false</value>
    </property>
    <property>
    <name>hive.metastore.uris</**name>
    <value>thrift://btirsvr69.**domain.test:9083</value>
    </property>
    <property>
    <name>hive.metastore.**warehouse.dir</name>
    <value>/user/hive/warehouse</**value>
    </property>
    <property>
    <name>hive.warehouse.subdir.**inherit.perms</name>
    <value>true</value>
    </property>
    <property>
    <name>mapred.reduce.tasks</**name>
    <value>-1</value>
    </property>
    <property>
    <name>hive.exec.reducers.**bytes.per.reducer</name>
    <value>1073741824</value>
    </property>
    <property>
    <name>hive.exec.reducers.max</**name>
    <value>999</value>
    </property>
    <property>
    <name>hive.metastore.execute.**setugi</name>
    <value>true</value>
    </property>
    <!--'hive.support.concurrency'**, originally set to 'true'
    (non-final), is
    overridden below by a safety valve-->
    <property>
    <name>hive.zookeeper.quorum</**name>

    <value>btirsvr70.domain.test,**btirsvr69.domain.test,**btirsvr74.domain.test</value>
    </property>
    <property>
    <name>hive.zookeeper.client.**port</name>
    <value>2181</value>
    </property>
    <property>
    <name>hive.server2.thrift.**port</name>
    <value>10000</value>
    </property>
    <property>
    <name>hive.server2.enable.**impersonation</name>
    <value>true</value>
    </property>
    <property>
    <name>fs.hdfs.impl.disable.**cache</name>
    <value>true</value>
    </property>
    <property>
    <name>hive.support.**concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.thrift.max.**worker.threads</name>
    <value>30</value>
    </property>
    </configuration>


    在 2013年9月8日星期日UTC+8下午12时04分43秒,**Harsh J写道:
    What is the exact name of the safety valve you modified and what
    generated file are you checking? For HS2, which is a service daemon,
    the right file to check would be at CM -> Hive -> Instances -> Hive
    Server 2 -> Process -> Show Config and Environment Files ->
    hive-site.xml.
    On Sun, Sep 8, 2013 at 8:16 AM, xiong xu wrote:
    Hi,

    I added some properties in HS2 hive-site.xml security valve, and
    restart
    HS2, but the auto-generated hive-site.xml is still not changed.
    What am I missing?

    Thanks,
    Xiong

    To unsubscribe from this group and stop receiving emails from it,
    send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it, send an
    email to scm-users+...@**cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it, send an
    email to scm-users+unsubscribe@cloudera.org.


    --
    Thanks,
    Darren

    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • 徐雄 at Sep 9, 2013 at 12:18 am
    CM version is 4.5.2.
    I did restart HS2 several times.
    The concurrency safety valve taking effect was set a month ago, the CM
    safety valve worked then.
    But now, it doesn't work anymore.

    2013/9/9 Darren Lo <dlo@cloudera.com>:
    Hi Xiong,

    It seems like some of your safety valve is taking effect (concurrency gets
    overridden to false), while others are not (authentication and aux jars).
    You already said you restarted HS2, which would otherwise have been my guess
    as to the problem. Might be worth restarting again just to be sure. Do you
    have other safety valves set for Hive that could be interfering?

    What CM version are you using?

    Thanks,
    Darren

    On Sun, Sep 8, 2013 at 6:01 AM, xiong xu wrote:

    I lookup this name from the
    html(http://.../cmf/services/135/config?groupParent=&groupKey=config.hive.group_hive1-HIVESERVER2-BASE)
    source:
    name="config.group_hive1-HIVESERVER2-BASE.hive_hs2_config_safety_valve"
    And, yes, I did restart HS2.

    在 2013年9月8日星期日UTC+8下午7时08分51秒,Harsh J写道:
    Hi,

    There are multiple service-specific safety valves. What is the exact
    name of the safety valve you modified, and did you restart the HS2
    process?
    On Sun, Sep 8, 2013 at 2:55 PM, xiong xu wrote:
    Here is what I put in the safety valve:
    <property>
    <name>hive.server2.thrift.max.worker.threads</name>
    <value>30</value>
    <description>Maximum number of Thrift worker threads</description>
    </property>
    <property>
    <name>hive.support.concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.authentication</name>
    <value>NOSASL</value>
    </property>
    <property>
    <name>hive.aux.jars.path</name>


    <value>file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/Rank.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/DateFunction.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/format.jar</value>
    </property>

    And here is the generated
    file(/cmf/process/3050/config?filename=hive-site.xml) contains:

    <?xml version="1.0" encoding="UTF-8"?>

    <!--Autogenerated by Cloudera CM on 2013-09-07T16:30:37.546Z-->
    <configuration>
    <property>
    <name>hive.metastore.local</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.metastore.uris</name>
    <value>thrift://btirsvr69.domain.test:9083</value>
    </property>
    <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
    </property>
    <property>
    <name>hive.warehouse.subdir.inherit.perms</name>
    <value>true</value>
    </property>
    <property>
    <name>mapred.reduce.tasks</name>
    <value>-1</value>
    </property>
    <property>
    <name>hive.exec.reducers.bytes.per.reducer</name>
    <value>1073741824</value>
    </property>
    <property>
    <name>hive.exec.reducers.max</name>
    <value>999</value>
    </property>
    <property>
    <name>hive.metastore.execute.setugi</name>
    <value>true</value>
    </property>
    <!--'hive.support.concurrency', originally set to 'true' (non-final),
    is
    overridden below by a safety valve-->
    <property>
    <name>hive.zookeeper.quorum</name>


    <value>btirsvr70.domain.test,btirsvr69.domain.test,btirsvr74.domain.test</value>
    </property>
    <property>
    <name>hive.zookeeper.client.port</name>
    <value>2181</value>
    </property>
    <property>
    <name>hive.server2.thrift.port</name>
    <value>10000</value>
    </property>
    <property>
    <name>hive.server2.enable.impersonation</name>
    <value>true</value>
    </property>
    <property>
    <name>fs.hdfs.impl.disable.cache</name>
    <value>true</value>
    </property>
    <property>
    <name>hive.support.concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.thrift.max.worker.threads</name>
    <value>30</value>
    </property>
    </configuration>


    在 2013年9月8日星期日UTC+8下午12时04分43秒,Harsh J写道:
    What is the exact name of the safety valve you modified and what
    generated file are you checking? For HS2, which is a service daemon,
    the right file to check would be at CM -> Hive -> Instances -> Hive
    Server 2 -> Process -> Show Config and Environment Files ->
    hive-site.xml.
    On Sun, Sep 8, 2013 at 8:16 AM, xiong xu wrote:
    Hi,

    I added some properties in HS2 hive-site.xml security valve, and
    restart
    HS2, but the auto-generated hive-site.xml is still not changed.
    What am I missing?

    Thanks,
    Xiong

    To unsubscribe from this group and stop receiving emails from it,
    send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it, send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it, send an
    email to scm-users+unsubscribe@cloudera.org.



    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Darren Lo at Sep 9, 2013 at 3:13 am
    Hi Xiong,

    I set up a CM4.5.2 cluster and put the same information into the HS2 safety
    valve, and I saw that the configs appeared in HS2's hive-site.xml. Here's
    the end of the file, which includes the aux jars:
    "
       <property>
         <name>hive.aux.jars.path</name>

    <value>file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/Rank.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/DateFunction.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/format.jar</value>
       </property>
    </configuration>
    "

    Do you have a role-level override that is taking precedence over your HS2
    group-level configuration? When you click on Hive - > Configuration -> View
    and Edit -> HiveServer2 (Base) -> Advanced, when you hover over text box of
    the safety valve for hive-site you should see something appear below the
    box saying "Override Instances". If it says "Overridden by 1 instance(s)"
    then you have a role-level override, which you probably want to remove. You
    can click on that text, and it will show you the override and an option to
    Edit Overrides. Select the HS2 role and make sure Inherited Value is
    selected in the dropdown, then click Apply.

    If that's not the problem, it's possible that your HS2 is in a different
    role config group, though that is far less likely. You can go to
    Configuration -> Role Groups to verify the correct group for HS2.

    Thanks,
    Darren

    On Sun, Sep 8, 2013 at 5:18 PM, 徐雄 wrote:

    CM version is 4.5.2.
    I did restart HS2 several times.
    The concurrency safety valve taking effect was set a month ago, the CM
    safety valve worked then.
    But now, it doesn't work anymore.

    2013/9/9 Darren Lo <dlo@cloudera.com>:
    Hi Xiong,

    It seems like some of your safety valve is taking effect (concurrency gets
    overridden to false), while others are not (authentication and aux jars).
    You already said you restarted HS2, which would otherwise have been my guess
    as to the problem. Might be worth restarting again just to be sure. Do you
    have other safety valves set for Hive that could be interfering?

    What CM version are you using?

    Thanks,
    Darren

    On Sun, Sep 8, 2013 at 6:01 AM, xiong xu wrote:

    I lookup this name from the
    html(http://
    .../cmf/services/135/config?groupParent=&groupKey=config.hive.group_hive1-HIVESERVER2-BASE)
    source:
    name="config.group_hive1-HIVESERVER2-BASE.hive_hs2_config_safety_valve"
    And, yes, I did restart HS2.

    在 2013年9月8日星期日UTC+8下午7时08分51秒,Harsh J写道:
    Hi,

    There are multiple service-specific safety valves. What is the exact
    name of the safety valve you modified, and did you restart the HS2
    process?
    On Sun, Sep 8, 2013 at 2:55 PM, xiong xu wrote:
    Here is what I put in the safety valve:
    <property>
    <name>hive.server2.thrift.max.worker.threads</name>
    <value>30</value>
    <description>Maximum number of Thrift worker threads</description>
    </property>
    <property>
    <name>hive.support.concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.authentication</name>
    <value>NOSASL</value>
    </property>
    <property>
    <name>hive.aux.jars.path</name>

    <value>file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/Rank.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/DateFunction.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/format.jar</value>
    </property>

    And here is the generated
    file(/cmf/process/3050/config?filename=hive-site.xml) contains:

    <?xml version="1.0" encoding="UTF-8"?>

    <!--Autogenerated by Cloudera CM on 2013-09-07T16:30:37.546Z-->
    <configuration>
    <property>
    <name>hive.metastore.local</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.metastore.uris</name>
    <value>thrift://btirsvr69.domain.test:9083</value>
    </property>
    <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
    </property>
    <property>
    <name>hive.warehouse.subdir.inherit.perms</name>
    <value>true</value>
    </property>
    <property>
    <name>mapred.reduce.tasks</name>
    <value>-1</value>
    </property>
    <property>
    <name>hive.exec.reducers.bytes.per.reducer</name>
    <value>1073741824</value>
    </property>
    <property>
    <name>hive.exec.reducers.max</name>
    <value>999</value>
    </property>
    <property>
    <name>hive.metastore.execute.setugi</name>
    <value>true</value>
    </property>
    <!--'hive.support.concurrency', originally set to 'true'
    (non-final),
    is
    overridden below by a safety valve-->
    <property>
    <name>hive.zookeeper.quorum</name>

    <value>btirsvr70.domain.test,btirsvr69.domain.test,btirsvr74.domain.test</value>
    </property>
    <property>
    <name>hive.zookeeper.client.port</name>
    <value>2181</value>
    </property>
    <property>
    <name>hive.server2.thrift.port</name>
    <value>10000</value>
    </property>
    <property>
    <name>hive.server2.enable.impersonation</name>
    <value>true</value>
    </property>
    <property>
    <name>fs.hdfs.impl.disable.cache</name>
    <value>true</value>
    </property>
    <property>
    <name>hive.support.concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.thrift.max.worker.threads</name>
    <value>30</value>
    </property>
    </configuration>


    在 2013年9月8日星期日UTC+8下午12时04分43秒,Harsh J写道:
    What is the exact name of the safety valve you modified and what
    generated file are you checking? For HS2, which is a service daemon,
    the right file to check would be at CM -> Hive -> Instances -> Hive
    Server 2 -> Process -> Show Config and Environment Files ->
    hive-site.xml.
    On Sun, Sep 8, 2013 at 8:16 AM, xiong xu wrote:
    Hi,

    I added some properties in HS2 hive-site.xml security valve, and
    restart
    HS2, but the auto-generated hive-site.xml is still not changed.
    What am I missing?

    Thanks,
    Xiong

    To unsubscribe from this group and stop receiving emails from it,
    send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it,
    send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it, send
    an
    email to scm-users+unsubscribe@cloudera.org.



    --
    Thanks,
    Darren


    --
    Thanks,
    Darren

    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • 徐雄 at Sep 9, 2013 at 3:50 am
    Thanks, Darren.
    You're right, that's a role-level override problem!
    But I still don't know why this role-level override happen.
    A override in metastore server will cause HS2 configuration overridden?


    2013/9/9 Darren Lo <dlo@cloudera.com>
    Hi Xiong,

    I set up a CM4.5.2 cluster and put the same information into the HS2
    safety valve, and I saw that the configs appeared in HS2's hive-site.xml.
    Here's the end of the file, which includes the aux jars:
    "
    <property>
    <name>hive.aux.jars.path</name>

    <value>file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/Rank.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/DateFunction.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/format.jar</value>
    </property>
    </configuration>
    "

    Do you have a role-level override that is taking precedence over your HS2
    group-level configuration? When you click on Hive - > Configuration -> View
    and Edit -> HiveServer2 (Base) -> Advanced, when you hover over text box of
    the safety valve for hive-site you should see something appear below the
    box saying "Override Instances". If it says "Overridden by 1 instance(s)"
    then you have a role-level override, which you probably want to remove. You
    can click on that text, and it will show you the override and an option to
    Edit Overrides. Select the HS2 role and make sure Inherited Value is
    selected in the dropdown, then click Apply.

    If that's not the problem, it's possible that your HS2 is in a different
    role config group, though that is far less likely. You can go to
    Configuration -> Role Groups to verify the correct group for HS2.

    Thanks,
    Darren

    On Sun, Sep 8, 2013 at 5:18 PM, 徐雄 wrote:

    CM version is 4.5.2.
    I did restart HS2 several times.
    The concurrency safety valve taking effect was set a month ago, the CM
    safety valve worked then.
    But now, it doesn't work anymore.

    2013/9/9 Darren Lo <dlo@cloudera.com>:
    Hi Xiong,

    It seems like some of your safety valve is taking effect (concurrency gets
    overridden to false), while others are not (authentication and aux jars).
    You already said you restarted HS2, which would otherwise have been my guess
    as to the problem. Might be worth restarting again just to be sure. Do you
    have other safety valves set for Hive that could be interfering?

    What CM version are you using?

    Thanks,
    Darren

    On Sun, Sep 8, 2013 at 6:01 AM, xiong xu wrote:

    I lookup this name from the
    html(http://
    .../cmf/services/135/config?groupParent=&groupKey=config.hive.group_hive1-HIVESERVER2-BASE)
    source:
    name="config.group_hive1-HIVESERVER2-BASE.hive_hs2_config_safety_valve"
    And, yes, I did restart HS2.

    在 2013年9月8日星期日UTC+8下午7时08分51秒,Harsh J写道:
    Hi,

    There are multiple service-specific safety valves. What is the exact
    name of the safety valve you modified, and did you restart the HS2
    process?
    On Sun, Sep 8, 2013 at 2:55 PM, xiong xu wrote:
    Here is what I put in the safety valve:
    <property>
    <name>hive.server2.thrift.max.worker.threads</name>
    <value>30</value>
    <description>Maximum number of Thrift worker threads</description>
    </property>
    <property>
    <name>hive.support.concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.authentication</name>
    <value>NOSASL</value>
    </property>
    <property>
    <name>hive.aux.jars.path</name>

    <value>file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/Rank.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/DateFunction.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/format.jar</value>
    </property>

    And here is the generated
    file(/cmf/process/3050/config?filename=hive-site.xml) contains:

    <?xml version="1.0" encoding="UTF-8"?>

    <!--Autogenerated by Cloudera CM on 2013-09-07T16:30:37.546Z-->
    <configuration>
    <property>
    <name>hive.metastore.local</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.metastore.uris</name>
    <value>thrift://btirsvr69.domain.test:9083</value>
    </property>
    <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
    </property>
    <property>
    <name>hive.warehouse.subdir.inherit.perms</name>
    <value>true</value>
    </property>
    <property>
    <name>mapred.reduce.tasks</name>
    <value>-1</value>
    </property>
    <property>
    <name>hive.exec.reducers.bytes.per.reducer</name>
    <value>1073741824</value>
    </property>
    <property>
    <name>hive.exec.reducers.max</name>
    <value>999</value>
    </property>
    <property>
    <name>hive.metastore.execute.setugi</name>
    <value>true</value>
    </property>
    <!--'hive.support.concurrency', originally set to 'true'
    (non-final),
    is
    overridden below by a safety valve-->
    <property>
    <name>hive.zookeeper.quorum</name>

    <value>btirsvr70.domain.test,btirsvr69.domain.test,btirsvr74.domain.test</value>
    </property>
    <property>
    <name>hive.zookeeper.client.port</name>
    <value>2181</value>
    </property>
    <property>
    <name>hive.server2.thrift.port</name>
    <value>10000</value>
    </property>
    <property>
    <name>hive.server2.enable.impersonation</name>
    <value>true</value>
    </property>
    <property>
    <name>fs.hdfs.impl.disable.cache</name>
    <value>true</value>
    </property>
    <property>
    <name>hive.support.concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.thrift.max.worker.threads</name>
    <value>30</value>
    </property>
    </configuration>


    在 2013年9月8日星期日UTC+8下午12时04分43秒,Harsh J写道:
    What is the exact name of the safety valve you modified and what
    generated file are you checking? For HS2, which is a service
    daemon,
    the right file to check would be at CM -> Hive -> Instances -> Hive
    Server 2 -> Process -> Show Config and Environment Files ->
    hive-site.xml.

    On Sun, Sep 8, 2013 at 8:16 AM, xiong xu <xu.xi...@gmail.com>
    wrote:
    Hi,

    I added some properties in HS2 hive-site.xml security valve, and
    restart
    HS2, but the auto-generated hive-site.xml is still not changed.
    What am I missing?

    Thanks,
    Xiong

    To unsubscribe from this group and stop receiving emails from it,
    send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it,
    send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it, send
    an
    email to scm-users+unsubscribe@cloudera.org.



    --
    Thanks,
    Darren


    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Darren Lo at Sep 9, 2013 at 4:04 am
    Hi Xiong,

    A role override will only override the group config, so I don't think HMS
    is affecting HS2, since they are entirely different role types. It's
    possible that someone accidentally set the HS2 role-level safety valve in
    the past instead of the group-level one. It's an easy mistake to make,
    though CM does show a popup warning if you look at role-level config when
    only one role is defined.

    Were you able to solve your problem?

    On Sun, Sep 8, 2013 at 8:50 PM, 徐雄 wrote:

    Thanks, Darren.
    You're right, that's a role-level override problem!
    But I still don't know why this role-level override happen.
    A override in metastore server will cause HS2 configuration overridden?


    2013/9/9 Darren Lo <dlo@cloudera.com>
    Hi Xiong,

    I set up a CM4.5.2 cluster and put the same information into the HS2
    safety valve, and I saw that the configs appeared in HS2's hive-site.xml.
    Here's the end of the file, which includes the aux jars:
    "
    <property>
    <name>hive.aux.jars.path</name>

    <value>file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/Rank.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/DateFunction.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/format.jar</value>
    </property>
    </configuration>
    "

    Do you have a role-level override that is taking precedence over your HS2
    group-level configuration? When you click on Hive - > Configuration -> View
    and Edit -> HiveServer2 (Base) -> Advanced, when you hover over text box of
    the safety valve for hive-site you should see something appear below the
    box saying "Override Instances". If it says "Overridden by 1 instance(s)"
    then you have a role-level override, which you probably want to remove. You
    can click on that text, and it will show you the override and an option to
    Edit Overrides. Select the HS2 role and make sure Inherited Value is
    selected in the dropdown, then click Apply.

    If that's not the problem, it's possible that your HS2 is in a different
    role config group, though that is far less likely. You can go to
    Configuration -> Role Groups to verify the correct group for HS2.

    Thanks,
    Darren

    On Sun, Sep 8, 2013 at 5:18 PM, 徐雄 wrote:

    CM version is 4.5.2.
    I did restart HS2 several times.
    The concurrency safety valve taking effect was set a month ago, the CM
    safety valve worked then.
    But now, it doesn't work anymore.

    2013/9/9 Darren Lo <dlo@cloudera.com>:
    Hi Xiong,

    It seems like some of your safety valve is taking effect (concurrency gets
    overridden to false), while others are not (authentication and aux jars).
    You already said you restarted HS2, which would otherwise have been my guess
    as to the problem. Might be worth restarting again just to be sure. Do you
    have other safety valves set for Hive that could be interfering?

    What CM version are you using?

    Thanks,
    Darren


    On Sun, Sep 8, 2013 at 6:01 AM, xiong xu <xu.xiong.cn@gmail.com>
    wrote:
    I lookup this name from the
    html(http://
    .../cmf/services/135/config?groupParent=&groupKey=config.hive.group_hive1-HIVESERVER2-BASE)
    source:
    name="config.group_hive1-HIVESERVER2-BASE.hive_hs2_config_safety_valve"
    And, yes, I did restart HS2.

    在 2013年9月8日星期日UTC+8下午7时08分51秒,Harsh J写道:
    Hi,

    There are multiple service-specific safety valves. What is the exact
    name of the safety valve you modified, and did you restart the HS2
    process?
    On Sun, Sep 8, 2013 at 2:55 PM, xiong xu wrote:
    Here is what I put in the safety valve:
    <property>
    <name>hive.server2.thrift.max.worker.threads</name>
    <value>30</value>
    <description>Maximum number of Thrift worker
    threads</description>
    </property>
    <property>
    <name>hive.support.concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.authentication</name>
    <value>NOSASL</value>
    </property>
    <property>
    <name>hive.aux.jars.path</name>

    <value>file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/Rank.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/DateFunction.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/format.jar</value>
    </property>

    And here is the generated
    file(/cmf/process/3050/config?filename=hive-site.xml) contains:

    <?xml version="1.0" encoding="UTF-8"?>

    <!--Autogenerated by Cloudera CM on 2013-09-07T16:30:37.546Z-->
    <configuration>
    <property>
    <name>hive.metastore.local</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.metastore.uris</name>
    <value>thrift://btirsvr69.domain.test:9083</value>
    </property>
    <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
    </property>
    <property>
    <name>hive.warehouse.subdir.inherit.perms</name>
    <value>true</value>
    </property>
    <property>
    <name>mapred.reduce.tasks</name>
    <value>-1</value>
    </property>
    <property>
    <name>hive.exec.reducers.bytes.per.reducer</name>
    <value>1073741824</value>
    </property>
    <property>
    <name>hive.exec.reducers.max</name>
    <value>999</value>
    </property>
    <property>
    <name>hive.metastore.execute.setugi</name>
    <value>true</value>
    </property>
    <!--'hive.support.concurrency', originally set to 'true'
    (non-final),
    is
    overridden below by a safety valve-->
    <property>
    <name>hive.zookeeper.quorum</name>

    <value>btirsvr70.domain.test,btirsvr69.domain.test,btirsvr74.domain.test</value>
    </property>
    <property>
    <name>hive.zookeeper.client.port</name>
    <value>2181</value>
    </property>
    <property>
    <name>hive.server2.thrift.port</name>
    <value>10000</value>
    </property>
    <property>
    <name>hive.server2.enable.impersonation</name>
    <value>true</value>
    </property>
    <property>
    <name>fs.hdfs.impl.disable.cache</name>
    <value>true</value>
    </property>
    <property>
    <name>hive.support.concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.thrift.max.worker.threads</name>
    <value>30</value>
    </property>
    </configuration>


    在 2013年9月8日星期日UTC+8下午12时04分43秒,Harsh J写道:
    What is the exact name of the safety valve you modified and what
    generated file are you checking? For HS2, which is a service
    daemon,
    the right file to check would be at CM -> Hive -> Instances ->
    Hive
    Server 2 -> Process -> Show Config and Environment Files ->
    hive-site.xml.

    On Sun, Sep 8, 2013 at 8:16 AM, xiong xu <xu.xi...@gmail.com>
    wrote:
    Hi,

    I added some properties in HS2 hive-site.xml security valve, and
    restart
    HS2, but the auto-generated hive-site.xml is still not changed.
    What am I missing?

    Thanks,
    Xiong

    To unsubscribe from this group and stop receiving emails from
    it,
    send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it,
    send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it,
    send an
    email to scm-users+unsubscribe@cloudera.org.



    --
    Thanks,
    Darren


    --
    Thanks,
    Darren

    --
    Thanks,
    Darren

    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Xiong xu at Sep 9, 2013 at 4:31 am
    Thank you, Darren.
    Problem solved, although I'm not sure about the real cause.

    在 2013年9月9日星期一UTC+8下午12时04分04秒,Darren Lo写道:
    Hi Xiong,

    A role override will only override the group config, so I don't think HMS
    is affecting HS2, since they are entirely different role types. It's
    possible that someone accidentally set the HS2 role-level safety valve in
    the past instead of the group-level one. It's an easy mistake to make,
    though CM does show a popup warning if you look at role-level config when
    only one role is defined.

    Were you able to solve your problem?

    On Sun, Sep 8, 2013 at 8:50 PM, 徐雄 <xux...@gsta.com <javascript:>> wrote:

    Thanks, Darren.
    You're right, that's a role-level override problem!
    But I still don't know why this role-level override happen.
    A override in metastore server will cause HS2 configuration overridden?


    2013/9/9 Darren Lo <d...@cloudera.com <javascript:>>
    Hi Xiong,

    I set up a CM4.5.2 cluster and put the same information into the HS2
    safety valve, and I saw that the configs appeared in HS2's hive-site.xml.
    Here's the end of the file, which includes the aux jars:
    "
    <property>
    <name>hive.aux.jars.path</name>

    <value>file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/Rank.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/DateFunction.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/format.jar</value>
    </property>
    </configuration>
    "

    Do you have a role-level override that is taking precedence over your
    HS2 group-level configuration? When you click on Hive - > Configuration ->
    View and Edit -> HiveServer2 (Base) -> Advanced, when you hover over text
    box of the safety valve for hive-site you should see something appear below
    the box saying "Override Instances". If it says "Overridden by 1
    instance(s)" then you have a role-level override, which you probably want
    to remove. You can click on that text, and it will show you the override
    and an option to Edit Overrides. Select the HS2 role and make sure
    Inherited Value is selected in the dropdown, then click Apply.

    If that's not the problem, it's possible that your HS2 is in a different
    role config group, though that is far less likely. You can go to
    Configuration -> Role Groups to verify the correct group for HS2.

    Thanks,
    Darren


    On Sun, Sep 8, 2013 at 5:18 PM, 徐雄 <xux...@gsta.com <javascript:>>wrote:
    CM version is 4.5.2.
    I did restart HS2 several times.
    The concurrency safety valve taking effect was set a month ago, the CM
    safety valve worked then.
    But now, it doesn't work anymore.

    2013/9/9 Darren Lo <d...@cloudera.com <javascript:>>:
    Hi Xiong,

    It seems like some of your safety valve is taking effect (concurrency gets
    overridden to false), while others are not (authentication and aux jars).
    You already said you restarted HS2, which would otherwise have been my guess
    as to the problem. Might be worth restarting again just to be sure. Do you
    have other safety valves set for Hive that could be interfering?

    What CM version are you using?

    Thanks,
    Darren


    On Sun, Sep 8, 2013 at 6:01 AM, xiong xu <xu.xi...@gmail.com<javascript:>>
    wrote:
    I lookup this name from the
    html(http://.../cmf/services/135/config?groupParent=&groupKey=config.hive.group_hive1-HIVESERVER2-BASE)
    source:
    name="config.group_hive1-HIVESERVER2-BASE.hive_hs2_config_safety_valve"
    And, yes, I did restart HS2.

    在 2013年9月8日星期日UTC+8下午7时08分51秒,Harsh J写道:
    Hi,

    There are multiple service-specific safety valves. What is the exact
    name of the safety valve you modified, and did you restart the HS2
    process?

    On Sun, Sep 8, 2013 at 2:55 PM, xiong xu <xu.xi...@gmail.com>
    wrote:
    Here is what I put in the safety valve:
    <property>
    <name>hive.server2.thrift.max.worker.threads</name>
    <value>30</value>
    <description>Maximum number of Thrift worker
    threads</description>
    </property>
    <property>
    <name>hive.support.concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.authentication</name>
    <value>NOSASL</value>
    </property>
    <property>
    <name>hive.aux.jars.path</name>

    <value>file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/Rank.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/DateFunction.jar,file:///opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hive/lib/format.jar</value>
    </property>

    And here is the generated
    file(/cmf/process/3050/config?filename=hive-site.xml) contains:

    <?xml version="1.0" encoding="UTF-8"?>

    <!--Autogenerated by Cloudera CM on 2013-09-07T16:30:37.546Z-->
    <configuration>
    <property>
    <name>hive.metastore.local</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.metastore.uris</name>
    <value>thrift://btirsvr69.domain.test:9083</value>
    </property>
    <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
    </property>
    <property>
    <name>hive.warehouse.subdir.inherit.perms</name>
    <value>true</value>
    </property>
    <property>
    <name>mapred.reduce.tasks</name>
    <value>-1</value>
    </property>
    <property>
    <name>hive.exec.reducers.bytes.per.reducer</name>
    <value>1073741824</value>
    </property>
    <property>
    <name>hive.exec.reducers.max</name>
    <value>999</value>
    </property>
    <property>
    <name>hive.metastore.execute.setugi</name>
    <value>true</value>
    </property>
    <!--'hive.support.concurrency', originally set to 'true'
    (non-final),
    is
    overridden below by a safety valve-->
    <property>
    <name>hive.zookeeper.quorum</name>

    <value>btirsvr70.domain.test,btirsvr69.domain.test,btirsvr74.domain.test</value>
    </property>
    <property>
    <name>hive.zookeeper.client.port</name>
    <value>2181</value>
    </property>
    <property>
    <name>hive.server2.thrift.port</name>
    <value>10000</value>
    </property>
    <property>
    <name>hive.server2.enable.impersonation</name>
    <value>true</value>
    </property>
    <property>
    <name>fs.hdfs.impl.disable.cache</name>
    <value>true</value>
    </property>
    <property>
    <name>hive.support.concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.thrift.max.worker.threads</name>
    <value>30</value>
    </property>
    </configuration>


    在 2013年9月8日星期日UTC+8下午12时04分43秒,Harsh J写道:
    What is the exact name of the safety valve you modified and what
    generated file are you checking? For HS2, which is a service
    daemon,
    the right file to check would be at CM -> Hive -> Instances ->
    Hive
    Server 2 -> Process -> Show Config and Environment Files ->
    hive-site.xml.

    On Sun, Sep 8, 2013 at 8:16 AM, xiong xu <xu.xi...@gmail.com>
    wrote:
    Hi,

    I added some properties in HS2 hive-site.xml security valve,
    and
    restart
    HS2, but the auto-generated hive-site.xml is still not changed.
    What am I missing?

    Thanks,
    Xiong

    To unsubscribe from this group and stop receiving emails from
    it,
    send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it,
    send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it,
    send an
    email to scm-users+...@cloudera.org <javascript:>.



    --
    Thanks,
    Darren


    --
    Thanks,
    Darren

    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Darren Lo at Sep 9, 2013 at 5:34 am
    Glad to hear that it's working for you now!

    If you use the enterprise or trial edition, you can see the configuration
    history and maybe learn when the HS2 override was set and what else was set
    at that time, as well as the user who made the change.

    Thanks,
    Darren

    On Sun, Sep 8, 2013 at 9:31 PM, xiong xu wrote:

    Thank you, Darren.
    Problem solved, although I'm not sure about the real cause.

    在 2013年9月9日星期一UTC+8下午12时04分04秒,Darren Lo写道:
    Hi Xiong,

    A role override will only override the group config, so I don't think HMS
    is affecting HS2, since they are entirely different role types. It's
    possible that someone accidentally set the HS2 role-level safety valve in
    the past instead of the group-level one. It's an easy mistake to make,
    though CM does show a popup warning if you look at role-level config when
    only one role is defined.

    Were you able to solve your problem?

    On Sun, Sep 8, 2013 at 8:50 PM, 徐雄 wrote:

    Thanks, Darren.
    You're right, that's a role-level override problem!
    But I still don't know why this role-level override happen.
    A override in metastore server will cause HS2 configuration overridden?


    2013/9/9 Darren Lo <d...@cloudera.com>
    Hi Xiong,

    I set up a CM4.5.2 cluster and put the same information into the HS2
    safety valve, and I saw that the configs appeared in HS2's hive-site.xml.
    Here's the end of the file, which includes the aux jars:
    "
    <property>
    <name>hive.aux.jars.path</**name>
    <value>file:///opt/cloudera/**parcels/CDH-4.2.1-1.cdh4.2.1.**
    p0.5/lib/hive/lib/Rank.jar,**file:///opt/cloudera/parcels/**
    CDH-4.2.1-1.cdh4.2.1.p0.5/lib/**hive/lib/DateFunction.jar,**
    file:///opt/cloudera/parcels/**CDH-4.2.1-1.cdh4.2.1.p0.5/lib/**
    hive/lib/format.jar</value>
    </property>
    </configuration>
    "

    Do you have a role-level override that is taking precedence over your
    HS2 group-level configuration? When you click on Hive - > Configuration ->
    View and Edit -> HiveServer2 (Base) -> Advanced, when you hover over text
    box of the safety valve for hive-site you should see something appear below
    the box saying "Override Instances". If it says "Overridden by 1
    instance(s)" then you have a role-level override, which you probably want
    to remove. You can click on that text, and it will show you the override
    and an option to Edit Overrides. Select the HS2 role and make sure
    Inherited Value is selected in the dropdown, then click Apply.

    If that's not the problem, it's possible that your HS2 is in a
    different role config group, though that is far less likely. You can go to
    Configuration -> Role Groups to verify the correct group for HS2.

    Thanks,
    Darren

    On Sun, Sep 8, 2013 at 5:18 PM, 徐雄 wrote:

    CM version is 4.5.2.
    I did restart HS2 several times.
    The concurrency safety valve taking effect was set a month ago, the CM
    safety valve worked then.
    But now, it doesn't work anymore.

    2013/9/9 Darren Lo <d...@cloudera.com>:
    Hi Xiong,

    It seems like some of your safety valve is taking effect
    (concurrency gets
    overridden to false), while others are not (authentication and aux jars).
    You already said you restarted HS2, which would otherwise have been my guess
    as to the problem. Might be worth restarting again just to be sure. Do you
    have other safety valves set for Hive that could be interfering?

    What CM version are you using?

    Thanks,
    Darren

    On Sun, Sep 8, 2013 at 6:01 AM, xiong xu wrote:

    I lookup this name from the
    html(http://.../cmf/services/**135/config?groupParent=&**
    groupKey=config.hive.group_**hive1-HIVESERVER2-BASE)
    source:
    name="config.group_hive1-**HIVESERVER2-BASE.hive_hs2_**
    config_safety_valve"
    And, yes, I did restart HS2.

    在 2013年9月8日星期日UTC+8下午7时08分51秒,**Harsh J写道:
    Hi,

    There are multiple service-specific safety valves. What is the
    exact
    name of the safety valve you modified, and did you restart the HS2
    process?

    On Sun, Sep 8, 2013 at 2:55 PM, xiong xu <xu.xi...@gmail.com>
    wrote:
    Here is what I put in the safety valve:
    <property>
    <name>hive.server2.thrift.max.**worker.threads</name>
    <value>30</value>
    <description>Maximum number of Thrift worker
    threads</description>
    </property>
    <property>
    <name>hive.support.**concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.**authentication</name>
    <value>NOSASL</value>
    </property>
    <property>
    <name>hive.aux.jars.path</**name>


    <value>file:///opt/cloudera/**parcels/CDH-4.2.1-1.cdh4.2.1.**
    p0.5/lib/hive/lib/Rank.jar,**file:///opt/cloudera/parcels/**
    CDH-4.2.1-1.cdh4.2.1.p0.5/lib/**hive/lib/DateFunction.jar,**
    file:///opt/cloudera/parcels/**CDH-4.2.1-1.cdh4.2.1.p0.5/lib/**
    hive/lib/format.jar</value>
    </property>

    And here is the generated
    file(/cmf/process/3050/config?**filename=hive-site.xml)
    contains:
    <?xml version="1.0" encoding="UTF-8"?>

    <!--Autogenerated by Cloudera CM on 2013-09-07T16:30:37.546Z-->
    <configuration>
    <property>
    <name>hive.metastore.local</**name>
    <value>false</value>
    </property>
    <property>
    <name>hive.metastore.uris</**name>
    <value>thrift://btirsvr69.**domain.test:9083</value>
    </property>
    <property>
    <name>hive.metastore.**warehouse.dir</name>
    <value>/user/hive/warehouse</**value>
    </property>
    <property>
    <name>hive.warehouse.subdir.**inherit.perms</name>
    <value>true</value>
    </property>
    <property>
    <name>mapred.reduce.tasks</**name>
    <value>-1</value>
    </property>
    <property>
    <name>hive.exec.reducers.**bytes.per.reducer</name>
    <value>1073741824</value>
    </property>
    <property>
    <name>hive.exec.reducers.max</**name>
    <value>999</value>
    </property>
    <property>
    <name>hive.metastore.execute.**setugi</name>
    <value>true</value>
    </property>
    <!--'hive.support.concurrency'**, originally set to 'true'
    (non-final),
    is
    overridden below by a safety valve-->
    <property>
    <name>hive.zookeeper.quorum</**name>


    <value>btirsvr70.domain.test,**btirsvr69.domain.test,**
    btirsvr74.domain.test</value>
    </property>
    <property>
    <name>hive.zookeeper.client.**port</name>
    <value>2181</value>
    </property>
    <property>
    <name>hive.server2.thrift.**port</name>
    <value>10000</value>
    </property>
    <property>
    <name>hive.server2.enable.**impersonation</name>
    <value>true</value>
    </property>
    <property>
    <name>fs.hdfs.impl.disable.**cache</name>
    <value>true</value>
    </property>
    <property>
    <name>hive.support.**concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.thrift.max.**worker.threads</name>
    <value>30</value>
    </property>
    </configuration>


    在 2013年9月8日星期日UTC+8下午12时04分43秒,**Harsh J写道:
    What is the exact name of the safety valve you modified and what
    generated file are you checking? For HS2, which is a service
    daemon,
    the right file to check would be at CM -> Hive -> Instances ->
    Hive
    Server 2 -> Process -> Show Config and Environment Files ->
    hive-site.xml.

    On Sun, Sep 8, 2013 at 8:16 AM, xiong xu <xu.xi...@gmail.com>
    wrote:
    Hi,

    I added some properties in HS2 hive-site.xml security valve,
    and
    restart
    HS2, but the auto-generated hive-site.xml is still not
    changed.
    What am I missing?

    Thanks,
    Xiong

    To unsubscribe from this group and stop receiving emails from
    it,
    send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from
    it, send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it,
    send an
    email to scm-users+...@**cloudera.org.



    --
    Thanks,
    Darren


    --
    Thanks,
    Darren

    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it, send an
    email to scm-users+unsubscribe@cloudera.org.


    --
    Thanks,
    Darren

    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Xiong xu at Sep 9, 2013 at 7:08 am
    Still wonder, if CM web console crashes or cannot be accessed, what
    fallback action should I take?
    What command line scripts we can rely on?

    在 2013年9月9日星期一UTC+8下午1时34分31秒,Darren Lo写道:
    Glad to hear that it's working for you now!

    If you use the enterprise or trial edition, you can see the configuration
    history and maybe learn when the HS2 override was set and what else was set
    at that time, as well as the user who made the change.

    Thanks,
    Darren


    On Sun, Sep 8, 2013 at 9:31 PM, xiong xu <xu.xi...@gmail.com <javascript:>
    wrote:
    Thank you, Darren.
    Problem solved, although I'm not sure about the real cause.

    在 2013年9月9日星期一UTC+8下午12时04分04秒,Darren Lo写道:
    Hi Xiong,

    A role override will only override the group config, so I don't think
    HMS is affecting HS2, since they are entirely different role types. It's
    possible that someone accidentally set the HS2 role-level safety valve in
    the past instead of the group-level one. It's an easy mistake to make,
    though CM does show a popup warning if you look at role-level config when
    only one role is defined.

    Were you able to solve your problem?

    On Sun, Sep 8, 2013 at 8:50 PM, 徐雄 wrote:

    Thanks, Darren.
    You're right, that's a role-level override problem!
    But I still don't know why this role-level override happen.
    A override in metastore server will cause HS2 configuration overridden?


    2013/9/9 Darren Lo <d...@cloudera.com>
    Hi Xiong,

    I set up a CM4.5.2 cluster and put the same information into the HS2
    safety valve, and I saw that the configs appeared in HS2's hive-site.xml.
    Here's the end of the file, which includes the aux jars:
    "
    <property>
    <name>hive.aux.jars.path</**name>
    <value>file:///opt/cloudera/**parcels/CDH-4.2.1-1.cdh4.2.1.**
    p0.5/lib/hive/lib/Rank.jar,**file:///opt/cloudera/parcels/**
    CDH-4.2.1-1.cdh4.2.1.p0.5/lib/**hive/lib/DateFunction.jar,**
    file:///opt/cloudera/parcels/**CDH-4.2.1-1.cdh4.2.1.p0.5/lib/**
    hive/lib/format.jar</value>
    </property>
    </configuration>
    "

    Do you have a role-level override that is taking precedence over your
    HS2 group-level configuration? When you click on Hive - > Configuration ->
    View and Edit -> HiveServer2 (Base) -> Advanced, when you hover over text
    box of the safety valve for hive-site you should see something appear below
    the box saying "Override Instances". If it says "Overridden by 1
    instance(s)" then you have a role-level override, which you probably want
    to remove. You can click on that text, and it will show you the override
    and an option to Edit Overrides. Select the HS2 role and make sure
    Inherited Value is selected in the dropdown, then click Apply.

    If that's not the problem, it's possible that your HS2 is in a
    different role config group, though that is far less likely. You can go to
    Configuration -> Role Groups to verify the correct group for HS2.

    Thanks,
    Darren

    On Sun, Sep 8, 2013 at 5:18 PM, 徐雄 wrote:

    CM version is 4.5.2.
    I did restart HS2 several times.
    The concurrency safety valve taking effect was set a month ago, the CM
    safety valve worked then.
    But now, it doesn't work anymore.

    2013/9/9 Darren Lo <d...@cloudera.com>:
    Hi Xiong,

    It seems like some of your safety valve is taking effect
    (concurrency gets
    overridden to false), while others are not (authentication and aux jars).
    You already said you restarted HS2, which would otherwise have been my guess
    as to the problem. Might be worth restarting again just to be sure. Do you
    have other safety valves set for Hive that could be interfering?

    What CM version are you using?

    Thanks,
    Darren


    On Sun, Sep 8, 2013 at 6:01 AM, xiong xu <xu.xi...@gmail.com>
    wrote:
    I lookup this name from the
    html(http://.../cmf/services/**135/config?groupParent=&**
    groupKey=config.hive.group_**hive1-HIVESERVER2-BASE)
    source:
    name="config.group_hive1-**HIVESERVER2-BASE.hive_hs2_**
    config_safety_valve"
    And, yes, I did restart HS2.

    在 2013年9月8日星期日UTC+8下午7时08分51秒,**Harsh J写道:
    Hi,

    There are multiple service-specific safety valves. What is the
    exact
    name of the safety valve you modified, and did you restart the HS2
    process?

    On Sun, Sep 8, 2013 at 2:55 PM, xiong xu <xu.xi...@gmail.com>
    wrote:
    Here is what I put in the safety valve:
    <property>
    <name>hive.server2.thrift.max.**worker.threads</name>
    <value>30</value>
    <description>Maximum number of Thrift worker
    threads</description>
    </property>
    <property>
    <name>hive.support.**concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.**authentication</name>
    <value>NOSASL</value>
    </property>
    <property>
    <name>hive.aux.jars.path</**name>


    <value>file:///opt/cloudera/**parcels/CDH-4.2.1-1.cdh4.2.1.**
    p0.5/lib/hive/lib/Rank.jar,**file:///opt/cloudera/parcels/**
    CDH-4.2.1-1.cdh4.2.1.p0.5/lib/**hive/lib/DateFunction.jar,**
    file:///opt/cloudera/parcels/**CDH-4.2.1-1.cdh4.2.1.p0.5/lib/**
    hive/lib/format.jar</value>
    </property>

    And here is the generated
    file(/cmf/process/3050/config?**filename=hive-site.xml)
    contains:
    <?xml version="1.0" encoding="UTF-8"?>

    <!--Autogenerated by Cloudera CM on 2013-09-07T16:30:37.546Z-->
    <configuration>
    <property>
    <name>hive.metastore.local</**name>
    <value>false</value>
    </property>
    <property>
    <name>hive.metastore.uris</**name>
    <value>thrift://btirsvr69.**domain.test:9083</value>
    </property>
    <property>
    <name>hive.metastore.**warehouse.dir</name>
    <value>/user/hive/warehouse</**value>
    </property>
    <property>
    <name>hive.warehouse.subdir.**inherit.perms</name>
    <value>true</value>
    </property>
    <property>
    <name>mapred.reduce.tasks</**name>
    <value>-1</value>
    </property>
    <property>
    <name>hive.exec.reducers.**bytes.per.reducer</name>
    <value>1073741824</value>
    </property>
    <property>
    <name>hive.exec.reducers.max</**name>
    <value>999</value>
    </property>
    <property>
    <name>hive.metastore.execute.**setugi</name>
    <value>true</value>
    </property>
    <!--'hive.support.concurrency'**, originally set to 'true'
    (non-final),
    is
    overridden below by a safety valve-->
    <property>
    <name>hive.zookeeper.quorum</**name>


    <value>btirsvr70.domain.test,**btirsvr69.domain.test,**
    btirsvr74.domain.test</value>
    </property>
    <property>
    <name>hive.zookeeper.client.**port</name>
    <value>2181</value>
    </property>
    <property>
    <name>hive.server2.thrift.**port</name>
    <value>10000</value>
    </property>
    <property>
    <name>hive.server2.enable.**impersonation</name>
    <value>true</value>
    </property>
    <property>
    <name>fs.hdfs.impl.disable.**cache</name>
    <value>true</value>
    </property>
    <property>
    <name>hive.support.**concurrency</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.thrift.max.**worker.threads</name>
    <value>30</value>
    </property>
    </configuration>


    在 2013年9月8日星期日UTC+8下午12时04分43秒,**Harsh J写道:
    What is the exact name of the safety valve you modified and
    what
    generated file are you checking? For HS2, which is a service
    daemon,
    the right file to check would be at CM -> Hive -> Instances ->
    Hive
    Server 2 -> Process -> Show Config and Environment Files ->
    hive-site.xml.

    On Sun, Sep 8, 2013 at 8:16 AM, xiong xu <xu.xi...@gmail.com>
    wrote:
    Hi,

    I added some properties in HS2 hive-site.xml security valve,
    and
    restart
    HS2, but the auto-generated hive-site.xml is still not
    changed.
    What am I missing?

    Thanks,
    Xiong

    To unsubscribe from this group and stop receiving emails
    from it,
    send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from
    it, send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it,
    send an
    email to scm-users+...@**cloudera.org.



    --
    Thanks,
    Darren


    --
    Thanks,
    Darren

    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it, send
    an email to scm-users+...@cloudera.org <javascript:>.


    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Darren Lo at Sep 9, 2013 at 3:39 pm
    You should have some plan for how to restore your CM server quickly. If
    your database is still available, you can start CM on a different host,
    then either use DNS tricks to make your new host have the same name and ip
    address as the old CM host, or update all CM agent config to point to the
    new CM host and restart the agents. You should also regularly take database
    backups (kept on a different machine than the database) that can be used to
    restore the database to a new host or an older state.

    Make sure you don't run two CM servers pointing to the same database at the
    same time.

    If you need to restart a process without CM, you can tell supervisord to
    restart it with the same config directory, though I forget the details
    there (haven't done it myself). You can modify config by editing the
    appropriate files in /var/run/cloudera-scm-agent/processes.

    Thanks,
    Darren

    On Mon, Sep 9, 2013 at 12:07 AM, xiong xu wrote:

    Still wonder, if CM web console crashes or cannot be accessed, what
    fallback action should I take?
    What command line scripts we can rely on?

    在 2013年9月9日星期一UTC+8下午1时34分31秒,Darren Lo写道:
    Glad to hear that it's working for you now!

    If you use the enterprise or trial edition, you can see the configuration
    history and maybe learn when the HS2 override was set and what else was set
    at that time, as well as the user who made the change.

    Thanks,
    Darren

    On Sun, Sep 8, 2013 at 9:31 PM, xiong xu wrote:

    Thank you, Darren.
    Problem solved, although I'm not sure about the real cause.

    在 2013年9月9日星期一UTC+8下午12时04分04秒,**Darren Lo写道:
    Hi Xiong,

    A role override will only override the group config, so I don't think
    HMS is affecting HS2, since they are entirely different role types. It's
    possible that someone accidentally set the HS2 role-level safety valve in
    the past instead of the group-level one. It's an easy mistake to make,
    though CM does show a popup warning if you look at role-level config when
    only one role is defined.

    Were you able to solve your problem?

    On Sun, Sep 8, 2013 at 8:50 PM, 徐雄 wrote:

    Thanks, Darren.
    You're right, that's a role-level override problem!
    But I still don't know why this role-level override happen.
    A override in metastore server will cause HS2 configuration overridden?


    2013/9/9 Darren Lo <d...@cloudera.com>
    Hi Xiong,

    I set up a CM4.5.2 cluster and put the same information into the HS2
    safety valve, and I saw that the configs appeared in HS2's hive-site.xml.
    Here's the end of the file, which includes the aux jars:
    "
    <property>
    <name>hive.aux.jars.path</**name**>
    <value>file:///opt/cloudera/**pa**rcels/CDH-4.2.1-1.cdh4.2.1.**
    p0.**5/lib/hive/lib/Rank.jar,**file:/**//opt/cloudera/parcels/**
    CDH-4.**2.1-1.cdh4.2.1.p0.5/lib/**hive/**lib/DateFunction.jar,**
    file:///**opt/cloudera/parcels/**CDH-4.2.**1-1.cdh4.2.1.p0.5/lib/**
    hive/**lib/format.jar</value>
    </property>
    </configuration>
    "

    Do you have a role-level override that is taking precedence over your
    HS2 group-level configuration? When you click on Hive - > Configuration ->
    View and Edit -> HiveServer2 (Base) -> Advanced, when you hover over text
    box of the safety valve for hive-site you should see something appear below
    the box saying "Override Instances". If it says "Overridden by 1
    instance(s)" then you have a role-level override, which you probably want
    to remove. You can click on that text, and it will show you the override
    and an option to Edit Overrides. Select the HS2 role and make sure
    Inherited Value is selected in the dropdown, then click Apply.

    If that's not the problem, it's possible that your HS2 is in a
    different role config group, though that is far less likely. You can go to
    Configuration -> Role Groups to verify the correct group for HS2.

    Thanks,
    Darren

    On Sun, Sep 8, 2013 at 5:18 PM, 徐雄 wrote:

    CM version is 4.5.2.
    I did restart HS2 several times.
    The concurrency safety valve taking effect was set a month ago, the
    CM
    safety valve worked then.
    But now, it doesn't work anymore.

    2013/9/9 Darren Lo <d...@cloudera.com>:
    Hi Xiong,

    It seems like some of your safety valve is taking effect
    (concurrency gets
    overridden to false), while others are not (authentication and aux jars).
    You already said you restarted HS2, which would otherwise have
    been my guess
    as to the problem. Might be worth restarting again just to be
    sure. Do you
    have other safety valves set for Hive that could be interfering?

    What CM version are you using?

    Thanks,
    Darren


    On Sun, Sep 8, 2013 at 6:01 AM, xiong xu <xu.xi...@gmail.com>
    wrote:
    I lookup this name from the
    html(http://.../cmf/services/**1**35/config?groupParent=&**
    groupKe**y=config.hive.group_**hive1-**HIVESERVER2-BASE)
    source:
    name="config.group_hive1-**HIVES**ERVER2-BASE.hive_hs2_**config_*
    *safety_valve"
    And, yes, I did restart HS2.

    在 2013年9月8日星期日UTC+8下午7时08分51秒,**Ha**rsh J写道:
    Hi,

    There are multiple service-specific safety valves. What is the
    exact
    name of the safety valve you modified, and did you restart the
    HS2
    process?

    On Sun, Sep 8, 2013 at 2:55 PM, xiong xu <xu.xi...@gmail.com>
    wrote:
    Here is what I put in the safety valve:
    <property>
    <name>hive.server2.thrift.max.****worker.threads</name>
    <value>30</value>
    <description>Maximum number of Thrift worker
    threads</description>
    </property>
    <property>
    <name>hive.support.**concurrency**</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.**authentica**tion</name>
    <value>NOSASL</value>
    </property>
    <property>
    <name>hive.aux.jars.path</**nam**e>


    <value>file:///opt/cloudera/**pa**rcels/CDH-4.2.1-1.cdh4.2.1.*
    *p0.**5/lib/hive/lib/Rank.jar,**file:/**//opt/cloudera/parcels/**
    CDH-4.**2.1-1.cdh4.2.1.p0.5/lib/**hive/**lib/DateFunction.jar,**
    file:///**opt/cloudera/parcels/**CDH-4.2.**1-1.cdh4.2.1.p0.5/lib/**
    hive/**lib/format.jar</value>
    </property>

    And here is the generated
    file(/cmf/process/3050/config?****filename=hive-site.xml)
    contains:
    <?xml version="1.0" encoding="UTF-8"?>

    <!--Autogenerated by Cloudera CM on 2013-09-07T16:30:37.546Z-->
    <configuration>
    <property>
    <name>hive.metastore.local</**na**me>
    <value>false</value>
    </property>
    <property>
    <name>hive.metastore.uris</**nam**e>
    <value>thrift://btirsvr69.**doma**in.test:9083</value>
    </property>
    <property>
    <name>hive.metastore.**warehouse**.dir</name>
    <value>/user/hive/warehouse</**v**alue>
    </property>
    <property>
    <name>hive.warehouse.subdir.**in**herit.perms</name>
    <value>true</value>
    </property>
    <property>
    <name>mapred.reduce.tasks</**nam**e>
    <value>-1</value>
    </property>
    <property>
    <name>hive.exec.reducers.**bytes**.per.reducer</name>
    <value>1073741824</value>
    </property>
    <property>
    <name>hive.exec.reducers.max</****name>
    <value>999</value>
    </property>
    <property>
    <name>hive.metastore.execute.**s**etugi</name>
    <value>true</value>
    </property>
    <!--'hive.support.concurrency'****, originally set to
    'true' (non-final),
    is
    overridden below by a safety valve-->
    <property>
    <name>hive.zookeeper.quorum</**n**ame>


    <value>btirsvr70.domain.test,**b**tirsvr69.domain.test,**
    btirsvr74**.domain.test</value>
    </property>
    <property>
    <name>hive.zookeeper.client.**po**rt</name>
    <value>2181</value>
    </property>
    <property>
    <name>hive.server2.thrift.**port**</name>
    <value>10000</value>
    </property>
    <property>
    <name>hive.server2.enable.**impe**rsonation</name>
    <value>true</value>
    </property>
    <property>
    <name>fs.hdfs.impl.disable.**cac**he</name>
    <value>true</value>
    </property>
    <property>
    <name>hive.support.**concurrency**</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.thrift.max.****worker.threads</name>
    <value>30</value>
    </property>
    </configuration>


    在 2013年9月8日星期日UTC+8下午12时04分43秒,**H**arsh J写道:
    What is the exact name of the safety valve you modified and
    what
    generated file are you checking? For HS2, which is a service
    daemon,
    the right file to check would be at CM -> Hive -> Instances
    -> Hive
    Server 2 -> Process -> Show Config and Environment Files ->
    hive-site.xml.

    On Sun, Sep 8, 2013 at 8:16 AM, xiong xu <xu.xi...@gmail.com>
    wrote:
    Hi,

    I added some properties in HS2 hive-site.xml security
    valve, and
    restart
    HS2, but the auto-generated hive-site.xml is still not
    changed.
    What am I missing?

    Thanks,
    Xiong

    To unsubscribe from this group and stop receiving emails
    from it,
    send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from
    it, send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from it,
    send an
    email to scm-users+...@**cloudera.org.



    --
    Thanks,
    Darren


    --
    Thanks,
    Darren

    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it, send
    an email to scm-users+...@**cloudera.org.


    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it, send an
    email to scm-users+unsubscribe@cloudera.org.


    --
    Thanks,
    Darren

    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Xiong xu at Sep 10, 2013 at 9:36 am
    Thanks for advice.
    And I can see that without CM there's much more work to do.

    在 2013年9月9日星期一UTC+8下午11时38分58秒,Darren Lo写道:
    You should have some plan for how to restore your CM server quickly. If
    your database is still available, you can start CM on a different host,
    then either use DNS tricks to make your new host have the same name and ip
    address as the old CM host, or update all CM agent config to point to the
    new CM host and restart the agents. You should also regularly take database
    backups (kept on a different machine than the database) that can be used to
    restore the database to a new host or an older state.

    Make sure you don't run two CM servers pointing to the same database at
    the same time.

    If you need to restart a process without CM, you can tell supervisord to
    restart it with the same config directory, though I forget the details
    there (haven't done it myself). You can modify config by editing the
    appropriate files in /var/run/cloudera-scm-agent/processes.

    Thanks,
    Darren


    On Mon, Sep 9, 2013 at 12:07 AM, xiong xu <xu.xi...@gmail.com<javascript:>
    wrote:
    Still wonder, if CM web console crashes or cannot be accessed, what
    fallback action should I take?
    What command line scripts we can rely on?

    在 2013年9月9日星期一UTC+8下午1时34分31秒,Darren Lo写道:
    Glad to hear that it's working for you now!

    If you use the enterprise or trial edition, you can see the
    configuration history and maybe learn when the HS2 override was set and
    what else was set at that time, as well as the user who made the change.

    Thanks,
    Darren

    On Sun, Sep 8, 2013 at 9:31 PM, xiong xu wrote:

    Thank you, Darren.
    Problem solved, although I'm not sure about the real cause.

    在 2013年9月9日星期一UTC+8下午12时04分04秒,**Darren Lo写道:
    Hi Xiong,

    A role override will only override the group config, so I don't think
    HMS is affecting HS2, since they are entirely different role types. It's
    possible that someone accidentally set the HS2 role-level safety valve in
    the past instead of the group-level one. It's an easy mistake to make,
    though CM does show a popup warning if you look at role-level config when
    only one role is defined.

    Were you able to solve your problem?

    On Sun, Sep 8, 2013 at 8:50 PM, 徐雄 wrote:

    Thanks, Darren.
    You're right, that's a role-level override problem!
    But I still don't know why this role-level override happen.
    A override in metastore server will cause HS2
    configuration overridden?


    2013/9/9 Darren Lo <d...@cloudera.com>
    Hi Xiong,

    I set up a CM4.5.2 cluster and put the same information into the HS2
    safety valve, and I saw that the configs appeared in HS2's hive-site.xml.
    Here's the end of the file, which includes the aux jars:
    "
    <property>
    <name>hive.aux.jars.path</**name**>
    <value>file:///opt/cloudera/**pa**rcels/CDH-4.2.1-1.cdh4.2.1.**
    p0.**5/lib/hive/lib/Rank.jar,**file:/**//opt/cloudera/parcels/**
    CDH-4.**2.1-1.cdh4.2.1.p0.5/lib/**hive/**lib/DateFunction.jar,**
    file:///**opt/cloudera/parcels/**CDH-4.2.**1-1.cdh4.2.1.p0.5/lib/**
    hive/**lib/format.jar</value>
    </property>
    </configuration>
    "

    Do you have a role-level override that is taking precedence over
    your HS2 group-level configuration? When you click on Hive - >
    Configuration -> View and Edit -> HiveServer2 (Base) -> Advanced, when you
    hover over text box of the safety valve for hive-site you should see
    something appear below the box saying "Override Instances". If it says
    "Overridden by 1 instance(s)" then you have a role-level override, which
    you probably want to remove. You can click on that text, and it will show
    you the override and an option to Edit Overrides. Select the HS2 role and
    make sure Inherited Value is selected in the dropdown, then click Apply.

    If that's not the problem, it's possible that your HS2 is in a
    different role config group, though that is far less likely. You can go to
    Configuration -> Role Groups to verify the correct group for HS2.

    Thanks,
    Darren

    On Sun, Sep 8, 2013 at 5:18 PM, 徐雄 wrote:

    CM version is 4.5.2.
    I did restart HS2 several times.
    The concurrency safety valve taking effect was set a month ago, the
    CM
    safety valve worked then.
    But now, it doesn't work anymore.

    2013/9/9 Darren Lo <d...@cloudera.com>:
    Hi Xiong,

    It seems like some of your safety valve is taking effect
    (concurrency gets
    overridden to false), while others are not (authentication and
    aux jars).
    You already said you restarted HS2, which would otherwise have
    been my guess
    as to the problem. Might be worth restarting again just to be
    sure. Do you
    have other safety valves set for Hive that could be interfering?

    What CM version are you using?

    Thanks,
    Darren


    On Sun, Sep 8, 2013 at 6:01 AM, xiong xu <xu.xi...@gmail.com>
    wrote:
    I lookup this name from the
    html(http://.../cmf/services/**1**35/config?groupParent=&**
    groupKe**y=config.hive.group_**hive1-**HIVESERVER2-BASE)
    source:
    name="config.group_hive1-**HIVES**ERVER2-BASE.hive_hs2_**config_
    **safety_valve"
    And, yes, I did restart HS2.

    在 2013年9月8日星期日UTC+8下午7时08分51秒,**Ha**rsh J写道:
    Hi,

    There are multiple service-specific safety valves. What is the
    exact
    name of the safety valve you modified, and did you restart the
    HS2
    process?

    On Sun, Sep 8, 2013 at 2:55 PM, xiong xu <xu.xi...@gmail.com>
    wrote:
    Here is what I put in the safety valve:
    <property>
    <name>hive.server2.thrift.max.****worker.threads</name>
    <value>30</value>
    <description>Maximum number of Thrift worker
    threads</description>
    </property>
    <property>
    <name>hive.support.**concurrency**</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.**authentica**tion</name>
    <value>NOSASL</value>
    </property>
    <property>
    <name>hive.aux.jars.path</**nam**e>


    <value>file:///opt/cloudera/**pa**rcels/CDH-4.2.1-1.cdh4.2.1.
    **p0.**5/lib/hive/lib/Rank.jar,**file:/**//opt/cloudera/parcels/**
    CDH-4.**2.1-1.cdh4.2.1.p0.5/lib/**hive/**lib/DateFunction.jar,**
    file:///**opt/cloudera/parcels/**CDH-4.2.**1-1.cdh4.2.1.p0.5/lib/**
    hive/**lib/format.jar</value>
    </property>

    And here is the generated
    file(/cmf/process/3050/config?****filename=hive-site.xml)
    contains:
    <?xml version="1.0" encoding="UTF-8"?>

    <!--Autogenerated by Cloudera CM on
    2013-09-07T16:30:37.546Z-->
    <configuration>
    <property>
    <name>hive.metastore.local</**na**me>
    <value>false</value>
    </property>
    <property>
    <name>hive.metastore.uris</**nam**e>
    <value>thrift://btirsvr69.**doma**in.test:9083</value>
    </property>
    <property>
    <name>hive.metastore.**warehouse**.dir</name>
    <value>/user/hive/warehouse</**v**alue>
    </property>
    <property>
    <name>hive.warehouse.subdir.**in**herit.perms</name>
    <value>true</value>
    </property>
    <property>
    <name>mapred.reduce.tasks</**nam**e>
    <value>-1</value>
    </property>
    <property>
    <name>hive.exec.reducers.**bytes**.per.reducer</name>
    <value>1073741824</value>
    </property>
    <property>
    <name>hive.exec.reducers.max</****name>
    <value>999</value>
    </property>
    <property>
    <name>hive.metastore.execute.**s**etugi</name>
    <value>true</value>
    </property>
    <!--'hive.support.concurrency'****, originally set to
    'true' (non-final),
    is
    overridden below by a safety valve-->
    <property>
    <name>hive.zookeeper.quorum</**n**ame>


    <value>btirsvr70.domain.test,**b**tirsvr69.domain.test,**
    btirsvr74**.domain.test</value>
    </property>
    <property>
    <name>hive.zookeeper.client.**po**rt</name>
    <value>2181</value>
    </property>
    <property>
    <name>hive.server2.thrift.**port**</name>
    <value>10000</value>
    </property>
    <property>
    <name>hive.server2.enable.**impe**rsonation</name>
    <value>true</value>
    </property>
    <property>
    <name>fs.hdfs.impl.disable.**cac**he</name>
    <value>true</value>
    </property>
    <property>
    <name>hive.support.**concurrency**</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.server2.thrift.max.****worker.threads</name>
    <value>30</value>
    </property>
    </configuration>


    在 2013年9月8日星期日UTC+8下午12时04分43秒,**H**arsh J写道:
    What is the exact name of the safety valve you modified and
    what
    generated file are you checking? For HS2, which is a service
    daemon,
    the right file to check would be at CM -> Hive -> Instances
    -> Hive
    Server 2 -> Process -> Show Config and Environment Files ->
    hive-site.xml.

    On Sun, Sep 8, 2013 at 8:16 AM, xiong xu <xu.xi...@gmail.com>
    wrote:
    Hi,

    I added some properties in HS2 hive-site.xml security
    valve, and
    restart
    HS2, but the auto-generated hive-site.xml is still not
    changed.
    What am I missing?

    Thanks,
    Xiong

    To unsubscribe from this group and stop receiving emails
    from it,
    send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from
    it, send
    an
    email to scm-users+...@cloudera.org.


    --
    Harsh J
    To unsubscribe from this group and stop receiving emails from
    it, send an
    email to scm-users+...@**cloudera.org.



    --
    Thanks,
    Darren


    --
    Thanks,
    Darren

    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it, send
    an email to scm-users+...@**cloudera.org.


    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it, send
    an email to scm-users+...@cloudera.org <javascript:>.


    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedSep 8, '13 at 2:46a
activeSep 10, '13 at 9:36a
posts16
users4
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase