Directory commands (available if you click on Hive and look in the actions
menu on the right), then the Start command for hive.
Then you should be able to re-try the first run command (find it in your
command history at http://<cm_host>:7180/cmf/commands/commands, click on
your first run command, and look for a retry button), or you can run each
of the listed commands manually, one by one.
On Wed, Apr 23, 2014 at 12:41 PM, Ivan Hsueh wrote:
Dear Darren,
It's seems work. I can start the metastore via UI.
Thu Apr 24 03:26:29 CST 2014
JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
using /usr/java/jdk1.7.0_45-cloudera as JAVA_HOME
using 5 as CDH_VERSION
using /opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hive as HIVE_HOME
using /var/run/cloudera-scm-agent/process/33-hive-HIVEMETASTORE as HIVE_CONF_DIR
using /opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop as HADOOP_HOME
using /var/run/cloudera-scm-agent/process/33-hive-HIVEMETASTORE/yarn-conf as HADOOP_CONF_DIR
Starting Hive Metastore Server
Both OS are CentOS 6.4.
If it does really work, can I just pass this process and keep running the
rest of process via CM UI?
<https://lh3.googleusercontent.com/-7Mkz3nBNeJs/U1gWaWKBePI/AAAAAAAAAhI/YgDpz6I1F-8/s1600/Screen+Shot+2014-04-24+at+3.31.40+AM.png>
Many thanks.
Darren Lo於 2014年4月24日星期四UTC+8上午3時23分24秒寫道：
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].Dear Darren,
It's seems work. I can start the metastore via UI.
Thu Apr 24 03:26:29 CST 2014
JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
using /usr/java/jdk1.7.0_45-cloudera as JAVA_HOME
using 5 as CDH_VERSION
using /opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hive as HIVE_HOME
using /var/run/cloudera-scm-agent/process/33-hive-HIVEMETASTORE as HIVE_CONF_DIR
using /opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop as HADOOP_HOME
using /var/run/cloudera-scm-agent/process/33-hive-HIVEMETASTORE/yarn-conf as HADOOP_CONF_DIR
Starting Hive Metastore Server
Both OS are CentOS 6.4.
If it does really work, can I just pass this process and keep running the
rest of process via CM UI?
<https://lh3.googleusercontent.com/-7Mkz3nBNeJs/U1gWaWKBePI/AAAAAAAAAhI/YgDpz6I1F-8/s1600/Screen+Shot+2014-04-24+at+3.31.40+AM.png>
Many thanks.
Darren Lo於 2014年4月24日星期四UTC+8上午3時23分24秒寫道：
Hi Ivan,
It looks like create table actually succeeded the first time. What
happens if you try to start the metastore (via the CM UI)?
Also, what operating system are you using?
Thanks,
Darren
an email to [email protected].It looks like create table actually succeeded the first time. What
happens if you try to start the metastore (via the CM UI)?
Also, what operating system are you using?
Thanks,
Darren
On Wed, Apr 23, 2014 at 11:50 AM, Ivan Hsueh wrote:
Dear Darren,
1.My Cloudera version is "Cloudera Express 5.0.0 (#215 built by jenkins
on 20140331-1424 git"
2.Yes, I used parcels for CDH binaries.
3.I used embedded PostgreSQL Database (default option). Sorry I can't
find the version info.
4.You are right too. First time when I ran the create tables command
seems no error. The log is as below
+ exec /opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hive/bin/schematool
-dbType postgres -initSchema -dbOpts postgres.filter.81
14/04/23 16:56:10 INFO Configuration.deprecation:
mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.
fileinputformat.input.dir.recursive
14/04/23 16:56:10 INFO Configuration.deprecation: mapred.max.split.size
is deprecated. Instead, use mapreduce.input.
fileinputformat.split.maxsize
14/04/23 16:56:10 INFO Configuration.deprecation: mapred.min.split.size
is deprecated. Instead, use mapreduce.input.
fileinputformat.split.minsize
14/04/23 16:56:10 INFO Configuration.deprecation:
mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.
fileinputformat.split.minsize.per.rack
14/04/23 16:56:10 INFO Configuration.deprecation:
mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.
fileinputformat.split.minsize.per.node
14/04/23 16:56:10 INFO Configuration.deprecation: mapred.reduce.tasks is
deprecated. Instead, use mapreduce.job.reduces
14/04/23 16:56:10 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution
is deprecated. Instead, use mapreduce.reduce.speculative
14/04/23 16:56:17 WARN conf.HiveConf: DEPRECATED: Configuration property
hive.metastore.local no longer has any effect. Make sure to provide a valid
value for hive.metastore.uris if you are connecting to a remote megastore.
So I try to check next step process (26-hdfs-NAMENODE-createdir) log.
There are some error messages.
+ echo 'Created directory /user/hive.'
+ /opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop-hdfs/bin/hdfs
--config /var/run/cloudera-scm-agent/process/26-hdfs-NAMENODE-createdir
dfs -chown hive:hive /user/hive
log4j:ERROR Could not find value for key log4j.appender.EventCounter
log4j:ERROR Could not instantiate appender named "EventCounter".
+ '[' 0 -eq 0 ']'
+ echo 'Changed ownership of /user/hive to hive:hive.'
+ /opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop-hdfs/bin/hdfs
--config /var/run/cloudera-scm-agent/process/26-hdfs-NAMENODE-createdir
dfs -chmod 1775 /user/hive
log4j:ERROR Could not find value for key log4j.appender.EventCounter
log4j:ERROR Could not instantiate appender named "EventCounter".
Darren Lo於 2014年4月24日星期四UTC+8上午1時59分25秒寫道：
To unsubscribe from this group and stop receiving emails from it, sendDear Darren,
1.My Cloudera version is "Cloudera Express 5.0.0 (#215 built by jenkins
on 20140331-1424 git"
2.Yes, I used parcels for CDH binaries.
3.I used embedded PostgreSQL Database (default option). Sorry I can't
find the version info.
4.You are right too. First time when I ran the create tables command
seems no error. The log is as below
+ exec /opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hive/bin/schematool
-dbType postgres -initSchema -dbOpts postgres.filter.81
14/04/23 16:56:10 INFO Configuration.deprecation:
mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.
fileinputformat.input.dir.recursive
14/04/23 16:56:10 INFO Configuration.deprecation: mapred.max.split.size
is deprecated. Instead, use mapreduce.input.
fileinputformat.split.maxsize
14/04/23 16:56:10 INFO Configuration.deprecation: mapred.min.split.size
is deprecated. Instead, use mapreduce.input.
fileinputformat.split.minsize
14/04/23 16:56:10 INFO Configuration.deprecation:
mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.
fileinputformat.split.minsize.per.rack
14/04/23 16:56:10 INFO Configuration.deprecation:
mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.
fileinputformat.split.minsize.per.node
14/04/23 16:56:10 INFO Configuration.deprecation: mapred.reduce.tasks is
deprecated. Instead, use mapreduce.job.reduces
14/04/23 16:56:10 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution
is deprecated. Instead, use mapreduce.reduce.speculative
14/04/23 16:56:17 WARN conf.HiveConf: DEPRECATED: Configuration property
hive.metastore.local no longer has any effect. Make sure to provide a valid
value for hive.metastore.uris if you are connecting to a remote megastore.
So I try to check next step process (26-hdfs-NAMENODE-createdir) log.
There are some error messages.
+ echo 'Created directory /user/hive.'
+ /opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop-hdfs/bin/hdfs
--config /var/run/cloudera-scm-agent/process/26-hdfs-NAMENODE-createdir
dfs -chown hive:hive /user/hive
log4j:ERROR Could not find value for key log4j.appender.EventCounter
log4j:ERROR Could not instantiate appender named "EventCounter".
+ '[' 0 -eq 0 ']'
+ echo 'Changed ownership of /user/hive to hive:hive.'
+ /opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop-hdfs/bin/hdfs
--config /var/run/cloudera-scm-agent/process/26-hdfs-NAMENODE-createdir
dfs -chmod 1775 /user/hive
log4j:ERROR Could not find value for key log4j.appender.EventCounter
log4j:ERROR Could not instantiate appender named "EventCounter".
Darren Lo於 2014年4月24日星期四UTC+8上午1時59分25秒寫道：
Hi Ivan,
You're right that the schema should have been clean when you first
started.
Can you confirm your Cloudera Manager version? Click on Support ->
About in the upper right. You may need to exit the wizard to see this link
(exit the wizard by clicking on the Cloudera Manager logo).
Did you use parcels for CDH binaries? (this is the default choice)
What version of postgresql was installed? ssh in to the machine and see
what the package manager says the version of postgresql is.
I suspect that the first time you ran the create tables command, the
error log had a different message. SSH in to the machine where the Hive
Metastore Server is running, then look in /var/run/cloudera-scm-agent/
process and find the directories with a name like
"52-hive-metastore-create-tables". The "52" part will be different on
your system. Find the earliest attempt, and look in the logs directory for
the stderr log. Please share the end of that log.
Thanks,
Darren
send an email to [email protected].You're right that the schema should have been clean when you first
started.
Can you confirm your Cloudera Manager version? Click on Support ->
About in the upper right. You may need to exit the wizard to see this link
(exit the wizard by clicking on the Cloudera Manager logo).
Did you use parcels for CDH binaries? (this is the default choice)
What version of postgresql was installed? ssh in to the machine and see
what the package manager says the version of postgresql is.
I suspect that the first time you ran the create tables command, the
error log had a different message. SSH in to the machine where the Hive
Metastore Server is running, then look in /var/run/cloudera-scm-agent/
process and find the directories with a name like
"52-hive-metastore-create-tables". The "52" part will be different on
your system. Find the earliest attempt, and look in the logs directory for
the stderr log. Please share the end of that log.
Thanks,
Darren
On Wed, Apr 23, 2014 at 10:48 AM, Ivan Hsueh wrote:
Thanks Darren,
It's weird, I installed CDH 5 from clean system.
There shouldn't be an existing table.
Another problem is I can't use schematool to initialise the DB.
Would you please describe more specifically how to clean schema.
Thanks again..
Darren Lo於 2014年4月24日星期四UTC+8上午12時50分04秒寫道：
To unsubscribe from this group and stop receiving emails from it,Thanks Darren,
It's weird, I installed CDH 5 from clean system.
There shouldn't be an existing table.
Another problem is I can't use schematool to initialise the DB.
Would you please describe more specifically how to clean schema.
Thanks again..
Darren Lo於 2014年4月24日星期四UTC+8上午12時50分04秒寫道：
Hi Ivan,
You've configured Hive to talk to a schema that already has tables,
as indicated by this message:
Error: ERROR: relation "BUCKETING_COLS" already exists
(state=42P07,code=0)
Configure Hive with a clean schema and this problem should go away.
Thanks,
Darren
send an email to [email protected].You've configured Hive to talk to a schema that already has tables,
as indicated by this message:
Error: ERROR: relation "BUCKETING_COLS" already exists
(state=42P07,code=0)
Configure Hive with a clean schema and this problem should go away.
Thanks,
Darren
On Wed, Apr 23, 2014 at 7:31 AM, Ivan Hsueh wrote:
Hi,
I try to build a LAB for Cloudera 5.0
There are 2 hosts in the same domain
describe as below,
Cloudera51>>Manager, Name Node
2GB RAM
50GB Space
Cloudera52>>Data Node
2GB RAM
50GB Space
I followed the "Cloudera-Manager-Installation-Guide.pdf", and used
"Installation Path A - Automated Installation by Cloudera Manager "
first, I installed the cloudera manager 5 on Cloudera51
Second, Choosed the cluster hosts
Third, Manager automatically assigned Cloudera51 is Name Node, and
Cloudera52 is Data Node.
fourth, Using Manager Web UI to install CDH5 on both Cloudera51 and
Cloudera52 (with Parcel)
I used embedded PostgreSQL Database
Everything is fine until Manager try to install Hive service.
It's always fail when creating megastore tables. (I had tried at
least 3 times)
Here is error log:
Failed to Create Hive Metastore Database Tables.
*Program:* hive/hive.sh ["create_metastore_tables"]
+ [[ create_metastore_tables == updatelocation ]]
+ [[ create_metastore_tables == create_metastore_tables ]]
+ [[ -z '' ]]
+ JDBC_JARS_CLASSPATH='/usr/share/cmf/lib/*:/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar:/usr/share/java/oracle-connector-java.jar'
+ [[ create_metastore_tables == create_metastore_tables ]]
+ '[' 5 -ge 5 ']'
+ exec /opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hive/bin/schematool -dbType postgres -initSchema -dbOpts postgres.filter.81
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
14/04/23 17:21:50 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.
Error: ERROR: relation "BUCKETING_COLS" already exists (state=42P07,code=0)
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
*** schemaTool failed ***
Can anyone help me to solved this problem?
Thanks a lot.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected].
To unsubscribe from this group and stop receiving emails from it,Hi,
I try to build a LAB for Cloudera 5.0
There are 2 hosts in the same domain
describe as below,
Cloudera51>>Manager, Name Node
2GB RAM
50GB Space
Cloudera52>>Data Node
2GB RAM
50GB Space
I followed the "Cloudera-Manager-Installation-Guide.pdf", and used
"Installation Path A - Automated Installation by Cloudera Manager "
first, I installed the cloudera manager 5 on Cloudera51
Second, Choosed the cluster hosts
Third, Manager automatically assigned Cloudera51 is Name Node, and
Cloudera52 is Data Node.
fourth, Using Manager Web UI to install CDH5 on both Cloudera51 and
Cloudera52 (with Parcel)
I used embedded PostgreSQL Database
Everything is fine until Manager try to install Hive service.
It's always fail when creating megastore tables. (I had tried at
least 3 times)
Here is error log:
Failed to Create Hive Metastore Database Tables.
*Program:* hive/hive.sh ["create_metastore_tables"]
+ [[ create_metastore_tables == updatelocation ]]
+ [[ create_metastore_tables == create_metastore_tables ]]
+ [[ -z '' ]]
+ JDBC_JARS_CLASSPATH='/usr/share/cmf/lib/*:/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar:/usr/share/java/oracle-connector-java.jar'
+ [[ create_metastore_tables == create_metastore_tables ]]
+ '[' 5 -ge 5 ']'
+ exec /opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hive/bin/schematool -dbType postgres -initSchema -dbOpts postgres.filter.81
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
14/04/23 17:21:50 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.
Error: ERROR: relation "BUCKETING_COLS" already exists (state=42P07,code=0)
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
*** schemaTool failed ***
Can anyone help me to solved this problem?
Thanks a lot.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected].
