FAQ
Hi Ivan,

It looks like create table actually succeeded the first time. What happens
if you try to start the metastore (via the CM UI)?

Also, what operating system are you using?

Thanks,
Darren

On Wed, Apr 23, 2014 at 11:50 AM, Ivan Hsueh wrote:

Dear Darren,

1.My Cloudera version is "Cloudera Express 5.0.0 (#215 built by jenkins on
20140331-1424 git"

2.Yes, I used parcels for CDH binaries.

3.I used embedded PostgreSQL Database (default option). Sorry I can't find
the version info.

4.You are right too. First time when I ran the create tables command seems
no error. The log is as below

+ exec
/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hive/bin/schematool
-dbType postgres -initSchema -dbOpts postgres.filter.81
14/04/23 16:56:10 INFO Configuration.deprecation:
mapred.input.dir.recursive is deprecated. Instead, use
mapreduce.input.fileinputformat.input.dir.recursive
14/04/23 16:56:10 INFO Configuration.deprecation: mapred.max.split.size is
deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize
14/04/23 16:56:10 INFO Configuration.deprecation: mapred.min.split.size is
deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
14/04/23 16:56:10 INFO Configuration.deprecation:
mapred.min.split.size.per.rack is deprecated. Instead, use
mapreduce.input.fileinputformat.split.minsize.per.rack
14/04/23 16:56:10 INFO Configuration.deprecation:
mapred.min.split.size.per.node is deprecated. Instead, use
mapreduce.input.fileinputformat.split.minsize.per.node
14/04/23 16:56:10 INFO Configuration.deprecation: mapred.reduce.tasks is
deprecated. Instead, use mapreduce.job.reduces
14/04/23 16:56:10 INFO Configuration.deprecation:
mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
mapreduce.reduce.speculative
14/04/23 16:56:17 WARN conf.HiveConf: DEPRECATED: Configuration property
hive.metastore.local no longer has any effect. Make sure to provide a valid
value for hive.metastore.uris if you are connecting to a remote megastore.

So I try to check next step process (26-hdfs-NAMENODE-createdir) log.
There are some error messages.

+ echo 'Created directory /user/hive.'
+
/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop-hdfs/bin/hdfs
--config /var/run/cloudera-scm-agent/process/26-hdfs-NAMENODE-createdir dfs
-chown hive:hive /user/hive
log4j:ERROR Could not find value for key log4j.appender.EventCounter
log4j:ERROR Could not instantiate appender named "EventCounter".
+ '[' 0 -eq 0 ']'
+ echo 'Changed ownership of /user/hive to hive:hive.'
+
/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop-hdfs/bin/hdfs
--config /var/run/cloudera-scm-agent/process/26-hdfs-NAMENODE-createdir dfs
-chmod 1775 /user/hive
log4j:ERROR Could not find value for key log4j.appender.EventCounter
log4j:ERROR Could not instantiate appender named "EventCounter".




Darren Lo於 2014年4月24日星期四UTC+8上午1時59分25秒寫道:
Hi Ivan,

You're right that the schema should have been clean when you first
started.

Can you confirm your Cloudera Manager version? Click on Support -> About
in the upper right. You may need to exit the wizard to see this link (exit
the wizard by clicking on the Cloudera Manager logo).

Did you use parcels for CDH binaries? (this is the default choice)

What version of postgresql was installed? ssh in to the machine and see
what the package manager says the version of postgresql is.

I suspect that the first time you ran the create tables command, the
error log had a different message. SSH in to the machine where the Hive
Metastore Server is running, then look in /var/run/cloudera-scm-agent/process
and find the directories with a name like "52-hive-metastore-create-tables".
The "52" part will be different on your system. Find the earliest attempt,
and look in the logs directory for the stderr log. Please share the end of
that log.

Thanks,
Darren

On Wed, Apr 23, 2014 at 10:48 AM, Ivan Hsueh wrote:

Thanks Darren,

It's weird, I installed CDH 5 from clean system.
There shouldn't be an existing table.

Another problem is I can't use schematool to initialise the DB.
Would you please describe more specifically how to clean schema.

Thanks again..


Darren Lo於 2014年4月24日星期四UTC+8上午12時50分04秒寫道:
Hi Ivan,

You've configured Hive to talk to a schema that already has tables, as
indicated by this message:
Error: ERROR: relation "BUCKETING_COLS" already exists
(state=42P07,code=0)

Configure Hive with a clean schema and this problem should go away.

Thanks,
Darren

On Wed, Apr 23, 2014 at 7:31 AM, Ivan Hsueh wrote:

Hi,

I try to build a LAB for Cloudera 5.0

There are 2 hosts in the same domain
describe as below,

Cloudera51»Manager, Name Node
2GB RAM
50GB Space

Cloudera52»Data Node
2GB RAM
50GB Space

I followed the "Cloudera-Manager-Installation-Guide.pdf", and used
"Installation Path A - Automated Installation by Cloudera Manager "

first, I installed the cloudera manager 5 on Cloudera51
Second, Choosed the cluster hosts
Third, Manager automatically assigned Cloudera51 is Name Node, and
Cloudera52 is Data Node.
fourth, Using Manager Web UI to install CDH5 on both Cloudera51 and
Cloudera52 (with Parcel)
I used embedded PostgreSQL Database

Everything is fine until Manager try to install Hive service.
It's always fail when creating megastore tables. (I had tried at least
3 times)

Here is error log:

Failed to Create Hive Metastore Database Tables.

*Program:* hive/hive.sh ["create_metastore_tables"]

+ [[ create_metastore_tables == updatelocation ]]
+ [[ create_metastore_tables == create_metastore_tables ]]
+ [[ -z '' ]]
+ JDBC_JARS_CLASSPATH='/usr/share/cmf/lib/*:/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar:/usr/share/java/oracle-connector-java.jar'
+ [[ create_metastore_tables == create_metastore_tables ]]
+ '[' 5 -ge 5 ']'
+ exec /opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hive/bin/schematool -dbType postgres -initSchema -dbOpts postgres.filter.81
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
14/04/23 17:21:45 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
14/04/23 17:21:50 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.
Error: ERROR: relation "BUCKETING_COLS" already exists (state=42P07,code=0)
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
*** schemaTool failed ***


Can anyone help me to solved this problem?
Thanks a lot.

To unsubscribe from this group and stop receiving emails from it, send
an email to scm-users+...@cloudera.org.
To unsubscribe from this group and stop receiving emails from it,
send an email to scm-users+...@cloudera.org.
To unsubscribe from this group and stop receiving emails from it, send
an email to scm-users+unsubscribe@cloudera.org.
To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 6 of 11 | next ›
Discussion Overview
groupscm-users @
categorieshadoop
postedApr 23, '14 at 2:31p
activeMay 16, '14 at 4:00p
posts11
users3
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase