Installing cloudera manger

On Tuesday, August 7, 2012 4:56:02 PM UTC-4, Megha Agrawal wrote:

Hi,

Can anyone please help me the installation of CLodera Manager 4.0 Free
Edition.

I am trying to install it using "cloudera-manager-installer.bin" and
getting the following error:
Failed to configure Embedded Service and Configuration Database. See
/var/log/cloudera-manager-installer/6.init-embedded-db.log for details.

When I look into log file I get following:
psql: FATAL: no pg_hba.conf entry for host "192.168.30.1", user
"cloudera-scm", database "postgres", SSL off


Now, to resolve this I have done following changes:
in postgresql.conf --------- did this : listen_addresses = '*'
in pg_hba.conf -------------- added:

host postgres cloudera-scm 192.168.30.1/24 trust
host all all 192.168.5.30/24 trust
host all all 192.168.30.30/24 trust
host postgres all 192.168.30.1/32 trust


But then also I get the above error.

Please help.

Thanks,
Megha

On Wed, Aug 8, 2012 at 9:55 AM, Megha Agrawal wrote:
To add on this is the content for db.log:
LOG: received smart shutdown request
LOG: shutting down
PANIC: could not open control file "global/pg_control": No such file or
directory
LOG: background writer process (PID 29561) was terminated by signal 6:
Aborted
LOG: terminating any other active server processes
LOG: abnormal database system shutdown
LOG: could not receive test message on socket for statistics collector:
Connection refused
LOG: disabling statistics collector for lack of working socket
WARNING: autovacuum not started because of misconfiguration
HINT: Enable the "track_counts" option.
LOG: database system was shut down at 2012-08-08 12:49:37 EDT
LOG: database system is ready to accept connections

FATAL: no pg_hba.conf entry for host "192.168.30.1", user "cloudera-scm",
database "postgres", SSL off
LOG: received smart shutdown request
LOG: shutting down
PANIC: could not open control file "global/pg_control": No such file or
directory
LOG: background writer process (PID 29838) was terminated by signal 6:
Aborted
LOG: terminating any other active server processes
LOG: abnormal database system shutdown

On Wednesday, August 8, 2012 1:04:02 PM UTC-4, Vikas Singh wrote:

Hi Megha,

Please send following files:
1. 6.init-embedded-db.log
2. pg_hba.conf (and its filesystem location)
3. postgresql.conf (and its filesystem location)

The error is occurring when CM is trying to bring up Postgresql
server. Which distribution of Linux are you using? Is there an
existing install of postgresql on your machine? If yes, uninstall that
and then try again.

- Vikas

On Wed, Aug 8, 2012 at 10:30 AM, Megha Agrawal wrote:
Hi Vikas,

Thanks for looking into issue.

I am using Ubuntu 12.04 LTS. And there isn't any postgresql already
installed on the system.

Here is the content for log file:
$sudo cat /var/log/cloudera-manager-installer/6.init-embedded-db.log
psql: FATAL: no pg_hba.conf entry for host "192.168.30.1", user
"cloudera-scm", database "postgres", SSL off
Location of pg_hba.conf: /etc/postgresql/8.4/main/pg_hba.conf
Content of pg_hba.conf:

# Database administrative login by Unix domain socket
local all postgres ident

# TYPE DATABASE USER CIDR-ADDRESS METHOD

# "local" is for Unix domain socket connections only
local all all ident
# IPv4 local connections:
host all all 127.0.0.1/32 md5

host postgres cloudera-scm 192.168.30.1/24 trust

host all all 192.168.5.30/24 trust
host all all 192.168.30.30/24 trust
host all all 192.168.30.1/24 trust
# IPv6 local connections:
host all all ::1/128 md5


Location of postgresql.conf: /etc/postgresql/8.4/main/postgresql.conf
Content of postgresql.conf:
# -----------------------------
# PostgreSQL configuration file
# -----------------------------
#
# This file consists of lines of the form:
#
# name = value
#
# (The "=" is optional.) Whitespace may be used. Comments are introduced
with
# "#" anywhere on a line. The complete list of parameter names and allowed
# values can be found in the PostgreSQL documentation.
#
# The commented-out settings shown in this file represent the default
values.
# Re-commenting a setting is NOT sufficient to revert it to the default
value;
# you need to reload the server.
#
# This file is read on server startup and when the server receives a SIGHUP
# signal. If you edit the file on a running system, you have to SIGHUP the
# server for the changes to take effect, or use "pg_ctl reload". Some
# parameters, which are marked below, require a server shutdown and restart
to
# take effect.
#
# Any parameter can also be given as a command-line option to the server,
e.g.,
# "postgres -c log_connections=on". Some parameters can be changed at run
time
# with the "SET" SQL command.
#
# Memory units: kB = kilobytes Time units: ms = milliseconds
# MB = megabytes s = seconds
# GB = gigabytes min = minutes
# h = hours
# d = days


#------------------------------------------------------------------------------
# FILE LOCATIONS
#------------------------------------------------------------------------------

# The default values of these variables are driven from the -D command-line
# option or PGDATA environment variable, represented here as ConfigDir.

data_directory = '/var/lib/postgresql/8.4/main' # use data in another
directory
# (change requires restart)
hba_file = '/etc/postgresql/8.4/main/pg_hba.conf' # host-based
authentication file
# (change requires restart)
ident_file = '/etc/postgresql/8.4/main/pg_ident.conf' # ident
configuration file
# (change requires restart)

# If external_pid_file is not explicitly set, no extra PID file is written.
external_pid_file = '/var/run/postgresql/8.4-main.pid' # write an
extra PID file
# (change requires restart)


#------------------------------------------------------------------------------
# CONNECTIONS AND AUTHENTICATION
#------------------------------------------------------------------------------

# - Connection Settings -

listen_addresses = '*' # what IP address(es) to listen on;
# comma-separated list of addresses;
# defaults to 'localhost', '*' = all
# (change requires restart)
port = 5432 # (change requires restart)
max_connections = 100 # (change requires restart)
# Note: Increasing max_connections costs ~400 bytes of shared memory per
# connection slot, plus lock space (see max_locks_per_transaction).
#superuser_reserved_connections = 3 # (change requires restart)
unix_socket_directory = '/var/run/postgresql' # (change requires
restart)
#unix_socket_group = '' # (change requires restart)
#unix_socket_permissions = 0777 # begin with 0 to use octal notation
# (change requires restart)
#bonjour_name = '' # defaults to the computer name
# (change requires restart)

# - Security and Authentication -

#authentication_timeout = 1min # 1s-600s
ssl = true # (change requires restart)
#ssl_ciphers = 'ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH' # allowed SSL ciphers
# (change requires restart)
#ssl_renegotiation_limit = 512MB # amount of data between renegotiations
#password_encryption = on
#db_user_namespace = off

# Kerberos and GSSAPI
#krb_server_keyfile = ''
#krb_srvname = 'postgres' # (Kerberos only)
#krb_caseins_users = off

# - TCP Keepalives -
# see "man 7 tcp" for details

#tcp_keepalives_idle = 0 # TCP_KEEPIDLE, in seconds;
# 0 selects the system default
#tcp_keepalives_interval = 0 # TCP_KEEPINTVL, in seconds;
# 0 selects the system default
#tcp_keepalives_count = 0 # TCP_KEEPCNT;
# 0 selects the system default


#------------------------------------------------------------------------------
# RESOURCE USAGE (except WAL)
#------------------------------------------------------------------------------

# - Memory -

shared_buffers = 24MB # min 128kB
# (change requires restart)
#temp_buffers = 8MB # min 800kB
#max_prepared_transactions = 0 # zero disables the feature
# (change requires restart)
# Note: Increasing max_prepared_transactions costs ~600 bytes of shared
memory
# per transaction slot, plus lock space (see max_locks_per_transaction).
# It is not advisable to set max_prepared_transactions nonzero unless you
# actively intend to use prepared transactions.
#work_mem = 1MB # min 64kB
#maintenance_work_mem = 16MB # min 1MB
#max_stack_depth = 2MB # min 100kB

# - Kernel Resource Usage -

#max_files_per_process = 1000 # min 25
# (change requires restart)
#shared_preload_libraries = '' # (change requires restart)

# - Cost-Based Vacuum Delay -

#vacuum_cost_delay = 0ms # 0-100 milliseconds
#vacuum_cost_page_hit = 1 # 0-10000 credits
#vacuum_cost_page_miss = 10 # 0-10000 credits
#vacuum_cost_page_dirty = 20 # 0-10000 credits
#vacuum_cost_limit = 200 # 1-10000 credits

# - Background Writer -

#bgwriter_delay = 200ms # 10-10000ms between rounds
#bgwriter_lru_maxpages = 100 # 0-1000 max buffers written/round
#bgwriter_lru_multiplier = 2.0 # 0-10.0 multipler on buffers
scanned/round

# - Asynchronous Behavior -

#effective_io_concurrency = 1 # 1-1000. 0 disables prefetching


#------------------------------------------------------------------------------
# WRITE AHEAD LOG
#------------------------------------------------------------------------------

# - Settings -

#fsync = on # turns forced synchronization on or off
#synchronous_commit = on # immediate fsync at commit
#wal_sync_method = fsync # the default is the first option
# supported by the operating system:
# open_datasync
# fdatasync (default on Linux)
# fsync
# fsync_writethrough
# open_sync
#full_page_writes = on # recover from partial page writes
#wal_buffers = 64kB # min 32kB
# (change requires restart)
#wal_writer_delay = 200ms # 1-10000 milliseconds

#commit_delay = 0 # range 0-100000, in microseconds
#commit_siblings = 5 # range 1-1000

# - Checkpoints -

#checkpoint_segments = 3 # in logfile segments, min 1, 16MB each
#checkpoint_timeout = 5min # range 30s-1h
#checkpoint_completion_target = 0.5 # checkpoint target duration, 0.0 -
1.0
#checkpoint_warning = 30s # 0 disables

# - Archiving -

#archive_mode = off # allows archiving to be done
# (change requires restart)
#archive_command = '' # command to use to archive a logfile segment
#archive_timeout = 0 # force a logfile segment switch after this
# number of seconds; 0 disables


#------------------------------------------------------------------------------
# QUERY TUNING
#------------------------------------------------------------------------------

# - Planner Method Configuration -

#enable_bitmapscan = on
#enable_hashagg = on
#enable_hashjoin = on
#enable_indexscan = on
#enable_mergejoin = on
#enable_nestloop = on
#enable_seqscan = on
#enable_sort = on
#enable_tidscan = on

# - Planner Cost Constants -

#seq_page_cost = 1.0 # measured on an arbitrary scale
#random_page_cost = 4.0 # same scale as above
#cpu_tuple_cost = 0.01 # same scale as above
#cpu_index_tuple_cost = 0.005 # same scale as above
#cpu_operator_cost = 0.0025 # same scale as above
#effective_cache_size = 128MB

# - Genetic Query Optimizer -

#geqo = on
#geqo_threshold = 12
#geqo_effort = 5 # range 1-10
#geqo_pool_size = 0 # selects default based on effort
#geqo_generations = 0 # selects default based on effort
#geqo_selection_bias = 2.0 # range 1.5-2.0

# - Other Planner Options -

#default_statistics_target = 100 # range 1-10000
#constraint_exclusion = partition # on, off, or partition
#cursor_tuple_fraction = 0.1 # range 0.0-1.0
#from_collapse_limit = 8
#join_collapse_limit = 8 # 1 disables collapsing of explicit
# JOIN clauses


#------------------------------------------------------------------------------
# ERROR REPORTING AND LOGGING
#------------------------------------------------------------------------------

# - Where to Log -

#log_destination = 'stderr' # Valid values are combinations of
# stderr, csvlog, syslog and eventlog,
# depending on platform. csvlog
# requires logging_collector to be on.

# This is used when logging to stderr:
#logging_collector = off # Enable capturing of stderr and csvlog
# into log files. Required to be on for
# csvlogs.
# (change requires restart)

# These are only used if logging_collector is on:
#log_directory = 'pg_log' # directory where log files are written,
# can be absolute or relative to PGDATA
#log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log' # log file name pattern,
# can include strftime() escapes
#log_truncate_on_rotation = off # If on, an existing log file of the
# same name as the new log file will be
# truncated rather than appended to.
# But such truncation only occurs on
# time-driven rotation, not on restarts
# or size-driven rotation. Default is
# off, meaning append to existing files
# in all cases.
#log_rotation_age = 1d # Automatic rotation of logfiles will
# happen after that time. 0 disables.
#log_rotation_size = 10MB # Automatic rotation of logfiles will
# happen after that much log output.
# 0 disables.

# These are relevant when logging to syslog:
#syslog_facility = 'LOCAL0'
#syslog_ident = 'postgres'

#silent_mode = off # Run server silently.
# DO NOT USE without syslog or
# logging_collector
# (change requires restart)


# - When to Log -

#client_min_messages = notice # values in order of decreasing detail:
# debug5
# debug4
# debug3
# debug2
# debug1
# log
# notice
# warning
# error

#log_min_messages = warning # values in order of decreasing detail:
# debug5
# debug4
# debug3
# debug2
# debug1
# info
# notice
# warning
# error
# log
# fatal
# panic

#log_error_verbosity = default # terse, default, or verbose messages

#log_min_error_statement = error # values in order of decreasing detail:
# debug5
# debug4
# debug3
# debug2
# debug1
# info
# notice
# warning
# error
# log
# fatal
# panic (effectively off)

#log_min_duration_statement = -1 # -1 is disabled, 0 logs all statements
# and their durations, > 0 logs only
# statements running at least this number
# of milliseconds


# - What to Log -

#debug_print_parse = off
#debug_print_rewritten = off
#debug_print_plan = off
#debug_pretty_print = on
#log_checkpoints = off
#log_connections = off
#log_disconnections = off
#log_duration = off
#log_hostname = off
log_line_prefix = '%t ' # special values:
# %u = user name
# %d = database name
# %r = remote host and port
# %h = remote host
# %p = process ID
# %t = timestamp without milliseconds
# %m = timestamp with milliseconds
# %i = command tag
# %c = session ID
# %l = session line number
# %s = session start timestamp
# %v = virtual transaction ID
# %x = transaction ID (0 if none)
# %q = stop here in non-session
# processes
# %% = '%'
# e.g. '<%u%%%d> '
#log_lock_waits = off # log lock waits >= deadlock_timeout
#log_statement = 'none' # none, ddl, mod, all
#log_temp_files = -1 # log temporary files equal or larger
# than the specified size in kilobytes;
# -1 disables, 0 logs all temp files
#log_timezone = unknown # actually, defaults to TZ environment
# setting


#------------------------------------------------------------------------------
# RUNTIME STATISTICS
#------------------------------------------------------------------------------

# - Query/Index Statistics Collector -

#track_activities = on
#track_counts = on
#track_functions = none # none, pl, all
#track_activity_query_size = 1024
#update_process_title = on
#stats_temp_directory = 'pg_stat_tmp'


# - Statistics Monitoring -

#log_parser_stats = off
#log_planner_stats = off
#log_executor_stats = off
#log_statement_stats = off


#------------------------------------------------------------------------------
# AUTOVACUUM PARAMETERS
#------------------------------------------------------------------------------

#autovacuum = on # Enable autovacuum subprocess? 'on'
# requires track_counts to also be on.
#log_autovacuum_min_duration = -1 # -1 disables, 0 logs all actions and
# their durations, > 0 logs only
# actions running at least this number
# of milliseconds.
#autovacuum_max_workers = 3 # max number of autovacuum subprocesses
#autovacuum_naptime = 1min # time between autovacuum runs
#autovacuum_vacuum_threshold = 50 # min number of row updates before
# vacuum
#autovacuum_analyze_threshold = 50 # min number of row updates before
# analyze
#autovacuum_vacuum_scale_factor = 0.2 # fraction of table size before
vacuum
#autovacuum_analyze_scale_factor = 0.1 # fraction of table size before
analyze
#autovacuum_freeze_max_age = 200000000 # maximum XID age before forced
vacuum
# (change requires restart)
#autovacuum_vacuum_cost_delay = 20ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
#autovacuum_vacuum_cost_limit = -1 # default vacuum cost limit for
# autovacuum, -1 means use
# vacuum_cost_limit


#------------------------------------------------------------------------------
# CLIENT CONNECTION DEFAULTS
#------------------------------------------------------------------------------

# - Statement Behavior -

#search_path = '"$user",public' # schema names
#default_tablespace = '' # a tablespace name, '' uses the default
#temp_tablespaces = '' # a list of tablespace names, '' uses
# only default tablespace
#check_function_bodies = on
#default_transaction_isolation = 'read committed'
#default_transaction_read_only = off
#session_replication_role = 'origin'
#statement_timeout = 0 # in milliseconds, 0 is disabled
#vacuum_freeze_min_age = 50000000
#vacuum_freeze_table_age = 150000000
#xmlbinary = 'base64'
#xmloption = 'content'

# - Locale and Formatting -

datestyle = 'iso, mdy'
#intervalstyle = 'postgres'
#timezone = unknown # actually, defaults to TZ environment
# setting
#timezone_abbreviations = 'Default' # Select the set of available time
zone
# abbreviations. Currently, there are
# Default
# Australia
# India
# You can create your own file in
# share/timezonesets/.
#extra_float_digits = 0 # min -15, max 2
#client_encoding = sql_ascii # actually, defaults to database
# encoding

# These settings are initialized by initdb, but they can be changed.
lc_messages = 'en_US.UTF-8' # locale for system error message
# strings
lc_monetary = 'en_US.UTF-8' # locale for monetary formatting
lc_numeric = 'en_US.UTF-8' # locale for number formatting
lc_time = 'en_US.UTF-8' # locale for time formatting

# default configuration for text search
default_text_search_config = 'pg_catalog.english'

# - Other Defaults -

#dynamic_library_path = '$libdir'
#local_preload_libraries = ''


#------------------------------------------------------------------------------
# LOCK MANAGEMENT
#------------------------------------------------------------------------------

#deadlock_timeout = 1s
#max_locks_per_transaction = 64 # min 10
# (change requires restart)
# Note: Each lock table slot uses ~270 bytes of shared memory, and there
are
# max_locks_per_transaction * (max_connections + max_prepared_transactions)
# lock table slots.


#------------------------------------------------------------------------------
# VERSION/PLATFORM COMPATIBILITY
#------------------------------------------------------------------------------

# - Previous PostgreSQL Versions -

#add_missing_from = off
#array_nulls = on
#backslash_quote = safe_encoding # on, off, or safe_encoding
#default_with_oids = off
#escape_string_warning = on
#regex_flavor = advanced # advanced, extended, or basic
#sql_inheritance = on
#standard_conforming_strings = off
#synchronize_seqscans = on

# - Other Platforms and Clients -

#transform_null_equals = off


#------------------------------------------------------------------------------
# CUSTOMIZED OPTIONS
#------------------------------------------------------------------------------

#custom_variable_classes = '' # list of custom variable class names

Thanks,
Megha

On Wed, Aug 8, 2012 at 8:12 PM, Vikas Singh wrote:

Hi Megha,

From the path you provided it seems like you have postgresql 8.4
installed. Please uninstall it. The version of Ubuntu you are using,
you should be getting postgresql 9.1 (see:
http://packages.ubuntu.com/precise/database/).

I think what is going on here is that CM installer is trying to
install 9.1 and having two version of Postgresql on the system is
confusing it when it is executing psql commands. Please make sure that
there is no other postgresql version in your system. After that you
can try to install postgresql yourself using apt-get or let CM
installer install it.

Vikas
or
6:
collector:
or
6:
Free
details.

On Wed, Aug 8, 2012 at 5:17 PM, Megha Agrawal wrote:
HI Vikas,

I have uninstalled all the versions of postgresql. CM tries to install it,
and gives the error.

There was CM already installed on the system, and I uninstalled it. Now when
I am trying to install it again, it is giving this psql error.

CM itself tries to install postgresql-8.4 version.

Thanks,
Megha




--
Regards
Megha Agrawal

On 08/09/2012 03:12 PM, Vikas Singh wrote:
Hi Megha,

It seems the cloudera database package depends on postgresql 8.4 so
installing 8.4 version seems to be right behavior. When installer
installs postgresql, it creates database in its own directory and
starts server on a separate port. The directory where it put database
files is : /var/lib/cloudera-scm-server-db/data

So if you can look into this directory for both pg_hba.conf and
postgresql.conf you may be able to find about the issue. Otherwise,
please attach these files to the reply.

Vikas

On Thu, Aug 9, 2012 at 12:40 PM, Megha wrote:
Hi Vikas,

I just saw that pg_hba.conf and postgresql.conf are there in
/var/lib/cloudera-scm-server-db/data. But I can access this directory only
after using sudo.

I have attached both of these files from data directory.

But these exist in /etc/postgresql/8.4/main/ too.

Thanks,
Megha

On Thu, Aug 9, 2012 at 1:28 PM, Megha wrote:
Hi Vikas,

This is the content of my /etc/hosts file:

127.0.0.1 hadoop-name.wwbp.org localhost
192.168.30.30 hadoop-name.wwbp.org hadoop-name name

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

192.168.30.31 hadoop-name-backup.wwbp.org name-backup
hadoop-name-backup
192.168.30.10 hadoop-data0.wwbp.org data0 hadoop-data0
192.168.30.11 hadoop-data1.wwbp.org data1 hadoop-data1
192.168.30.12 hadoop-data2.wwbp.org data2 hadoop-data2
192.168.30.13 hadoop-data3.wwbp.org data3 hadoop-data3

192.168.5.10 wwbp.org wwbp

Basically, the server I am installing cloudera is on two networks and has
following IPs 192.168.5.30 and 192.168.30.30.

Thanks,
Megha

On Thursday, August 9, 2012 4:46:25 PM UTC-4, Vikas Singh wrote:

Hi Megha,

Please try this:

1. Take a backup of /etc/hosts
2. Get rid of hadoop-name.wwbp.org from localhost line (so the first
line should be "127.0.0.1 localhost")
3. Retry installing CM

CM executes postgresql command by using host as localhost (psql
--host=localhost <cmd>) and it seems like it is somehow resolving
localhost to the 192.168.30.1 IP address.

Vikas

On Thu, Aug 9, 2012 at 1:28 PM, Megha <[email protected] <javascript:>>
wrote:
install
Now
using,
that
already
/var/log/cloudera-manager-installer/6.init-embedded-db.log
ident
and
default
a
and
changed
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
in
write
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
memory
max_locks_per_transaction).
SSL
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
unless
buffers
prefetching
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
off
writes
duration,
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
explicit
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
of
name
file
decreasing
decreasing
deadlock_timeout
larger
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
delay
for
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
environment
available
database
changed.
formatting
formatting
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
and
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
class
wrote:
4.0
"cloudera-manager-installer.bin"
Database.
user
'*'

On Thu, Aug 9, 2012 at 2:46 PM, Megha wrote:
Hi Vikas,

This is what I get when I try to reverse lookup and login into psql:


[email protected]:~$ psql --host=localhost --username=root
--dbname=postgres
psql: FATAL: no pg_hba.conf entry for host "192.168.5.30", user "root",
database "postgres", SSL on
FATAL: no pg_hba.conf entry for host "192.168.5.30", user "root", database
"postgres", SSL off

[email protected]:~$ nslookup 192.168.5.30
Server: 128.91.254.4
Address: 128.91.254.4#53

** server can't find 30.5.168.192.in-addr.arpa.: NXDOMAIN

[email protected]:~$ nslookup 192.168.30.1
Server: 128.91.254.4
Address: 128.91.254.4#53

** server can't find 1.30.168.192.in-addr.arpa.: NXDOMAIN

[email protected]:~$ nslookup 192.168.30.30
Server: 128.91.254.4
Address: 128.91.254.4#53

** server can't find 30.30.168.192.in-addr.arpa.: NXDOMAIN

[email protected]:~$ hostname
hadoop-name


Actually, this server is on two networks: 192.168.5.x and 192.168.30.x. The
5.x has internet connected to it and 30.x doesn't. So this node also works
as a gateway for nodes on 30.x network.
My data nodes are on 30.x network whereas name-server and
secondary-name-server are on 5.x and 30.x both. Now, data nodes uses
192.168.30.1 as default gateway IP and 192.168.30.30 and 192.168.5.30 are
its own IPs.

Thanks,
Megha




On 08/09/2012 05:35 PM, Vikas Singh wrote:

Hi Megha,

No, I don't think you need to add that to hosts file. You need to find
out from where you are getting these IP addresses. It used to be
'192.168.30.1' but now it is '192.168.5.30'. What hostname do you get
when you do a reverse lookup on '192.168.5.30' or '192.168.30.1'
(nslookup <ip_address>)?

I would suggest starting postgresql service on your system and see if
you can login to the database using following command.

psql --host=localhost --username=root --dbname=postgres

I think you need to sort out networking before trying to install CM.

Vikas

On Thu, Aug 9, 2012 at 2:16 PM, Megha Agrawal wrote:

Hi Vikas,

I changes the /etc.hosts file and it still gives the following error:

[email protected]:~$ cat
/var/log/cloudera-manager-installer/6.init-embedded-db.log
psql: FATAL: no pg_hba.conf entry for host "192.168.5.30", user
"cloudera-scm", database "postgres", SSL off

Do I need to include 192.168.5.30 also in the host file?

Thanks,
Megha


On Thursday, August 9, 2012 4:46:25 PM UTC-4, Vikas Singh wrote:

Hi Megha,

Please try this:

1. Take a backup of /etc/hosts
2. Get rid of hadoop-name.wwbp.org from localhost line (so the first
line should be "127.0.0.1 localhost")
3. Retry installing CM

CM executes postgresql command by using host as localhost (psql
--host=localhost <cmd>) and it seems like it is somehow resolving
localhost to the 192.168.30.1 IP address.

Vikas

On Thu, Aug 9, 2012 at 1:28 PM, Megha wrote:

Hi Vikas,

This is the content of my /etc/hosts file:

127.0.0.1 hadoop-name.wwbp.org localhost
192.168.30.30 hadoop-name.wwbp.org hadoop-name name

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

192.168.30.31 hadoop-name-backup.wwbp.org name-backup
hadoop-name-backup
192.168.30.10 hadoop-data0.wwbp.org data0 hadoop-data0
192.168.30.11 hadoop-data1.wwbp.org data1 hadoop-data1
192.168.30.12 hadoop-data2.wwbp.org data2 hadoop-data2
192.168.30.13 hadoop-data3.wwbp.org data3 hadoop-data3

192.168.5.10 wwbp.org wwbp

Basically, the server I am installing cloudera is on two networks and
has
following IPs 192.168.5.30 and 192.168.30.30.

Thanks,
Megha



On 08/09/2012 04:24 PM, Vikas Singh wrote:

Hi Megha,

The CM database will pick up properties from
/var/lib/cloudera-scm-server-db/data directory. The default
pg_hba.conf we generate contains permission for all users to connect
to this database from localhost (ip address 127.0.0.1).

What's content of your /etc/hosts file. What's the ip address that
localhost is mapped to?

Vikas

On Thu, Aug 9, 2012 at 12:40 PM, Megha wrote:

Hi Vikas,

I just saw that pg_hba.conf and postgresql.conf are there in
/var/lib/cloudera-scm-server-db/data. But I can access this directory
only
after using sudo.

I have attached both of these files from data directory.

But these exist in /etc/postgresql/8.4/main/ too.

Thanks,
Megha





On 08/09/2012 03:12 PM, Vikas Singh wrote:

Hi Megha,

It seems the cloudera database package depends on postgresql 8.4 so
installing 8.4 version seems to be right behavior. When installer
installs postgresql, it creates database in its own directory and
starts server on a separate port. The directory where it put database
files is : /var/lib/cloudera-scm-server-db/data

So if you can look into this directory for both pg_hba.conf and
postgresql.conf you may be able to find about the issue. Otherwise,
please attach these files to the reply.

Vikas

On Wed, Aug 8, 2012 at 5:17 PM, Megha Agrawal wrote:

HI Vikas,

I have uninstalled all the versions of postgresql. CM tries to
install
it,
and gives the error.

There was CM already installed on the system, and I uninstalled it.
Now
when
I am trying to install it again, it is giving this psql error.

CM itself tries to install postgresql-8.4 version.

Thanks,
Megha


On Wed, Aug 8, 2012 at 8:12 PM, Vikas Singh wrote:

Hi Megha,

From the path you provided it seems like you have postgresql 8.4
installed. Please uninstall it. The version of Ubuntu you are
using,
you should be getting postgresql 9.1 (see:
http://packages.ubuntu.com/precise/database/).

I think what is going on here is that CM installer is trying to
install 9.1 and having two version of Postgresql on the system is
confusing it when it is executing psql commands. Please make sure
that
there is no other postgresql version in your system. After that you
can try to install postgresql yourself using apt-get or let CM
installer install it.

Vikas

On Wed, Aug 8, 2012 at 10:30 AM, Megha Agrawal wrote:

Hi Vikas,

Thanks for looking into issue.

I am using Ubuntu 12.04 LTS. And there isn't any postgresql
already
installed on the system.

Here is the content for log file:
$sudo cat
/var/log/cloudera-manager-installer/6.init-embedded-db.log
psql: FATAL: no pg_hba.conf entry for host "192.168.30.1", user
"cloudera-scm", database "postgres", SSL off

Location of pg_hba.conf: /etc/postgresql/8.4/main/pg_hba.conf
Content of pg_hba.conf:

# Database administrative login by Unix domain socket
local all postgres
ident

# TYPE DATABASE USER CIDR-ADDRESS METHOD

# "local" is for Unix domain socket connections only
local all all ident
# IPv4 local connections:
host all all 127.0.0.1/32 md5

host postgres cloudera-scm 192.168.30.1/24 trust

host all all 192.168.5.30/24 trust
host all all 192.168.30.30/24 trust
host all all 192.168.30.1/24 trust
# IPv6 local connections:
host all all ::1/128 md5


Location of postgresql.conf:
/etc/postgresql/8.4/main/postgresql.conf
Content of postgresql.conf:
# -----------------------------
# PostgreSQL configuration file
# -----------------------------
#
# This file consists of lines of the form:
#
# name = value
#
# (The "=" is optional.) Whitespace may be used. Comments are
introduced
with
# "#" anywhere on a line. The complete list of parameter names
and
allowed
# values can be found in the PostgreSQL documentation.
#
# The commented-out settings shown in this file represent the
default
values.
# Re-commenting a setting is NOT sufficient to revert it to the
default
value;
# you need to reload the server.
#
# This file is read on server startup and when the server receives
a
SIGHUP
# signal. If you edit the file on a running system, you have to
SIGHUP
the
# server for the changes to take effect, or use "pg_ctl reload".
Some
# parameters, which are marked below, require a server shutdown
and
restart
to
# take effect.
#
# Any parameter can also be given as a command-line option to the
server,
e.g.,
# "postgres -c log_connections=on". Some parameters can be
changed
at
run
time
# with the "SET" SQL command.
#
# Memory units: kB = kilobytes Time units: ms =
milliseconds
# MB = megabytes s = seconds
# GB = gigabytes min = minutes
# h = hours
# d = days






#------------------------------------------------------------------------------
# FILE LOCATIONS




#------------------------------------------------------------------------------

# The default values of these variables are driven from the -D
command-line
# option or PGDATA environment variable, represented here as
ConfigDir.

data_directory = '/var/lib/postgresql/8.4/main' # use data
in
another
directory
# (change requires restart)
hba_file = '/etc/postgresql/8.4/main/pg_hba.conf' # host-based
authentication file
# (change requires restart)
ident_file = '/etc/postgresql/8.4/main/pg_ident.conf' # ident
configuration file
# (change requires restart)

# If external_pid_file is not explicitly set, no extra PID file is
written.
external_pid_file = '/var/run/postgresql/8.4-main.pid' #
write
an
extra PID file
# (change requires restart)






#------------------------------------------------------------------------------
# CONNECTIONS AND AUTHENTICATION




#------------------------------------------------------------------------------

# - Connection Settings -

listen_addresses = '*' # what IP address(es) to listen on;
# comma-separated list of addresses;
# defaults to 'localhost', '*' = all
# (change requires restart)
port = 5432 # (change requires restart)
max_connections = 100 # (change requires restart)
# Note: Increasing max_connections costs ~400 bytes of shared
memory
per
# connection slot, plus lock space (see
max_locks_per_transaction).
#superuser_reserved_connections = 3 # (change requires restart)
unix_socket_directory = '/var/run/postgresql' # (change
requires
restart)
#unix_socket_group = '' # (change requires restart)
#unix_socket_permissions = 0777 # begin with 0 to use octal
notation
# (change requires restart)
#bonjour_name = '' # defaults to the computer name
# (change requires restart)

# - Security and Authentication -

#authentication_timeout = 1min # 1s-600s
ssl = true # (change requires restart)
#ssl_ciphers = 'ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH' # allowed
SSL
ciphers
# (change requires restart)
#ssl_renegotiation_limit = 512MB # amount of data between
renegotiations
#password_encryption = on
#db_user_namespace = off

# Kerberos and GSSAPI
#krb_server_keyfile = ''
#krb_srvname = 'postgres' # (Kerberos only)
#krb_caseins_users = off

# - TCP Keepalives -
# see "man 7 tcp" for details

#tcp_keepalives_idle = 0 # TCP_KEEPIDLE, in seconds;
# 0 selects the system default
#tcp_keepalives_interval = 0 # TCP_KEEPINTVL, in seconds;
# 0 selects the system default
#tcp_keepalives_count = 0 # TCP_KEEPCNT;
# 0 selects the system default






#------------------------------------------------------------------------------
# RESOURCE USAGE (except WAL)




#------------------------------------------------------------------------------

# - Memory -

shared_buffers = 24MB # min 128kB
# (change requires restart)
#temp_buffers = 8MB # min 800kB
#max_prepared_transactions = 0 # zero disables the feature
# (change requires restart)
# Note: Increasing max_prepared_transactions costs ~600 bytes of
shared
memory
# per transaction slot, plus lock space (see
max_locks_per_transaction).
# It is not advisable to set max_prepared_transactions nonzero
unless
you
# actively intend to use prepared transactions.
#work_mem = 1MB # min 64kB
#maintenance_work_mem = 16MB # min 1MB
#max_stack_depth = 2MB # min 100kB

# - Kernel Resource Usage -

#max_files_per_process = 1000 # min 25
# (change requires restart)
#shared_preload_libraries = '' # (change requires restart)

# - Cost-Based Vacuum Delay -

#vacuum_cost_delay = 0ms # 0-100 milliseconds
#vacuum_cost_page_hit = 1 # 0-10000 credits
#vacuum_cost_page_miss = 10 # 0-10000 credits
#vacuum_cost_page_dirty = 20 # 0-10000 credits
#vacuum_cost_limit = 200 # 1-10000 credits

# - Background Writer -

#bgwriter_delay = 200ms # 10-10000ms between rounds
#bgwriter_lru_maxpages = 100 # 0-1000 max buffers
written/round
#bgwriter_lru_multiplier = 2.0 # 0-10.0 multipler on
buffers
scanned/round

# - Asynchronous Behavior -

#effective_io_concurrency = 1 # 1-1000. 0 disables
prefetching






#------------------------------------------------------------------------------
# WRITE AHEAD LOG




#------------------------------------------------------------------------------

# - Settings -

#fsync = on # turns forced synchronization on or
off
#synchronous_commit = on # immediate fsync at commit
#wal_sync_method = fsync # the default is the first option
# supported by the operating system:
# open_datasync
# fdatasync (default on Linux)
# fsync
# fsync_writethrough
# open_sync
#full_page_writes = on # recover from partial page
writes
#wal_buffers = 64kB # min 32kB
# (change requires restart)
#wal_writer_delay = 200ms # 1-10000 milliseconds

#commit_delay = 0 # range 0-100000, in microseconds
#commit_siblings = 5 # range 1-1000

# - Checkpoints -

#checkpoint_segments = 3 # in logfile segments, min 1, 16MB
each
#checkpoint_timeout = 5min # range 30s-1h
#checkpoint_completion_target = 0.5 # checkpoint target
duration,
0.0
-
1.0
#checkpoint_warning = 30s # 0 disables

# - Archiving -

#archive_mode = off # allows archiving to be done
# (change requires restart)
#archive_command = '' # command to use to archive a logfile
segment
#archive_timeout = 0 # force a logfile segment switch after
this
# number of seconds; 0 disables






#------------------------------------------------------------------------------
# QUERY TUNING




#------------------------------------------------------------------------------

# - Planner Method Configuration -

#enable_bitmapscan = on
#enable_hashagg = on
#enable_hashjoin = on
#enable_indexscan = on
#enable_mergejoin = on
#enable_nestloop = on
#enable_seqscan = on
#enable_sort = on
#enable_tidscan = on

# - Planner Cost Constants -

#seq_page_cost = 1.0 # measured on an arbitrary scale
#random_page_cost = 4.0 # same scale as above
#cpu_tuple_cost = 0.01 # same scale as above
#cpu_index_tuple_cost = 0.005 # same scale as above
#cpu_operator_cost = 0.0025 # same scale as above
#effective_cache_size = 128MB

# - Genetic Query Optimizer -

#geqo = on
#geqo_threshold = 12
#geqo_effort = 5 # range 1-10
#geqo_pool_size = 0 # selects default based on effort
#geqo_generations = 0 # selects default based on effort
#geqo_selection_bias = 2.0 # range 1.5-2.0

# - Other Planner Options -

#default_statistics_target = 100 # range 1-10000
#constraint_exclusion = partition # on, off, or partition
#cursor_tuple_fraction = 0.1 # range 0.0-1.0
#from_collapse_limit = 8
#join_collapse_limit = 8 # 1 disables collapsing of
explicit
# JOIN clauses






#------------------------------------------------------------------------------
# ERROR REPORTING AND LOGGING




#------------------------------------------------------------------------------

# - Where to Log -

#log_destination = 'stderr' # Valid values are combinations
of
# stderr, csvlog, syslog and eventlog,
# depending on platform. csvlog
# requires logging_collector to be on.

# This is used when logging to stderr:
#logging_collector = off # Enable capturing of stderr and
csvlog
# into log files. Required to be on for
# csvlogs.
# (change requires restart)

# These are only used if logging_collector is on:
#log_directory = 'pg_log' # directory where log files are
written,
# can be absolute or relative to PGDATA
#log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log' # log file
name
pattern,
# can include strftime() escapes
#log_truncate_on_rotation = off # If on, an existing log
file
of
the
# same name as the new log file will be
# truncated rather than appended to.
# But such truncation only occurs on
# time-driven rotation, not on restarts
# or size-driven rotation. Default is
# off, meaning append to existing files
# in all cases.
#log_rotation_age = 1d # Automatic rotation of logfiles
will
# happen after that time. 0 disables.
#log_rotation_size = 10MB # Automatic rotation of logfiles
will
# happen after that much log output.
# 0 disables.

# These are relevant when logging to syslog:
#syslog_facility = 'LOCAL0'
#syslog_ident = 'postgres'

#silent_mode = off # Run server silently.
# DO NOT USE without syslog or
# logging_collector
# (change requires restart)


# - When to Log -

#client_min_messages = notice # values in order of
decreasing
detail:
# debug5
# debug4
# debug3
# debug2
# debug1
# log
# notice
# warning
# error

#log_min_messages = warning # values in order of decreasing
detail:
# debug5
# debug4
# debug3
# debug2
# debug1
# info
# notice
# warning
# error
# log
# fatal
# panic

#log_error_verbosity = default # terse, default, or verbose
messages

#log_min_error_statement = error # values in order of
decreasing
detail:
# debug5
# debug4
# debug3
# debug2
# debug1
# info
# notice
# warning
# error
# log
# fatal
# panic (effectively off)

#log_min_duration_statement = -1 # -1 is disabled, 0 logs all
statements
# and their durations, > 0 logs only
# statements running at least this number
# of milliseconds


# - What to Log -

#debug_print_parse = off
#debug_print_rewritten = off
#debug_print_plan = off
#debug_pretty_print = on
#log_checkpoints = off
#log_connections = off
#log_disconnections = off
#log_duration = off
#log_hostname = off
log_line_prefix = '%t ' # special values:
# %u = user name
# %d = database name
# %r = remote host and port
# %h = remote host
# %p = process ID
# %t = timestamp without milliseconds
# %m = timestamp with milliseconds
# %i = command tag
# %c = session ID
# %l = session line number
# %s = session start timestamp
# %v = virtual transaction ID
# %x = transaction ID (0 if none)
# %q = stop here in non-session
# processes
# %% = '%'
# e.g. '<%u%%%d> '
#log_lock_waits = off # log lock waits >=
deadlock_timeout
#log_statement = 'none' # none, ddl, mod, all
#log_temp_files = -1 # log temporary files equal or
larger
# than the specified size in kilobytes;
# -1 disables, 0 logs all temp files
#log_timezone = unknown # actually, defaults to TZ
environment
# setting






#------------------------------------------------------------------------------
# RUNTIME STATISTICS




#------------------------------------------------------------------------------

# - Query/Index Statistics Collector -

#track_activities = on
#track_counts = on
#track_functions = none # none, pl, all
#track_activity_query_size = 1024
#update_process_title = on
#stats_temp_directory = 'pg_stat_tmp'


# - Statistics Monitoring -

#log_parser_stats = off
#log_planner_stats = off
#log_executor_stats = off
#log_statement_stats = off






#------------------------------------------------------------------------------
# AUTOVACUUM PARAMETERS




#------------------------------------------------------------------------------

#autovacuum = on # Enable autovacuum subprocess? 'on'
# requires track_counts to also be on.
#log_autovacuum_min_duration = -1 # -1 disables, 0 logs all
actions
and
# their durations, > 0 logs only
# actions running at least this number
# of milliseconds.
#autovacuum_max_workers = 3 # max number of autovacuum
subprocesses
#autovacuum_naptime = 1min # time between autovacuum runs
#autovacuum_vacuum_threshold = 50 # min number of row updates
before
# vacuum
#autovacuum_analyze_threshold = 50 # min number of row updates
before
# analyze
#autovacuum_vacuum_scale_factor = 0.2 # fraction of table size
before
vacuum
#autovacuum_analyze_scale_factor = 0.1 # fraction of table size
before
analyze
#autovacuum_freeze_max_age = 200000000 # maximum XID age before
forced
vacuum
# (change requires restart)
#autovacuum_vacuum_cost_delay = 20ms # default vacuum cost
delay
for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
#autovacuum_vacuum_cost_limit = -1 # default vacuum cost limit
for
# autovacuum, -1 means use
# vacuum_cost_limit






#------------------------------------------------------------------------------
# CLIENT CONNECTION DEFAULTS




#------------------------------------------------------------------------------

# - Statement Behavior -

#search_path = '"$user",public' # schema names
#default_tablespace = '' # a tablespace name, '' uses the
default
#temp_tablespaces = '' # a list of tablespace names, ''
uses
# only default tablespace
#check_function_bodies = on
#default_transaction_isolation = 'read committed'
#default_transaction_read_only = off
#session_replication_role = 'origin'
#statement_timeout = 0 # in milliseconds, 0 is disabled
#vacuum_freeze_min_age = 50000000
#vacuum_freeze_table_age = 150000000
#xmlbinary = 'base64'
#xmloption = 'content'

# - Locale and Formatting -

datestyle = 'iso, mdy'
#intervalstyle = 'postgres'
#timezone = unknown # actually, defaults to TZ
environment
# setting
#timezone_abbreviations = 'Default' # Select the set of
available
time
zone
# abbreviations. Currently, there are
# Default
# Australia
# India
# You can create your own file in
# share/timezonesets/.
#extra_float_digits = 0 # min -15, max 2
#client_encoding = sql_ascii # actually, defaults to
database
# encoding

# These settings are initialized by initdb, but they can be
changed.
lc_messages = 'en_US.UTF-8' # locale for system error
message
# strings
lc_monetary = 'en_US.UTF-8' # locale for monetary
formatting
lc_numeric = 'en_US.UTF-8' # locale for number
formatting
lc_time = 'en_US.UTF-8' # locale for time
formatting

# default configuration for text search
default_text_search_config = 'pg_catalog.english'

# - Other Defaults -

#dynamic_library_path = '$libdir'
#local_preload_libraries = ''






#------------------------------------------------------------------------------
# LOCK MANAGEMENT




#------------------------------------------------------------------------------

#deadlock_timeout = 1s
#max_locks_per_transaction = 64 # min 10
# (change requires restart)
# Note: Each lock table slot uses ~270 bytes of shared memory,
and
there
are
# max_locks_per_transaction * (max_connections +
max_prepared_transactions)
# lock table slots.






#------------------------------------------------------------------------------
# VERSION/PLATFORM COMPATIBILITY




#------------------------------------------------------------------------------

# - Previous PostgreSQL Versions -

#add_missing_from = off
#array_nulls = on
#backslash_quote = safe_encoding # on, off, or safe_encoding
#default_with_oids = off
#escape_string_warning = on
#regex_flavor = advanced # advanced, extended, or basic
#sql_inheritance = on
#standard_conforming_strings = off
#synchronize_seqscans = on

# - Other Platforms and Clients -

#transform_null_equals = off






#------------------------------------------------------------------------------
# CUSTOMIZED OPTIONS




#------------------------------------------------------------------------------

#custom_variable_classes = '' # list of custom variable
class
names

Thanks,
Megha



On Wednesday, August 8, 2012 1:04:02 PM UTC-4, Vikas Singh wrote:

Hi Megha,

Please send following files:
1. 6.init-embedded-db.log
2. pg_hba.conf (and its filesystem location)
3. postgresql.conf (and its filesystem location)

The error is occurring when CM is trying to bring up Postgresql
server. Which distribution of Linux are you using? Is there an
existing install of postgresql on your machine? If yes, uninstall
that
and then try again.

- Vikas

On Wed, Aug 8, 2012 at 9:55 AM, Megha Agrawal
wrote:

To add on this is the content for db.log:
LOG: received smart shutdown request
LOG: shutting down
PANIC: could not open control file "global/pg_control": No such
file
or
directory
LOG: background writer process (PID 29561) was terminated by
signal
6:
Aborted
LOG: terminating any other active server processes
LOG: abnormal database system shutdown
LOG: could not receive test message on socket for statistics
collector:
Connection refused
LOG: disabling statistics collector for lack of working socket
WARNING: autovacuum not started because of misconfiguration
HINT: Enable the "track_counts" option.
LOG: database system was shut down at 2012-08-08 12:49:37 EDT
LOG: database system is ready to accept connections

FATAL: no pg_hba.conf entry for host "192.168.30.1", user
"cloudera-scm",
database "postgres", SSL off
LOG: received smart shutdown request
LOG: shutting down
PANIC: could not open control file "global/pg_control": No such
file
or
directory
LOG: background writer process (PID 29838) was terminated by
signal
6:
Aborted
LOG: terminating any other active server processes
LOG: abnormal database system shutdown




On Tuesday, August 7, 2012 4:56:02 PM UTC-4, Megha Agrawal
wrote:

Hi,

Can anyone please help me the installation of CLodera Manager
4.0
Free
Edition.

I am trying to install it using
"cloudera-manager-installer.bin"
and
getting the following error:
Failed to configure Embedded Service and Configuration
Database.
See
/var/log/cloudera-manager-installer/6.init-embedded-db.log for
details.

When I look into log file I get following:
psql: FATAL: no pg_hba.conf entry for host "192.168.30.1",
user
"cloudera-scm", database "postgres", SSL off


Now, to resolve this I have done following changes:
in postgresql.conf --------- did this : listen_addresses =
'*'
in pg_hba.conf -------------- added:

host postgres cloudera-scm 192.168.30.1/24 trust
host all all 192.168.5.30/24 trust
host all all 192.168.30.30/24 trust
host postgres all 192.168.30.1/32 trust


But then also I get the above error.

Please help.

Thanks,
Megha



--
Regards
Megha Agrawal