FAQ
My mistake Darren. I had only set the logging level at the role level, and
I was looking at the service level logging settings. I have set them all to
WARN level now. I also set it to only retain 48 hours of logs in each of
the monitor services. I noticed another setting to purge job details after
14 days though, so I set this to 7 days.
Under the service level, I don't see anywhere to limit the total logs size
though. All I can see is a few parts (eg. Reports Manager Max Log Size)that limit to 200MB by default. How do I limit the total size of
cloudera-scm-server-db?

Here's a list of the top tables by size from PostgreSQL:
                 relname | relpages
----------------------------------------+----------
  commands | 134
  pg_attribute | 68
  pg_proc | 54
  pg_depend | 48
  pg_toast_16982 | 45
  pg_depend_reference_index | 40
  pg_depend_depender_index | 39
  configs | 33
  pg_proc_proname_args_nsp_index | 26
  pg_statistic | 26
  audits | 26
  pg_toast_2618 | 25
  pg_attribute_relid_attnam_index | 22
  pg_description | 21
  idx_command_parent | 18
  idx_command_schedule | 16
  pg_description_o_c_o_index | 16
  idx_command_host | 16
  processes | 16
  idx_command_cluster | 16
  pg_attribute_relid_attnum_index | 14

On Tuesday, August 20, 2013 6:43:49 PM UTC+1, Darren Lo wrote:

Hi Jaykay,

I don't know why these configs would have changed, other than a manual
action from CM user (via UI or API). How did you set them in the first
place? Did you set them at service-level, or role-level? You can tell if
you've done a role-level override by looking at the service-level configs
(click on the Mgmt service, then on Configuration -> View and Edit). It
will have a note next to any config that has a role-level override. It's
possible that you set a role-level override before, and now when you
inspected the config you were looking at service-level, which still said
INFO. If you have the Enterprise or Trial edition, you can also look in the
config revision history to see exactly what happened to your configs.

Since you are concerned about the max log size as well as the file growth
rate, you may want to configure the log settings to decrease the size per
file or the maximum number of files kept for each service. This can be
configured in the logs tab for each role.

Thanks,
Darren


On Tue, Aug 20, 2013 at 5:38 AM, Jaykay <johnk...@gmail.com <javascript:>>wrote:
Hi all,

I came back to work today after the weekend to see that the
logs(cloudera-scm-server-db) have gone from 4.5GB to 32GB, which is the
full amount of free disk space. It seems that the log growth had slowed but
is has now gone back to unmanageable. When I looked at the monitoring
service settings, the logging level had reverted back to INFO on all of
them. I had set it to WARN last week. Any idea what would cause it to
revert?

On Wednesday, August 14, 2013 3:31:30 PM UTC+1, Darren Lo wrote:

If you need a db admin for the embedded postgresql db, that login is in
/var/lib/cloudera-scm-server-**db/data/generated_password.**txt.

On Wed, Aug 14, 2013 at 7:24 AM, Serega Sheypak wrote:

you are welcome :)


2013/8/14 Jaykay <johnk...@gmail.com>

Thanks again Joe.
Thanks to all who helped me out here, Sergey and Gerd too. I have a
lot to learn with this stuff.
On Wednesday, August 14, 2013 2:04:19 PM UTC+1, Joe Travaglini wrote:

it should be located in /etc/cloudera-scm-server/db.**pr**operties.

On Wed, Aug 14, 2013 at 8:50 AM, Jaykay wrote:

Yeah default didn't work.
On Wednesday, August 14, 2013 1:43:47 PM UTC+1, Serega Sheypak wrote:

did you try admin/admin?


2013/8/14 Jaykay <johnk...@gmail.com>

Thanks Joe. I ended up rolling back to a snapshot, as it's a VM. If
I want to connect to postgres to look at db sizes, any idea what the
default postgres login is with a Cloudera install?


On Wednesday, August 14, 2013 1:00:27 PM UTC+1, Joe Travaglini
wrote:
IMO the quickest way to purge space is to find a log file that's
large.

du /var/log | sort -rn | less

echo "" > /path/to/that/file

This will free space immediately even if a process has an open fd
to write to that log file. If you simply rm the file while the process
writing to it is still running, it will not free the disk space.

On Wed, Aug 14, 2013 at 7:47 AM, Jaykay wrote:

Too late, all my disk space is gone again, so I can't change
anything in the GUI. Is there a way to purge the data from
cloudera-scm-server-db? I would like to connect to the db to see which
tables are taking up space, but I am out of space now, so psql can't run
properly.

On Wednesday, August 14, 2013 7:42:25 AM UTC+1, Gerd Koenig
wrote:
Yes, there is a way to configure the min. log level.
To configure it, e.g. for the Datanodes, navigate to the
configuration property "DataNode Logging Th reshold" via
"Service HDFS" => "Configuration" => "View and Edit" => "Default" =>
"Datanode (Default)" => "Logs"

br....Gerd....

On 13 August 2013 16:15, Jaykay wrote:

Is there a way to set CM to log in less verbose way? I notice
that I can search the logs, and filter by min. log level, which includes
info, debug, warn, error, fatal.
I would like to set it to only record error and fatal.

On Tuesday, August 13, 2013 1:12:04 PM UTC+1, Serega Sheypak
wrote:
We didn't do something special. It should be cleaned in a
while...


2013/8/13 Jaykay <johnk...@gmail.com>
Correction to my earlier post:
Thanks Serega, it is indeed the
/var/lib/cloudera-scm-server-**d************b that is
taking up all the space.
Host and Service Monitor expiration periods are both set to
7days only. I will change this down to 3days and see how it looks then.
I have changed it down to 3days, but the db is still the
same size. Do I need to do some kind of purge to get the size down?

On Tuesday, August 13, 2013 11:31:05 AM UTC+1, Serega
Sheypak wrote:
If it loks this way:
[root@prod-scm run]# du -sh /var/lib/* | sort -h
4,0K /var/lib/fprint
4,0K /var/lib/games
4,0K /var/lib/logrotate.status
4,0K /var/lib/misc
4,0K /var/lib/net-snmp
4,0K /var/lib/ntp
4,0K /var/lib/random-seed
4,0K /var/lib/rsyslog
8,0K /var/lib/dbus
8,0K /var/lib/plymouth
8,0K /var/lib/udev
12K /var/lib/prelink
12K /var/lib/samba
12K /var/lib/stateless
16K /var/lib/dhclient
16K /var/lib/pgsql
20K /var/lib/authconfig
28K /var/lib/polkit-1
208K /var/lib/alternatives
868K /var/lib/readahead
3,3M /var/lib/mlocate
6,8M /var/lib/yum
38M /var/lib/rpm
*90G /var/lib/cloudera-scm-server-db*

Then you need to tune management service

Host Monitor Data Expiration Period (hours)
Service Monitor Data Expiration Period (hours)

and so on...
We also had such problem. We are planning to migrate
management service to standalone VM with standalone PostGre SQL server.


2013/8/13 Jaykay <johnk...@gmail.com>
I upgraded my Cloudera Manager server from 4.2 to 4.6.
Since upgrading, I have lost over 20GB in disk space in about a day. I
assume it is from the logging that comes with the latest version. Any way
to change it so that logging takes up less disk space?

--
Thanks,
Darren
To unsubscribe from this group and stop receiving emails from it, send
an email to scm-users+...@cloudera.org <javascript:>.


--
Thanks,
Darren
To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

Search Discussions

  • Gerd König at Aug 21, 2013 at 12:46 pm
    Jaykay,
    Postgres doesn't offer an option to limit the max. used space (afaik).
    Therefore the limit is the one from the underlying OS and respectively the
    partition-/LVM-size where your database resides.


    2013/8/21 Jaykay <johnkellyie@gmail.com>
    My mistake Darren. I had only set the logging level at the role level, and
    I was looking at the service level logging settings. I have set them all to
    WARN level now. I also set it to only retain 48 hours of logs in each of
    the monitor services. I noticed another setting to purge job details after
    14 days though, so I set this to 7 days.
    Under the service level, I don't see anywhere to limit the total logs size
    though. All I can see is a few parts (eg. Reports Manager Max Log Size)that limit to 200MB by default. How do I limit the total size of
    cloudera-scm-server-db?

    Here's a list of the top tables by size from PostgreSQL:
    relname | relpages
    ----------------------------------------+----------
    commands | 134
    pg_attribute | 68
    pg_proc | 54
    pg_depend | 48
    pg_toast_16982 | 45
    pg_depend_reference_index | 40
    pg_depend_depender_index | 39
    configs | 33
    pg_proc_proname_args_nsp_index | 26
    pg_statistic | 26
    audits | 26
    pg_toast_2618 | 25
    pg_attribute_relid_attnam_index | 22
    pg_description | 21
    idx_command_parent | 18
    idx_command_schedule | 16
    pg_description_o_c_o_index | 16
    idx_command_host | 16
    processes | 16
    idx_command_cluster | 16
    pg_attribute_relid_attnum_index | 14


    On Tuesday, August 20, 2013 6:43:49 PM UTC+1, Darren Lo wrote:

    Hi Jaykay,

    I don't know why these configs would have changed, other than a manual
    action from CM user (via UI or API). How did you set them in the first
    place? Did you set them at service-level, or role-level? You can tell if
    you've done a role-level override by looking at the service-level configs
    (click on the Mgmt service, then on Configuration -> View and Edit). It
    will have a note next to any config that has a role-level override. It's
    possible that you set a role-level override before, and now when you
    inspected the config you were looking at service-level, which still said
    INFO. If you have the Enterprise or Trial edition, you can also look in the
    config revision history to see exactly what happened to your configs.

    Since you are concerned about the max log size as well as the file growth
    rate, you may want to configure the log settings to decrease the size per
    file or the maximum number of files kept for each service. This can be
    configured in the logs tab for each role.

    Thanks,
    Darren

    On Tue, Aug 20, 2013 at 5:38 AM, Jaykay wrote:

    Hi all,

    I came back to work today after the weekend to see that the
    logs(cloudera-scm-server-db) have gone from 4.5GB to 32GB, which is the
    full amount of free disk space. It seems that the log growth had slowed but
    is has now gone back to unmanageable. When I looked at the monitoring
    service settings, the logging level had reverted back to INFO on all of
    them. I had set it to WARN last week. Any idea what would cause it to
    revert?

    On Wednesday, August 14, 2013 3:31:30 PM UTC+1, Darren Lo wrote:

    If you need a db admin for the embedded postgresql db, that login is in
    /var/lib/cloudera-scm-server-**d**b/data/generated_password.**txt.

    On Wed, Aug 14, 2013 at 7:24 AM, Serega Sheypak wrote:

    you are welcome :)


    2013/8/14 Jaykay <johnk...@gmail.com>

    Thanks again Joe.
    Thanks to all who helped me out here, Sergey and Gerd too. I have a
    lot to learn with this stuff.
    On Wednesday, August 14, 2013 2:04:19 PM UTC+1, Joe Travaglini wrote:

    it should be located in /etc/cloudera-scm-server/db.**pr****
    operties.

    On Wed, Aug 14, 2013 at 8:50 AM, Jaykay wrote:

    Yeah default didn't work.

    On Wednesday, August 14, 2013 1:43:47 PM UTC+1, Serega Sheypak
    wrote:
    did you try admin/admin?


    2013/8/14 Jaykay <johnk...@gmail.com>

    Thanks Joe. I ended up rolling back to a snapshot, as it's a VM.
    If I want to connect to postgres to look at db sizes, any idea what the
    default postgres login is with a Cloudera install?


    On Wednesday, August 14, 2013 1:00:27 PM UTC+1, Joe Travaglini
    wrote:
    IMO the quickest way to purge space is to find a log file that's
    large.

    du /var/log | sort -rn | less

    echo "" > /path/to/that/file

    This will free space immediately even if a process has an open
    fd to write to that log file. If you simply rm the file while the process
    writing to it is still running, it will not free the disk space.

    On Wed, Aug 14, 2013 at 7:47 AM, Jaykay wrote:

    Too late, all my disk space is gone again, so I can't change
    anything in the GUI. Is there a way to purge the data from
    cloudera-scm-server-db? I would like to connect to the db to see which
    tables are taking up space, but I am out of space now, so psql can't run
    properly.

    On Wednesday, August 14, 2013 7:42:25 AM UTC+1, Gerd Koenig
    wrote:
    Yes, there is a way to configure the min. log level.
    To configure it, e.g. for the Datanodes, navigate to the
    configuration property "DataNode Logging Th reshold" via
    "Service HDFS" => "Configuration" => "View and Edit" => "Default" =>
    "Datanode (Default)" => "Logs"

    br....Gerd....

    On 13 August 2013 16:15, Jaykay wrote:

    Is there a way to set CM to log in less verbose way? I notice
    that I can search the logs, and filter by min. log level, which includes
    info, debug, warn, error, fatal.
    I would like to set it to only record error and fatal.

    On Tuesday, August 13, 2013 1:12:04 PM UTC+1, Serega Sheypak
    wrote:
    We didn't do something special. It should be cleaned in a
    while...


    2013/8/13 Jaykay <johnk...@gmail.com>
    Correction to my earlier post:
    Thanks Serega, it is indeed the
    /var/lib/cloudera-scm-server-**d**************b that is
    taking up all the space.
    Host and Service Monitor expiration periods are both set
    to 7days only. I will change this down to 3days and see how it looks then.
    I have changed it down to 3days, but the db is still the
    same size. Do I need to do some kind of purge to get the size down?

    On Tuesday, August 13, 2013 11:31:05 AM UTC+1, Serega
    Sheypak wrote:
    If it loks this way:
    [root@prod-scm run]# du -sh /var/lib/* | sort -h
    4,0K /var/lib/fprint
    4,0K /var/lib/games
    4,0K /var/lib/logrotate.status
    4,0K /var/lib/misc
    4,0K /var/lib/net-snmp
    4,0K /var/lib/ntp
    4,0K /var/lib/random-seed
    4,0K /var/lib/rsyslog
    8,0K /var/lib/dbus
    8,0K /var/lib/plymouth
    8,0K /var/lib/udev
    12K /var/lib/prelink
    12K /var/lib/samba
    12K /var/lib/stateless
    16K /var/lib/dhclient
    16K /var/lib/pgsql
    20K /var/lib/authconfig
    28K /var/lib/polkit-1
    208K /var/lib/alternatives
    868K /var/lib/readahead
    3,3M /var/lib/mlocate
    6,8M /var/lib/yum
    38M /var/lib/rpm
    *90G /var/lib/cloudera-scm-server-db*

    Then you need to tune management service

    Host Monitor Data Expiration Period (hours)
    Service Monitor Data Expiration Period (hours)

    and so on...
    We also had such problem. We are planning to migrate
    management service to standalone VM with standalone PostGre SQL server.


    2013/8/13 Jaykay <johnk...@gmail.com>
    I upgraded my Cloudera Manager server from 4.2 to 4.6.
    Since upgrading, I have lost over 20GB in disk space in about a day. I
    assume it is from the logging that comes with the latest version. Any way
    to change it so that logging takes up less disk space?

    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it, send
    an email to scm-users+...@**cloudera.org.


    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it, send an
    email to scm-users+unsubscribe@cloudera.org.


    --
    Gerd Koenig
    Data Engineer / Big Data Analytics
    ____________________________________

    YMC AG
    Sonnenstr. 4
    CH-8280 Kreuzlingen
    Switzerland

    Tel +41 (0)71 / 508 24 74
    Fax +41 (0)71 / 560 53 89

    http://twitter.com/gerd_koenig
    www.ymc.ch <http://www.ymc.ch>
    ____________________________________

    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Jaykay at Aug 21, 2013 at 12:56 pm
    Hi Gerd,
    Yes the OS is limiting the size. However, I would like to configure
    Cloudera in such a way as it doesn't use up all my available space with
    just a few days of logs. Just looking for the best approach for this.
    On Wednesday, August 21, 2013 1:46:34 PM UTC+1, Gerd König wrote:

    Jaykay,
    Postgres doesn't offer an option to limit the max. used space (afaik).
    Therefore the limit is the one from the underlying OS and respectively the
    partition-/LVM-size where your database resides.


    2013/8/21 Jaykay <johnk...@gmail.com <javascript:>>
    My mistake Darren. I had only set the logging level at the role level,
    and I was looking at the service level logging settings. I have set them
    all to WARN level now. I also set it to only retain 48 hours of logs in
    each of the monitor services. I noticed another setting to purge job
    details after 14 days though, so I set this to 7 days.
    Under the service level, I don't see anywhere to limit the total logs
    size though. All I can see is a few parts (eg. Reports Manager Max Log
    Size) that limit to 200MB by default. How do I limit the total size of
    cloudera-scm-server-db?

    Here's a list of the top tables by size from PostgreSQL:
    relname | relpages
    ----------------------------------------+----------
    commands | 134
    pg_attribute | 68
    pg_proc | 54
    pg_depend | 48
    pg_toast_16982 | 45
    pg_depend_reference_index | 40
    pg_depend_depender_index | 39
    configs | 33
    pg_proc_proname_args_nsp_index | 26
    pg_statistic | 26
    audits | 26
    pg_toast_2618 | 25
    pg_attribute_relid_attnam_index | 22
    pg_description | 21
    idx_command_parent | 18
    idx_command_schedule | 16
    pg_description_o_c_o_index | 16
    idx_command_host | 16
    processes | 16
    idx_command_cluster | 16
    pg_attribute_relid_attnum_index | 14


    On Tuesday, August 20, 2013 6:43:49 PM UTC+1, Darren Lo wrote:

    Hi Jaykay,

    I don't know why these configs would have changed, other than a manual
    action from CM user (via UI or API). How did you set them in the first
    place? Did you set them at service-level, or role-level? You can tell if
    you've done a role-level override by looking at the service-level configs
    (click on the Mgmt service, then on Configuration -> View and Edit). It
    will have a note next to any config that has a role-level override. It's
    possible that you set a role-level override before, and now when you
    inspected the config you were looking at service-level, which still said
    INFO. If you have the Enterprise or Trial edition, you can also look in the
    config revision history to see exactly what happened to your configs.

    Since you are concerned about the max log size as well as the file
    growth rate, you may want to configure the log settings to decrease the
    size per file or the maximum number of files kept for each service. This
    can be configured in the logs tab for each role.

    Thanks,
    Darren

    On Tue, Aug 20, 2013 at 5:38 AM, Jaykay wrote:

    Hi all,

    I came back to work today after the weekend to see that the
    logs(cloudera-scm-server-db) have gone from 4.5GB to 32GB, which is the
    full amount of free disk space. It seems that the log growth had slowed but
    is has now gone back to unmanageable. When I looked at the monitoring
    service settings, the logging level had reverted back to INFO on all of
    them. I had set it to WARN last week. Any idea what would cause it to
    revert?

    On Wednesday, August 14, 2013 3:31:30 PM UTC+1, Darren Lo wrote:

    If you need a db admin for the embedded postgresql db, that login is
    in /var/lib/cloudera-scm-server-**d**b/data/generated_password.**txt.

    On Wed, Aug 14, 2013 at 7:24 AM, Serega Sheypak wrote:

    you are welcome :)


    2013/8/14 Jaykay <johnk...@gmail.com>

    Thanks again Joe.
    Thanks to all who helped me out here, Sergey and Gerd too. I have a
    lot to learn with this stuff.
    On Wednesday, August 14, 2013 2:04:19 PM UTC+1, Joe Travaglini wrote:

    it should be located in /etc/cloudera-scm-server/db.**pr****
    operties.

    On Wed, Aug 14, 2013 at 8:50 AM, Jaykay wrote:

    Yeah default didn't work.

    On Wednesday, August 14, 2013 1:43:47 PM UTC+1, Serega Sheypak
    wrote:
    did you try admin/admin?


    2013/8/14 Jaykay <johnk...@gmail.com>

    Thanks Joe. I ended up rolling back to a snapshot, as it's a VM.
    If I want to connect to postgres to look at db sizes, any idea what the
    default postgres login is with a Cloudera install?


    On Wednesday, August 14, 2013 1:00:27 PM UTC+1, Joe Travaglini
    wrote:
    IMO the quickest way to purge space is to find a log file
    that's large.

    du /var/log | sort -rn | less

    echo "" > /path/to/that/file

    This will free space immediately even if a process has an open
    fd to write to that log file. If you simply rm the file while the process
    writing to it is still running, it will not free the disk space.

    On Wed, Aug 14, 2013 at 7:47 AM, Jaykay wrote:

    Too late, all my disk space is gone again, so I can't change
    anything in the GUI. Is there a way to purge the data from
    cloudera-scm-server-db? I would like to connect to the db to see which
    tables are taking up space, but I am out of space now, so psql can't run
    properly.

    On Wednesday, August 14, 2013 7:42:25 AM UTC+1, Gerd Koenig
    wrote:
    Yes, there is a way to configure the min. log level.
    To configure it, e.g. for the Datanodes, navigate to the
    configuration property "DataNode Logging Th reshold" via
    "Service HDFS" => "Configuration" => "View and Edit" => "Default" =>
    "Datanode (Default)" => "Logs"

    br....Gerd....

    On 13 August 2013 16:15, Jaykay wrote:

    Is there a way to set CM to log in less verbose way? I
    notice that I can search the logs, and filter by min. log level, which
    includes info, debug, warn, error, fatal.
    I would like to set it to only record error and fatal.

    On Tuesday, August 13, 2013 1:12:04 PM UTC+1, Serega Sheypak
    wrote:
    We didn't do something special. It should be cleaned in a
    while...


    2013/8/13 Jaykay <johnk...@gmail.com>
    Correction to my earlier post:
    Thanks Serega, it is indeed the
    /var/lib/cloudera-scm-server-**d**************b that is
    taking up all the space.
    Host and Service Monitor expiration periods are both set
    to 7days only. I will change this down to 3days and see how it looks then.
    I have changed it down to 3days, but the db is still the
    same size. Do I need to do some kind of purge to get the size down?

    On Tuesday, August 13, 2013 11:31:05 AM UTC+1, Serega
    Sheypak wrote:
    If it loks this way:
    [root@prod-scm run]# du -sh /var/lib/* | sort -h
    4,0K /var/lib/fprint
    4,0K /var/lib/games
    4,0K /var/lib/logrotate.status
    4,0K /var/lib/misc
    4,0K /var/lib/net-snmp
    4,0K /var/lib/ntp
    4,0K /var/lib/random-seed
    4,0K /var/lib/rsyslog
    8,0K /var/lib/dbus
    8,0K /var/lib/plymouth
    8,0K /var/lib/udev
    12K /var/lib/prelink
    12K /var/lib/samba
    12K /var/lib/stateless
    16K /var/lib/dhclient
    16K /var/lib/pgsql
    20K /var/lib/authconfig
    28K /var/lib/polkit-1
    208K /var/lib/alternatives
    868K /var/lib/readahead
    3,3M /var/lib/mlocate
    6,8M /var/lib/yum
    38M /var/lib/rpm
    *90G /var/lib/cloudera-scm-server-db*

    Then you need to tune management service

    Host Monitor Data Expiration Period (hours)
    Service Monitor Data Expiration Period (hours)

    and so on...
    We also had such problem. We are planning to migrate
    management service to standalone VM with standalone PostGre SQL server.


    2013/8/13 Jaykay <johnk...@gmail.com>
    I upgraded my Cloudera Manager server from 4.2 to 4.6.
    Since upgrading, I have lost over 20GB in disk space in about a day. I
    assume it is from the logging that comes with the latest version. Any way
    to change it so that logging takes up less disk space?

    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it, send
    an email to scm-users+...@**cloudera.org.


    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it, send
    an email to scm-users+...@cloudera.org <javascript:>.


    --
    Gerd Koenig
    Data Engineer / Big Data Analytics
    ____________________________________

    YMC AG
    Sonnenstr. 4
    CH-8280 Kreuzlingen
    Switzerland

    Tel +41 (0)71 / 508 24 74
    Fax +41 (0)71 / 560 53 89

    http://twitter.com/gerd_koenig
    www.ymc.ch <http://www.ymc.ch>
    ____________________________________
    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Jaykay at Aug 21, 2013 at 4:16 pm
    Sorry, here's the correct sizes of the dbs, amon is the one taking up all
    the space.

       datname | size
    -----------+---------
      template1 | 5408 kB
      template0 | 5408 kB
      postgres | 5408 kB
      scm | 30 MB
      smon | 3025 MB
      amon | 27 GB
      hmon | 937 MB
      rman | 5408 kB
      nav | 5408 kB
      hive | 7062 kB

    That's Activity Monitor database. So I changed a few settings from it. I
    changed Purge Activities Data at This Age and Purge Attempts Data at This
    Age from 14 days to 72 hours, and Activity Monitor Event Publication Log
    Quiet Time Period from 1min to 30mins. So amon is down from 27GB to 2.5GB
    now.
    On Wednesday, August 21, 2013 1:56:29 PM UTC+1, Jaykay wrote:

    Hi Gerd,
    Yes the OS is limiting the size. However, I would like to configure
    Cloudera in such a way as it doesn't use up all my available space with
    just a few days of logs. Just looking for the best approach for this.
    On Wednesday, August 21, 2013 1:46:34 PM UTC+1, Gerd König wrote:

    Jaykay,
    Postgres doesn't offer an option to limit the max. used space (afaik).
    Therefore the limit is the one from the underlying OS and respectively the
    partition-/LVM-size where your database resides.


    2013/8/21 Jaykay <johnk...@gmail.com>
    My mistake Darren. I had only set the logging level at the role level,
    and I was looking at the service level logging settings. I have set them
    all to WARN level now. I also set it to only retain 48 hours of logs in
    each of the monitor services. I noticed another setting to purge job
    details after 14 days though, so I set this to 7 days.
    Under the service level, I don't see anywhere to limit the total logs
    size though. All I can see is a few parts (eg. Reports Manager Max Log
    Size) that limit to 200MB by default. How do I limit the total size of
    cloudera-scm-server-db?

    Here's a list of the top tables by size from PostgreSQL:
    relname | relpages
    ----------------------------------------+----------
    commands | 134
    pg_attribute | 68
    pg_proc | 54
    pg_depend | 48
    pg_toast_16982 | 45
    pg_depend_reference_index | 40
    pg_depend_depender_index | 39
    configs | 33
    pg_proc_proname_args_nsp_index | 26
    pg_statistic | 26
    audits | 26
    pg_toast_2618 | 25
    pg_attribute_relid_attnam_index | 22
    pg_description | 21
    idx_command_parent | 18
    idx_command_schedule | 16
    pg_description_o_c_o_index | 16
    idx_command_host | 16
    processes | 16
    idx_command_cluster | 16
    pg_attribute_relid_attnum_index | 14


    On Tuesday, August 20, 2013 6:43:49 PM UTC+1, Darren Lo wrote:

    Hi Jaykay,

    I don't know why these configs would have changed, other than a manual
    action from CM user (via UI or API). How did you set them in the first
    place? Did you set them at service-level, or role-level? You can tell if
    you've done a role-level override by looking at the service-level configs
    (click on the Mgmt service, then on Configuration -> View and Edit). It
    will have a note next to any config that has a role-level override. It's
    possible that you set a role-level override before, and now when you
    inspected the config you were looking at service-level, which still said
    INFO. If you have the Enterprise or Trial edition, you can also look in the
    config revision history to see exactly what happened to your configs.

    Since you are concerned about the max log size as well as the file
    growth rate, you may want to configure the log settings to decrease the
    size per file or the maximum number of files kept for each service. This
    can be configured in the logs tab for each role.

    Thanks,
    Darren

    On Tue, Aug 20, 2013 at 5:38 AM, Jaykay wrote:

    Hi all,

    I came back to work today after the weekend to see that the
    logs(cloudera-scm-server-db) have gone from 4.5GB to 32GB, which is the
    full amount of free disk space. It seems that the log growth had slowed but
    is has now gone back to unmanageable. When I looked at the monitoring
    service settings, the logging level had reverted back to INFO on all of
    them. I had set it to WARN last week. Any idea what would cause it to
    revert?

    On Wednesday, August 14, 2013 3:31:30 PM UTC+1, Darren Lo wrote:

    If you need a db admin for the embedded postgresql db, that login is
    in /var/lib/cloudera-scm-server-**d**b/data/generated_password.**txt.


    On Wed, Aug 14, 2013 at 7:24 AM, Serega Sheypak <serega....@gmail.com
    wrote:
    you are welcome :)


    2013/8/14 Jaykay <johnk...@gmail.com>

    Thanks again Joe.
    Thanks to all who helped me out here, Sergey and Gerd too. I have a
    lot to learn with this stuff.

    On Wednesday, August 14, 2013 2:04:19 PM UTC+1, Joe Travaglini
    wrote:
    it should be located in /etc/cloudera-scm-server/db.**pr****
    operties.

    On Wed, Aug 14, 2013 at 8:50 AM, Jaykay wrote:

    Yeah default didn't work.

    On Wednesday, August 14, 2013 1:43:47 PM UTC+1, Serega Sheypak
    wrote:
    did you try admin/admin?


    2013/8/14 Jaykay <johnk...@gmail.com>

    Thanks Joe. I ended up rolling back to a snapshot, as it's a VM.
    If I want to connect to postgres to look at db sizes, any idea what the
    default postgres login is with a Cloudera install?


    On Wednesday, August 14, 2013 1:00:27 PM UTC+1, Joe Travaglini
    wrote:
    IMO the quickest way to purge space is to find a log file
    that's large.

    du /var/log | sort -rn | less

    echo "" > /path/to/that/file

    This will free space immediately even if a process has an open
    fd to write to that log file. If you simply rm the file while the process
    writing to it is still running, it will not free the disk space.

    On Wed, Aug 14, 2013 at 7:47 AM, Jaykay wrote:

    Too late, all my disk space is gone again, so I can't change
    anything in the GUI. Is there a way to purge the data from
    cloudera-scm-server-db? I would like to connect to the db to see which
    tables are taking up space, but I am out of space now, so psql can't run
    properly.

    On Wednesday, August 14, 2013 7:42:25 AM UTC+1, Gerd Koenig
    wrote:
    Yes, there is a way to configure the min. log level.
    To configure it, e.g. for the Datanodes, navigate to the
    configuration property "DataNode Logging Th reshold" via
    "Service HDFS" => "Configuration" => "View and Edit" => "Default" =>
    "Datanode (Default)" => "Logs"

    br....Gerd....

    On 13 August 2013 16:15, Jaykay wrote:

    Is there a way to set CM to log in less verbose way? I
    notice that I can search the logs, and filter by min. log level, which
    includes info, debug, warn, error, fatal.
    I would like to set it to only record error and fatal.

    On Tuesday, August 13, 2013 1:12:04 PM UTC+1, Serega
    Sheypak wrote:
    We didn't do something special. It should be cleaned in a
    while...


    2013/8/13 Jaykay <johnk...@gmail.com>
    Correction to my earlier post:
    Thanks Serega, it is indeed the
    /var/lib/cloudera-scm-server-**d**************b that is
    taking up all the space.
    Host and Service Monitor expiration periods are both set
    to 7days only. I will change this down to 3days and see how it looks then.
    I have changed it down to 3days, but the db is still the
    same size. Do I need to do some kind of purge to get the size down?

    On Tuesday, August 13, 2013 11:31:05 AM UTC+1, Serega
    Sheypak wrote:
    If it loks this way:
    [root@prod-scm run]# du -sh /var/lib/* | sort -h
    4,0K /var/lib/fprint
    4,0K /var/lib/games
    4,0K /var/lib/logrotate.status
    4,0K /var/lib/misc
    4,0K /var/lib/net-snmp
    4,0K /var/lib/ntp
    4,0K /var/lib/random-seed
    4,0K /var/lib/rsyslog
    8,0K /var/lib/dbus
    8,0K /var/lib/plymouth
    8,0K /var/lib/udev
    12K /var/lib/prelink
    12K /var/lib/samba
    12K /var/lib/stateless
    16K /var/lib/dhclient
    16K /var/lib/pgsql
    20K /var/lib/authconfig
    28K /var/lib/polkit-1
    208K /var/lib/alternatives
    868K /var/lib/readahead
    3,3M /var/lib/mlocate
    6,8M /var/lib/yum
    38M /var/lib/rpm
    *90G /var/lib/cloudera-scm-server-db*

    Then you need to tune management service

    Host Monitor Data Expiration Period (hours)
    Service Monitor Data Expiration Period (hours)

    and so on...
    We also had such problem. We are planning to migrate
    management service to standalone VM with standalone PostGre SQL server.


    2013/8/13 Jaykay <johnk...@gmail.com>
    I upgraded my Cloudera Manager server from 4.2 to 4.6.
    Since upgrading, I have lost over 20GB in disk space in about a day. I
    assume it is from the logging that comes with the latest version. Any way
    to change it so that logging takes up less disk space?

    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it,
    send an email to scm-users+...@**cloudera.org.


    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it, send
    an email to scm-users+...@cloudera.org.


    --
    Gerd Koenig
    Data Engineer / Big Data Analytics
    ____________________________________

    YMC AG
    Sonnenstr. 4
    CH-8280 Kreuzlingen
    Switzerland

    Tel +41 (0)71 / 508 24 74
    Fax +41 (0)71 / 560 53 89

    http://twitter.com/gerd_koenig
    www.ymc.ch <http://www.ymc.ch>
    ____________________________________
    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Darren Lo at Aug 21, 2013 at 5:05 pm
    Hi Jaykay,

    If you want to retain more data, then you can also consider moving off of
    the embedded db, increasing the space allowed for /var/lib, or moving the
    contents of /var/lib/cloudera-scm-server-db/data to somewhere with more
    space and creating a symlink from the old location to the new one (so CM
    doesn't even know it moved). You should stop Hive, all management services,
    Cloudera Manager, and then the cloudera manager database before moving the
    database.

    Thanks,
    Darren

    On Wed, Aug 21, 2013 at 9:16 AM, Jaykay wrote:

    Sorry, here's the correct sizes of the dbs, amon is the one taking up all
    the space.

    datname | size
    -----------+---------
    template1 | 5408 kB
    template0 | 5408 kB
    postgres | 5408 kB
    scm | 30 MB
    smon | 3025 MB
    amon | 27 GB
    hmon | 937 MB
    rman | 5408 kB
    nav | 5408 kB
    hive | 7062 kB

    That's Activity Monitor database. So I changed a few settings from it. I
    changed Purge Activities Data at This Age and Purge Attempts Data at This
    Age from 14 days to 72 hours, and Activity Monitor Event Publication Log
    Quiet Time Period from 1min to 30mins. So amon is down from 27GB to 2.5GB
    now.
    On Wednesday, August 21, 2013 1:56:29 PM UTC+1, Jaykay wrote:

    Hi Gerd,
    Yes the OS is limiting the size. However, I would like to configure
    Cloudera in such a way as it doesn't use up all my available space with
    just a few days of logs. Just looking for the best approach for this.
    On Wednesday, August 21, 2013 1:46:34 PM UTC+1, Gerd König wrote:

    Jaykay,
    Postgres doesn't offer an option to limit the max. used space (afaik).
    Therefore the limit is the one from the underlying OS and respectively the
    partition-/LVM-size where your database resides.


    2013/8/21 Jaykay <johnk...@gmail.com>
    My mistake Darren. I had only set the logging level at the role level,
    and I was looking at the service level logging settings. I have set them
    all to WARN level now. I also set it to only retain 48 hours of logs in
    each of the monitor services. I noticed another setting to purge job
    details after 14 days though, so I set this to 7 days.
    Under the service level, I don't see anywhere to limit the total logs
    size though. All I can see is a few parts (eg. Reports Manager Max Log
    Size) that limit to 200MB by default. How do I limit the total size of
    cloudera-scm-server-db?

    Here's a list of the top tables by size from PostgreSQL:
    relname | relpages
    ------------------------------**----------+----------
    commands ** | 134
    pg_attribute ** | 68
    pg_proc ** | 54
    pg_depend ** | 48
    pg_toast_16982 ** | 45
    pg_depend_reference_index ** | 40
    pg_depend_depender_index ** | 39
    configs ** | 33
    pg_proc_proname_args_nsp_**index | 26
    pg_statistic ** | 26
    audits ** | 26
    pg_toast_2618 ** | 25
    pg_attribute_relid_attnam_**index | 22
    pg_description ** | 21
    idx_command_parent ** | 18
    idx_command_schedule ** | 16
    pg_description_o_c_o_index ** | 16
    idx_command_host ** | 16
    processes ** | 16
    idx_command_cluster ** | 16
    pg_attribute_relid_attnum_**index | 14


    On Tuesday, August 20, 2013 6:43:49 PM UTC+1, Darren Lo wrote:

    Hi Jaykay,

    I don't know why these configs would have changed, other than a manual
    action from CM user (via UI or API). How did you set them in the first
    place? Did you set them at service-level, or role-level? You can tell if
    you've done a role-level override by looking at the service-level configs
    (click on the Mgmt service, then on Configuration -> View and Edit). It
    will have a note next to any config that has a role-level override. It's
    possible that you set a role-level override before, and now when you
    inspected the config you were looking at service-level, which still said
    INFO. If you have the Enterprise or Trial edition, you can also look in the
    config revision history to see exactly what happened to your configs.

    Since you are concerned about the max log size as well as the file
    growth rate, you may want to configure the log settings to decrease the
    size per file or the maximum number of files kept for each service. This
    can be configured in the logs tab for each role.

    Thanks,
    Darren

    On Tue, Aug 20, 2013 at 5:38 AM, Jaykay wrote:

    Hi all,

    I came back to work today after the weekend to see that the
    logs(cloudera-scm-server-db) have gone from 4.5GB to 32GB, which is the
    full amount of free disk space. It seems that the log growth had slowed but
    is has now gone back to unmanageable. When I looked at the monitoring
    service settings, the logging level had reverted back to INFO on all of
    them. I had set it to WARN last week. Any idea what would cause it to
    revert?

    On Wednesday, August 14, 2013 3:31:30 PM UTC+1, Darren Lo wrote:

    If you need a db admin for the embedded postgresql db, that login is
    in /var/lib/cloudera-scm-server-**d****b/data/generated_password.**
    txt.


    On Wed, Aug 14, 2013 at 7:24 AM, Serega Sheypak <
    serega....@gmail.com> wrote:
    you are welcome :)


    2013/8/14 Jaykay <johnk...@gmail.com>

    Thanks again Joe.
    Thanks to all who helped me out here, Sergey and Gerd too. I have
    a lot to learn with this stuff.

    On Wednesday, August 14, 2013 2:04:19 PM UTC+1, Joe Travaglini
    wrote:
    it should be located in /etc/cloudera-scm-server/db.**pr******
    operties.

    On Wed, Aug 14, 2013 at 8:50 AM, Jaykay wrote:

    Yeah default didn't work.

    On Wednesday, August 14, 2013 1:43:47 PM UTC+1, Serega Sheypak
    wrote:
    did you try admin/admin?


    2013/8/14 Jaykay <johnk...@gmail.com>

    Thanks Joe. I ended up rolling back to a snapshot, as it's a
    VM. If I want to connect to postgres to look at db sizes, any idea what the
    default postgres login is with a Cloudera install?


    On Wednesday, August 14, 2013 1:00:27 PM UTC+1, Joe Travaglini
    wrote:
    IMO the quickest way to purge space is to find a log file
    that's large.

    du /var/log | sort -rn | less

    echo "" > /path/to/that/file

    This will free space immediately even if a process has an
    open fd to write to that log file. If you simply rm the file while the
    process writing to it is still running, it will not free the disk space.

    On Wed, Aug 14, 2013 at 7:47 AM, Jaykay wrote:

    Too late, all my disk space is gone again, so I can't change
    anything in the GUI. Is there a way to purge the data from
    cloudera-scm-server-db? I would like to connect to the db to see which
    tables are taking up space, but I am out of space now, so psql can't run
    properly.

    On Wednesday, August 14, 2013 7:42:25 AM UTC+1, Gerd Koenig
    wrote:
    Yes, there is a way to configure the min. log level.
    To configure it, e.g. for the Datanodes, navigate to the
    configuration property "DataNode Logging Th reshold" via
    "Service HDFS" => "Configuration" => "View and Edit" => "Default" =>
    "Datanode (Default)" => "Logs"

    br....Gerd....

    On 13 August 2013 16:15, Jaykay wrote:

    Is there a way to set CM to log in less verbose way? I
    notice that I can search the logs, and filter by min. log level, which
    includes info, debug, warn, error, fatal.
    I would like to set it to only record error and fatal.

    On Tuesday, August 13, 2013 1:12:04 PM UTC+1, Serega
    Sheypak wrote:
    We didn't do something special. It should be cleaned in a
    while...


    2013/8/13 Jaykay <johnk...@gmail.com>
    Correction to my earlier post:
    Thanks Serega, it is indeed the
    /var/lib/cloudera-scm-server-**d****************b that
    is taking up all the space.
    Host and Service Monitor expiration periods are both
    set to 7days only. I will change this down to 3days and see how it looks
    then.
    I have changed it down to 3days, but the db is still the
    same size. Do I need to do some kind of purge to get the size down?

    On Tuesday, August 13, 2013 11:31:05 AM UTC+1, Serega
    Sheypak wrote:
    If it loks this way:
    [root@prod-scm run]# du -sh /var/lib/* | sort -h
    4,0K /var/lib/fprint
    4,0K /var/lib/games
    4,0K /var/lib/logrotate.status
    4,0K /var/lib/misc
    4,0K /var/lib/net-snmp
    4,0K /var/lib/ntp
    4,0K /var/lib/random-seed
    4,0K /var/lib/rsyslog
    8,0K /var/lib/dbus
    8,0K /var/lib/plymouth
    8,0K /var/lib/udev
    12K /var/lib/prelink
    12K /var/lib/samba
    12K /var/lib/stateless
    16K /var/lib/dhclient
    16K /var/lib/pgsql
    20K /var/lib/authconfig
    28K /var/lib/polkit-1
    208K /var/lib/alternatives
    868K /var/lib/readahead
    3,3M /var/lib/mlocate
    6,8M /var/lib/yum
    38M /var/lib/rpm
    *90G /var/lib/cloudera-scm-server-db*

    Then you need to tune management service

    Host Monitor Data Expiration Period (hours)
    Service Monitor Data Expiration Period (hours)

    and so on...
    We also had such problem. We are planning to migrate
    management service to standalone VM with standalone PostGre SQL server.


    2013/8/13 Jaykay <johnk...@gmail.com>
    I upgraded my Cloudera Manager server from 4.2 to
    4.6. Since upgrading, I have lost over 20GB in disk space in about a day. I
    assume it is from the logging that comes with the latest version. Any way
    to change it so that logging takes up less disk space?

    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it,
    send an email to scm-users+...@**cloudera.org.


    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it, send
    an email to scm-users+...@cloudera.org.


    --
    Gerd Koenig
    Data Engineer / Big Data Analytics
    ______________________________**______

    YMC AG
    Sonnenstr. 4
    CH-8280 Kreuzlingen
    Switzerland

    Tel +41 (0)71 / 508 24 74
    Fax +41 (0)71 / 560 53 89

    http://twitter.com/gerd_koenig
    www.ymc.ch <http://www.ymc.ch>
    ______________________________**______
    To unsubscribe from this group and stop receiving emails from it, send
    an email to scm-users+unsubscribe@cloudera.org.


    --
    Thanks,
    Darren

    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Jaykay at Aug 21, 2013 at 5:09 pm
    Good to know Darren
    On Wednesday, August 21, 2013 6:05:26 PM UTC+1, Darren Lo wrote:

    Hi Jaykay,

    If you want to retain more data, then you can also consider moving off of
    the embedded db, increasing the space allowed for /var/lib, or moving the
    contents of /var/lib/cloudera-scm-server-db/data to somewhere with more
    space and creating a symlink from the old location to the new one (so CM
    doesn't even know it moved). You should stop Hive, all management services,
    Cloudera Manager, and then the cloudera manager database before moving the
    database.

    Thanks,
    Darren


    On Wed, Aug 21, 2013 at 9:16 AM, Jaykay <johnk...@gmail.com <javascript:>>wrote:
    Sorry, here's the correct sizes of the dbs, amon is the one taking up all
    the space.

    datname | size
    -----------+---------
    template1 | 5408 kB
    template0 | 5408 kB
    postgres | 5408 kB
    scm | 30 MB
    smon | 3025 MB
    amon | 27 GB
    hmon | 937 MB
    rman | 5408 kB
    nav | 5408 kB
    hive | 7062 kB

    That's Activity Monitor database. So I changed a few settings from it. I
    changed Purge Activities Data at This Age and Purge Attempts Data at
    This Age from 14 days to 72 hours, and Activity Monitor Event
    Publication Log Quiet Time Period from 1min to 30mins. So amon is down
    from 27GB to 2.5GB now.
    On Wednesday, August 21, 2013 1:56:29 PM UTC+1, Jaykay wrote:

    Hi Gerd,
    Yes the OS is limiting the size. However, I would like to configure
    Cloudera in such a way as it doesn't use up all my available space with
    just a few days of logs. Just looking for the best approach for this.
    On Wednesday, August 21, 2013 1:46:34 PM UTC+1, Gerd König wrote:

    Jaykay,
    Postgres doesn't offer an option to limit the max. used space (afaik).
    Therefore the limit is the one from the underlying OS and respectively the
    partition-/LVM-size where your database resides.


    2013/8/21 Jaykay <johnk...@gmail.com>
    My mistake Darren. I had only set the logging level at the role level,
    and I was looking at the service level logging settings. I have set them
    all to WARN level now. I also set it to only retain 48 hours of logs in
    each of the monitor services. I noticed another setting to purge job
    details after 14 days though, so I set this to 7 days.
    Under the service level, I don't see anywhere to limit the total logs
    size though. All I can see is a few parts (eg. Reports Manager Max
    Log Size) that limit to 200MB by default. How do I limit the total
    size of cloudera-scm-server-db?

    Here's a list of the top tables by size from PostgreSQL:
    relname | relpages
    ------------------------------**----------+----------
    commands ** | 134
    pg_attribute ** | 68
    pg_proc ** | 54
    pg_depend ** | 48
    pg_toast_16982 ** | 45
    pg_depend_reference_index ** | 40
    pg_depend_depender_index ** | 39
    configs ** | 33
    pg_proc_proname_args_nsp_**index | 26
    pg_statistic ** | 26
    audits ** | 26
    pg_toast_2618 ** | 25
    pg_attribute_relid_attnam_**index | 22
    pg_description ** | 21
    idx_command_parent ** | 18
    idx_command_schedule ** | 16
    pg_description_o_c_o_index ** | 16
    idx_command_host ** | 16
    processes ** | 16
    idx_command_cluster ** | 16
    pg_attribute_relid_attnum_**index | 14


    On Tuesday, August 20, 2013 6:43:49 PM UTC+1, Darren Lo wrote:

    Hi Jaykay,

    I don't know why these configs would have changed, other than a
    manual action from CM user (via UI or API). How did you set them in the
    first place? Did you set them at service-level, or role-level? You can tell
    if you've done a role-level override by looking at the service-level
    configs (click on the Mgmt service, then on Configuration -> View and
    Edit). It will have a note next to any config that has a role-level
    override. It's possible that you set a role-level override before, and now
    when you inspected the config you were looking at service-level, which
    still said INFO. If you have the Enterprise or Trial edition, you can also
    look in the config revision history to see exactly what happened to your
    configs.

    Since you are concerned about the max log size as well as the file
    growth rate, you may want to configure the log settings to decrease the
    size per file or the maximum number of files kept for each service. This
    can be configured in the logs tab for each role.

    Thanks,
    Darren

    On Tue, Aug 20, 2013 at 5:38 AM, Jaykay wrote:

    Hi all,

    I came back to work today after the weekend to see that the
    logs(cloudera-scm-server-db) have gone from 4.5GB to 32GB, which is the
    full amount of free disk space. It seems that the log growth had slowed but
    is has now gone back to unmanageable. When I looked at the monitoring
    service settings, the logging level had reverted back to INFO on all of
    them. I had set it to WARN last week. Any idea what would cause it to
    revert?

    On Wednesday, August 14, 2013 3:31:30 PM UTC+1, Darren Lo wrote:

    If you need a db admin for the embedded postgresql db, that login
    is in /var/lib/cloudera-scm-server-**d****
    b/data/generated_password.**txt.


    On Wed, Aug 14, 2013 at 7:24 AM, Serega Sheypak <
    serega....@gmail.com> wrote:
    you are welcome :)


    2013/8/14 Jaykay <johnk...@gmail.com>

    Thanks again Joe.
    Thanks to all who helped me out here, Sergey and Gerd too. I have
    a lot to learn with this stuff.

    On Wednesday, August 14, 2013 2:04:19 PM UTC+1, Joe Travaglini
    wrote:
    it should be located in /etc/cloudera-scm-server/db.**pr******
    operties.

    On Wed, Aug 14, 2013 at 8:50 AM, Jaykay wrote:

    Yeah default didn't work.

    On Wednesday, August 14, 2013 1:43:47 PM UTC+1, Serega Sheypak
    wrote:
    did you try admin/admin?


    2013/8/14 Jaykay <johnk...@gmail.com>

    Thanks Joe. I ended up rolling back to a snapshot, as it's a
    VM. If I want to connect to postgres to look at db sizes, any idea what the
    default postgres login is with a Cloudera install?


    On Wednesday, August 14, 2013 1:00:27 PM UTC+1, Joe
    Travaglini wrote:
    IMO the quickest way to purge space is to find a log file
    that's large.

    du /var/log | sort -rn | less

    echo "" > /path/to/that/file

    This will free space immediately even if a process has an
    open fd to write to that log file. If you simply rm the file while the
    process writing to it is still running, it will not free the disk space.

    On Wed, Aug 14, 2013 at 7:47 AM, Jaykay wrote:

    Too late, all my disk space is gone again, so I can't
    change anything in the GUI. Is there a way to purge the data from
    cloudera-scm-server-db? I would like to connect to the db to see which
    tables are taking up space, but I am out of space now, so psql can't run
    properly.

    On Wednesday, August 14, 2013 7:42:25 AM UTC+1, Gerd Koenig
    wrote:
    Yes, there is a way to configure the min. log level.
    To configure it, e.g. for the Datanodes, navigate to the
    configuration property "DataNode Logging Th reshold" via
    "Service HDFS" => "Configuration" => "View and Edit" => "Default" =>
    "Datanode (Default)" => "Logs"

    br....Gerd....

    On 13 August 2013 16:15, Jaykay wrote:

    Is there a way to set CM to log in less verbose way? I
    notice that I can search the logs, and filter by min. log level, which
    includes info, debug, warn, error, fatal.
    I would like to set it to only record error and fatal.

    On Tuesday, August 13, 2013 1:12:04 PM UTC+1, Serega
    Sheypak wrote:
    We didn't do something special. It should be cleaned in
    a while...


    2013/8/13 Jaykay <johnk...@gmail.com>
    Correction to my earlier post:
    Thanks Serega, it is indeed the
    /var/lib/cloudera-scm-server-**d****************b
    that is taking up all the space.
    Host and Service Monitor expiration periods are both
    set to 7days only. I will change this down to 3days and see how it looks
    then.
    I have changed it down to 3days, but the db is still
    the same size. Do I need to do some kind of purge to get the size down?

    On Tuesday, August 13, 2013 11:31:05 AM UTC+1, Serega
    Sheypak wrote:
    If it loks this way:
    [root@prod-scm run]# du -sh /var/lib/* | sort -h
    4,0K /var/lib/fprint
    4,0K /var/lib/games
    4,0K /var/lib/logrotate.status
    4,0K /var/lib/misc
    4,0K /var/lib/net-snmp
    4,0K /var/lib/ntp
    4,0K /var/lib/random-seed
    4,0K /var/lib/rsyslog
    8,0K /var/lib/dbus
    8,0K /var/lib/plymouth
    8,0K /var/lib/udev
    12K /var/lib/prelink
    12K /var/lib/samba
    12K /var/lib/stateless
    16K /var/lib/dhclient
    16K /var/lib/pgsql
    20K /var/lib/authconfig
    28K /var/lib/polkit-1
    208K /var/lib/alternatives
    868K /var/lib/readahead
    3,3M /var/lib/mlocate
    6,8M /var/lib/yum
    38M /var/lib/rpm
    *90G /var/lib/cloudera-scm-server-db*

    Then you need to tune management service

    Host Monitor Data Expiration Period (hours)
    Service Monitor Data Expiration Period (hours)

    and so on...
    We also had such problem. We are planning to migrate
    management service to standalone VM with standalone PostGre SQL server.


    2013/8/13 Jaykay <johnk...@gmail.com>
    I upgraded my Cloudera Manager server from 4.2 to
    4.6. Since upgrading, I have lost over 20GB in disk space in about a day. I
    assume it is from the logging that comes with the latest version. Any way
    to change it so that logging takes up less disk space?

    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it,
    send an email to scm-users+...@**cloudera.org.


    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it,
    send an email to scm-users+...@cloudera.org.


    --
    Gerd Koenig
    Data Engineer / Big Data Analytics
    ______________________________**______

    YMC AG
    Sonnenstr. 4
    CH-8280 Kreuzlingen
    Switzerland

    Tel +41 (0)71 / 508 24 74
    Fax +41 (0)71 / 560 53 89

    http://twitter.com/gerd_koenig
    www.ymc.ch <http://www.ymc.ch>
    ______________________________**______
    To unsubscribe from this group and stop receiving emails from it, send
    an email to scm-users+...@cloudera.org <javascript:>.


    --
    Thanks,
    Darren
    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedAug 21, '13 at 12:38p
activeAug 21, '13 at 5:09p
posts6
users3
websitecloudera.com
irc#hadoop

3 users in discussion

Jaykay: 4 posts Darren Lo: 1 post Gerd König: 1 post

People

Translate

site design / logo © 2022 Grokbase