Good morning all CM administrators,
I've a CDH5.0 cluster, installed via CM Agent 5.0. I checked CM's
Parcels window and there are a few downloadable service's upgrade
available; namely CDH5.0.1 and Impala 1.3.1. I wish to download and upgrade
these services via CM but is gated by the Parcel Directory -
"/opt/cloudera/parcels" space issue.
My cluster resides on AWS EC2 and CM installation defaulted each
instance with 10GB local FS space. When cluster first setup and configured,
approximately 5GB or 50% of every instance ( NameNode and DataNode ) local
FS in cluster is used. All is well at this point.
When I perform a service upgrade via CM's Download-Distribute-Activate,
the instances' local FS gets exhausted, 95% because files get created in
folder "/opt/cloudera/parcels" and "/opt/cloudera/parcel-cache/". I
understand that upon activating a service upgrade, I can safely remove
files in "/opt/cloudera/parcel-cache/" but even with this cleanup, my
instances' local FS utilization is still up around 80%+. Impacts of this
scenario are 1) prevent efficient workflow execution, 2 ) no more space to
upgrade other services and 3 ) CM services' monitoring constantly showing
RED status due to local instance space low issue.
My question is :
1 - In order to alleviate the instance local FS space restriction issue,
I'm considering mount a new disk to each instance, NameNode and DataNode(s)
and set in CM, Parcel Directory to this new mount disk. For example
"/newmountdisk/cloudera/parcels". Hence all service upgrade package
distribution, files creation and activation will now on this new mount
disk. I will keep instance local FS for service logging only and staging
area for workflow processing. Will this configuration works ?
Note : Upgrading to CDH5.0.1 via CM Parcel prevented my cluster's SQOOP
service from starting up due to some JDBC upgrade not allowable if my
memory didn't fail me.
To unsubscribe from this group and stop receiving emails from it, send an email to email@example.com.