There's a couple of separate issues bundled up in your question, which I'll
try and separate out to answer.
1) Can I set up a Hadoop cluster using just puppet?
Of course you can, but this isn't the forum for that. This is for
discussion of Cloudera Manager and how you
might use it to achieve various things. If you really want to install CDH
without using Cloudera Manager, the
cdh-user mailing list would be a good resource.
2) What's this puppet-scm thing?
puppet-scm is/was a set of puppet recipes for installing Cloudera Manager
itself. Once that's done, you would
use Cloudera Manager to complete the set up of the Hadoop cluster. However,
puppet-scm is stale and will
not correctly set up the latest versions of CM, so it's not a turn-key
3) Can I use Cloudera Manager to set up a cluster without using the UI for
The short answer is yes, but you have to get CM installed first. The
scm-installer.bin program can be
automated with a few command line arguments to get the server installed,
but installing the CM agent
on your cluster nodes has to be done with the UI or with an external tool
like puppet, which brings us back
Once you've addressed that, one way or another, you can use the CM APIhttp://cloudera.github.com/cm_api/
to automate setting up of services, roles, configs, etc.
3b) Note that in CM releases prior to 4.5, you are also responsible for
installing CDH packages if you cannot
or won't use the wizard UI - there is no API automation for this. In 4.5,
it becomes possible to do through the
parcel mechanism which is exposed through the API. This is all part of the
v3 API which we may not have
properly published the python bindings for yet.
Hope that helps,
On 10 January 2013 08:23, Alex McLintock wrote:
I've joined this list some weeks ago to learn about Cloudera Manager,
but I don't want to be doing everything through a GUI.
Ideally I want my Hadoop cluster set up through Puppet. I see that
Cloudera have some puppet configs called puppet-scm.
Is this the right place to discuss them? I've not heard of ANYONE
using them yet...