FAQ
I think I found out what's going on. So it seems like when a parcel is
redistributed, it goes directly from a DISTRIBUTING stage to ACTIVATED
stage.

Current parcel stage: [ACTIVATED]
Current parcel stage: [ACTIVATED] <----- Was already activated on the
cluster
Current parcel stage: [DISTRIBUTING] <----- Redistribution starts on new
hosts
Current parcel stage: [DISTRIBUTING]
Current parcel stage: [DISTRIBUTING]
Current parcel stage: [DISTRIBUTING]
Current parcel stage: [DISTRIBUTING]
<snip>
Current parcel stage: [DISTRIBUTING]
Current parcel stage: [DISTRIBUTING]
Current parcel stage: [DISTRIBUTING]
Current parcel stage: [DISTRIBUTING]
Current parcel stage: [ACTIVATED] <----- ??
Current parcel stage: [ACTIVATED]
Current parcel stage: [ACTIVATED]
Current parcel stage: [ACTIVATED]

My current code waits for it to move to a DISTRIBUTED state first before
activating it. Since it never achieves that stage, it just sits there
waiting.

Is this something that should be expected or is a bug expected to be fixed
in later CM versions?

Thanks,

On Tue, Mar 4, 2014 at 4:24 PM, kulkarni.swarnim@gmail.com wrote:

I'll try it out. Thanks Vikram for taking time to help me out.

On Tue, Mar 4, 2014 at 4:14 PM, Vikram Bajaj wrote:

No, it does not at least in my scripts. I do have a script to add new
dataNodes and it works as expected. i.e it distributes the parcel to the
new hosts and activates it.

You can test it out, by adding a new host from the cloudera manager and
then check the parcel status from the cloudera manager parcels link.


On Tue, Mar 4, 2014 at 2:11 PM, kulkarni.swarnim@gmail.com <
kulkarni.swarnim@gmail.com> wrote:
Thanks Vikram.

So I am actually consuming the Java API but my code is very similar to
what you have.

So in this case, if I call the distribute_parcel method again to
redistribute them, wouldn't the "parcel = cluster.get_parcel(product,
version)" return state of parcel as ACTIVATED and then would directly break
out of the loop? Or am I missing something. This is the exact problem I am
facing.

Thanks again.

On Tue, Mar 4, 2014 at 3:52 PM, Vikram Bajaj wrote:

This should help

def distribute_parcel(cluster, product, version):



# If available remotely, start the downloading the parcel

cmd = None

while True:

parcel = cluster.get_parcel(product, version)

LOG.info('Parcel Status: %s' %(parcel.stage))

if parcel.stage == 'UNAVAILABLE':

LOG.info('Giving CM time to find parcel information')

LOG.info('Parcel Status: %s' %(parcel.stage))

if parcel.stage == 'AVAILABLE_REMOTELY':

LOG.info('Beginning Parcel %s : %s download.' %(product,
version))

cmd = parcel.start_download()

if parcel.stage == 'DOWNLOADING':

LOG.info("download progress: %s / %s" %
(parcel.state.progress, parcel.state.totalProgress))

if parcel.stage == 'DOWNLOADED':

LOG.info("Starting distribution of %s parcel version %s to
cluster %s" % (product, version, cluster.name))

cmd = parcel.start_distribution()

if parcel.stage == 'DISTRIBUTING':

LOG.info("distribution progress: %s / %s" %
(parcel.state.progress, parcel.state.totalProgress))

if parcel.stage == 'DISTRIBUTED':

LOG.info("Starting activation of %s parcel version %s for
cluster %s" % (product, version, cluster.name))

cmd = parcel.activate()

if parcel.stage == 'ACTIVATED':

LOG.info("activated - %s parcel version %s on cluster %s"% (product, version,
cluster.name))

break

if cmd:

if not cmd.wait(CMD_TIMEOUT).success:

LOG.critical("PAR:501 Time out during parcel
operation. Exiting now. %s " %(cmd.resultMessage))

sys.exit(501)

if parcel.state.errors:

LOG.critical('PAR:500 Exception in parcel operation %s
Message %s' %(parcel.state.errors, cmd.resultMessage))

sys.exit(500)

time.sleep(15) # check again in 15 seconds


On Tue, Mar 4, 2014 at 1:04 PM, kulkarni.swarnim@gmail.com <
kulkarni.swarnim@gmail.com> wrote:
Thanks Vikram. This is exactly what my expectation was as well.

The point where I got stuck with this approach is since all these
parcel commands(download, distribute, activate) are async in nature, in
case of redistribution how do we ensure their successful completion? Even
if the parcel is getting download/distributed only on new hosts, the state
of parcel still remains "Activated" since it was previously activated on
other hosts.

Thanks,

On Tue, Mar 4, 2014 at 2:51 PM, Vikram Bajaj wrote:

You have to re distribute the parcel, just like you did for the other
hosts. If the hosts already have the parcel and is activated it will not
distribute it to them, but only to the new hosts

parcel = cluster.get_parcel(product, version)

cmd = parcel.start_download()

parcel.start_distribution()

cmd = parcel.activate()


Regards,

Vikram


On Tue, Mar 4, 2014 at 12:47 PM, kulkarni.swarnim@gmail.com <
kulkarni.swarnim@gmail.com> wrote:
Hello,

First of all thanks for the prompt response I have been getting from
this community. Really appreciate it.

I have a use case where I am trying to use the cloudera manager API
to add a new host to an existing cluster that already has bunch of parcels
deployed and activated. I am able to add the new host and assign roles to
it but not able to start them because it doesn't have those parcels
deployed. Is there anything in the API that will help to deploy parcels on
a specific host? I hunted for it in the API for quite a while but couldn't
find one. All I could find was using the ParcelsResource to deploy them on
the cluster itself.

Thanks,

--
Swarnim

To unsubscribe from this group and stop receiving emails from it,
send an email to scm-users+unsubscribe@cloudera.org.

--
Swarnim

--
Swarnim

--
Swarnim


--
Swarnim

To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 8 of 9 | next ›
Discussion Overview
groupscm-users @
categorieshadoop
postedMar 4, '14 at 8:47p
activeMar 5, '14 at 6:47p
posts9
users3
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase