FAQ
Hi kulkarni

This is expected. The parcel was already and still is activated
throughout the process, so as soon as distribution completes to the
newly added hosts, it is back to activated.

Herman

On Wed, Mar 5, 2014 at 9:04 AM, kulkarni.swarnim@gmail.com
wrote:
I think I found out what's going on. So it seems like when a parcel is
redistributed, it goes directly from a DISTRIBUTING stage to ACTIVATED
stage.

Current parcel stage: [ACTIVATED]
Current parcel stage: [ACTIVATED] <----- Was already activated on the
cluster
Current parcel stage: [DISTRIBUTING] <----- Redistribution starts on new
hosts
Current parcel stage: [DISTRIBUTING]
Current parcel stage: [DISTRIBUTING]
Current parcel stage: [DISTRIBUTING]
Current parcel stage: [DISTRIBUTING]
<snip>
Current parcel stage: [DISTRIBUTING]
Current parcel stage: [DISTRIBUTING]
Current parcel stage: [DISTRIBUTING]
Current parcel stage: [DISTRIBUTING]
Current parcel stage: [ACTIVATED] <----- ??
Current parcel stage: [ACTIVATED]
Current parcel stage: [ACTIVATED]
Current parcel stage: [ACTIVATED]

My current code waits for it to move to a DISTRIBUTED state first before
activating it. Since it never achieves that stage, it just sits there
waiting.

Is this something that should be expected or is a bug expected to be fixed
in later CM versions?

Thanks,


On Tue, Mar 4, 2014 at 4:24 PM, kulkarni.swarnim@gmail.com
wrote:
I'll try it out. Thanks Vikram for taking time to help me out.

On Tue, Mar 4, 2014 at 4:14 PM, Vikram Bajaj wrote:

No, it does not at least in my scripts. I do have a script to add new
dataNodes and it works as expected. i.e it distributes the parcel to the new
hosts and activates it.

You can test it out, by adding a new host from the cloudera manager and
then check the parcel status from the cloudera manager parcels link.


On Tue, Mar 4, 2014 at 2:11 PM, kulkarni.swarnim@gmail.com
wrote:
Thanks Vikram.

So I am actually consuming the Java API but my code is very similar to
what you have.

So in this case, if I call the distribute_parcel method again to
redistribute them, wouldn't the "parcel = cluster.get_parcel(product,
version)" return state of parcel as ACTIVATED and then would directly break
out of the loop? Or am I missing something. This is the exact problem I am
facing.

Thanks again.


On Tue, Mar 4, 2014 at 3:52 PM, Vikram Bajaj <paphillon@gmail.com>
wrote:
This should help

def distribute_parcel(cluster, product, version):



# If available remotely, start the downloading the parcel

cmd = None

while True:

parcel = cluster.get_parcel(product, version)

LOG.info('Parcel Status: %s' %(parcel.stage))

if parcel.stage == 'UNAVAILABLE':

LOG.info('Giving CM time to find parcel information')

LOG.info('Parcel Status: %s' %(parcel.stage))

if parcel.stage == 'AVAILABLE_REMOTELY':

LOG.info('Beginning Parcel %s : %s download.' %(product,
version))

cmd = parcel.start_download()

if parcel.stage == 'DOWNLOADING':

LOG.info("download progress: %s / %s" %
(parcel.state.progress, parcel.state.totalProgress))

if parcel.stage == 'DOWNLOADED':

LOG.info("Starting distribution of %s parcel version %s to
cluster %s" % (product, version, cluster.name))

cmd = parcel.start_distribution()

if parcel.stage == 'DISTRIBUTING':

LOG.info("distribution progress: %s / %s" %
(parcel.state.progress, parcel.state.totalProgress))

if parcel.stage == 'DISTRIBUTED':

LOG.info("Starting activation of %s parcel version %s for
cluster %s" % (product, version, cluster.name))

cmd = parcel.activate()

if parcel.stage == 'ACTIVATED':

LOG.info("activated - %s parcel version %s on cluster %s" %
(product, version, cluster.name))

break

if cmd:

if not cmd.wait(CMD_TIMEOUT).success:

LOG.critical("PAR:501 Time out during parcel operation.
Exiting now. %s " %(cmd.resultMessage))

sys.exit(501)

if parcel.state.errors:

LOG.critical('PAR:500 Exception in parcel operation %s
Message %s' %(parcel.state.errors, cmd.resultMessage))

sys.exit(500)

time.sleep(15) # check again in 15 seconds



On Tue, Mar 4, 2014 at 1:04 PM, kulkarni.swarnim@gmail.com
wrote:
Thanks Vikram. This is exactly what my expectation was as well.

The point where I got stuck with this approach is since all these
parcel commands(download, distribute, activate) are async in nature, in case
of redistribution how do we ensure their successful completion? Even if the
parcel is getting download/distributed only on new hosts, the state of
parcel still remains "Activated" since it was previously activated on other
hosts.

Thanks,


On Tue, Mar 4, 2014 at 2:51 PM, Vikram Bajaj <paphillon@gmail.com>
wrote:
You have to re distribute the parcel, just like you did for the other
hosts. If the hosts already have the parcel and is activated it will not
distribute it to them, but only to the new hosts

parcel = cluster.get_parcel(product, version)

cmd = parcel.start_download()

parcel.start_distribution()

cmd = parcel.activate()


Regards,

Vikram



On Tue, Mar 4, 2014 at 12:47 PM, kulkarni.swarnim@gmail.com
wrote:
Hello,

First of all thanks for the prompt response I have been getting from
this community. Really appreciate it.

I have a use case where I am trying to use the cloudera manager API
to add a new host to an existing cluster that already has bunch of parcels
deployed and activated. I am able to add the new host and assign roles to it
but not able to start them because it doesn't have those parcels deployed.
Is there anything in the API that will help to deploy parcels on a specific
host? I hunted for it in the API for quite a while but couldn't find one.
All I could find was using the ParcelsResource to deploy them on the cluster
itself.

Thanks,

--
Swarnim

To unsubscribe from this group and stop receiving emails from it,
send an email to scm-users+unsubscribe@cloudera.org.


--
Swarnim


--
Swarnim


--
Swarnim



--
Swarnim

To unsubscribe from this group and stop receiving emails from it, send an
email to scm-users+unsubscribe@cloudera.org.
To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

Search Discussions

Discussion Posts

Previous

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 9 of 9 | next ›
Discussion Overview
groupscm-users @
categorieshadoop
postedMar 4, '14 at 8:47p
activeMar 5, '14 at 6:47p
posts9
users3
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase