FAQ
Hi,

I'm currently using Oozie workflow to trigger a mapreduce job.

Does Hue/Oozie provide an API or a web service to be able to track the
progress of the workflow in terms of percentage that is shown on it's UI ?

Like 33% complete ... and so on.

Any help is appreciated.

To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+unsubscribe@cloudera.org.

Search Discussions

  • Abraham Elmahrek at Dec 12, 2013 at 7:51 am
    Hey there,

    Take a look at http://oozie.apache.org/docs/3.3.2/WebServicesAPI.html for
    documentation on Oozie's web service layer. Here is how Hue calculates
    progress of a workflow:
    https://github.com/cloudera/hue/blob/master/desktop/libs/liboozie/src/liboozie/types.py#L258
    .

    Hue actually updates its UI dynamically over AJAX and a REST/JSON API. You
    might able to hook into that? It should be 'GET
    /oozie/list_oozie_workflow/<job id>?format=json'.

    -Abe

    On Wed, Dec 11, 2013 at 11:34 PM, wrote:

    Hi,

    I'm currently using Oozie workflow to trigger a mapreduce job.

    Does Hue/Oozie provide an API or a web service to be able to track the
    progress of the workflow in terms of percentage that is shown on it's UI ?

    Like 33% complete ... and so on.

    Any help is appreciated.

    To unsubscribe from this group and stop receiving emails from it, send an
    email to hue-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+unsubscribe@cloudera.org.
  • Gokhale, Saurabh at Dec 12, 2013 at 8:55 pm
    Hi,

    Thanks for your reply.

    I checked the Oozie client API. I couldn't find an API which will provide
    me with list of WorkflowActions, when I input the workflow id.

    Could you please point me to that API ?

    Thanks.





    Regards,
    Saurabh Gokhale


    On Thu, Dec 12, 2013 at 9:16 PM, Romain Rigaux wrote:

    You could get from the Oozie Job Id, the list of actions of the Workflow,
    then in each Action the MR Job ID if there is one: externalId field

    Notice that is it non trivial to report an overall progress of the
    workflow as some actions won't necessarily be ran. Oozie does not give an
    indicator out of the box.

    Romain

    On Thu, Dec 12, 2013 at 4:07 AM, HadoopDev wrote:

    Hello,

    Thanks for the reply.

    I'm currently using Java Oozie Client. All I have is the access to
    workflow Id.

    But to be able to get the job progress through #mapprogress(), or
    #reduceprogress() methods of JobClient, I would require the actual job id
    (not the workflow id)

    Is there any API that can give me actual hadoop job id by inputting oozie
    workflow id ?



    On Thursday, 12 December 2013 13:21:36 UTC+5:30, abe wrote:

    Hey there,

    Take a look at http://oozie.apache.org/docs/3.3.2/WebServicesAPI.htmlfor documentation on Oozie's web service layer. Here is how Hue calculates
    progress of a workflow: https://github.com/cloudera/hue/blob/master/
    desktop/libs/liboozie/src/liboozie/types.py#L258.

    Hue actually updates its UI dynamically over AJAX and a REST/JSON API.
    You might able to hook into that? It should be 'GET
    /oozie/list_oozie_workflow/<job id>?format=json'.

    -Abe

    On Wed, Dec 11, 2013 at 11:34 PM, wrote:

    Hi,

    I'm currently using Oozie workflow to trigger a mapreduce job.

    Does Hue/Oozie provide an API or a web service to be able to track the
    progress of the workflow in terms of percentage that is shown on it's UI ?

    Like 33% complete ... and so on.

    Any help is appreciated.

    To unsubscribe from this group and stop receiving emails from it, send
    an email to hue-user+u...@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to hue-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+unsubscribe@cloudera.org.
  • Romain Rigaux at Dec 12, 2013 at 9:00 pm
    I am talking about the Oozie REST API, I am not sure for the Oozie Client
    API but I did not see it on the CLI.

    Romain

    On Thu, Dec 12, 2013 at 12:54 PM, Gokhale, Saurabh wrote:

    Hi,

    Thanks for your reply.

    I checked the Oozie client API. I couldn't find an API which will provide
    me with list of WorkflowActions, when I input the workflow id.

    Could you please point me to that API ?

    Thanks.





    Regards,
    Saurabh Gokhale


    On Thu, Dec 12, 2013 at 9:16 PM, Romain Rigaux wrote:

    You could get from the Oozie Job Id, the list of actions of the Workflow,
    then in each Action the MR Job ID if there is one: externalId field

    Notice that is it non trivial to report an overall progress of the
    workflow as some actions won't necessarily be ran. Oozie does not give an
    indicator out of the box.

    Romain

    On Thu, Dec 12, 2013 at 4:07 AM, HadoopDev wrote:

    Hello,

    Thanks for the reply.

    I'm currently using Java Oozie Client. All I have is the access to
    workflow Id.

    But to be able to get the job progress through #mapprogress(), or
    #reduceprogress() methods of JobClient, I would require the actual job id
    (not the workflow id)

    Is there any API that can give me actual hadoop job id by inputting
    oozie workflow id ?



    On Thursday, 12 December 2013 13:21:36 UTC+5:30, abe wrote:

    Hey there,

    Take a look at http://oozie.apache.org/docs/3.3.2/WebServicesAPI.htmlfor documentation on Oozie's web service layer. Here is how Hue calculates
    progress of a workflow: https://github.com/cloudera/hue/blob/master/
    desktop/libs/liboozie/src/liboozie/types.py#L258.

    Hue actually updates its UI dynamically over AJAX and a REST/JSON API.
    You might able to hook into that? It should be 'GET
    /oozie/list_oozie_workflow/<job id>?format=json'.

    -Abe

    On Wed, Dec 11, 2013 at 11:34 PM, wrote:

    Hi,

    I'm currently using Oozie workflow to trigger a mapreduce job.

    Does Hue/Oozie provide an API or a web service to be able to track the
    progress of the workflow in terms of percentage that is shown on it's UI ?

    Like 33% complete ... and so on.

    Any help is appreciated.

    To unsubscribe from this group and stop receiving emails from it, send
    an email to hue-user+u...@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it,
    send an email to hue-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+unsubscribe@cloudera.org.
  • Robert Kanter at Dec 12, 2013 at 10:38 pm
    This should be doable by either the Java or REST APIs, as well as the CLI
    (in fact, the CLI uses the Java API which uses the REST API). There are
    obviously some differences in formatting and usage though.

    To get the list of WorkflowActions form a Workflow Job ID in the Java
    Client API, you can do:
    List<WorkflowAction> = OozieClient.getJobInfo(String JobId).getActions();

    Here is the Javadoc for the Client API:
    http://archive.cloudera.com/cdh4/cdh/4/oozie/client/apidocs/index.html

    The equivalent call for the REST API is shown here:
    http://archive.cloudera.com/cdh4/cdh/4/oozie/WebServicesAPI.html#Job_Information

    The equivalent call for the CLI is shown here (though you’d have to do a
    bunch of parsing to get specific info as its designed to be human-readable):
    http://archive.cloudera.com/cdh4/cdh/4/oozie/DG_CommandLineTool.html#Checking_the_Status_of_a_Workflow_Coordinator_or_Bundle_Job_or_a_Coordinator_Action


    - Robert

    On Thu, Dec 12, 2013 at 1:00 PM, Romain Rigaux wrote:

    I am talking about the Oozie REST API, I am not sure for the Oozie Client
    API but I did not see it on the CLI.

    Romain


    On Thu, Dec 12, 2013 at 12:54 PM, Gokhale, Saurabh <
    saurabh.gokhale1@gmail.com> wrote:
    Hi,

    Thanks for your reply.

    I checked the Oozie client API. I couldn't find an API which will provide
    me with list of WorkflowActions, when I input the workflow id.

    Could you please point me to that API ?

    Thanks.





    Regards,
    Saurabh Gokhale


    On Thu, Dec 12, 2013 at 9:16 PM, Romain Rigaux wrote:

    You could get from the Oozie Job Id, the list of actions of the
    Workflow, then in each Action the MR Job ID if there is one: externalId
    field

    Notice that is it non trivial to report an overall progress of the
    workflow as some actions won't necessarily be ran. Oozie does not give an
    indicator out of the box.

    Romain

    On Thu, Dec 12, 2013 at 4:07 AM, HadoopDev wrote:

    Hello,

    Thanks for the reply.

    I'm currently using Java Oozie Client. All I have is the access to
    workflow Id.

    But to be able to get the job progress through #mapprogress(), or
    #reduceprogress() methods of JobClient, I would require the actual job id
    (not the workflow id)

    Is there any API that can give me actual hadoop job id by inputting
    oozie workflow id ?



    On Thursday, 12 December 2013 13:21:36 UTC+5:30, abe wrote:

    Hey there,

    Take a look at http://oozie.apache.org/docs/3.3.2/WebServicesAPI.htmlfor documentation on Oozie's web service layer. Here is how Hue calculates
    progress of a workflow: https://github.com/cloudera/hue/blob/master/
    desktop/libs/liboozie/src/liboozie/types.py#L258.

    Hue actually updates its UI dynamically over AJAX and a REST/JSON API.
    You might able to hook into that? It should be 'GET
    /oozie/list_oozie_workflow/<job id>?format=json'.

    -Abe

    On Wed, Dec 11, 2013 at 11:34 PM, wrote:

    Hi,

    I'm currently using Oozie workflow to trigger a mapreduce job.

    Does Hue/Oozie provide an API or a web service to be able to track
    the progress of the workflow in terms of percentage that is shown on it's
    UI ?

    Like 33% complete ... and so on.

    Any help is appreciated.

    To unsubscribe from this group and stop receiving emails from it,
    send an email to hue-user+u...@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it,
    send an email to hue-user+unsubscribe@cloudera.org.
    --

    ---
    You received this message because you are subscribed to the Google Groups
    "CDH Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to cdh-user+unsubscribe@cloudera.org.
    For more options, visit
    https://groups.google.com/a/cloudera.org/groups/opt_out.
    To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+unsubscribe@cloudera.org.
  • Gokhale, Saurabh at Dec 13, 2013 at 9:29 am
    Thanks Robert. I used the same approach and is now working fine.

    Thank you all for your help and support.




    On Fri, Dec 13, 2013 at 4:08 AM, Robert Kanter wrote:

    This should be doable by either the Java or REST APIs, as well as the CLI
    (in fact, the CLI uses the Java API which uses the REST API). There are
    obviously some differences in formatting and usage though.

    To get the list of WorkflowActions form a Workflow Job ID in the Java
    Client API, you can do:
    List<WorkflowAction> = OozieClient.getJobInfo(String JobId).getActions();

    Here is the Javadoc for the Client API:
    http://archive.cloudera.com/cdh4/cdh/4/oozie/client/apidocs/index.html

    The equivalent call for the REST API is shown here:
    http://archive.cloudera.com/cdh4/cdh/4/oozie/WebServicesAPI.html#Job_Information

    The equivalent call for the CLI is shown here (though you’d have to do a
    bunch of parsing to get specific info as its designed to be human-readable):

    http://archive.cloudera.com/cdh4/cdh/4/oozie/DG_CommandLineTool.html#Checking_the_Status_of_a_Workflow_Coordinator_or_Bundle_Job_or_a_Coordinator_Action


    - Robert

    On Thu, Dec 12, 2013 at 1:00 PM, Romain Rigaux wrote:

    I am talking about the Oozie REST API, I am not sure for the Oozie Client
    API but I did not see it on the CLI.

    Romain


    On Thu, Dec 12, 2013 at 12:54 PM, Gokhale, Saurabh <
    saurabh.gokhale1@gmail.com> wrote:
    Hi,

    Thanks for your reply.

    I checked the Oozie client API. I couldn't find an API which will
    provide me with list of WorkflowActions, when I input the workflow id.

    Could you please point me to that API ?

    Thanks.





    Regards,
    Saurabh Gokhale


    On Thu, Dec 12, 2013 at 9:16 PM, Romain Rigaux wrote:

    You could get from the Oozie Job Id, the list of actions of the
    Workflow, then in each Action the MR Job ID if there is one: externalId
    field

    Notice that is it non trivial to report an overall progress of the
    workflow as some actions won't necessarily be ran. Oozie does not give an
    indicator out of the box.

    Romain

    On Thu, Dec 12, 2013 at 4:07 AM, HadoopDev wrote:

    Hello,

    Thanks for the reply.

    I'm currently using Java Oozie Client. All I have is the access to
    workflow Id.

    But to be able to get the job progress through #mapprogress(), or
    #reduceprogress() methods of JobClient, I would require the actual job id
    (not the workflow id)

    Is there any API that can give me actual hadoop job id by inputting
    oozie workflow id ?



    On Thursday, 12 December 2013 13:21:36 UTC+5:30, abe wrote:

    Hey there,

    Take a look at http://oozie.apache.org/docs/3.3.2/WebServicesAPI.htmlfor documentation on Oozie's web service layer. Here is how Hue calculates
    progress of a workflow: https://github.com/cloudera/hue/blob/master/
    desktop/libs/liboozie/src/liboozie/types.py#L258.

    Hue actually updates its UI dynamically over AJAX and a REST/JSON
    API. You might able to hook into that? It should be 'GET
    /oozie/list_oozie_workflow/<job id>?format=json'.

    -Abe

    On Wed, Dec 11, 2013 at 11:34 PM, wrote:

    Hi,

    I'm currently using Oozie workflow to trigger a mapreduce job.

    Does Hue/Oozie provide an API or a web service to be able to track
    the progress of the workflow in terms of percentage that is shown on it's
    UI ?

    Like 33% complete ... and so on.

    Any help is appreciated.

    To unsubscribe from this group and stop receiving emails from it,
    send an email to hue-user+u...@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it,
    send an email to hue-user+unsubscribe@cloudera.org.
    --

    ---
    You received this message because you are subscribed to the Google Groups
    "CDH Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to cdh-user+unsubscribe@cloudera.org.
    For more options, visit
    https://groups.google.com/a/cloudera.org/groups/opt_out.
    To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+unsubscribe@cloudera.org.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouphue-user @
categorieshadoop
postedDec 12, '13 at 7:34a
activeDec 13, '13 at 9:29a
posts6
users4
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase