Dear all,

I'm design a process that is splitted on some tasks (one queue for each
piece of process). To a sample, suppose I have a message that can treated
for many steps: Retrieve Information, enrich information, Save on Database,
but each step can occurs or not, so I've thought about on two approaches :

1 - Orchestration: In this case a central component will receive the
message and decide for each step, after execution of each step this
component will decide the next step.
2 - Choreography: In this case each step must to know the next step and
send message for next queue

Does anybody has some opinion about the better approach? Does anybody knows
some tool/framework that works top of Rabbit that manage it (Java if
possible) ?


Best regards


--
Isa?as Barroso
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120107/1cc8bf5b/attachment.htm>

Search Discussions

  • Ciprian Dorin Craciun at Jan 8, 2012 at 2:02 pm

    On Sat, Jan 7, 2012 at 22:38, Isaias Barroso wrote:
    Dear all,

    I'm design a process that is splitted on some tasks (one queue for each
    piece of process). To a sample, suppose I have a message that can treated
    for many steps: Retrieve Information, enrich information, Save on Database,
    but each step can occurs or not, so I've thought about on two approaches :

    1 - Orchestration: In this case a central component will receive the message
    and decide for each step, after execution of each step this component will
    decide the next step.
    2 - Choreography: In this case each step must to know the next step and send
    message for next queue

    Does anybody has some opinion about the better approach? Does anybody knows
    some tool/framework that works top of Rabbit that manage it (Java if
    possible) ?

    Best regards
    --
    Isa?as Barroso

    I've also pondered about the same topic a while ago... (Though I
    haven't tried it in practice.) As such my options were the following:

    a) (the easiest one) Go with the choreography approach, but
    instead of naming the queue of the next service (and send to a direct
    exchange), just send it to an exchange named by the next step's
    purpose (like "enrich", "validate", etc.). Maybe make this a topic
    exchange and put enough information in the routing key to allow
    "debugging" by being able to register another queue and "capture" the
    traffic.

    b) (the "declarative" approach) Why not have the orchestration
    approach (in a declarative flavor) combined with choreography:
    * let's assume that you have a task which goes through a set of
    processes (transformations or just validations, etc.) which can be
    described as decision tree (a binary tree of actions, where each node
    is a process and the two children (one for success and one for
    failure) are the next process to be executed);
    * now you just put this decision tree inside the "envelope" of
    each message, send it to the root process;
    * this process executes its logic and based on the outcome
    (success or failure), extracts one of it's subtrees, puts that in the
    new envelope and together with the new message sends it to the root of
    the new tree; (evidently at some point the new tree would be empty,
    which means it should stop sending it further;)
    * (you could again use "purpose" names for exchanges instead of
    direct queues;)
    * you could extend this to a graph (maybe containing cycles, thus
    obtaining loops), by saying that each node has two links to other
    nodes;
    * of course yet another extension would be to allow "labeled" node
    links, i.e. each process outputs two things: a next-step string label,
    and the new message, and now you select from the labeled links the one
    matching the label; (maybe have something as a default link);

    Thus with option b) you obtain both the advantages of choreography
    (i.e. no middle man to become a single point of failure or
    bottleneck), but also the advantages of orchestration (i.e. each
    message having it's own "route" through the processes). At the same
    time you keep the processors "dumb", none knowing of another.

    Hope this helps.
    Ciprian.

    P.S.: If you like this approach (b), you could go even further and
    instead of using a static decision tree (or graph) you could
    substitute that with an embedded language or machine which allows
    generic operations on both the old and new message to decide the next
    actions. (I could expand on this if someone is interested.)

    A practical (not very elegant) Java approach:
    * define a Java interface with a method like `public AmqpTarget
    decide (MyMessage old, MyMessage new)` (or something similar), where
    `AmqpTarget` has two fields for exchange and routing key;
    a) if all the processes share the same code and are upgraded at
    the same time, just instantiate a serializable object implementing
    that interface and put that in the envelope instead of the decision
    tree;
    b) if you don't have shared code or you want to keep your options opened:
    * for each different "workflow" define a new public "top" class
    (i.e. not nested or anonymous) implementing that interface with as few
    dependencies as possible;
    * in the envelope instead of the decision tree put the byte-code
    of this class (you can use the
    `MyWorkflowClass.class.getResourceStream(MyWorkflowClass.class.getCanonicalName())`
    and the canonical name; (be careful to use caching or you'll get
    `PermGenError` or similar;)
    * to instantiate the class on the receiver just create a custom
    classloader which when asked for the bytecode of that known class just
    returns the bytes previously stored;
  • Isaias Barroso at Jan 8, 2012 at 2:46 pm
    Hi Ciprian,


    I've also pondered about the same topic a while ago... (Though I
    haven't tried it in practice.) As such my options were the following:

    a) (the easiest one) Go with the choreography approach, but
    instead of naming the queue of the next service (and send to a direct
    exchange), just send it to an exchange named by the next step's
    purpose (like "enrich", "validate", etc.). Maybe make this a topic
    exchange and put enough information in the routing key to allow
    "debugging" by being able to register another queue and "capture" the
    traffic.
    Yes, I agree with you, send to a exchange is more apropriate

    b) (the "declarative" approach) Why not have the orchestration
    approach (in a declarative flavor) combined with choreography:
    * let's assume that you have a task which goes through a set of
    processes (transformations or just validations, etc.) which can be
    described as decision tree (a binary tree of actions, where each node
    is a process and the two children (one for success and one for
    failure) are the next process to be executed);
    * now you just put this decision tree inside the "envelope" of
    each message, send it to the root process;
    * this process executes its logic and based on the outcome
    (success or failure), extracts one of it's subtrees, puts that in the
    new envelope and together with the new message sends it to the root of
    the new tree; (evidently at some point the new tree would be empty,
    which means it should stop sending it further;)
    * (you could again use "purpose" names for exchanges instead of
    direct queues;)
    * you could extend this to a graph (maybe containing cycles, thus
    obtaining loops), by saying that each node has two links to other
    nodes;
    * of course yet another extension would be to allow "labeled" node
    links, i.e. each process outputs two things: a next-step string label,
    and the new message, and now you select from the labeled links the one
    matching the label; (maybe have something as a default link);

    Thus with option b) you obtain both the advantages of choreography
    (i.e. no middle man to become a single point of failure or
    bottleneck), but also the advantages of orchestration (i.e. each
    message having it's own "route" through the processes). At the same
    time you keep the processors "dumb", none knowing of another.
    Very interesting this approach, and better the possibility to eliminate a
    possible SPOF.


    Hope this helps.
    Ciprian.

    P.S.: If you like this approach (b), you could go even further and
    instead of using a static decision tree (or graph) you could
    substitute that with an embedded language or machine which allows
    generic operations on both the old and new message to decide the next
    actions. (I could expand on this if someone is interested.)

    I think that (b) approach can be more flexible for large and complex
    process and it sounds good.

    Maybe this topic can generate good discussions and patterns for complex
    process management.

    Best regards



    A practical (not very elegant) Java approach:
    * define a Java interface with a method like `public AmqpTarget
    decide (MyMessage old, MyMessage new)` (or something similar), where
    `AmqpTarget` has two fields for exchange and routing key;
    a) if all the processes share the same code and are upgraded at
    the same time, just instantiate a serializable object implementing
    that interface and put that in the envelope instead of the decision
    tree;
    b) if you don't have shared code or you want to keep your options
    opened:
    * for each different "workflow" define a new public "top" class
    (i.e. not nested or anonymous) implementing that interface with as few
    dependencies as possible;
    * in the envelope instead of the decision tree put the byte-code
    of this class (you can use the

    `MyWorkflowClass.class.getResourceStream(MyWorkflowClass.class.getCanonicalName())`
    and the canonical name; (be careful to use caching or you'll get
    `PermGenError` or similar;)
    * to instantiate the class on the receiver just create a custom
    classloader which when asked for the bytecode of that known class just
    returns the bytes previously stored;


    --
    Isa?as Barroso
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120108/4e56504b/attachment.htm>
  • Isaias Barroso at Jan 8, 2012 at 2:52 pm
    Hi Ciprian,

    I've seen that Spring Integration with AMQP module can be a solution for
    Orchestration, I'll do a deep analysis on it. But I think is more restrict
    that mixed approach (b)

    Best regards


    On Sun, Jan 8, 2012 at 12:46 PM, Isaias Barroso wrote:

    Hi Ciprian,


    I've also pondered about the same topic a while ago... (Though I
    haven't tried it in practice.) As such my options were the following:

    a) (the easiest one) Go with the choreography approach, but
    instead of naming the queue of the next service (and send to a direct
    exchange), just send it to an exchange named by the next step's
    purpose (like "enrich", "validate", etc.). Maybe make this a topic
    exchange and put enough information in the routing key to allow
    "debugging" by being able to register another queue and "capture" the
    traffic.
    Yes, I agree with you, send to a exchange is more apropriate

    b) (the "declarative" approach) Why not have the orchestration
    approach (in a declarative flavor) combined with choreography:
    * let's assume that you have a task which goes through a set of
    processes (transformations or just validations, etc.) which can be
    described as decision tree (a binary tree of actions, where each node
    is a process and the two children (one for success and one for
    failure) are the next process to be executed);
    * now you just put this decision tree inside the "envelope" of
    each message, send it to the root process;
    * this process executes its logic and based on the outcome
    (success or failure), extracts one of it's subtrees, puts that in the
    new envelope and together with the new message sends it to the root of
    the new tree; (evidently at some point the new tree would be empty,
    which means it should stop sending it further;)
    * (you could again use "purpose" names for exchanges instead of
    direct queues;)
    * you could extend this to a graph (maybe containing cycles, thus
    obtaining loops), by saying that each node has two links to other
    nodes;
    * of course yet another extension would be to allow "labeled" node
    links, i.e. each process outputs two things: a next-step string label,
    and the new message, and now you select from the labeled links the one
    matching the label; (maybe have something as a default link);

    Thus with option b) you obtain both the advantages of choreography
    (i.e. no middle man to become a single point of failure or
    bottleneck), but also the advantages of orchestration (i.e. each
    message having it's own "route" through the processes). At the same
    time you keep the processors "dumb", none knowing of another.
    Very interesting this approach, and better the possibility to eliminate a
    possible SPOF.


    Hope this helps.
    Ciprian.

    P.S.: If you like this approach (b), you could go even further and
    instead of using a static decision tree (or graph) you could
    substitute that with an embedded language or machine which allows
    generic operations on both the old and new message to decide the next
    actions. (I could expand on this if someone is interested.)

    I think that (b) approach can be more flexible for large and complex
    process and it sounds good.

    Maybe this topic can generate good discussions and patterns for complex
    process management.

    Best regards



    A practical (not very elegant) Java approach:
    * define a Java interface with a method like `public AmqpTarget
    decide (MyMessage old, MyMessage new)` (or something similar), where
    `AmqpTarget` has two fields for exchange and routing key;
    a) if all the processes share the same code and are upgraded at
    the same time, just instantiate a serializable object implementing
    that interface and put that in the envelope instead of the decision
    tree;
    b) if you don't have shared code or you want to keep your options
    opened:
    * for each different "workflow" define a new public "top" class
    (i.e. not nested or anonymous) implementing that interface with as few
    dependencies as possible;
    * in the envelope instead of the decision tree put the byte-code
    of this class (you can use the

    `MyWorkflowClass.class.getResourceStream(MyWorkflowClass.class.getCanonicalName())`
    and the canonical name; (be careful to use caching or you'll get
    `PermGenError` or similar;)
    * to instantiate the class on the receiver just create a custom
    classloader which when asked for the bytecode of that known class just
    returns the bytes previously stored;


    --
    Isa?as Barroso

    --
    Isa?as Barroso
    Consultor SAP NetWeaver / Oracle (SOA / ADF) / JEE
    Belo Horizonte - MG
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120108/0b70a406/attachment.htm>

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprabbitmq-discuss @
categoriesrabbitmq
postedJan 7, '12 at 8:38p
activeJan 8, '12 at 2:52p
posts4
users2
websiterabbitmq.com
irc#rabbitmq

People

Translate

site design / logo © 2022 Grokbase