FAQ
I'm using the SimPy package to run simulations. Anyone who's used this
package knows that the way it simulates process concurrency is through the
clever use of yield statements. Some of the code in my programs is very
complex and contains several repeating sequences of yield statements. I
want to combine these sequences into common functions. The problem of
course, is that once a yield gets put into a function, the function is now a
generator and its behavior changes. Is there any elegant way to do this? I
suppose I can do things like ping-pong yield statements, but that solutions
seems even uglier than having a very flat, single main routine with
repeating sequences.

Thanks in advance.

Search Discussions

  • Steven D'Aprano at Jun 5, 2011 at 12:56 am

    On Sat, 04 Jun 2011 14:27:32 -0400, TommyVee wrote:

    I'm using the SimPy package to run simulations. Anyone who's used this
    package knows that the way it simulates process concurrency is through
    the clever use of yield statements. Some of the code in my programs is
    very complex and contains several repeating sequences of yield
    statements. I want to combine these sequences into common functions.
    The problem of course, is that once a yield gets put into a function,
    the function is now a generator and its behavior changes. Is there any
    elegant way to do this?

    I don't quite understand the nature of your problem, but it seems to me
    that it is easily solved by simply not using yield in the common
    functions. Instead of:


    def main():
    for x in sequence:
    if a:
    y = a+b+c+d+e
    yield y
    elif b:
    y = a+b+c+d+f
    yield y
    else:
    y = a+b+c+d
    yield y+1

    (where each of the lines y = ... is meant to be a big block of mostly
    common code), factor out the similar parts into one or more external
    functions:

    def f(a, b, c, d):
    return a+b+c+d # big block of common code

    def main():
    for x in sequence:
    if a:
    y = f(a, b, c, d) # call the common part
    y += e
    yield y
    elif b:
    y = f(a, b, c, d)
    y += f
    yield y
    else:
    y = f(a, b, c, d)
    y += 1
    yield y


    If this is not what you're talking about, an example may help.

    I suppose I can do things like ping-pong yield statements,
    I have no idea what you think that means, but another alternative is to
    loop over the generator output and re-yield it:

    for x in f():
    yield x

    A nice piece of syntax that has been proposed for Python is "yield from",
    which will do the same thing, but you can't use that yet.


    --
    Steven
  • Gregory Ewing at Jun 5, 2011 at 1:43 am

    Steven D'Aprano wrote:

    A nice piece of syntax that has been proposed for Python is "yield from",
    which will do the same thing, but you can't use that yet.
    Unless you're impatient enough to compile your own
    Python with my patch applied:

    http://www.cosc.canterbury.ac.nz/greg.ewing/python/yield-from/yield_from.html

    --
    Greg
  • Jack Diederich at Jun 5, 2011 at 2:06 am

    On Sat, Jun 4, 2011 at 9:43 PM, Gregory Ewing wrote:
    Steven D'Aprano wrote:
    A nice piece of syntax that has been proposed for Python is "yield from",
    which will do the same thing, but you can't use that yet.
    You can also patch the library to always return lists instead of generators.

    def seriously_I_just_want_lists(func):
    def replacement(*args, **kwargs):
    return list(func(*args, **kwargs))
    return replacement

    import otherlib
    otherlib.somefunc = seriously_I_just_want_lists(otherlib.somefunc)

    But I'd avoid it where you can. While the idea of a list is simpler
    than the idea of an iterable (and easier to get your head around)
    iterables promise less and that makes them more flexible and easier to
    composite.

    -Jack
  • TommyVee at Jun 6, 2011 at 12:11 am
    "Gregory Ewing" wrote in message news:95059eFuroU1 at mid.individual.net...

    Steven D'Aprano wrote:
    A nice piece of syntax that has been proposed for Python is "yield from",
    which will do the same thing, but you can't use that yet.
    Unless you're impatient enough to compile your own
    Python with my patch applied:

    http://www.cosc.canterbury.ac.nz/greg.ewing/python/yield-from/yield_from.html

    --
    Greg

    Sounds like the "yield from" is the about this best solution. Thanks for
    all your help.
  • Jan Decaluwe at Jun 5, 2011 at 9:52 am

    On 06/04/2011 08:27 PM, TommyVee wrote:
    I'm using the SimPy package to run simulations. Anyone who's used
    this package knows that the way it simulates process concurrency is
    through the clever use of yield statements. Some of the code in my
    programs is very complex and contains several repeating sequences of
    yield statements. I want to combine these sequences into common
    functions. The problem of course, is that once a yield gets put into
    a function, the function is now a generator and its behavior changes.
    Is there any elegant way to do this? I suppose I can do things like
    ping-pong yield statements, but that solutions seems even uglier than
    having a very flat, single main routine with repeating sequences.
    In MyHDL, this is supported by the possibility to yield generators,
    which are then handled by the simulation engine.

    Jan


    --
    Jan Decaluwe - Resources bvba - http://www.jandecaluwe.com
    Python as a HDL: http://www.myhdl.org
    VHDL development, the modern way: http://www.sigasi.com
    Analog design automation: http://www.mephisto-da.com
    World-class digital design: http://www.easics.com
  • Thomas Rachel at Jun 6, 2011 at 9:07 am

    Am 04.06.2011 20:27 schrieb TommyVee:
    I'm using the SimPy package to run simulations. Anyone who's used this
    package knows that the way it simulates process concurrency is through
    the clever use of yield statements. Some of the code in my programs is
    very complex and contains several repeating sequences of yield
    statements. I want to combine these sequences into common functions.
    Which are then generators.
    The problem of course, is that once a yield gets put into a function,
    the function is now a generator and its behavior changes.
    Isn't your "main" function a generator as well?

    Is there any elegant way to do this? I suppose I can do things like
    ping-pong yield statements, but that solutions seems even uglier than
    having a very flat, single main routine with repeating sequences.
    I'm not sure if I got it right, but I think you could emulate this
    "yield from" with a decorator:

    def subgen1(): yield 1; yield 2;
    def subgen2(): yield 1; yield 2;

    Instead of doing now

    def allgen():
    for i in subgen1(): yield i
    for i in subgen2(): yield i

    you as well could do:

    def yield_from(f):
    def wrapper(*a, **k):
    for sub in f(*a, **k):
    for i in sub:
    yield i
    return wrapper

    @yield_from
    def allgen():
    yield subgen1()
    yield subgen2()

    (Untested.)


    Thomas
  • TommyVee at Jun 8, 2011 at 12:41 am
    "Thomas Rachel" wrote in message news:isi5dk$8h1$1 at r03.glglgl.eu...

    Am 04.06.2011 20:27 schrieb TommyVee:
    I'm using the SimPy package to run simulations. Anyone who's used this
    package knows that the way it simulates process concurrency is through
    the clever use of yield statements. Some of the code in my programs is
    very complex and contains several repeating sequences of yield
    statements. I want to combine these sequences into common functions.
    Which are then generators.
    The problem of course, is that once a yield gets put into a function,
    the function is now a generator and its behavior changes.
    Isn't your "main" function a generator as well?

    Is there any elegant way to do this? I suppose I can do things like
    ping-pong yield statements, but that solutions seems even uglier than
    having a very flat, single main routine with repeating sequences.
    I'm not sure if I got it right, but I think you could emulate this
    "yield from" with a decorator:

    def subgen1(): yield 1; yield 2;
    def subgen2(): yield 1; yield 2;

    Instead of doing now

    def allgen():
    for i in subgen1(): yield i
    for i in subgen2(): yield i

    you as well could do:

    def yield_from(f):
    def wrapper(*a, **k):
    for sub in f(*a, **k):
    for i in sub:
    yield i
    return wrapper

    @yield_from
    def allgen():
    yield subgen1()
    yield subgen2()

    (Untested.)


    Thomas

    Yes, the main function is a generator. Give me a day or two to absorb your
    code. But in the meantime, perhaps a quick explanation of the SimPy package
    is in order. Also, here's the link if you're interested:

    http://simpy.sourceforge.net/simpy_overview.htm

    The way the package works is that you instantiate what I'll call an "actor"
    class, inheriting from one of the SimPy base classes. Once you instantiate
    the actor class(es), it it "registered" in SimPy. Then when you start the
    simulation, SimPy's main routine will begin to "dispatch" each of the actor
    classes by driving what they call a "PEM" method in the actor class. Within
    that method, you simulate the activity of the actor. The way that you tell
    SimPy that you are "busy", or that you want to request or release a
    resource, is with a yield statement. Here's a little example:

    from SimPy.Simulation import *

    svc1 = Resource(capacity=1)
    svc2 = Resource(capacity=1)

    class Customer(Process):
    def PEM(self):

    print now(), "I am starting..."

    print now(), "I am requesting service1"
    yield request, self, svc1

    print now(), "I got service 1, now I'm going ask it to do 10 ticks
    worth of work"
    yield hold, self, 10

    print now(), "Service 1's work is done, I am now releasing him"
    yield release, self, svc1

    print now(), "Now I am requesting service2"
    yield request, self, svc2

    print now(), "I got service 2, now I'm going ask him to do 5 ticks
    worth of work"
    yield hold, self, 5

    print now(), "Service 2's work is done, I am now releasing him"
    yield release, self, svc2

    print now(), "I am ending..."

    initialize()

    # Create a customer and "register" (activate) him to SimPy
    c = Customer()
    activate(c, c.PEM())

    # Pass control to the SimPy main dispatch routine, and run the simulation
    for 100 ticks
    simulate(until0)

    Note that when you do the yields, you actually send back information to the
    SimPy main routine. For example, when you do a "yield hold, self, 10",
    control will be yielded to SimPy, he will simulate the passage of 10 time
    ticks, and redispatch you (do a "next" on your PEM). Similar deal with the
    "yield request, self, svc1". When you yield to SimPy, he will attempt to
    obtain the resource for you, and if it is already taken by another actor,
    you will wait in a queue until released. At that time, SimPy will then
    redispatch you. Obviously, this is a trivial example, since it only
    involves the creation of a single actor. In a real simulation, there could
    be hundreds of these things running "concurrently", all vying for resources,
    holding them for varying amounts of time, etc. SimPy also uses yield
    statements to simulate other things too, like waiting for a signal from
    another actor.

    In the example you see above, note that I am "repeating" a generic sequence
    twice, e.g. "yield request, self, svcX", followed by "yield hold, self,
    time", followed by "yield release, self, svcX". In a workflow example, you
    may have literally dozens of services that you may hit in serial or
    parallel. Because of the generator "restriction", you'd have to code these
    three statements repeatedly in the PEM routine. It would be nice to just
    have a class method which did:

    def UseService(self, svcName, svcTime):
    yield request, self, svcName
    yield hold, self, svcTime
    yield release, self, svcName

    Then you can just call it in your main routine, passing the desired
    parameters (e.g., "UseService(svc1, 10)").

    Anyway, hopefully you'll see that there's a reason for my original question.
    Let me absorb what you sent and see if it makes sense.

    Thanks, Tom
  • Joel at Jun 20, 2011 at 10:04 pm

    On Jun 4, 2:27?pm, "TommyVee" wrote:
    I'm using the SimPy package to run simulations. Anyone who's used this
    package knows that the way it simulates process concurrency is through the
    clever use of yield statements. Some of the code in my programs is very
    complex and contains several repeating sequences of yield statements. ?I
    want to combine these sequences into common functions. ?The problem of
    course, is that once a yield gets put into a function, the function is now a
    generator and its behavior changes. ?Is there any elegant way to do this? ?I
    suppose I can do things like ping-pong yield statements, but that solutions
    seems even uglier than having a very flat, single main routine with
    repeating sequences.

    Thanks in advance.
    I actually found a reasonable answer to this, I think. If one of the
    called functions contains a yield, that function is by definition a
    generator, and will test as such with 'if
    type(result)==types.GeneratorType:'. This makes it possible for the
    function that calls the subroutine to either do its own yield, or to
    yield the result of the function's next 'yield' statement. I've got a
    class the run method of which calls the 'cycle' method of its derived
    class, as follows:

    while True:
    result= self.cycle(now()) # result may or may not be a
    generator
    if type(result)==types.GeneratorType:
    # Next is a generator, meaning it includes a
    'yield'.
    # Otherwise, result should be None and cycle is a
    simple
    # function.
    try:
    yield result.next()
    while True:
    yield result.next()
    except StopIteration:
    pass

    else:
    # self.cycle() was a simple function, returning None-
    it's
    # done now.
    pass
    # It is time for this process to go back to sleep; all the
    yields
    # in self.cycle() have been processed.
    yield hold,self,self.nextCycleTime-now()
  • Terry Reedy at Jun 21, 2011 at 1:42 am

    On 6/20/2011 6:04 PM, Joel wrote:
    On Jun 4, 2:27 pm, "TommyVee"wrote:
    I'm using the SimPy package to run simulations. Anyone who's used this
    package knows that the way it simulates process concurrency is through the
    clever use of yield statements. Some of the code in my programs is very
    complex and contains several repeating sequences of yield statements. I
    want to combine these sequences into common functions. The problem of
    course, is that once a yield gets put into a function, the function is now a
    generator and its behavior changes.
    ...
    I actually found a reasonable answer to this, I think. If one of the
    called functions contains a yield, that function is by definition a
    generator,
    A nomenclature note: a function with yield is a 'generator function'. It
    is an instance of the 'function' class, same as for any other def
    statement (or lambda expression). It returns an instance of class
    'generator, as you note here
    and will test as such with 'if type(result)==types.GeneratorType:'.
    It is the result, and not the function, that is the generator (a type of
    iterator).

    --
    Terry Jan Reedy
  • Joel at Jun 21, 2011 at 3:03 am

    On Jun 20, 9:42?pm, Terry Reedy wrote:
    A nomenclature note: a function with yield is a 'generator function'. It
    is an instance of the 'function' class, same as for any other def
    statement (or lambda expression). It returns an instance of class
    'generator, as you note here

    ?> and will test as such with 'if type(result)==types.GeneratorType:'.

    It is the result, and not the function, that is the generator (a type of
    iterator).

    --
    Terry Jan Reedy
    Yes, quite right. I stand corrected.

    -Joel

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedJun 4, '11 at 6:27p
activeJun 21, '11 at 3:03a
posts11
users8
websitepython.org

People

Translate

site design / logo © 2022 Grokbase