FAQ
Has anybody thought about adding Gearman support to the fpm sapi?
Managing Gearman workers is pretty much identical to managing fastcgi
workers, so it doesn't seem like much of a stretch.

-Rasmus

Search Discussions

  • Reilly, Patrick at Jun 10, 2010 at 3:15 pm
    Rasmus,

    I can work on it if no one has it already.

    Warm Regards,
    Patrick
    On Thu, Jun 10, 2010 at 7:56 AM, Rasmus Lerdorf wrote:
    Has anybody thought about adding Gearman support to the fpm sapi?
    Managing Gearman workers is pretty much identical to managing fastcgi
    workers, so it doesn't seem like much of a stretch.

    -Rasmus

    --
    PHP Internals - PHP Runtime Development Mailing List
    To unsubscribe, visit: http://www.php.net/unsub.php
  • Brian Moon at Jun 10, 2010 at 3:20 pm
    The big difference is that different processes do different jobs in
    Gearman. All PHP workers in fpm just run what ever code is handed to
    them. How do you handle that?


    Brian.
    --------
    http://brian.moonspot.net/
    On 6/10/10 9:56 AM, Rasmus Lerdorf wrote:
    Has anybody thought about adding Gearman support to the fpm sapi?
    Managing Gearman workers is pretty much identical to managing fastcgi
    workers, so it doesn't seem like much of a stretch.

    -Rasmus
  • Frederic Hardy at Jun 10, 2010 at 3:30 pm

    On 06/10/2010 17:20, Brian Moon wrote:
    The big difference is that different processes do different jobs in
    Gearman. All PHP workers in fpm just run what ever code is handed to
    them. How do you handle that?
    Threading ;) ?
    On 6/10/10 9:56 AM, Rasmus Lerdorf wrote:
    Has anybody thought about adding Gearman support to the fpm sapi?
    Managing Gearman workers is pretty much identical to managing fastcgi
    workers, so it doesn't seem like much of a stretch.
    Best regards,
    Fred

    --
    ========================================================================
    Frédéric Hardy : Architecte d'application/Admin. système/Ergonome
    Status : En recherche d'emploi
    CV : http://blog.mageekbox.net/public/cv.frederic.hardy.pdf
    Blog : http://blog.mageekbox.net
    Twitter : http://twitter.com/mageekguy
    ========================================================================
  • Rasmus Lerdorf at Jun 10, 2010 at 3:46 pm

    On 6/10/10 8:30 AM, Frederic Hardy wrote:
    On 06/10/2010 17:20, Brian Moon wrote:
    The big difference is that different processes do different jobs in
    Gearman. All PHP workers in fpm just run what ever code is handed to
    them. How do you handle that?
    Threading ;) ?
    Definitely not. Gearman is actually the answer to a lot of problems
    where people think they need threads.

    As far as different workers doing different jobs, I don't see the issue
    here. fpm doesn't need to know the details. All it has to know is that
    it needs to maintain pools of php processes. Each pool would correspond
    to a Gearman job. The workers can take care of registering themselves
    with gearmand. All fpm has to do is start up and maintain N processes
    per worker pool.

    [gearman]
    worker1.path = /path/to/worker1.php
    worker1.pm = dynamic
    worker1.pm.max_children = 10
    worker1.pm.start_servers = 5

    worker2.path = /path/to/worker2.php
    worker2.pm = dynamic
    worker2.pm.max_children = 4
    worker2.pm.start_servers = 2

    ...

    or however you do multiple nested conf blocks in the fpm .ini file.

    In fact, this may not need to be gearman-specific. We could just call
    it cli job management or something like that if we left it completely up
    to the workers to register with the gearmand. Alternatively, we could
    make fpm Gearman-aware and register the workers from there, but I am not
    sure I see much of an advantage to that other than making the worker
    code slightly simpler.

    -Rasmus
  • Brian Moon at Jun 10, 2010 at 4:31 pm

    [gearman]
    worker1.path = /path/to/worker1.php
    worker1.pm = dynamic
    worker1.pm.max_children = 10
    worker1.pm.start_servers = 5

    worker2.path = /path/to/worker2.php
    worker2.pm = dynamic
    worker2.pm.max_children = 4
    worker2.pm.start_servers = 2
    So, this is a proposed new entry in the ini format? I am a bit ignorant
    about fpm, I admit. This would require a good bit of fiddling for our
    gearman set up to work with it. We run X workers that know how to do all
    jobs available and then 1 + n (defined by ini file) that only do one job
    for each available job. This gives us good coverage with controlled
    process load. I guess it would just require a lot more configuration
    than we have now. For example, could we pass parameters to the script?
    Having a separate script plus a separate class for every function
    quickly makes a lot of files.

    I will simply stay tuned and see where it goes.

    FWIW, this is my GearmanManager that basically does this already via
    PHP+pcntl. http://github.com/brianlmoon/GearmanManager. It can not
    dynamically ramp up more workers like FPM can. But, I would never run it
    that way anyhow. I like my processes under control.

    Brian.
  • Rasmus Lerdorf at Jun 10, 2010 at 4:38 pm

    On 6/10/10 9:31 AM, Brian Moon wrote:
    [gearman]
    worker1.path = /path/to/worker1.php
    worker1.pm = dynamic
    worker1.pm.max_children = 10
    worker1.pm.start_servers = 5

    worker2.path = /path/to/worker2.php
    worker2.pm = dynamic
    worker2.pm.max_children = 4
    worker2.pm.start_servers = 2
    So, this is a proposed new entry in the ini format? I am a bit ignorant
    about fpm, I admit. This would require a good bit of fiddling for our
    gearman set up to work with it. We run X workers that know how to do all
    jobs available and then 1 + n (defined by ini file) that only do one job
    for each available job. This gives us good coverage with controlled
    process load. I guess it would just require a lot more configuration
    than we have now. For example, could we pass parameters to the script?
    Having a separate script plus a separate class for every function
    quickly makes a lot of files.

    I will simply stay tuned and see where it goes.

    FWIW, this is my GearmanManager that basically does this already via
    PHP+pcntl. http://github.com/brianlmoon/GearmanManager. It can not
    dynamically ramp up more workers like FPM can. But, I would never run it
    that way anyhow. I like my processes under control.
    Yes, I know, I have seen it. Process management and signal handling
    like that really isn't something that should be done in PHP as far as I
    am concerned. And FPM already does all of this and more. It just needs
    a slight tweak to be able to manage any sort of php process.

    -Rasmus
  • Tjerk Anne Meesters at Jun 10, 2010 at 3:53 pm
    All "fpm" workers would be called using the same function name (e.g.
    "runphp"). This allows you to reuse a gearmand instance for other
    workers too.

    The advantages of fpm vs gearman would be mostly performance (it uses
    ping and requires an underlying layer on top of tcp/ip), followed by
    dynamic spawning and "crash recovery". Think there are more but I
    can't access the website from over here ;)

    On 6/10/10, Brian Moon wrote:
    The big difference is that different processes do different jobs in
    Gearman. All PHP workers in fpm just run what ever code is handed to
    them. How do you handle that?


    Brian.
    --------
    http://brian.moonspot.net/
    On 6/10/10 9:56 AM, Rasmus Lerdorf wrote:
    Has anybody thought about adding Gearman support to the fpm sapi?
    Managing Gearman workers is pretty much identical to managing fastcgi
    workers, so it doesn't seem like much of a stretch.

    -Rasmus
    --
    PHP Internals - PHP Runtime Development Mailing List
    To unsubscribe, visit: http://www.php.net/unsub.php

    --
    --
    Tjerk
  • Rasmus Lerdorf at Jun 10, 2010 at 4:00 pm

    On 6/10/10 8:53 AM, Tjerk Anne Meesters wrote:
    All "fpm" workers would be called using the same function name (e.g.
    "runphp"). This allows you to reuse a gearmand instance for other
    workers too.
    I don't think we would need to require that. Obviously you could build
    your Gearman worker such that it registers "runphp" and has a big switch
    in it to run various types of jobs and fpm would just need to manage a
    single pool of workers. But, assuming fpm could manage multiple pools,
    I don't see the problem with having different types of workers.

    -Rasmus

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupphp-internals @
categoriesphp
postedJun 10, '10 at 2:56p
activeJun 10, '10 at 4:38p
posts9
users5
websitephp.net

People

Translate

site design / logo © 2022 Grokbase