FAQ
Hello all,

Before I ask the question a couple of notes :

* This question is for implementing a script inside the Wing IDE. For
some reason using the subprocess module doesn't work so I need a
solution that doesn't use this module.
* The platform is Windows and I'm happy with a Windoze only solution.
:-)

I would like to execute subprocesses asynchronously and capture stdout
/ stderr as a single stream.

If I use "os.popen3(executable)" it gives me separate pipes for stdout
and stderr, but reads from them are blocking.

The output on stdout and stderr may be interleaved and I would like to
display them *as* they arrive. That means I can't just read from them
and output the results.

The only solution I can think of is to read from both a character at a
time on two separate threads, putting the data into a queue. A separate
thread could pull characters off the queue and display them. (I don't
need to differentiate between stdout and stderr when I display.)

Can anyone think of a better solution ?

My current code works, but *doesn't* capture stderr :

from threading import Thread

pipe = os.popen(executable)

def DisplayOutput():
while True:
output = pipe.read(1)
if not output:
break
display(output)

Thread(target=DisplayOutput).start()

All the best,


Fuzzyman
http://www.voidspace.org.uk/python/articles.shtml

Search Discussions

  • Gabriel Genellina at Jan 7, 2007 at 8:17 pm

    On 7 ene, 16:33, "Fuzzyman" wrote:
    Hello all,

    Before I ask the question a couple of notes :

    * This question is for implementing a script inside the Wing IDE. For
    some reason using the subprocess module doesn't work so I need a
    solution that doesn't use this module.
    * The platform is Windows and I'm happy with a Windoze only solution.
    :-)

    I would like to execute subprocesses asynchronously and capture stdout
    / stderr as a single stream.

    If I use "os.popen3(executable)" it gives me separate pipes for stdout
    and stderr, but reads from them are blocking.

    The output on stdout and stderr may be interleaved and I would like to
    display them *as* they arrive. That means I can't just read from them
    and output the results.

    The only solution I can think of is to read from both a character at a
    time on two separate threads, putting the data into a queue. A separate
    thread could pull characters off the queue and display them. (I don't
    need to differentiate between stdout and stderr when I display.)

    Can anyone think of a better solution ?

    My current code works, but *doesn't* capture stderr :

    from threading import Thread

    pipe = os.popen(executable)

    def DisplayOutput():
    while True:
    output = pipe.read(1)
    if not output:
    break
    display(output)

    Thread(target=DisplayOutput).start()

    All the best,

    Fuzzymanhttp://www.voidspace.org.uk/python/articles.shtml
    Try using popen4 instead. But since you already have a thread, it may
    be better to use popen2 and two threads to read from stdout and stderr
    to avoid a potential deadlock; they can put read lines into a Queue,
    and DisplayOutput just get these lines in order. (See the warnings in
    the popen2 module documentation).

    --
    Gabriel Genellina
  • Fuzzyman at Jan 7, 2007 at 8:37 pm

    Gabriel Genellina wrote:
    On 7 ene, 16:33, "Fuzzyman" wrote:
    [snip..]
    My current code works, but *doesn't* capture stderr :

    from threading import Thread

    pipe = os.popen(executable)

    def DisplayOutput():
    while True:
    output = pipe.read(1)
    if not output:
    break
    display(output)

    Thread(target=DisplayOutput).start()

    All the best,

    Fuzzyman
    http://www.voidspace.org.uk/python/articles.shtml
    Try using popen4 instead. But since you already have a thread, it may
    be better to use popen2 and two threads to read from stdout and stderr
    to avoid a potential deadlock; they can put read lines into a Queue,
    and DisplayOutput just get these lines in order. (See the warnings in
    the popen2 module documentation).
    popen4 works great, I didn't even know it existed.

    Two threads and a queue sounds horrible.

    Thanks

    Fuzzyman
    http://www.voidspace.org.uk/python/articles.shtml
    --
    Gabriel Genellina
  • Gabriel Genellina at Jan 7, 2007 at 8:44 pm

    On 7 ene, 17:37, "Fuzzyman" wrote:

    Two threads and a queue sounds horrible.
    But unfortunately it's the only way if you don't control how the child
    process behaves.
    (It's not soooo ugly afterwards... certainly would be worse if you had
    to syncronize both reading threads and the display thread using
    semaphores by hand.)

    --
    Gabriel Genellina

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedJan 7, '07 at 7:33p
activeJan 7, '07 at 8:44p
posts4
users2
websitepython.org

2 users in discussion

Fuzzyman: 2 posts Gabriel Genellina: 2 posts

People

Translate

site design / logo © 2022 Grokbase