FAQ
Let's say I have this Python file called loop.py:

import sys
print 'hi'
sys.stdout.flush()
while 1:
pass

And I want to call it from another Python process and read the value
'hi'. How would I do it?

So far I have tried this:
proc = subprocess.Popen('python /home/chiefinnovator/loop.py',shell=True,stdin=subprocess.PIPE,stdout=subprocess.PIPE)
proc.stdout.read()
But it just hangs at read()

proc.communicate() also just hangs. What am I doing wrong? Please
advise.

Thanks,

Greg


From http Sun Sep 23 05:46:21 2007
From: http (Paul Rubin)
Date: 22 Sep 2007 20:46:21 -0700
Subject: Getting rid of bitwise operators in Python 3?
References: <46f49740$0$32514$4c368faf@roadrunner.com>
<miaJi.28220$eY.3184@newssvr13.news.prodigy.net>
<1190484589.890198.266430@y42g2000hsy.googlegroups.com>
<13fbahtem90fp99@corp.supernews.com>
<1190506229.092718.41690@g4g2000hsf.googlegroups.com>
<7xps0achlg.fsf@ruckus.brouhaha.com>
<1190517836.766461.29570@g4g2000hsf.googlegroups.com>
Message-ID: <7x8x6y3uya.fsf@ruckus.brouhaha.com>

richyjsm at gmail.com writes:
Well okay, I take that back---I wouldn't mind *writing* it; I just
wouldn't expect to get much speed from *running* it.
I don't see why not. It's just the basic Python long arithmetic which
is coded in C. I tested it against gmpy (carefully written asm code
and tuned algorithms) and gmpy's modexp was around 4x faster, but
Python's is plenty fast enough for many purposes. The 4x ratio is
certainly lower than the usual ratio between Python code and C code.

Search Discussions

  • Gregpinero at Sep 23, 2007 at 4:16 am

    On Sep 22, 11:28 pm, "gregpin... at gmail.com" wrote:
    Let's say I have this Python file called loop.py:

    import sys
    print 'hi'
    sys.stdout.flush()
    while 1:
    pass

    And I want to call it from another Python process and read the value
    'hi'. How would I do it?

    So far I have tried this:
    proc = subprocess.Popen('python /home/chiefinnovator/loop.py',shell=True,stdin=subprocess.PIPE,stdout=subprocess.PIPE)
    proc.stdout.read()
    But it just hangs at read()

    proc.communicate() also just hangs. What am I doing wrong? Please
    advise.
    Well, using this subclass of subprocess.Popen fixes the problem
    (http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440554)

    I don't understand how it works though. Would anyone mind
    explaining? I'm thinking there's something fundamental about Unix
    processes I'm not understanding.

    -Greg
  • Lawrence D'Oliveiro at Sep 24, 2007 at 2:57 am
    In message <1190521003.928634.48900 at d55g2000hsg.googlegroups.com>,
    gregpinero at gmail.com wrote:
    Well, using this subclass of subprocess.Popen fixes the problem
    (http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440554)

    I don't understand how it works though. Would anyone mind
    explaining? I'm thinking there's something fundamental about Unix
    processes I'm not understanding.
    The "select" calls are checking that the subprocess is ready to read before
    writing to its stdin, or that it has something to write before reading from
    its stdout.

    The O_NONBLOCK fcntl simply says not to wait if there is nothing more to be
    read, just return what was already read.
  • Suresh Babu Kolla at Sep 23, 2007 at 5:57 am

    gregpinero at gmail.com wrote:
    Let's say I have this Python file called loop.py:

    import sys
    print 'hi'
    sys.stdout.flush()
    while 1:
    pass

    And I want to call it from another Python process and read the value
    'hi'. How would I do it?

    So far I have tried this:
    proc = subprocess.Popen('python /home/chiefinnovator/loop.py',shell=True,stdin=subprocess.PIPE,stdout=subprocess.PIPE)
    proc.stdout.read()
    From python documentation

    `read([size])'
    Read at most SIZE bytes from the file (less if the read hits `EOF'
    before obtaining SIZE bytes). If the SIZE argument is negative or
    omitted, read all data until `EOF' is reached. The bytes are
    returned as a string object. An empty string is returned when
    `EOF' is encountered immediately. (For certain files, like ttys,
    it makes sense to continue reading after an `EOF' is hit.) Note
    that this method may call the underlying C function `fread()' more
    than once in an effort to acquire as close to SIZE bytes as
    possible. Also note that when in non-blocking mode, less data than
    what was requested may be returned, even if no SIZE parameter was
    given.

    read call in your code is waiting for EOF, since the script never exits
    EOF is not reached.

    Change read code to

    proc.stdout.readline()

    or

    remove while 1 loop from loop.py.

    HTH
    Kolla
  • Karthik Gurusamy at Sep 23, 2007 at 6:58 am

    On Sep 22, 8:28 pm, "gregpin... at gmail.com" wrote:
    Let's say I have this Python file called loop.py:

    import sys
    print 'hi'
    sys.stdout.flush()
    Add sys.stdout.close()
    while 1:
    pass

    And I want to call it from another Python process and read the value
    'hi'. How would I do it?

    So far I have tried this:
    proc = subprocess.Popen('python /home/chiefinnovator/loop.py',shell=True,stdin=subprocess.PIPE,stdout=subprocess.PIPE)
    proc.stdout.read()
    But it just hangs at read()

    proc.communicate() also just hangs. What am I doing wrong? Please
    advise.
    Since your loop.py is still alive and hasn't closed its stdout, the
    caller continues to wait for EOF (it doesn't know if loop.py is done
    generating all its output)

    Karthik
    Thanks,

    Greg
  • Gregpinero at Sep 24, 2007 at 9:22 pm

    On Sep 23, 2:58 am, Karthik Gurusamy wrote:
    On Sep 22, 8:28 pm, "gregpin... at gmail.com" wrote:

    Let's say I have this Python file called loop.py:
    import sys
    print 'hi'
    sys.stdout.flush()
    Add sys.stdout.close()
    Adding sys.stdout.close() and removing sys.stdout.flush() seems to
    make it work. But can the while loop still use sys.stdout later on?
    Do I have to reopen it?

    Thanks,

    Greg
  • Karthik Gurusamy at Sep 25, 2007 at 2:42 am

    On Sep 24, 2:22 pm, "gregpin... at gmail.com" wrote:
    On Sep 23, 2:58 am, Karthik Gurusamy wrote:

    On Sep 22, 8:28 pm, "gregpin... at gmail.com" <gregpin... at gmail.com>
    wrote:
    Let's say I have this Python file called loop.py:
    import sys
    print 'hi'
    sys.stdout.flush()
    Add sys.stdout.close()
    Adding sys.stdout.close() and removing sys.stdout.flush() seems to
    make it work. But can the while loop still use sys.stdout later on?
    Do I have to reopen it?
    Once you close a file-object, you cannot use it. You'll get exception
    if you try.
    I quickly tried the fdopen on id 1 (stdout in unix like OS) and it
    seems to work
    import sys
    sys.stdout.close()
    import os
    print 'hi'
    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    ValueError: I/O operation on closed file
    sys.stdout=os.fdopen(1, 'w')
    print 'hi'
    hi
    >>>

    BUT you may want to re-think your design. Note that your caller
    process will anyway stop reading further from your loop.py process
    the moment it sees the "first" EOF. So even if you enable loop.py to
    generate more output (thru' the above fdopen), the caller process is
    not going to see the newly generated data.

    If your requirement is to collect all output from loop.py then you
    can't do it if loop.py has an infinite loop across its data generation
    points (ie it can generate data after the infinite loop -- which
    anyway doesn't sense).

    If all you want is not to get blocked, try one of the select solutions
    or read a small amount at a time (which you can guarantee to be
    available). Yet another solution would be you could set up an alarm
    and get out of the blocking read if the alarm fires.

    Karthik
    Thanks,

    Greg
  • Lawrence D'Oliveiro at Sep 24, 2007 at 2:53 am
    In message <1190518129.356844.242710 at 19g2000hsx.googlegroups.com>,
    gregpinero at gmail.com wrote:
    Let's say I have this Python file called loop.py:

    import sys
    print 'hi'
    sys.stdout.flush()
    while 1:
    pass

    And I want to call it from another Python process and read the value
    'hi'. How would I do it?

    So far I have tried this:
    proc = subprocess.Popen('python
    /home/chiefinnovator/loop.py',shell=True,stdin=subprocess.PIPE,stdout=subprocess.PIPE)
    proc.stdout.read()
    But it just hangs at read()
    That's because you didn't tell it how much to read, so by default it tries
    to read until EOF <http://docs.python.org/lib/bltin-file-objects.html>. But
    since the subprocess is still running and hasn't closed its stdout, the
    pipe has not reached EOF.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedSep 23, '07 at 3:28a
activeSep 25, '07 at 2:42a
posts8
users4
websitepython.org

People

Translate

site design / logo © 2022 Grokbase