Grokbase › Groups › Python › tutor › May 2005
FAQ
Hi,
I am working on getting comfortable with Python, and am trying to rewrite
some of my old (clumsy) bash scripts into python.

I am having a little problem with how to pipe a variable's contents to an
external command. Essentially, I have a variable that contains the
contents of a variable that I want to feed to a series of external commands
(clamav and spamassassin).

EMAIL contains the full email message, and CLAMAV contains the path to the
clamscan bin (and necessary flags). I have tested this on the command line
and it works (so I know the command syntax is correct), but in my python
script it is clear that nothing is getting piped to the scanner.

CLAMAV_h= os.popen(CLAMAV, 'w')
CLAMAV_h.write(EMAIL)
CLAM_RESULT=CLAMAV_h.close()
print CLAM_RESULT

I can make this work by writing to a temp file and scanning that, but I
would like to avoid having to write to and read from a temp file at every
step. I expect I am making a very basic error, stemming from my tackling a
(perhaps?) more advanced problem at this early stage. But one can only get
so far with "Hello World"....

Many thanks,

Jeff


* * * * * * *
Jeffrey Rice || jeffrice at finity.org || www.finity.org

Search Discussions

  • Danny Yoo at May 12, 2005 at 6:16 am ⇧

    On Wed, 11 May 2005, Jeffrey Rice wrote:


    I am having a little problem with how to pipe a variable's contents to an
    external command. Essentially, I have a variable that contains the
    contents of a variable that I want to feed to a series of external commands
    (clamav and spamassassin).
    Hi Jeffrey,

    Ok, let's take a look.

    CLAMAV_h= os.popen(CLAMAV, 'w')
    CLAMAV_h.write(EMAIL)
    CLAM_RESULT=CLAMAV_h.close()
    print CLAM_RESULT
    Ah, ok. The value of the 'close()' here isn't defined, so you probably
    won't get anything. Let me double check that:

    ######
    import os
    f = os.popen('ls')
    result = f.close()
    type(result)
    <type 'NoneType'>
    ######


    Yeah, no value. By the way, we can also look this up by reading the
    documentation on file-like objects:

    http://www.python.org/doc/lib/bltin-file-objects.html


    Anyway, so trying to get a result from close() is a no-go. But when we
    open up a process, we can get back both an "input" file and an "output
    file". os.popen() just gives us the output stream, since unidirectional
    communication is the common case. But there are variations of popen that
    can give us both:

    http://www.python.org/doc/lib/module-popen2.html

    You're probably looking for 'popen2', which gives us both streams.
    Here's an example with the word-counting Unix program 'wc':

    ######
    child_out, child_in = popen2.popen2("wc")
    child_in.write("hello world\nthis is a test\ngoodbye!\n")
    child_in.close()
    result = child_out.read()
    child_out.close()
    result
    ' 3 7 36\n'
    ######

    Three lines, seven words, and thirty-six characters. Sounds right.
    *grin*


    That being said, if you haven't used either popen2 or subprocess set, you
    may want to try subprocess first, as there are some subtleties to using
    popen2 that aren't obvious at first.

    (Some of these subtleties are documented in:
    http://www.python.org/doc/lib/popen2-flow-control.html)

    The 'subprocess' module is described here:

    http://www.python.org/doc/lib/module-subprocess.html

    and should be fairly easy to work with. I haven't yet installed Python
    2.4 on my home machine, so I'll need to delegate to someone else on the
    Tutor list here for a subprocess example. *grin*


    If you have more questions, please feel free to ask!
  • Jeffrey Rice at May 15, 2005 at 6:41 pm ⇧

    At 04:17 PM 5/13/2005, Alan Gauld wrote:
    AS a general pont all uppercase usually means a consrtant in Python.
    Variable names typically start lowercase and either use capital
    letters to separate words likeThis, or use underscores like_this.

    Variables starting with an uppercase letter tend to indicate class
    names - although thats not universally true, even in the standard
    library! But the ALLUPPER thing for constants is pretty universal
    so its worth adopting IMHO.
    Thanks for the style pointers. I will make these changes -- I realize how
    important consistency is, since I use other people's code as a learning
    tool with great frequency.
    So what happens? Do you get any error messages?
    What value is CLAM_RESULT showing?
    What value are you expecting? (The actual return value from
    popen/close is not clearly defined in general.)
    Finally, although you close the pipe it may be worth trying
    a flush() first just to clear the buffer.
    I have gotten this working, with much help (especially from Danny). Here
    is what the code I settled on looks like:

    ###
    working = email.message_from_file(open(emailfile))

    clamav = popen2.Popen3(clamav_cmd,1)
    clamav.tochild.write(working.as_string())
    clamav.tochild.close()
    clamav_result = clamav.fromchild.read().lstrip('stream: ')
    clamav.fromchild.close
    clamav_exit = os.WEXITSTATUS(clamav.wait())
    ###

    This gives me a string (clamav_result) that contains the output from the
    scanner: either OK if it was clean, or the name of the virus
    detected. (it is prefixed with "stream: " to indicate that stdin was
    scanned, so I strip that off)
    clamav_exit contains the exit code, converted using WEXITSTATUS: either 0,
    1, or 2 for OK, infected, or error respectively.

    This seems to be working out just fine, and gives me a lot more flexibility
    that the bash script I was using! (as a note, I am reading the email into
    an email object because I perform other actions, like adding
    headers. Otherwise, it would be simpler to have clamav read the file
    directly rather than the stream.)

    Thanks to everyone for your patience and advice!

    Jeff

    * * * * * * *
    Jeffrey Rice || jeffrice at finity.org || www.finity.org
  • Danny Yoo at May 15, 2005 at 10:29 pm ⇧

    Here is what the code I settled on looks like:

    ###
    working = email.message_from_file(open(emailfile))

    clamav = popen2.Popen3(clamav_cmd,1)
    clamav.tochild.write(working.as_string())
    clamav.tochild.close()
    clamav_result = clamav.fromchild.read().lstrip('stream: ')
    clamav.fromchild.close
    ^^^^^^^^^^^^^^^^^^^^^^


    Hi Jeffrey,

    Careful: you may need parens there on your 'close' line. (clothesline?
    *grin*)

    As written, that last statement doesn't fire off the close() method.
    This might be suprising because some other languages make parens optional.
    But we need them in Python because Python makes it very easy to get
    function references:

    ######
    def calculatePi():
    ... return 22/7
    ...
    calculatePi
    <function calculatePi at 0x402dae64>
    ######

    Without parens, we just get the function reference. We still need to
    "call" that function value by using parens, which trigger the function to
    fire off:

    ######
    calculatePi()
    3
    ######


    Otherwise, your program looks good! I'm glad that it's working for you
    now.


    Best of wishes!
  • Jeffrey Rice at May 16, 2005 at 12:18 am ⇧

    At 04:29 PM 5/15/2005, Danny Yoo wrote:
    clamav.fromchild.close
    ^^^^^^^^^^^^^^^^^^^^^^
    Careful: you may need parens there on your 'close' line. (clothesline?
    *grin*)

    As written, that last statement doesn't fire off the close() method.
    This might be suprising because some other languages make parens optional.
    But we need them in Python because Python makes it very easy to get
    function references:
    The program appears to work fine as written, in fact, but apparently was
    not doing exactly what I thought. I assume the stream would be closed at
    the end of the program if I did not do it explicitly, but it is
    sloppy. Interesting that the following clawav.wait() statement seem to
    function fine even though the close statement lacked its parens.

    Jeff

    * * * * * * *
    Jeffrey Rice || jeffrice at finity.org || www.finity.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouptutor @
categoriespython
postedMay 12, '05 at 4:39a
activeMay 16, '05 at 12:18a
posts5
users2
websitepython.org

2 users in discussion

Jeffrey Rice: 3 posts Danny Yoo: 2 posts

Content

People

Support

Translate

site design / logo © 2023 Grokbase