FAQ
Hello List,



Would appreciate some insight/help, ran out of ideas...



BRIEF OVERVIEW:



I am trying to create a simple webserver gui wrapper for a set of
scripts I developed to test some of our firewalls here at Cisco. Being
that the total amount of engineer that will ever probably use this is 4
people and my limited python experience I just decided to do a quick
cgi-bin solution. I control the machine with the webserver on em0 where
I do my fw testing on em1/em2.



Browser goes to http://webserver/main.py -> main.py executes a
script->tests.py ->test.py imports my engine v6tester_main.py which is a
series of functions I wrote. tests.py kicks of whatever test main.py
wanted.



THE PROBLEM:



When I execute the scripts from the command line (#python main.py) it
generates it fine (albeit slowly), it prints all the html code out
including the script. The 'core' part of the script dumbed down to the
lowest level is->



proc = subprocess.Popen(['/usr/local/bin/python', 'tests.py'],
stdout=subprocess.PIPE)

output = proc.stdout.read()

print output

proc.stdout.close()



When I open main.py and execute the script it just hangs... it seems to
execute the script (I see pcap fires on the interface that I am testing
on the firewall) but its not executing correctly... or loading the
entire webpage...the webpage keeps chugging along and eventually gives
me an error timeout.



I know it's not a permissions issue or setup issue b/c I did a proof of
concept where I just fired one simple pcap and it works fine (reported
back just like it would if I ran it on the command line).. it has
something to do with either the amount of prints out the script is
doing, or the timing. I see no problems except the timeout (nothing in
logs: /var/log/http-error.log). My script takes about 8 secounds to
run. It does use threading but I wouldn't think that would mess it up.



BTW: I posted here if this helps anyone:
http://stackoverflow.com/questions/9524758/cgi-bin-timing-timeout-on-fre
ebsd-apache22





Thanks in advance for any ideas. I can include the whole main.py if
that would help.



============================

Sean Cavanaugh

Cisco Systems

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20120302/b89b39a1/attachment.html>

Search Discussions

  • Chris Rebert at Mar 2, 2012 at 8:22 pm

    On Fri, Mar 2, 2012 at 12:09 PM, Sean Cavanaugh (scavanau) wrote:
    <snip>
    THE PROBLEM:

    When I execute the scripts from the command line (#python main.py) it
    generates it fine (albeit slowly), it prints all the html code out including
    the script.? The ?core? part of the script dumbed down to the lowest level
    is->

    ??????? proc = subprocess.Popen(['/usr/local/bin/python', 'tests.py'],
    stdout=subprocess.PIPE)
    ??????? output = proc.stdout.read()
    Note the red warning box about possible deadlock with .stdout.read()
    and friends:
    http://docs.python.org/library/subprocess.html#popen-objects
    ??????? print output
    ??????? proc.stdout.close()
    As the docs advise, try using .communicate()
    [http://docs.python.org/library/subprocess.html#subprocess.Popen.communicate
    ] instead:
    proc = subprocess.Popen(?)
    out, err = proc.communicate()
    print out
    When I open main.py and execute the script it just hangs? it seems to
    execute the script (I see pcap fires on the interface that I am testing on
    the firewall) but its not executing correctly? or loading the entire
    webpage?the webpage keeps chugging along and eventually gives me an error
    timeout.
    The hanging makes me suspect that the aforementioned deadlock is occurring.

    Cheers,
    Chris
  • Sean Cavanaugh (scavanau) at Mar 2, 2012 at 9:05 pm
    Hey Chris,

    Thanks for your quick reply! I switched my code to->

    proc = subprocess.Popen(['/usr/local/bin/python', 'tests.py'], stdout=subprocess.PIPE)
    out, err = proc.communicate()
    print out
    proc.stdout.close()

    It still dead locked. Interestingly enough When I did a #python tests.py on the command line even that was taking awhile to print out to the command line so I decided to restart my webserver... wow from mucking before something must have been running in the background still. I got the script down to 2 seconds or so...

    Now it still works but faster when I do #python main.py it generates all the text to the command line but the website still hangs when I go to http://webserver/main.py... I am not sure what is going wrong... no error in the /var/log except for the eventual timeout after a couple minutes goes by.

    -S

    -----Original Message-----
    From: chris at rebertia.com [mailto:chris at rebertia.com] On Behalf Of Chris Rebert
    Sent: Friday, March 02, 2012 3:23 PM
    To: Sean Cavanaugh (scavanau)
    Cc: python-list at python.org
    Subject: Re: Python - CGI-BIN - Apache Timeout Problem

    On Fri, Mar 2, 2012 at 12:09 PM, Sean Cavanaugh (scavanau)
    wrote:
    <snip>
    THE PROBLEM:

    When I execute the scripts from the command line (#python main.py) it
    generates it fine (albeit slowly), it prints all the html code out including
    the script.? The ?core? part of the script dumbed down to the lowest level
    is->

    ??????? proc = subprocess.Popen(['/usr/local/bin/python', 'tests.py'],
    stdout=subprocess.PIPE)
    ??????? output = proc.stdout.read()
    Note the red warning box about possible deadlock with .stdout.read()
    and friends:
    http://docs.python.org/library/subprocess.html#popen-objects
    ??????? print output
    ??????? proc.stdout.close()
    As the docs advise, try using .communicate()
    [http://docs.python.org/library/subprocess.html#subprocess.Popen.communicate
    ] instead:
    proc = subprocess.Popen(?)
    out, err = proc.communicate()
    print out
    When I open main.py and execute the script it just hangs? it seems to
    execute the script (I see pcap fires on the interface that I am testing on
    the firewall) but its not executing correctly? or loading the entire
    webpage?the webpage keeps chugging along and eventually gives me an error
    timeout.
    The hanging makes me suspect that the aforementioned deadlock is occurring.

    Cheers,
    Chris
  • Sean Cavanaugh (scavanau) at Mar 2, 2012 at 9:43 pm
    Hey All,

    So maybe this part is important (after doing some troubleshooting) hopefully not everyone has beers in hand already since its Friday :-)

    The way the code works if you want to send through the firewall (i.e. SERVER->FIREWALL->SERVER) I split the process into two threads. One is listening on the egress, then I send on the ingress. The main waits until the thread finishes (it times out after 2 seconds). I had to do this b/c scapy (the library I used) won't let me send pcaps while I receive them. This lets me see packets on both sides (i.e. did that sort of internet traffic go through the firewall). The reason I think this could be a problem is when I ran a test where I sent to the firewall (rather than through it) and my program does not need to thread the webserver works fine......

    Suggestions anyone?

    -S

    -----Original Message-----
    From: chris at rebertia.com [mailto:chris at rebertia.com] On Behalf Of Chris Rebert
    Sent: Friday, March 02, 2012 3:23 PM
    To: Sean Cavanaugh (scavanau)
    Cc: python-list at python.org
    Subject: Re: Python - CGI-BIN - Apache Timeout Problem

    On Fri, Mar 2, 2012 at 12:09 PM, Sean Cavanaugh (scavanau)
    wrote:
    <snip>
    THE PROBLEM:

    When I execute the scripts from the command line (#python main.py) it
    generates it fine (albeit slowly), it prints all the html code out including
    the script.? The ?core? part of the script dumbed down to the lowest level
    is->

    ??????? proc = subprocess.Popen(['/usr/local/bin/python', 'tests.py'],
    stdout=subprocess.PIPE)
    ??????? output = proc.stdout.read()
    Note the red warning box about possible deadlock with .stdout.read()
    and friends:
    http://docs.python.org/library/subprocess.html#popen-objects
    ??????? print output
    ??????? proc.stdout.close()
    As the docs advise, try using .communicate()
    [http://docs.python.org/library/subprocess.html#subprocess.Popen.communicate
    ] instead:
    proc = subprocess.Popen(?)
    out, err = proc.communicate()
    print out
    When I open main.py and execute the script it just hangs? it seems to
    execute the script (I see pcap fires on the interface that I am testing on
    the firewall) but its not executing correctly? or loading the entire
    webpage?the webpage keeps chugging along and eventually gives me an error
    timeout.
    The hanging makes me suspect that the aforementioned deadlock is occurring.

    Cheers,
    Chris
  • Sean Cavanaugh (scavanau) at Mar 4, 2012 at 6:07 pm
    Thanks Chris,

    I isolated it using logging

    import logging
    logging.basicConfig(filename="test3.log", level=logging.INFO)
    then logging.info('sniffer got to point A')

    and going through my code until I isolated the problem to a function with scapy called sniff. For some reason this function operates very weirdly on FreeBSD9. I have already submitted an email to the scapy mailing list. Going to try to replicate this on Fedora but I doubt I will see this problem. Even from the command line the sniff() function is not working correctly on FreeBSD 9.

    -S

    -----Original Message-----
    From: chris at rebertia.com [mailto:chris at rebertia.com] On Behalf Of Chris Rebert
    Sent: Friday, March 02, 2012 3:23 PM
    To: Sean Cavanaugh (scavanau)
    Cc: python-list at python.org
    Subject: Re: Python - CGI-BIN - Apache Timeout Problem

    On Fri, Mar 2, 2012 at 12:09 PM, Sean Cavanaugh (scavanau)
    wrote:
    <snip>
    THE PROBLEM:

    When I execute the scripts from the command line (#python main.py) it
    generates it fine (albeit slowly), it prints all the html code out including
    the script.? The ?core? part of the script dumbed down to the lowest level
    is->

    ??????? proc = subprocess.Popen(['/usr/local/bin/python', 'tests.py'],
    stdout=subprocess.PIPE)
    ??????? output = proc.stdout.read()
    Note the red warning box about possible deadlock with .stdout.read()
    and friends:
    http://docs.python.org/library/subprocess.html#popen-objects
    ??????? print output
    ??????? proc.stdout.close()
    As the docs advise, try using .communicate()
    [http://docs.python.org/library/subprocess.html#subprocess.Popen.communicate
    ] instead:
    proc = subprocess.Popen(?)
    out, err = proc.communicate()
    print out
    When I open main.py and execute the script it just hangs? it seems to
    execute the script (I see pcap fires on the interface that I am testing on
    the firewall) but its not executing correctly? or loading the entire
    webpage?the webpage keeps chugging along and eventually gives me an error
    timeout.
    The hanging makes me suspect that the aforementioned deadlock is occurring.

    Cheers,
    Chris

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedMar 2, '12 at 8:09p
activeMar 4, '12 at 6:07p
posts5
users2
websitepython.org

People

Translate

site design / logo © 2022 Grokbase