FAQ
I have a listening socket, self.lsock, which is used in an accept() call
as follows


   endpoint = self.lsock.accept()


The problem is that if control-C is pressed it is not recognised until
something connects to that socket. Only when the accept() call returns
is the signal seen.


The question, then, is how to get the signal to break out of the
accept() call. This is currently on Windows but I would like it to run
on Unix too. I see from the web that this type of thing is a common
problem with the underlying C libraries but I cannot quite relate the
posts I have found to Python.


Any ideas?


James

Search Discussions

  • Chris Angelico at Sep 10, 2015 at 6:36 pm

    On Fri, Sep 11, 2015 at 4:24 AM, James Harris wrote:
    I have a listening socket, self.lsock, which is used in an accept() call as
    follows

    endpoint = self.lsock.accept()

    The problem is that if control-C is pressed it is not recognised until
    something connects to that socket. Only when the accept() call returns is
    the signal seen.

    The question, then, is how to get the signal to break out of the accept()
    call. This is currently on Windows but I would like it to run on Unix too. I
    see from the web that this type of thing is a common problem with the
    underlying C libraries but I cannot quite relate the posts I have found to
    Python.

    What version of Python are you using? Also (in case it matters), what
    version of Windows?


    Have you tested on any Unix system? I just tried on my Linux, and
    Ctrl-C interrupted the accept() straight away, so this is quite
    probably a Windows-only issue.


    Can you produce an absolute minimal demo program? I'd try something like this:


    import socket
    s = socket.socket()
    s.listen(1)
    s.accept()


    which is what worked for me (interactively, Python 2.7.9 and 3.6.0a0,
    Debian Linux).


    ChrisA
  • James Harris at Sep 10, 2015 at 7:11 pm
    "Chris Angelico" <rosuav@gmail.com> wrote in message
    news:mailman.332.1441910212.8327.python-list at python.org...
    On Fri, Sep 11, 2015 at 4:24 AM, James Harris
    wrote:
    I have a listening socket, self.lsock, which is used in an accept()
    call as
    follows

    endpoint = self.lsock.accept()

    The problem is that if control-C is pressed it is not recognised
    until
    something connects to that socket. Only when the accept() call
    returns is
    the signal seen.

    The question, then, is how to get the signal to break out of the
    accept()
    call. This is currently on Windows but I would like it to run on Unix
    too. I
    see from the web that this type of thing is a common problem with the
    underlying C libraries but I cannot quite relate the posts I have
    found to
    Python.
    What version of Python are you using? Also (in case it matters), what
    version of Windows?

    Good point. It turns out that it does matter. I have one implementation
    which fails (Windows) and one which works (Linux). The Linux one breaks
    out on Control-C. The Windows one does not recognise Control-C until the
    accept() call returns.


    The implementations are:


    $ uname -srm
    Linux 3.18.7-v7+ armv7l
    $ python -V
    Python 2.7.3


    And


    c:\>ver
    Microsoft Windows XP [Version 5.1.2600]
    c:\>python -V
    Python 2.7.9

    Have you tested on any Unix system? I just tried on my Linux, and
    Ctrl-C interrupted the accept() straight away,

    Thanks.

    so this is quite probably a Windows-only issue.

    That turns out to be exactly right.

    Can you produce an absolute minimal demo program? I'd try something
    like this:

    import socket
    s = socket.socket()
    s.listen(1)
    s.accept()

    Yes:


    port = 8880
    import socket
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.bind(("", port))
    s.listen(1)
    endpoint = s.accept()

    which is what worked for me (interactively, Python 2.7.9 and 3.6.0a0,
    Debian Linux).

    On Linux I get


    $ python socktest.py
    ^CTraceback (most recent call last):
       File "socktest.py", line 6, in <module>
         endpoint = s.accept()
       File "/usr/lib/python2.7/socket.py", line 202, in accept
         sock, addr = self._sock.accept()
    KeyboardInterrupt
    $


    On Windows I get


    S:\>python socktest.py
    Traceback (most recent call last):
       File "socktest.py", line 6, in <module>
         endpoint = s.accept()
       File "C:\Python27\lib\socket.py", line 202, in accept
         sock, addr = self._sock.accept()
    KeyboardInterrupt


    S:\>


    However, on Windows the recognition of Control-C does not happen until
    after something connects to the socket.


    I will carry on researching it but maybe the above gives a clue to those
    in the know...!


    James
  • Chris Angelico at Sep 10, 2015 at 7:26 pm

    On Fri, Sep 11, 2015 at 5:11 AM, James Harris wrote:
    S:\>python socktest.py
    Traceback (most recent call last):
    File "socktest.py", line 6, in <module>
    endpoint = s.accept()
    File "C:\Python27\lib\socket.py", line 202, in accept
    sock, addr = self._sock.accept()
    KeyboardInterrupt

    S:\>

    However, on Windows the recognition of Control-C does not happen until after
    something connects to the socket.

    I will carry on researching it but maybe the above gives a clue to those in
    the know...!

    This is a known problem on Windows. I can't remember what the best
    solution was, but there's a chance something got into 2.7.10, as it
    was fairly recent. There's a significantly better chance that
    something's different in Python 3.x. You may find it worth grabbing a
    few different versions of Python and trying the same code on all of
    them.


    You may run into issues with XP, though. For instance, Python 3.5
    doesn't support it, and (IIRC) won't install at all; 3.4 does work, as
    will all releases of 2.7.x. Worst case, grab yourself a Windows 7 and
    try a few tests.


    But a quick test on one of my VMs, with 3.4 on Win 7, didn't show any
    change. It's entirely possible that a blocking socket-accept call will
    continue to block. There is one rather silly option, and that's to use
    select() to effectively poll for Ctrl-C... or, possibly better, have a
    separate program that shuts down your server (by connecting to it,
    which thus breaks the stranglehold).


    Of course, switching over to Unix is also a good option...


    ChrisA
  • James Harris at Sep 10, 2015 at 8:12 pm
    "Chris Angelico" <rosuav@gmail.com> wrote in message
    news:mailman.337.1441913195.8327.python-list at python.org...
    On Fri, Sep 11, 2015 at 5:11 AM, James Harris
    wrote:

    ...

    However, on Windows the recognition of Control-C does not happen
    until after
    something connects to the socket.

    ...

    This is a known problem on Windows.

    ...

    It's entirely possible that a blocking socket-accept call will
    continue to block. There is one rather silly option, and that's to use
    select() to effectively poll for Ctrl-C... or, possibly better, have a
    separate program that shuts down your server (by connecting to it,
    which thus breaks the stranglehold).

    Thanks for your help, Chris. Using select() is a very good option. I
    tried first without a timeout and even then this version of Windows does
    not recognise Control-C until after the select() call returns (which
    needs similar prompting as with the accept() call. However, select()
    with a timeout allows the code to work both on Windows and Linux.
    Hooray!


    For anyone else who is looking for this the earlier test code was
    changed to


    port = 8880
    import select
    import socket
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.setblocking(0)
    s.bind(("", port))
    s.listen(1)
    while 1:
       ready = select.select((s,), (), (), 0.5)
       #print '(ready %s)' % repr(ready)
       if (ready[0]):
         try:
           endpoint = s.accept()
         except socket.error, details:
           print 'Ignoring socket error:', repr(details)
           continue
         print '(endpoint %s)' % repr(endpoint)
         if endpoint:
           break


    James
  • Chris Angelico at Sep 11, 2015 at 2:01 am

    On Fri, Sep 11, 2015 at 6:12 AM, James Harris wrote:
    Thanks for your help, Chris. Using select() is a very good option. I tried
    first without a timeout and even then this version of Windows does not
    recognise Control-C until after the select() call returns (which needs
    similar prompting as with the accept() call. However, select() with a
    timeout allows the code to work both on Windows and Linux. Hooray!

    For anyone else who is looking for this the earlier test code was changed to

    port = 8880
    import select
    import socket
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.setblocking(0)
    s.bind(("", port))
    s.listen(1)
    while 1:
    ready = select.select((s,), (), (), 0.5)
    #print '(ready %s)' % repr(ready)
    if (ready[0]):
    try:
    endpoint = s.accept()
    except socket.error, details:
    print 'Ignoring socket error:', repr(details)
    continue
    print '(endpoint %s)' % repr(endpoint)
    if endpoint:
    break

    (Your indentation is looking like a single space, here. I would
    suggest indenting a bit more, for readability; but it might just be an
    artefact of posting.)


    This is what I meant when I said you would be polling. Effectively,
    you wake up your program every half-second, check if Ctrl-C has been
    pressed, and if it hasn't, you go back to sleep again. This is pretty
    inefficient.


    Assuming you don't need stdin for any other purpose, one solution
    would be to spin off a thread that simply watches for a keyboard
    signal. I tested this on Windows 7 with 2.7.10 and 3.4.3, and it
    appears to work:


    import socket
    import threading


    # Python 2/3 compat
    try: input = raw_input
    except NameError: pass


    PORT = 8880
    mainsock = socket.socket()
    mainsock.bind(("", PORT))
    mainsock.listen(1)


    def console():
         """Constantly read from stdin and discard"""
         try:
             while "moar console": input()
         except (KeyboardInterrupt, EOFError):
             socket.socket().connect(("127.0.0.1",PORT))


    threading.Thread(target=console).start()


    while "moar sockets":
         s = mainsock.accept()
         print("New connection: %r" % s)
         # Do whatever you want with the connection
         s.close()




    As long as you have _some_ thread reading from the console, you can
    get woken up by Ctrl-C. When that happens, it simply fires off a quick
    socket connection to itself, thus waking up the main thread; and then
    the main thread sees the KeyboardInterrupt. (I'm not sure why, but
    instead of seeing KeyboardInterrupt in the console thread, I've been
    seeing EOFError. Since I don't particularly care to explore the
    problem, I just wrote the except clause to catch both.)


    This eliminates the polling, but you have to sacrifice stdin to do it.
    It may be worth bracketing all of that code with a platform check -
    don't bother doing all this unless you're on Windows. Up to you.


    Stupid Windows.


    ChrisA
  • Grant Edwards at Sep 11, 2015 at 1:50 pm

    On 2015-09-11, Chris Angelico wrote:


    This is what I meant when I said you would be polling. Effectively,
    you wake up your program every half-second, check if Ctrl-C has been
    pressed, and if it hasn't, you go back to sleep again. This is pretty
    inefficient.

    Though it offends one's engineering sensibilities[1], it's just not
    that inefficient. I'd bet money you won't even be able to measure the
    difference in CPU usage. Waking up twice per second and immediately
    calling select() again on any hardware/OS built in the past 50 years
    is going completely negligible (as long as you can ignore the smell).


    Even waking up ten times per second won't be noticeable.


    Waking up every millisecond or two might be noticeable.

    Stupid Windows.

    Agreed.


    [1] If offenses to engineering sensibility were of concern, then
         he wouldn't be using Windows in the first place. ;)


    --
    Grant Edwards grant.b.edwards Yow! PIZZA!!
                                       at
                                   gmail.com
  • Marko Rauhamaa at Sep 11, 2015 at 2:00 pm

    Grant Edwards <invalid@invalid.invalid>:

    On 2015-09-11, Chris Angelico wrote:
    This is what I meant when I said you would be polling. Effectively,
    you wake up your program every half-second, check if Ctrl-C has been
    pressed, and if it hasn't, you go back to sleep again. This is pretty
    inefficient.
    Though it offends one's engineering sensibilities[1], it's just not
    that inefficient. I'd bet money you won't even be able to measure the
    difference in CPU usage. Waking up twice per second and immediately
    calling select() again on any hardware/OS built in the past 50 years
    is going completely negligible (as long as you can ignore the smell).

    Even waking up ten times per second won't be noticeable.

    Waking up every millisecond or two might be noticeable.

    It can add up. In particular, it can prevent the CPU from staying in the
    low-power mode, especially on battery-powered devices.


       <URL: https://lwn.net/Articles/549580/>


    Another environment where such polling circuses can actually overwhelm a
    CPU is virtual machines. When dozens or hundreds of VMs are each polling
    left and right, the host may not get much actual work done.




    Marko
  • Chris Angelico at Sep 11, 2015 at 2:27 pm

    On Fri, Sep 11, 2015 at 11:50 PM, Grant Edwards wrote:
    On 2015-09-11, Chris Angelico wrote:

    This is what I meant when I said you would be polling. Effectively,
    you wake up your program every half-second, check if Ctrl-C has been
    pressed, and if it hasn't, you go back to sleep again. This is pretty
    inefficient.
    Though it offends one's engineering sensibilities[1], it's just not
    that inefficient. I'd bet money you won't even be able to measure the
    difference in CPU usage. Waking up twice per second and immediately
    calling select() again on any hardware/OS built in the past 50 years
    is going completely negligible (as long as you can ignore the smell).

    Even waking up ten times per second won't be noticeable.

    True (although, as Marko says, it can add up). The other difference,
    though, is that I like to keep "cope-with-stupidity" code to itself -
    ideally, the clean path shouldn't be infected with it. Waking up
    periodically with select(), when you otherwise just want a blocking
    accept(), affects all your code. Spinning off a thread that monitors
    stdin and signals the main thread when it's time to shut down can be
    kept to a single block of code, guarded by some sort of platform
    check:


    import socket


    PORT = 8880
    mainsock = socket.socket()
    mainsock.bind(("", PORT))
    mainsock.listen(1)


    if Windows: # however you want to go about checking this
         import threading


         # Python 2/3 compat
         try: input = raw_input
         except NameError: pass


         def console():
             """Constantly read from stdin and discard"""
             try:
                 while "moar console": input()
             except (KeyboardInterrupt, EOFError):
                 socket.socket().connect(("127.0.0.1",PORT))


         threading.Thread(target=console).start()


    while "moar sockets":
         s = mainsock.accept()
         print("New connection: %r" % s)
         # Do whatever you want with the connection
         s.close()




    As well as keeping the Unix variant stupidity-free, this allows you to
    vary your platform check in the event that a future version of Windows
    allows blocking calls to be interrupted. Or if a future version of
    Python implements this kind of check globally. Or if a Python built
    with Cygwin doesn't exhibit this behaviour. Or if PyPy... you get the
    idea. The clean path is clearly delineated, and hopefully, the
    complexity of your code will increase linearly with the number of
    problems you're coping with, rather than having each patch affect each
    other.


    ChrisA
  • James Harris at Sep 11, 2015 at 5:14 pm
    "Grant Edwards" <invalid@invalid.invalid> wrote in message
    news:msum6c$hv$1 at reader1.panix.com...
    On 2015-09-11, Chris Angelico wrote:

    This is what I meant when I said you would be polling. Effectively,
    you wake up your program every half-second, check if Ctrl-C has been
    pressed, and if it hasn't, you go back to sleep again. This is pretty
    inefficient.
    Though it offends one's engineering sensibilities[1], it's just not
    that inefficient.

    Indeed.

    I'd bet money you won't even be able to measure the
    difference in CPU usage.

    You would win your bet.

    Waking up twice per second and immediately
    calling select() again on any hardware/OS built in the past 50 years
    is going completely negligible (as long as you can ignore the smell).

    CPU usage is not the only issue but it is a consideration. I tested it
    before posting the code and couldn't see any significant increase in CPU
    use. To give it a better test I have just left running for a couple of
    hours or so. I am pleased to say it currently reports a cumulative total
    of 0:00:00, i.e. it has not clocked up even a second of CPU time yet!


    ...

    [1] If offenses to engineering sensibility were of concern, then
    he wouldn't be using Windows in the first place. ;)

    LOL. I know that's tongue in cheek but I tend to favour portability over
    most other things. So running on Windows as well as Unix is, in my book,
    a Good Thing.


    James
  • James Harris at Sep 11, 2015 at 11:15 pm
    "James Harris" <james.harris.1@gmail.com> wrote in message
    news:msv21t$n1m$1 at dont-email.me...
    "Grant Edwards" <invalid@invalid.invalid> wrote in message
    news:msum6c$hv$1 at reader1.panix.com...

    ...

    Waking up twice per second and immediately
    calling select() again on any hardware/OS built in the past 50 years
    is going completely negligible (as long as you can ignore the smell).
    CPU usage is not the only issue but it is a consideration. I tested it
    before posting the code and couldn't see any significant increase in
    CPU use. To give it a better test I have just left running for a
    couple of hours or so. I am pleased to say it currently reports a
    cumulative total of 0:00:00, i.e. it has not clocked up even a second
    of CPU time yet!

    I am beginning to wonder if time is being accounted properly. It has now
    been running 8 hours and still shows CPU time as zero.


    James

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedSep 10, '15 at 6:24p
activeSep 11, '15 at 11:15p
posts11
users4
websitepython.org

People

Translate

site design / logo © 2019 Grokbase