FAQ
Hi,

I've just started to learn python (I've been using perl for some years).

How do I redirect ALL stderr stuff to syslog, even stderr from
external programs that don't explicitly change their own stderr?

Say I have a program called foo:

#!/usr/bin/python
import syslog
import os, sys
class logstderr:
def write(self, data):
syslog.syslog('STDERR: %s' % data)
syslog.openlog('test[%u]' % os.getpid() )
sys.stderr=logstderr()
cmd='ls -al asdfsdf'
os.system(cmd)
bar('foo')

And bar is a nonexistent function.

If I run test I get:
./test
ls: cannot access asdfsdf: No such file or directory

And in /var/log/messages I get:
Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: Traceback (most recent
call last):
Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: File "./foo", line
11, in <module>
Aug 5 21:08:35 linux-9k3z test[2186]: STDERR:
Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: bar('foo')
Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: NameError
Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: :
Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: name 'bar' is not defined
Aug 5 21:08:35 linux-9k3z test[2186]: STDERR:


What I want is the "ls: cannot access asdfsdf: No such file or
directory" message to go to syslog instead:
e.g.
Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: ls: cannot access
asdfsdf: No such file or directory
Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: Traceback (most recent
call last):
Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: File "./foo", line
11, in <module>
Aug 5 21:08:35 linux-9k3z test[2186]: STDERR:
Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: bar('foo')
Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: NameError
Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: :
Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: name 'bar' is not defined
Aug 5 21:08:35 linux-9k3z test[2186]: STDERR:

Explanation:

I do not normally redirect STDERR and STDOUT to /dev/null for daemons I write.

Since in _theory_ nothing should be "leaking" out, if stuff does leak
out in practice, something is not quite right.

So I want all such "leaks" be redirected to syslog (or my logging
routines), so that I can see the bugs/warnings - whether in my
program or other programs/modules I call/use.

Sorry if this has been dealt with before - I haven't found the
solution in my searches though.

I do NOT want to resort to this:

#!/bin/sh
/bin/foo 2>&1 | logger -t "test: STDERR/STDOUT"

:)

Thanks,

Link.

Search Discussions

  • Matthew Woodcraft at Aug 5, 2008 at 7:29 pm
    In article <mailman.1134.1217944475.922.python-list at python.org>,
    How do I redirect ALL stderr stuff to syslog, even stderr from
    external programs that don't explicitly change their own stderr?
    Sending messages to syslog involves more than writing to a file
    descriptor, so there's no way to make this happen without having some
    process read the external programs' output and send it to syslog (which
    is basically the 'logger' method that you said you didn't like).

    Since in _theory_ nothing should be "leaking" out, if stuff does leak
    out in practice, something is not quite right.
    So I want all such "leaks" be redirected to syslog (or my logging
    routines), so that I can see the bugs/warnings - whether in my
    program or other programs/modules I call/use.
    One reasonable way to do this is to have a separate 'breakage log' file
    for this kind of message, and point standard error there. This will
    catch output from external programs, and any output to standard error
    which libraries in your main program (or the Python interpreter) decide
    to produce.

    You'd do that with something like this (untested):

    STDERR = 2
    se = os.open("breakage.log", os.O_WRONLY|os.O_APPEND)
    os.dup2(se, STDERR)
    os.close(se)

    You can also use this breakage log for errors from your own program, if
    there are any cases where you think things might be too messed up for
    you to want to rely on your normal logging routines.

    Then since the breakage log should normally be empty, you can write a
    cronjob or something to keep an eye on it and notify you if anything
    appears there.

    -M-
  • Lincoln Yeoh at Aug 6, 2008 at 1:44 am

    At 03:29 AM 8/6/2008, Matthew Woodcraft wrote:
    In article <mailman.1134.1217944475.922.python-list at python.org>,
    How do I redirect ALL stderr stuff to syslog, even stderr from
    external programs that don't explicitly change their own stderr?
    Sending messages to syslog involves more than writing to a file
    descriptor, so there's no way to make this happen without having some
    process read the external programs' output and send it to syslog (which
    is basically the 'logger' method that you said you didn't like).
    I actually don't mind resorting to logger, I just don't want to wrap
    the python program in a shell script.

    Anyway the following appears to work, not sure if there are any
    gotchas (I'm new to this).

    (a,b)=popen2.popen2('/bin/logger -t "test[%u]: STDERR"' % os.getpid())
    os.dup2(b.fileno(),sys.stderr.fileno())
    (a,b1)=popen2.popen2('/bin/logger -t "test[%u]: STDOUT"' % os.getpid())
    os.dup2(b1.fileno(),sys.stdout.fileno())

    It seems subprocess is to be preferred over popen2. subprocess is
    more verbose for this.

    With perl I normally do (close your eyes if you are allergic to perl :) ):

    open(STDERR,"|/bin/logger -t \"${PROGNAME}[$$]: STDERR\"") or die
    "Error: Unable to redirect STDERR to logger!";
    open(STDOUT,"|/bin/logger -t \"${PROGNAME}[$$]: STDOUT\"") or die
    "Error: Unable to redirect STDOUT to logger!";
    You'd do that with something like this (untested):

    STDERR = 2
    se = os.open("breakage.log", os.O_WRONLY|os.O_APPEND)
    os.dup2(se, STDERR)
    os.close(se)

    You can also use this breakage log for errors from your own program, if
    there are any cases where you think things might be too messed up for
    you to want to rely on your normal logging routines.
    I've seen that but rejected it because it will not provide
    timestamps, and the logs end up in different places - some to syslog
    and some to this file.

    Having the logs go to the same place also gives a hint on the order
    of events - it's not 100% in order of course, but usually good
    enough, and much better than not even knowing what hour each line occurred.

    Regards,
    Link.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedAug 5, '08 at 1:47p
activeAug 6, '08 at 1:44a
posts3
users2
websitepython.org

2 users in discussion

Lincoln Yeoh: 2 posts Matthew Woodcraft: 1 post

People

Translate

site design / logo © 2022 Grokbase