Normally I'd use the Linux command
tee for that:
prog | tee prog.out saves a copy of the output to the
file prog.out as well as printing it. That worked fine until
I added something that needed to prompt the user for an answer.
That doesn't work when you're piping through tee: the output gets
buffered and doesn't show up when you need it to, even if you try
to flush() it explicitly.
I investigated shell-based solutions: the output I need is on
sterr, while Python's raw_input() user prompt uses stdout, so
if I could get the shell to send stderr through tee without stdout,
that would have worked. My preferred shell, tcsh, can't do this at all,
but bash supposedly can. But the best examples I could find on the
web, like the arcane
prog 2>&1 >&3 3>&- | tee prog.out 3>&-
I considered using /dev/tty or opening a pty, but those calls only work on Linux and Unix and the program is otherwise cross-platform.
What I really wanted was a class that acts like a standard
but when you write to it it writes to two places: the log file and stderr.
I found an example of someone
to write a Python tee class, but it didn't work: it worked for
write() but not for print
I am greatly indebted to KirkMcDonald of #python for finding the problem.
In the Python source implementing
PyFile_WriteObject (line 2447) checks the object's type, and if it's
subclassed from the built-in
file object, it writes
directly to the object's
fd instead of calling
The solution is to use composition rather than inheritance. Don't make your
file-like class inherit from
file, but instead include a
file object inside it. Like this:
import sys class tee : def __init__(self, _fd1, _fd2) : self.fd1 = _fd1 self.fd2 = _fd2 def __del__(self) : if self.fd1 != sys.stdout and self.fd1 != sys.stderr : self.fd1.close() if self.fd2 != sys.stdout and self.fd2 != sys.stderr : self.fd2.close() def write(self, text) : self.fd1.write(text) self.fd2.write(text) def flush(self) : self.fd1.flush() self.fd2.flush() stderrsav = sys.stderr outputlog = open(logfilename, "w") sys.stderr = tee(stderrsav, outputlog)
And it works!
print >>sys.stderr, "Hello, world" now
goes to the file as well as stderr, and
works to prompt the user for input.
In general, I'm told, it's not safe to inherit from
Python's built-in objects like
file, because they tend
to make assumptions instead of making virtual calls to your
overloaded methods. What happened here will happen for other objects too.
So use composition instead when extending Python's built-in types.
[ 09:48 Apr 16, 2010 More programming | permalink to this entry | comments ]