code, thoughts, and learnings

Logging to both a file and stdout in Python

03 Nov 2014

If you have a set of log messages that you always want sent to both stdout and a log file, you can easily set up a python logger to do exactly that:

import logging
import sys

logger = logging.getLogger('mylogger')

On the other hand if you’re debugging something, you may have littered your code with a flurry of temporary print statements. You want to watch your screen as the action happens, but you also want the opportunity to slush through the output afterwards. One way to make that happen is with the tee command:

# code
favorite_foods = ["corn dogs", "mac and cheese", "donuts"]
print favorite_foods

#command line
python | tee some_output.log

But there’s a catch with this approach. By default Python buffers output to stdout, so if you have a lot of print statements, you may end up seeing them lag behind until they’re suddenly flushed out. The quick and dirty solution is to run python unbuffered:

python -u | tee some_output.log

The -u flag will skip buffering for the entire python process which could be a hit performance-wise, but not much of a concern when you’re only debugging.