If you find yourself having troubles debugging your code, or wondering what went wrong, then you should start logging events in your python code.
logging library you can basically record what actions is your code doing, i.e making a web request, reading a file, monitoring something, etc. It can help you to narrow down your faulty code for debugging.
Moreover, logging is not only helpful for debugging, but it is also helpful for collaboration, and many platforms use the logging module in your code so you navigate between events easily. It will not only help you on your projects, but also in professional environments.
You can start by importing the
logging library in your python shell and after importing it you can log different event levels such as
In : import logging
I assume you noticed that our
info event wasn’t logged. This is because the default logging level is set to record events equals to warnings and above. There are 4 types of levels you should know about.
We can allow the logging module to log
info records using the
In : logging.getLogger().setLevel(logging.INFO)
However, what we should be doing, is to instantiate a logger class. This will allow us to set up the format, a handler and the logging level without having to call it from the main module.
Loggers have the following attributes and methods. Note that Loggers should NEVER be instantiated directly, but always through the module-level function
logging.getLogger(name). Multiple calls to
getLogger()with the same name will always return a reference to the same Logger object.
— Python 3 Docs
We can instantiate logger using the method
getLogger(), we just have to provide a name for that logger.
Usually what you will want to do is to provide a name to the
getLogger() function, this name can be the special variable
__name__. This way you can identify from which module is the log coming from.
In : import logging
After you have instantiated your logger class, you have to define a handler. Now, a handler is what tells the logger where it should store the logs. The two most common options are the
FileHandler and the
StreamHandler, but there are other options as well.
In : import logging
However, I would like to add more formatting to the logs, in this case, a time, a level name, and a message will suffice. Luckily, this is fairly easy using the
In : log_format = logging.Formatter("%(asctime)s - %(levelname)s: %(message)s")
You can see a list of attribute names in the python documentation here.
Let’s gather our thoughts and put this scenario. Let’s say we have a
TXT file in which we have some Urls and we have build a script that goes through each one of them to test if they are up and running, throwing error codes (HTTP 5xx) or maybe completely down.
We also want to log everything into a file, more specifically
INFO levels and up, but we also want to log only
WARNING events into our terminal so we don’t fill it up with information we don’t care about.
To do this we are going to split our script into 2 modules. Is not necessary but I don’t want to cramp it up everything into a single file. Let’s first create a
TXT file and fill it out with Urls:
Now, let’s create a python script and call it
load_logger.py. Here we’ll define our logger instance, the formatter, and handlers appended to it with their respective logging level.
Now that we have defined our logger, let’s create our main script
site_monitor.py. What this file will do is:
- Load the event logger
- Read our file domains.txt
- Test the status code of each destination
- Log the event of the status code
To run our script you can call it using the command line:
$ python site_monitor.py domains.txt
After that you will see that only
WARNING events are being recorded in the console, but in our
status.log file you will see
INFO logs and up.
In this example, I’m executing code and I’m using the command
tail with the flag
-f to see what’s being recorded in that file.
As you can see, in our example we only log
WARNING events in our terminal, but we are able to dump everything in our
If you want to take a closer look at the code, I’ve set up a sample repository at: https://github.com/franccesco/status_monitor_example
Feel free to reach out if you have any doubts.