Develop and Publish With Poetry

I’ve been trying to publish my packages to PyPi so people can access my software more easily. But I have to be honest, Python’s publishing system is not the best out there and it has to improve quite a lot.

Wandering around I stumbled upon Poetry, which is a python packager and dependency manager created by Sébastien Eustace for people that doesn’t want to lose their head managing a Python project.

Let’s say that we want to make a small command line application that checks the status of a web page, let’s call it checkstat. So how can we create, develop, and publish our software with Poetry?

Installing Poetry

Installing Poetry is really easy with Pip.

$ pip install poetry

Now we are able to see Poetry’s options.

$ poetry
Poetry 0.11.4

Usage:
  command [options] [arguments]

Options:
  -h, --help                      Display this help message
  -q, --quiet                     Do not output any message
  -V, --version                   Display this application version
      --ansi                      Force ANSI output
      --no-ansi                   Disable ANSI output
  -n, --no-interaction            Do not ask any interactive question
  -v|vv|vvv, --verbose[=VERBOSE]  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Available commands:
  about          Short information about Poetry.
  add            Add a new dependency to pyproject.toml.
  build          Builds a package, as a tarball and a wheel by default.
  check          Checks the validity of the pyproject.toml file.
  config         Sets/Gets config options.

--- SNIP ---

 debug
  debug:info     Shows debug information.
  debug:resolve  Debugs dependency resolution.
 self
  self:update    Updates poetry to the latest version.

Creating our package and add dependencies

First we have to create our package with the new command, this will create a file structure that we can modify later.

# Create a new package
$ poetry new checkstat
Created package checkstat in checkstat

# Enter the new package directory
$ cd checkstat

# Prints the directory tree
$ tree
.
├── checkstat
│   └── __init__.py
├── pyproject.toml
├── README.rst
└── tests
    ├── __init__.py
    └── test_checkstat.py

2 directories, 5 files

As we can see, the only argument needed to create a directory structure for our little project is the argument new followed by our project’s name. This will also generate a template for our project called pyproject.toml which is used as a replacement of the most feared (at least by me) setup.py.

[tool.poetry]
name = "checkstat"
version = "0.1.0"
description = ""
authors = ["Franccesco Orozco <[email protected]>"]

[tool.poetry.dependencies]
python = "*"

[tool.poetry.dev-dependencies]
pytest = "^3.0"

As you can see, it describes basic information about the project, dependencies and development dependencies. We will get back to the TOML file shortly.

What we want to do next is to create the virtual environment of our project and install the development dependencies with the command install.

# Create package isolated virtualenv and install dependencies
$ poetry install
Creating virtualenv checkstat-py3.7 in /home/franccesco/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies... (1.6s)


Package operations: 7 installs, 0 updates, 0 removals

Writing lock file

  - Installing six (1.11.0)
  - Installing atomicwrites (1.1.5)
  - Installing attrs (18.1.0)
  - Installing more-itertools (4.3.0)
  - Installing pluggy (0.7.1)
  - Installing py (1.5.4)
  - Installing pytest (3.7.1)

Now that we have installed the requirements and initiated the virtual environment let’s add more libraries to our project, shall we? Let’s add Click, Colorama and Requests.

# Add more requirements
$ poetry add click colorama requests
Using version ^6.7 for click
Using version ^0.3.9 for colorama
Using version ^2.19 for requests

--- SNIP ---

  - Installing urllib3 (1.23)
  - Installing click (6.7)
  - Installing requests (2.19.1)

There you go, now if we check the our pyproject.toml again, you can see that it has the new package dependencies.

[tool.poetry.dependencies]
python = "*"
click = "^6.7"
colorama = "^0.3.9"
requests = "^2.19"

Developing our module and our CLI

Now we’re ready to develop our checkstat module and command line interface. Let’s start by making a test with Pytest. For this, let’s open test_checkstat.py inside our tests folder.

Here we’re going to set an expectation. Our module checkstat should have a method called is_up which should return True if the webpage returns the code 200 for any other code it should return False. Here’s the test:

import checkstat


def test_checkstat():
    """Test if checkstat module returns True on 200 code."""
    assert checkstat.is_up('https://codingdose.info')

Now if we run this test with pytest it will show red because we haven’t build any modules yet, but for the sake of demonstration, let’s show the red test first.

# Running our test
$ poetry run pytest

Now, let’s build the module shall we? Let’s make a file called checkstat.py under the checkstat directory and define a is_up method that returns True if the webpage is reachable and returns a 200 HTTP code.

# checkstat/checkstat.py
import requests


def is_up(webpage):
    """Return True if 200 code was received, else return False."""
    try:
        req = requests.get(webpage)

    # On connection error, return False.
    except requests.exceptions.ConnectionError:
        return False
    # Connection was successful, return True on 200 code, else return False.
    else:
        if req.status_code == 200:
            return True
        return False

Now let’s add our module to our initialization package so it gets correctly loaded.

# checkstat/__init__.py
from .checkstat import is_up

Let’s re-run our test to see if its passing now.

Perfect, now that our test is passing we can go ahead and make our command line script. Let’s create a file called cli.py inside our checkstat folder.

# checkstat/cli.py
import click
import checkstat


@click.command()
@click.argument('host')
def main(host):
    """CLI for checkstat package."""
    print('Status: ', end='')
    if checkstat.is_up(host):
        click.secho('Up and running.', bold=True, fg='green')
    else:
        click.secho('Server is down.', bold=True, fg='red')


if __name__ == '__main__':
    main()

Perfect, now we have completed our CLI, but we haven’t tried it yet, how do we make it run on the terminal without calling python x_file.py? Easy-peasy, let’s edit the pyproject.toml and add a script section.

[tool.poetry]
name = "checkstat"
version = "0.1.0"
description = ""
authors = ["Franccesco Orozco <[email protected]>"]

--- SNIP ---

# New section
[tool.poetry.scripts]
checkstat = "checkstat.cli:main"

What does checkstat.cli:main means? It means: Hey python, whenever I type checkstat I want you to execute the function main() inside of the module cli.py in the package checkstat. Now we must tell Poetry that we want install the package in develop mode.

This “develop” mode is useful because this way, every time we edit our package, be our CLI or module, we wouldn’t have to install the package again and again; We can see our changes reflected right away in execution.

$ poetry develop
Installing dependencies from lock file                                                                               
                                                                                                                     
Nothing to install or update                                                                                         
                                                                                                                     
Installing checkstat (0.1.0)

Now that our package is installed in our virtual environment and we have set a script for it, we can execute it directly with Poetry.

Let’s check the status code of another web page that returns a 500 Internal Error code.

Awesome! Our little command line application is ready to be published.

Publishing our tool

Publishing our tool to the PyPi is really effortless, let’s try to do it right now. First of all we have to build our package.

$ poetry build
Building checkstat (0.1.0)                                                                                           
 - Building sdist                                                                                                    
 - Built checkstat-0.1.0.tar.gz                                                                                      
                                                                                                                     
 - Building wheel                                                                                                    
 - Built checkstat-0.1.0-py2.py3-none-any.whl

Now we have to upload it to the pypi.org repository.

$ poetry publish
Username: franccesco
Password: **********

 - Uploading checkstat-0.1.0-py2.py3-none-any.whl 100%
 - Uploading checkstat-0.1.0.tar.gz 100%

And that’s it! Our tools is now available to anyone using pip to install packages. Let’s check it out: https://pypi.org/project/checkstat/

Installing our tool

Let’s go and install our tool to see if it’s available now.

Collecting checkstat
  Downloading https://files.pythonhosted.org/packages/7a/76/3cf9a476224e3a1962c29c1eb7bedbc71eb1322df62c1cf48b36df48c58e/checkstat-0.1.0-py2.py3-none-any.whl
Requirement already satisfied: colorama<0.4.0,>=0.3.9 in ./.pyenv/versions/3.7.0/lib/python3.7/site-packages (from checkstat) (0.3.9)
Requirement already satisfied: requests<3.0,>=2.19 in ./.pyenv/versions/3.7.0/lib/python3.7/site-packages (from checkstat) (2.19.1)

--- SNIP ---

Installing collected packages: checkstat
Successfully installed checkstat-0.1.0

And let’s execute it.

It works!

Conclusion

I hope you like the entry, today we have learn how to create, develop and ship a very basic CLI. If you have any issues in the process remember to comment below. Have fun!

Further reading

• Poetry
• Command Line Applications
• Structuring Your Project
• Requests
• Click
• Pytest