Whether you’re a data scientist, a software developer or an engineer, you’re likely encountering Python on a daily basis. Maybe you already know the basics — or perhaps you’re quite proficient at this point. What happens after you learn Python and build a working project? You probably want to share, deploy or distribute your code publicly for others to install and use. So…how do you do that?
PyPI to the rescue! First things first…
What Is PyPI?
Unfortunately, the resources to distribute your work through PyPI are limited and not often complete. In this article, I’ll simplify the process of distributing a package to PyPI in five simple steps using setuptools.
5 Easy Steps to Package and Publish Your Python Code to PyPI
- Get your code files ready.
- Prepare your supporting files.
- Build your package locally.
- Upload your package to TestPyPI.
- Distribute your work on PyPI.
Step 1: Get Your Code Files Ready
To package your work, you need, well…work. Once you’ve completed your code, modularized and tested it, you will need to put it in a specific hierarchy before you start packing and publishing it. Here’s a simple project hierarchy you can start with:
packageName ├── LICENSE ├── projectName │ ├── __init__.py │ ├── module_1.py │ ├── module_1.py │ └── module_2.py ├── README ├── pyproject.toml └── setup file
In some packages, the
packageName and the
projectName are the same, but that’s not necessary. The
packageName is used in the
pip install command, so it will be
packageName, while the project name is what’s used in the
import command after you install the package. Once you’ve finished the installation, you type
If your project requires further depth in the hierarchy, make sure to include a
__init__.py file in every hierarchy to make it importable. Basically, if your directory contains
an __init__.py file, you can then import the content of this file. For example, consider the following directory:
mydir ├── dir │ ├── __init__.py │ └── module_1.py └── some other files
mydir is on your path ( the one you choose when you install Python ) you will be able to import the code
module_1.py in your code files simply as:
import dir.module_1 #Or from dir import module_1
When you run Python, if the
__init__.py file was not in the directory, the interpreter will no longer look for modules within that directory. This will lead to an import error if you try to import these modules. The
__init__.py file, in most cases, is an empty file. Sometimes it can include simpler names for the submodules.
Step 2: Prepare Your Supporting Files
Now that your files are clean and sorted, we can add the supporting files. You will need four or five supporting files to complete your package files.
The Setup File
This file includes metadata about the project including its author, its repository, a description of the project, the license under which it’s published and more. There are two types of setup files: static and dynamic.
- Static (
setup.cfg): This means the file is the same every time you install the package. It has a format that’s easy to read and understand.
- Dynamic (
setup.py): Some items within the file are dynamic or determined at the time of installation.
The official Python packaging website strongly suggests using static setup files and only using dynamic when absolutely necessary. For that reason, I’ll focus on the static setup file. Here’s a template of a
setup.cfg file you can use.
[metadata] name = NAME_OF_YOUR_PROJECT version = 0.0.1 author = YOUR_NAME author_email = YOUR_EMAIL description = WHAT IS THE REASON YOU BUILD THIS PROJECT AND WHAT IT DOES long_description = file: README.md long_description_content_type = text/markdown url = GITHUB REPOSITORY LINK classifiers = Programming Language :: Python :: 3 License :: OSI Approved :: Apache Software License Operating System :: OS Independent [options] packages = find: python_requires = >=3.7 include_package_data = True
[options] sections deal with dependencies. The line
Package = find: works automatically to detect your package dependencies.
This is a simple, short and often fixed file that includes Python's tools to create a PyPI package. It basically explains to PyPI that we’re planning to use setuptools and wheels to distribute and build our package.
[build-system] requires = [ “setuptools>=42”, “wheel” ] build-backend = “setuptools.build_meta”
A License File
There are many different open-source licenses. Choosing which one you want depends on your goal and the nature of your project; you always find help if you’re struggling to decide which license to use.
Once you choose your license, you must add it to the
setup file in a PyPI accepted format. In the above example, I used the Apache license. You can find all supported license formats here.
A README File
The README file often includes detailed information about the project, its installation and maybe a usage example. In PyPI packages, the README files often take one of four forms:
README README.txtREADME.rst README.md
The files are either plain text, a reStructuredText or a markdown file. Whichever file you decide to use, it will be used as the project description on the package page on PyPI.
This is an optional file, and you only need to include it if you have non-code files in your package. If you do, you need to write them down in this file so the package installer knows where to find them.
Step 3: Build Your Package Locally
You’re almost done. Before you upload your package to PyPI, you need to build it locally and make sure you’re not missing any files. You also want to make sure you don’t have any errors in your code or supporting files. So, from the directory of your package, run the command line.
If you don’t have the
wheel tool, you will need to install it and have the latest version of the
pip install wheel py -m pip install --upgrade build
Now that you have the tool to build the package, you can start building your project.
py -m build
If all your files are in order, this command should produce many lines of commands and end with no error.
Step 4: Upload Your Package to TestPyPI
Just because your package was successfully built locally doesn’t mean it will be smooth sailing when you try to
pip install it. So, this is just a test and debug step; you can skip it if you want. Uploading your package to testPyPI will let you
pip install it just to test it out.
So, go ahead and register for both PyPI and testPyPI. I should point out that these two are completely independent and don’t share a database. Once you’re registered, you will have a username and password; make sure you remember them because you’ll use them to upload your package.
Now, install Twine, which is a tool to help you create the package.
pip install twine
To upload your project to testPyPI, type in the following command:
py -m twine upload --repository testpypi dist/*
Which should result in something similar to:
Uploading distributions to https://test.pypi.org/legacy/ Enter your username: [your username] Enter your password: Uploading yourpkg_YOUR_USERNAME_HERE-0.0.1-py3-none-any.whl 100%|█████████████████████| 4.65k/4.65k [00:01<00:00, 2.88kB/s] Uploading yourpkg_YOUR_USERNAME_HERE-0.0.1.tar.gz 100%|█████████████████████| 4.25k/4.25k [00:01<00:00, 3.05kB/s]
You can then install your package in a virtual environment and test to make sure it works properly. If it does, we can move it to the final step.
Step 5: Distribute Your Work on PyPI
Once you make sure your package works on testPyPI you can go ahead and upload it to PyPI. If it’s the first time you’re uploading this package, then you can use the following command to upload it.
py -m twine upload --repository PyPI dist/*
But if you already have a package published and you just need to upload a new version of it, you should use this command.
py -m twine upload --skip-existing dist/*
And voila, your package is uploaded and ready to use.