A Complete Guide to Using Python Progress Bars

We’ve all been there. Our code is running a job, and it’s going to take a while. Scratching our head, we pray that our code isn’t stuck in some infinite loop. When dealing with large data sets, however, even the simplest operations can take hours to complete. After all, a watched pot never boils, and a watched loop never breaks

If you didn’t guess already, I am an impatient guy. Using an ETA or progress bar while executing code works wonders for my anxiety.

Fortunately, there are libraries out there that provide this functionality using only a couple of lines. In this tutorial, I will introduce four libraries and compare them based on a variety of factors.

How to Set Up a Progress Bar in Python

The code used in this tutorial was tested on an Ubuntu system with Python 3.6.9 installed.

I would highly recommend setting up a virtual environment with all the required libraries for testing. Here is how you can do it.

$ virtualenv progress_env
$ source ./progress_env/bin/activate
$ pip3 install progress progressbar2 alive-progress tqdm

The four libraries that we are going to work with include: 

1. Progress
2. Progressbar2
3. Alive-progress
4. tqdm

More on Python: 13 Python Code Snippets You Need to Know

Progress Bar Using Progress

Progress is a fairly simple and intuitive library. It provides a wide variety of styles to pick from. All progress bars can be integrated into our loops using either a context manager or by wrapping up an iterable object into a method. Let’s take a look at some of them.

A simple progress bar that is filled with hash.

from time import sleep
from progress.bar import Bar

with Bar('Processing...') as bar:
    for i in range(100):
        sleep(0.02)
        bar.next()

We first import the bar class from the progress.bar module and create its object. We supply the prefix 'Processing...' that will be added to the front of our progress bar. To update the progress bar, we use the next() method at the end of each iteration.

There are multiple arguments available to us like fill and suffix. Let’s try modifying the fill from # to @ and also look at the ETA.

from time import sleep
from progress.bar import Bar

with Bar('Loading', fill='@', suffix='%(percent).1f%% - %(eta)ds') as bar:
    for i in range(100):
        sleep(0.02)
        bar.next()

There are times when we don’t know how long an operation is going to take. In such cases, we can use spinners to visualize the process rather than progress bars.

from time import sleep
from progress.spinner import MoonSpinner

with MoonSpinner('Processing…') as bar:
    for i in range(100):
        sleep(0.02)
        bar.next()

Progress Bars Using Progressbar2

Python-Progressbar (aka Progressbar2) is an extremely popular and easy to use module. It provides some very powerful features like auto-resizing. You can control the format of the progress-bar in the form of widgets like “AbsoluteETA” and “AdaptiveETA.”

Let’s take a look at a basic use-case:

from time import sleep
from progressbar import progressbar

for i in progressbar(range(100)):
    sleep(0.02)

Although in the beginning we installed the library as progressbar2, it will be imported as progressbar.

Combining Progress-Bars With Print Output

One thing that we didn’t discuss earlier with the progress library: What happens when the for loop contains a print function?

Let’s see:

from time import sleep
from progress.bar import Bar

with Bar('Processing',max = 5) as bar:
    for i in range(5):
        sleep(0.1)
        print('\n',i)
        bar.next()

As you can see, this will make the progress bar very messy, and sadly, there is no workaround in the progress library for this. However, Progressbar2 allows us to redirect standard output.

from time import sleep
from progressbar import progressbar
for i in progressbar(range(100), redirect_stdout=True):
    print('Some text', i)
    sleep(0.1)

Building Animated Progress Bars With Alive-Progress

If you love animations and you want to make your progress bars look interactive, then look further. This library contains some of the coolest looking progress-bars available. Take a look:

(Left: Alive-Progress Available Progress-Bars, Right: Alive-Progress Available Spinners (Source: Documentation))

In terms of code, it’s pretty similar. We will present a default progress-bar, and then one with a couple modifications:

from alive_progress import alive_bar
from time import sleep

with alive_bar(100) as bar:   # default setting
    for i in range(100):
        sleep(0.03)
        bar()                        # call after consuming one item

# using bubble bar and notes spinner
with alive_bar(200, bar = 'bubbles', spinner = 'notes2') as bar:
    for i in range(200):
        sleep(0.03)
        bar()                        # call after consuming one item

As you can see above, we get information regarding ETA and iterations per second along with the progress bar. The Github documentation includes details regarding some really powerful features that this library has.

Progress Bars inTQDM

If you need a no-bullshit, speedy yet powerful progress-bar, look no further than TQDM.

TQDM provides a fast framework with a lot of customization and information. Even with all the features, it’s still extremely simple to set up. Take a look:

from tqdm import tqdm
from time import sleep

for i in tqdm(range(100)):
    sleep(0.02)

By default, you get info such as percentage, ETA and even iterables per second. TQDM works on all major platforms like Linux, Windows and Mac. It integrates seamlessly in a console, a graphical user interface (GUI) and even in IPython/Jupyter notebooks.

IPython/Jupyter is supported via the tqdm.notebook sub-module. Let’s take an example with nested progress bars:

This feature with multiple progress-bars is unique to TQDM.

There’s also a pandas integration:

import pandas as pd
import numpy as np
from tqdm import tqdm

df = pd.DataFrame(np.random.randint(0, 1000, (100000, 600)))

tqdm.pandas(desc="my bar!")

df.progress_apply(lambda x: x**2)

TQDM also has an experimental GUI-Version called tqdm-gui.

TQDM is an extremely important project for individuals working in data science. 

More on Python: PIP Command Not Found on Windows: A Guide

Advantages of Progress Bars

Progress bars provide us with:

1. A visual cue that processing is underway.
2. A reliable estimate of the code execution time. This is extremely important when training on large data sets.
3. Indication if the program is stuck in an error and must be stopped.

TQDM has the lowest overhead of all the libraries discussed above at about 60 ns per iteration. By comparison, the well-established ProgressBar2 has an 800 ns/iter overhead.

Most of these libraries don’t have any dependencies.

Progressbar2 and alive-progress officially support print statements along with the progress bar.

In this tutorial, we saw how with only a few lines of code, you can implement progress bars in your python scripts.