The "What", "When" and "How" of Multithreading

“Idleness is a form of passivity in an active universe.”
― Michael Bassey Johnson, Master of Maxims

Multi-core processors and smart Operating systems enable our computers to perform multiple operations at the same time - this not only improves the user experience but also boosts the temporal performance of the programs being executed.

Like most of the humanity, Software engineers tend to have a linear thinking - where response to a step must be elicited before another step is taken. This linearity in the logic makes it a bit trickier to take the advantage of multi-core architecture. Here, I intend to tap into the world of threads, a small yet powerful part of an operating system.

What is multithreading?

Getting the basics right

  • Process: Every program that is being executed on a machine. A program is nothing more than a piece of code that can be executed by a computer and when it is loaded onto the memory, it becomes a process. At any given time, a process can have one of the following five states:

    • Start: Process initialisation
    • Ready: Process waiting to be asssigned to a processor by the Operating System for execution
    • Running: Process assigned to a processor by OS scheduler
    • Waiting: Process temporarily stopped waiting for a resource, for example: downloading a file from internet
    • Terminated: The execution completed its execution, or terminated by the OS
  • Thread: At an atomic level a process comprises of many threads executing simultaneously and sharing various resources such as memory. Each thread can be managed independently by the OS’s scheduler.
  • Main Thread: Every Process in running state triggers at least one thread, called the main thread.

Now, since we have the stage all set, it the time to pull the curtains on the protagonist of this post - “Multithreading”:

Multithreading is the ability of a CPU to execute multiple threads within a single process in a concurrent manner to maximize its utilisation.

Phew, that was not so hard after all! Was it? In simpler terms, it is a way to make sure that the CPU never sits ideal and ensures that one part of our process does not end up blocking the other part - this where the software engineers can make design decisions.

For example, the browser you are reading this blog post on runs multiple threads: one for rendering the content of this blog, another to load more resources from a server and so on.

When to use multithreading?

As mentioned before, using multiple threads can speed things up by doing things in parallel and ensuring maximum utilisation of the CPU. There are a couple of questions to consider before putting multithreading to use:

  • Is the process CPU-bound? If the program is responsible for performing some sequential CPU intensive task then, multithreading will be useless. For example, processing an image from state 1 to state 2 requires some matrix transformations making it a CPU intensive process, multithreading may not be that beneficial in this case.
  • Is the process IO-bound? If the program is blocked because of waiting for some input/output (except user input), multithreading can greatly speed things up. For example, if the program is supposed to download 200 images from a website, instead of looping over the list of image URL one by one, we can use multithreading to access multiple URLs at the same time.

    Note: Web Crawlers can greatly benefit from multithreading - especially the fetcher and the downloader component in a web crawler.

  • Does the underlying hardware support multiple operations? If the underlying hardware does not support running multiple operations simultaneously, the operating system will just create a perception of having the tasks run at the same time.

“How” of Multithreading: An illustration using Python

You can download the code from here.

After developing a basic understanding of what is multithreading and when to use it, now is the time to dive right into the how to implement using Python. Here, we’ll be exploring Python’s ThreadPoolExecutor. This was introduced into the language in version 3.2 and provides an intuitive high-level interface for executing IO-bound tasks asynchronously.

We will try to download 15 images from https://unsplash.com/ first using single thread and then, using ThreadPoolExecutor. It is a good way of illustrating how fetching web resources in a synchronous fashion can create a bottleneck. By using ThreadPoolExecutor we can improve the perormance by fetching multple resources concurrently, mitigating the bottleneck created by waiting for response from each URL.

Using single thread

In this example we iterate over the list of available urls and make a GET request to each one using requests [link to be added] get method. In order to measure the time taken by this opeation we will use time module.

Below we have created a function called make_request() - it makes a get request to each URL that is passed to it and returns the content bytes to the caller. We will not handle any errors for the sake of simplicity.

import requests
import time

url_list = [...]

def make_request(url):
  res = requests.get(url)
  return res

t1 = time.time()

for i, url in enumerate(url_list):
  res = make_request(url)
  print(f"Downloaded item \033[32m{i + 1} of {len(url_list)}\033[m")

t2 = time.time()

print(f"---- Downloaded {len(url_list)} items in {round(t2 - t1)} seconds----")

Output:

Using ThreadPoolExecutor for multithreading

In this example we will firstly instantiate ThreadPoolExecutor object and submit it the task of making requests using our makerequest()_ function defined in previous example. We will instantiate a ThreadPoolExecutor as a context manger like so:

with ThreadPoolExecutor(max_workers=4) as executor:

Below the ThreadPoolExectuor’s map method will automatically map each url in the urllist and call makerequest() function in a different thread - improving the overall performance of the script.

Here’s the code:

import requests
import time
from concurrent.futures import ThreadPoolExecutor

url_list = [...

def make_request(url):
  res = requests.get(url)
  return res

t1 = time.time()

with ThreadPoolExecutor() as executor:
  results = executor.map(make_request, url_list)
  for result in results:
  print(type(result))

print(f"---- Downloaded {len(url_list)} items in {round(t2 - t1)} seconds----")

Output:

We have printed the type of each downloaded image to make sure that the resource was successfully downloaded. Unlike in the previous example, the downloading did not happen iteratively, each request was made using a thread in a concurrent manner and the whole IO process took about 1 second, which is a stellar improvement in the execution.

Further Reading:

A gentle introduction to multithreading

Benefits of threads

An Intro to Threading in Python

CPU Basics: Multiple CPUs, Cores, and Hyper-Threading Explained

Multi-Threaded Crawler in Python

Processes by uic.edu



Published 7 Jan 2020

Comments

Comments


Rishabh Madan on Twitter