How to implement multithreading in Python to run tasks in parallel

Parallel task programming is crucial in the development of efficient applications. Python, although it does not have true parallel execution due to the Global Interpreter Lock (GIL), offers effective tools to implement multithreading. In this article, we will explore how to implement multithreading in Python to run tasks in parallel, optimizing your application for better performance.

What is Multithreading?

Multithreading is the ability of a program to execute multiple threads of execution concurrently. Each thread represents a sequence of execution that can run independently. This allows, for example, performing several tasks simultaneously in an application, enhancing efficiency and response time.

Advantages of Multithreading

• Better efficiency: It can perform multiple operations at the same time, making better use of system resources.
• Improved application responsiveness: Applications can remain interactive while background tasks are being performed.
• Facilitates concurrent task programming: It allows for dividing a problem into subproblems that can be solved simultaneously.

threading Library in Python

Python provides the threading library, which facilitates the creation and management of threads. With this library, you can easily create, start, and manage threads.

Installation

The threading library is included in the standard Python library, so there is no need to install anything additional. However, make sure you have Python installed on your system.

Basic Multithreading Example

Next, let's look at a basic example that illustrates how to use the threading library to run multiple tasks in parallel.

import threading
import time

def task(name):
    print(f'Task {name} started')
    time.sleep(2)
    print(f'Task {name} completed')

# Create threads
thread1 = threading.Thread(target=task, args=("1",))
thread2 = threading.Thread(target=task, args=("2",))

# Start threads
thread1.start()
thread2.start()

# Wait for threads to finish
thread1.join()
thread2.join()

print('All tasks completed')

Code Analysis

1. Imports: Importing threading and time.
2. Defining the Task: Creating a function called task that simulates assigned work with a wait time.
3. Creating Threads: Using threading.Thread to create threads, specifying the target function and arguments.
4. Starting and Synchronization: With start(), we begin the threads and use join() to wait for them to finish.

Handling Shared Data

One of the issues with multithreading is concurrent access to shared data, which can lead to race conditions. To handle this problem, Python provides synchronization mechanisms like Lock.

Using Lock

import threading

# Create a Lock object
lock = threading.Lock()
counter = 0

def increment():
    global counter
    for _ in range(100000):
        with lock:  # Acquire the lock
            counter += 1  # Critical section

# Create threads
thread1 = threading.Thread(target=increment)
thread2 = threading.Thread(target=increment)

# Start threads
thread1.start()
thread2.start()

# Wait for threads to finish
thread1.join()
thread2.join()

print(f'Final counter: {counter}')

Explanation of the Example

1. Lock: We create a Lock object that is used to control access to the shared variable counter.
2. increment Function: Increments the counter using a lock to prevent other threads from accessing the critical section while it's in use.
3. Creating and Managing Threads: Similar to the previous example but focused on ensuring the integrity of counter.

Limitations of Multithreading in Python

Although multithreading is useful, there are areas where it is not the ideal solution:

Global Interpreter Lock (GIL)

The GIL is a mechanism that ensures only one thread executes Python bytecode at a time. This means that multithreading cannot fully leverage multi-core systems for CPU-bound operations. In such cases, using multiprocessing is recommended.

CPU and Resource Usage

For tasks that are CPU-intensive, such as complex mathematical calculations, it's better to use the multiprocessing module, which allows real parallel execution by creating separate processes instead of threads.

Conclusion

Multithreading in Python is a powerful tool that allows the execution of tasks in parallel, improving efficiency and performance of applications. While it presents certain challenges, such as handling shared data and the limitations of the GIL, with the proper use of the threading library and synchronization mechanisms, highly effective applications can be created. 

For tasks that require true parallelism, the multiprocessing module is an alternative you should consider. With a correct implementation of these tools, you will be well-prepared to handle concurrent tasks in your Python projects.