How to work with the command line using subprocess in Python

The command line has been an important part of software development and system administration for many years. Python’s subprocess module allows users to work with the command line smoothly and powerfully, connecting Python scripts with external processes.

Understand subprocess basics

Before diving into advanced usage, let’s establish a foundation by understanding the basics of the subprocess module. The module allows us to spawn new processes, connect to their input/output/error pipes, and obtain their return codes as given below:

import subprocess
result = subprocess.run(["ls", "-l"], capture_output=True, text=True)
print(result.stdout)

In the example above, we use subprocess.run() to execute the ls -l command and capture its output.

Handle input and output streams

One of the subprocess module’s strengths are handling input and output streams. We can communicate with the subprocess by feeding input to its standard input stream and capturing its output from the standard output stream, as follows:

import subprocess
# Input: Writing to the subprocess
input_data = "Hello\nfrom\nsubprocess"
result = subprocess.run(
    ["grep", "subprocess"], input=input_data, capture_output=True, text=True
)
# Output: Reading from the subprocess
output = result.stdout
print(output)

The code example above demonstrates how to interact with the subprocess by providing input and capturing the output using the standard input and output streams.

Advanced features—pipes and chains

The subprocess module supports the creation of complex pipelines by connecting the output of one command as the input to another. This allows for sophisticated command line workflows directly from Python, as given in the following example:

import subprocess
# Creating a pipeline: ls -l | grep "ed"
first = subprocess.run(["ls", "-l"], capture_output=True, text=True)
result = subprocess.run(["grep", "ed"], input=first.stdout, capture_output=True, text=True)
print(result.stdout)

In the code example above, we use the output of the ls -l command as input for the grep ed command, demonstrating the ability of Python’s subprocess module to create powerful command line chains.

Handle errors and exceptions

Robust error handling is essential in command line interactions. Therefore, we’ll also see the subprocess module, allowing us to capture and handle errors gracefully, as shown in the following example:

import subprocess
try:
    subprocess.run(["touch"], check=True)
except subprocess.CalledProcessError as e:
    print(f"Command failed with return code {e.returncode}")
    print(e.stderr)

In the code example above, we use the check=True parameter to raise a CalledProcessError in case the command returns a non-zero exit code. This enables effective error handling in our Python scripts.

Conclusion

Python’s subprocess module is a versatile tool for interacting with the command line, allowing developers to execute commands, handle input/output streams, create complex pipelines, and manage errors effectively. As we explore the subprocess module further, we’ll find it a useful to automate and improve the workflows.

Free Resources

Copyright ©2025 Educative, Inc. All rights reserved