How to work with the command line using subprocess in Python
The command line has been an important part of software development and system administration for many years. Python’s subprocess module allows users to work with the command line smoothly and powerfully, connecting Python scripts with external processes.
Understand subprocess basics
Before diving into advanced usage, let’s establish a foundation by understanding the basics of the subprocess module. The module allows us to spawn new processes, connect to their input/output/error pipes, and obtain their return codes as given below:
import subprocessresult = subprocess.run(["ls", "-l"], capture_output=True, text=True)print(result.stdout)
In the example above, we use subprocess.run() to execute the ls -l command and capture its output.
Handle input and output streams
One of the subprocess module’s strengths are handling input and output streams. We can communicate with the subprocess by feeding input to its standard input stream and capturing its output from the standard output stream, as follows:
import subprocess# Input: Writing to the subprocessinput_data = "Hello\nfrom\nsubprocess"result = subprocess.run(["grep", "subprocess"], input=input_data, capture_output=True, text=True)# Output: Reading from the subprocessoutput = result.stdoutprint(output)
The code example above demonstrates how to interact with the subprocess by providing input and capturing the output using the standard input and output streams.
Advanced features—pipes and chains
The subprocess module supports the creation of complex pipelines by connecting the output of one command as the input to another. This allows for sophisticated command line workflows directly from Python, as given in the following example:
import subprocess# Creating a pipeline: ls -l | grep "ed"first = subprocess.run(["ls", "-l"], capture_output=True, text=True)result = subprocess.run(["grep", "ed"], input=first.stdout, capture_output=True, text=True)print(result.stdout)
In the code example above, we use the output of the ls -l command as input for the grep ed command, demonstrating the ability of Python’s subprocess module to create powerful command line chains.
Handle errors and exceptions
Robust error handling is essential in command line interactions. Therefore, we’ll also see the subprocess module, allowing us to capture and handle errors gracefully, as shown in the following example:
import subprocesstry:subprocess.run(["touch"], check=True)except subprocess.CalledProcessError as e:print(f"Command failed with return code {e.returncode}")print(e.stderr)
In the code example above, we use the check=True parameter to raise a CalledProcessError in case the command returns a non-zero exit code. This enables effective error handling in our Python scripts.
Conclusion
Python’s subprocess module is a versatile tool for interacting with the command line, allowing developers to execute commands, handle input/output streams, create complex pipelines, and manage errors effectively. As we explore the subprocess module further, we’ll find it a useful to automate and improve the workflows.
