How to use cURL in Python
cURL is a powerful command-line tool that developers often rely on for various tasks. In this Answer, we will learn how to use the cURL command in Python and explore some of its advanced use cases.
To extend the limits of cURL and use it in different applications, Python provides us with two ways that enable us to combine the control and features of cURL. Let’s discuss both of these methods.
Using the subprocess module
To use cURL in Python, one approach is to leverage the subprocess module, which allows the execution of external commands from within Python scripts.
Code example
import subprocessdef curl_request(url):# Define the command to execute using curlcommand = ['curl', '-s', '-o', '-', url]# Execute the curl command and capture the outputresult = subprocess.run(command, capture_output=True, text=True)# Return the stdout of the curl commandreturn result.stdout# Make a curl request to https://www.google.com/response = curl_request('https://www.google.com/')# Make a curl request to https://www.google.com/print(response)
Code explanation
Line 1: We import the
subprocessmodule, which allows us to execute external commands from within Python.Line 2: We define a function called
curl_requestthat takes a URL as an argument.Line 5: We construct a list called
command, which contains the command-line arguments for the cURL command. In this example, we use the-soption to silence the progress meter, and the-o-option to output the response tostdout.Line 8: We use the
subprocess.run()function to execute the cURL command. Thecapture_output=Trueargument captures the command’sstdout, and thetext=Trueargument ensures that the output is returned as a string.Line 11: We return the stdout of the cURL command as the response from the
curl_requestfunction.Lines 14–17: We invoke the
curl_requestfunction with a URL and print the response.
Using the pycURL library
The pycURL library provides a convenient way to make HTTP requests, interact with web services, and handle different protocols like HTTP, HTTPS, FTP, and more. The pycURL library is built on top of the libcURL library, a highly reliable and widely-used networking library that supports a vast range of protocols and features.
Code example
import pycurlfrom io import BytesIO# Create a new cURL objectcurl = pycurl.Curl()# Set the URL to fetchcurl.setopt(curl.URL, 'https://www.google.com/')# Create a BytesIO object to store the responsebuffer = BytesIO()curl.setopt(curl.WRITEDATA, buffer)# Perform the requestcurl.perform()# Get the response bodyresponse = buffer.getvalue()# Print the responseprint(response.decode('utf-8'))# Close the cURL objectcurl.close()
Code explanation
Line 1: We import the
pycURLmodule.Line 2: We import the
BytesIOmodule to handle the binary data.Line 5: We create a new
cURLobject to make HTTP requests.Line 8: We set the URL to be requested using the
.setopt()method.Line 11: We create a
BytesIOobject to store the response data.Line 12: We set the variable that will store the data.
Line 15: We perform the request.
Line 18: We get the response and store it in the
responsevariable.Lines 21–24: We print the response and close the
cURLobject.
Free Resources