Search⌘ K
AI Features
Home
Courses
Python 201 - Interactively Learn Advanced Concepts in Python 3

urllib.request

Explore how to use the urllib.request module to open and fetch URLs. Learn to access HTTP response details like headers and status codes, handle redirects, download files, and customize user-agent strings for web requests in Python.

We'll cover the following...

The urllib.request module is primarily used for opening and fetching URLs. Let’s take a look at some of the things we can do with the urlopen function:

Python 3.5
import urllib.request
url = urllib.request.urlopen('https://www.google.com/')
print (url.geturl())
#'https://www.google.com/'
print (url.info())
#<http.client.HTTPMessage object at 0x7fddc2de04e0>
header = url.info()
print (header.as_string())
print (url.getcode())
#200

Here we import our module and ask it to open Google’s URL. Now we have an HTTPResponse object that we can interact with. The first thing we do is call the geturl method which will return the URL of the resource that was retrieved. This is useful for finding out if we followed a redirect.

Next we call info, which will return meta-data about the page, such as headers. Because of this, we assign that result to our ...