Trusted answers to developer questions

What is the urllib module in Python 3?

Get Started With Data Science

Learn the fundamentals of Data Science with this free course. Future-proof your career by adding Data Science skills to your toolkit — or prepare to land a job in AI, Machine Learning, or Data Analysis.

Urllib is a Python 3 package that allows you to access, and interact with, websites using their URL’s (Uniform Resource Locator). It has several modules for working with URL’s, these are shown in the illustration below:

svg viewer

urllib.request

Using urllib.request, with urlopen, allows you to open the specified URL. This is shown in the code snippet below:

from urllib.request import urlopen
myURL = urlopen("http://www.google.com/")
print(myURL.read())

Once the URL has been opened, the read() function is used to get the entire HTML code for the webpage.


urllib.parse

The code snippet below shows the usage of urllib.parse:

from urllib.parse import urlparse
parsedUrl = urlparse('https://www.educative.io/track/python-for-programmers')
print(parsedUrl)

The URL is split into its components such as the protocol scheme used, the network location netloc and the path to the webpage.


urllib.error

This module is used to catch exceptions encountered from url.request. These exceptions, or errors, are classified as follows:

  1. URL Error, which is raised when the URL is incorrect, or when there is an internet connectivity issue.
  2. HTTP Error, which is raised because of HTTP errors such as 404 (request not found) and 403 (request forbidden). The following code snippet shows the usage of urlib.error:
from urllib.request import urlopen, HTTPError, URLError
try:
myURL = urlopen("http://ww.educative.xyz/")
except HTTPError as e:
print('HTTP Error code: ', e.code)
except URLError as e:
print('URL Error: ', e.reason)
else:
print('No Error.')

A request to open http://ww.educative.xyz/ is caught by the URLError exception; the URL is invalid. Experiment with the exceptions by opening different URL’s.

RELATED TAGS

urllib
python3
module
Copyright ©2024 Educative, Inc. All rights reserved
Did you find this helpful?