How does the tomllib library work
The tomllib library is a new module in the Python 3.11 standard library for parsing TOML (Tom’s Obvious Minimal Language) files. TOML is a simple, human-readable format for configuration files. It’s becoming increasingly popular in the Python community because many popular tools and frameworks, such as Poetry, use it. The tomllib library provides several advantages over using third-party TOML parsers:
It’s a part of the Python standard library, so there is no need to install additional packages.
It’s fast and efficient.
It’s well-tested and maintained.
It’s compatible with the TOML 1.0 specification.
What is tomllib used for?
The tomllib library is useful for any project that needs to read TOML configuration files. Here are some specific examples of how tomllib can be used:
To parse the configuration file(e.g., pyproject.toml) for a Python project
For a Python package, parse the configuration file(e.g., setup.cfg or pyproject.toml)
To parse the configuration file for a Python tool or framework (e.g.,
Poetry).To parse TOML-formatted data files.
To use tomllib, install it using pip, as follows:
pip install tomllib
Once tomllib is installed, it can be imported into the Python program to start parsing and writing TOML files.
Parsing TOML files with tomllib
The tomllib library provides two functions for parsing TOML files:
load(): This reads a TOML file from disk and returns a Python dictionary.loads()(a.k.aload-s): This reads a TOML string from memory and returns a Python dictionary.
The load() function
To parse a TOML file, we can use the tomllib.load() function. This function takes the path to the TOML file as an argument and returns a Python dictionary containing the parsed TOML data.
For example, the following code parses the config.toml file and stores the parsed data in the config variable:
import tomllib# Load a TOML file from diskwith open("config.toml", "rb") as f:config = tomllib.load(f)# Access the values in the TOML dictionaryprint(config["tool"]["poetry"]["name"])
Here’s the explanation:
Line 1: We import the
tomllibmodule.Line 4: This line opens the
config.tomlfile in binary mode and reads its contents in a variablef. Thewithstatement ensures that the file is closed automatically when the code block exits.Line 5: The
tomllib.load()function reads the contents of the filefand parses it into a Python dictionary. The dictionary is then assigned to the variableconfig.Line 7: This code accesses the value of the
namekey in thepoetrytable of thetooltable of theconfigdictionary. The value is then printed to the console.
The loads() function
The loads() function reads a TOML string from memory and returns a Python dictionary. For example, the following code parses the TOML string and stores the parsed data in the output variable:
import tomllib# Load a TOML string from memorytoml_str = """[tool.poetry]name = "my-project"version = "1.0.0""""output = tomllib.loads(toml_str)# Access the values in the TOML dictionaryprint(output["tool"]["poetry"]["version"])
Here’s the explanation:
Line 1: We import the
tomllibmodule.Line 4–8: This creates a TOML string in the variable
toml_str. The TOML string contains a singletool.poetrytable with two key-value pairs:nameandversion.Line 9: The
tomllib.loads()function is used to parse the TOML string intoml_strinto a Python dictionary. The dictionary is then assigned to the variableoutput.Line 11: The value of the
versionkey is accessed in thepoetrytable of thetooltable of theoutputdictionary. The value is then printed to the console.
The load() function is mostly used unless there’s a specific reason to use loads(). The load() function is the more common function in use and is easier to understand.
Handling errors
When parsing a TOML file, several errors can occur, such as:
Syntax errors: These are errors in the structure of the TOML file, such as missing commas or unbalanced square brackets.
Type errors: These are problems in the values of the TOML file, such as an integer being assigned to a string variable.
Schema errors: These are errors in the structure of the TOML file according to a specific schema.
The tomllib library raises a TOMLDecodeError exception if it encounters an error while parsing a TOML file. This exception contains information about the error, such as the line number and column number where the error occurred.
We can handle this exception by catching it and taking appropriate action. For example, we can log the error and continue parsing the file, or we can abort parsing the file altogether.
Here’s an example of how to handle a TOMLDecodeError exception:
import tomllibtry:with open("config.toml", "rb") as f:config = tomllib.load(f)except tomllib.TOMLDecodeError as e:print(e.args[0])exit(1)
Here’s an explanation:
Line 1: This line imports the
tomllibmodule.Line 3–7: This code block uses a
trystatement to attempt to load the TOML fileconfig.toml. If the file is loaded successfully, the code will continue to the next line. However, if there’s an error loading the file, the code will catch theTOMLDecodeErrorexception and print the error message to the console. The code will then exit the program with a failure code of 1.
If tomllib is used in a production environment, it’s important to handle errors gracefully. These errors should not be ignored as they could lead to unexpected behavior.
Comparison of tomllib with other TOML parsers
Some other TOML parsers available for Python include tomli and pytoml. The tomllib library is the most recent TOML parser to be added to the Python standard library, and it’s generally considered to be the most reliable and efficient TOML parser.
Here’s a table comparing tomllib to other popular TOML parsers:
Comparison of TOML Parsers
Parser | Advantages | Disadvantages |
| Included in the Python standard library, fast and efficient, well-tested, maintained, and compatible with TOML 1.0 specification | Doesn’t have the functionality to write new TOML files |
| Supports writing TOML files, is fast and efficient, and has a large and active community | Not present in the Python standard library. |
| Supports writing TOML files, and has a simple and easy-to-use API | Not as fast and efficient as |
Limitations of tomllib
The tomllib library is a powerful and reliable TOML parser, but it does have a few limitations:
tomllibdoesn’t support writing TOML files, meaning it can’t be used to create new TOML files.tomllibdoesn’t support all the features of the TOML 1.1 specification. For example,tomllibdoes not support embedded TOML tables.
The tomllib library is a relatively new TOML parser, but it’s quickly becoming the most popular TOML parser for Python. The tomllib team is actively developing the parser and planning to add support for additional features in the future. Overall, tomllib is a great choice for parsing TOML files in Python.
Free Resources