Generator Expressions
Explore Python generator expressions to transform sequences lazily, reducing memory usage compared to list comprehensions. Understand syntax differences, usage with next() and loops, single-use nature, and how chaining generators enables efficient data pipelines. Gain skills to process large or infinite datasets with minimal memory overhead.
We have already seen how generator functions and the yield keyword allow us to create iterators that produce values lazily. While this approach is powerful, defining a full function can feel excessive for simple, one-off transformations.
In many cases, we want to transform a sequence on the fly, without storing the results in memory, much like a list comprehension, but without its eager behavior. Python addresses this need with generator expressions.
A generator expression looks almost identical to a list comprehension, but uses parentheses instead of brackets. This small syntactic change turns an eager list builder into a lazy data stream, producing values only when they are requested. As a result, generator expressions allow us to process large or even infinite datasets efficiently and with minimal memory usage.
From brackets to parentheses
A generator expression uses almost the same syntax as a list comprehension, with one crucial difference: it is enclosed in parentheses (()) instead of square brackets ([]). This small syntactic change leads to a fundamental difference in behavior.
A list comprehension is eager. It ...