Search⌘ K
AI Features
Home
Courses
Python FTW: Under the Hood

Let's Make a Giant String!

Explore how to create large strings efficiently in Python by understanding the performance costs of different concatenation methods. Learn to use timeit for timing, avoid inefficient plus operations, and apply best practices like using join for better performance and cleaner code.

We'll cover the following...

How long does it take to build a giant string? Let’s find out.

Python 3.5
def add_string_with_plus(iters):
    s = ""
    for i in range(iters):
        s += "xyz"
    assert len(s) == 3*iters
def add_bytes_with_plus(iters):
    s = b""
    for i in range(iters):
        s += b"xyz"
    assert len(s) == 3*iters
def add_string_with_format(iters):
    fs = "{}"*iters
    s = fs.format(*(["xyz"]*iters))
    assert len(s) == 3*iters
def add_string_with_join(iters):
    l = []
    for i in range(iters):
        l.append("xyz")
    s = "".join(l)
    assert len(s) == 3*iters
def convert_list_to_string(l, iters):
    s = "".join(l)
    assert len(s) == 3*iters

Let’s use 1000 iterations to build a string.

The time is given in seconds.

Python 3.5
import timeit
NUM_ITERS = 1000
print(timeit.timeit('add_string_with_plus(NUM_ITERS)', number=1000, globals=globals()))
print(timeit.timeit('add_bytes_with_plus(NUM_ITERS)', number=1000, globals=globals()))
print(timeit.timeit('add_string_with_format(NUM_ITERS)', number=1000, globals=globals()))
print(timeit.timeit('add_string_with_join(NUM_ITERS)', number=1000, globals=globals()))
l = ["xyz"]*NUM_ITERS
print(timeit.timeit('convert_list_to_string(l, NUM_ITERS)', number=1000, globals=globals()))

Let’s increase the number of iterations by a factor of 10.

Python 3.5
import timeit
NUM_ITERS = 10000
print(timeit.timeit('add_string_with_plus(NUM_ITERS)', number=1000, globals=globals()))
print(timeit.timeit('add_bytes_with_plus(NUM_ITERS)', number=1000, globals=globals()))
print(timeit.timeit('add_string_with_format(NUM_ITERS)', number=1000, globals=globals()))
print(timeit.timeit('add_string_with_join(NUM_ITERS)', number=1000, globals=globals()))
l = ["xyz"]*NUM_ITERS
print(timeit.timeit('convert_list_to_string(l, NUM_ITERS)', number=1000, globals=globals()))

Explanation

  • You can read more about timeit or %timeit through these links. They are used to measure the execution time of code pieces.
  • Don’t use + for generating long strings—in Python, str is immutable, so the left and right strings have to be copied into the new string for every pair of concentrations.

  • If you concatenate four strings of length 1010, the number ...