[Tut] Python List Concatenation: Add (+) vs INPLACE Add (+=) vs extend()

[xSicKxBot]

Python List Concatenation: Add (+) vs INPLACE Add (+=) vs extend()

A wildly popular operation you’ll find in any (non-trivial) code base is to concatenate lists—but there are multiple methods to accomplish this. Master coders will always choose the right method for the right problem.

This tutorial shows you the difference between three methods to concatenate lists:

• Concatenate two lists with the
  Code:

  +
  operator. For example, the expression
  Code:

  [1, 2, 3] + [4, 5]
  results in a new list
  Code:

  [1, 2, 3, 4, 5]
  . More here.
  
  
• Concatenate two lists with the
  Code:

  +=
  operator. This operation is inplace which means that you don’t create a new list and the result of the expression
  Code:

  lst += [4, 5]
  is to add the elements on the right to the existing list object
  Code:

  lst
  . More here.
  
  
• Concatenate two lists with the
  Code:

  extend()
  method of Python lists. Like
  Code:

  +=
  , this method modifies an existing list in place. So the result of
  Code:

  lst.extend([4, 5])
  adds the elements
  Code:

  4
  and
  Code:

  5
  to the list
  Code:

  lst
  . More here.
  

To summarize: the difference between the

Code:

+

method and the

Code:

+=

and

Code:

extend()

methods is that the former creates a new list and the latter modify an existing list object in-place.

You can quickly compare those three methods in the following interactive code shell:

https://repl.it/@finxter/listinplaceaddv...?lite=true

Puzzle: Can you already figure out the outputs of this code snippet?

Fear not if you can’t! I’ll explain you each detailed example next.

Method 1: Add (+)



The standard way of adding two lists is to use the

Code:

+

operator like this:

Code:

# METHOD 1: ADD + lst = ['Alice', 'Bob', 'Ann'] lst_new = lst + [42, 21] print(lst) print(lst_new)

While the

Code:

+

operator is the most readable one (especially for beginner coders), it’s not the best choice in most scenarios. The reason is that it creates a new list each time you call it. This can become very slow and I’ve seen many practical code snippets where the list data structure used with the

Code:

+

operator is the bottleneck of the whole algorithm.

In the above code snippet, you create two list objects in memory—even though your goal is probably just to update the existing list

Code:

['Alice', 'Bob', 'Ann']

.

This can be nicely demonstrated in the code visualization tool:

https://pythontutor.com/iframe-embed.htm...nces=false

Just keep clicking “Next” until the second list appears in memory.

Method 2: INPLACE Add (+=)

The

Code:

+=

operator is not well understood by the Pythoncommunity. Many of my students (join us for free) believe the add operation

Code:

lst += [3, 4]

is just short for

Code:

lst = lst + [3, 4]

. This is wrong and I’ll demonstrate it in the following example:

Code:

# METHOD 2: INPLACE ADD += lst = ['Alice', 'Bob', 'Ann'] lst_old = lst lst += [42, 21] print(lst) print(lst_old)

Again, you can visualize the memory objects with the following interactive tool (click “Next”):

https://pythontutor.com/iframe-embed.htm...nces=false

The takeaway is that the

Code:

+=

operation performs INPLACE add. It changes an existing list object rather than creating a new one. This makes it more efficient in the majority of cases. Only if you absolutely need to create a new list, you should use the

Code:

+

operator. In all other cases, you should use the

Code:

+=

operator or the

Code:

extend()

method.

Speaking of which…

Method 3: Extend()

Like the previous method

Code:

+=

, the

Code:

list.extend(iterable)

method adds a number of elements to the end of a list. The methodoperators in-place so no new list object is created.

Code:

# METHOD 3: EXTEND() lst = ['Alice', 'Bob', 'Ann'] lst_old = lst lst.extend([42, 21]) print(lst) print(lst_old)

Here’s the interactive memory visualization:

https://pythontutor.com/iframe-embed.htm...nces=false

Click “Next” and explore how the memory allocation “unfolds” as the execution proceeds.

Speed Comparison Benchmark

Having understood the differences of the three methods

Code:

+

vs

Code:

+=

vs

Code:

extend()

, you may ask: what’s the fastest?

To help you understand why it’s important to choose the best method, I’ve performed a detailed speed benchmark on my Intel i7 (8th Gen) Notebook (8GB RAM) concatenating lists with increasing sizes using the three methods described previously.

Here’s the result:



The plot shows that with increasing list size, the runtime difference between the

Code:

+

method (Method 1), and the

Code:

+=

and

Code:

extend()

methods (Methods 2 and 3) becomes increasingly evident. The former creates a new list for each concatenation operation—and this slows it down.

Result: Thus, both INPLACE methods

Code:

+=

and

Code:

extend()

are more than 30% faster than the

Code:

+

method for list concatenation.

You can reproduce the result with the following code snippet:

Code:

import time # Compare runtime of three methods list_sizes = [i * 300000 for i in range(40)] runtimes_1 = [] # Method 1: + Operator runtimes_2 = [] # Method 2: += Operator runtimes_3 = [] # Method 3: extend() for size in list_sizes: to_add = list(range(size)) # Get time stamps time_0 = time.time() lst = [1] lst = lst + to_add time_1 = time.time() lst = [1] lst += to_add time_2 = time.time() lst = [1] lst.extend(to_add) time_3 = time.time() # Calculate runtimes runtimes_1.append((size, time_1 - time_0)) runtimes_2.append((size, time_2 - time_1)) runtimes_3.append((size, time_3 - time_2)) # Plot everything import matplotlib.pyplot as plt import numpy as np runtimes_1 = np.array(runtimes_1) runtimes_2 = np.array(runtimes_2) runtimes_3 = np.array(runtimes_3) print(runtimes_1) print(runtimes_2) print(runtimes_3) plt.plot(runtimes_1[:,0], runtimes_1[:,1], label='Method 1: +') plt.plot(runtimes_2[:,0], runtimes_2[:,1], label='Method 2: +=') plt.plot(runtimes_3[:,0], runtimes_3[:,1], label='Method 3: extend()') plt.xlabel('list size') plt.ylabel('runtime (seconds)') plt.legend() plt.savefig('speed.jpg') plt.show()

If you liked this tutorial, join my free email list where I’ll send you the most comprehensive FREE Python email academy right in your INBOX.

Join the Finxter Community Now!

Where to Go From Here?

Enough theory, let’s get some practice!

To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

Practice projects is how you sharpen your saw in coding!

Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?

Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

Join my free webinar “How to Build Your High-Income Skill Python” and watch how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!