Create an account


Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
[Tut] Python List Concatenation: Add (+) vs INPLACE Add (+=) vs extend()

#1
Python List Concatenation: Add (+) vs INPLACE Add (+=) vs extend()

<div><p>A wildly popular operation you’ll find in any (non-trivial) code base is to <a rel="noreferrer noopener" href="https://blog.finxter.com/concatenate-lists-in-python/" target="_blank">concatenate lists</a>—but there are multiple methods to accomplish this. Master coders will always choose the right method for the right problem. </p>
<p>This tutorial shows you the difference between three methods to concatenate <a href="https://blog.finxter.com/python-lists/" target="_blank" rel="noreferrer noopener">lists</a>:</p>
<ul>
<li><strong>Concatenate two lists with the <code>+</code> operator. </strong>For example, the expression <code>[1, 2, 3] + [4, 5]</code> results in a new list <code>[1, 2, 3, 4, 5]</code>. <a rel="noreferrer noopener" href="https://blog.finxter.com/concatenate-lists-in-python/" target="_blank">More here.</a></li>
<li><strong>Concatenate two lists with the <code>+=</code> operator.</strong> This operation is inplace which means that you don’t create a new list and the result of the expression <code>lst += [4, 5]</code> is to add the elements on the right to the existing list object <code>lst</code>. <a rel="noreferrer noopener" href="https://blog.finxter.com/concatenate-lists-in-python/" target="_blank">More here.</a></li>
<li><strong>Concatenate two lists with the <code>extend()</code> method of Python lists. </strong>Like <code>+=</code>, this method modifies an existing list in place. So the result of <code>lst.extend([4, 5])</code> adds the elements <code>4</code> and <code>5</code> to the list <code>lst</code>. <a href="https://blog.finxter.com/python-list-extend/" target="_blank" rel="noreferrer noopener">More here.</a></li>
</ul>
<p><strong>To summarize: the difference between the <code>+</code> method and the <code>+=</code> and <code>extend()</code> methods is that the former creates a new list and the latter modify an existing list object in-place.</strong></p>
<p>You can quickly compare those three methods in the following interactive code shell:</p>
<figure><iframe src="https://repl.it/@finxter/listinplaceaddvsaddvsextend?lite=true" allowfullscreen="true" width="100%" height="1000px"></iframe></figure>
<p><strong>Puzzle</strong>: Can you already figure out the outputs of this code snippet?</p>
<p>Fear not if you can’t! I’ll explain you each detailed example next.</p>
<h2>Method 1: Add (+)</h2>
<figure class="wp-block-embed-youtube wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio">
<div class="wp-block-embed__wrapper">
<div class="ast-oembed-container"><iframe title="How to Concatenate Lists in Python? [Interactive Guide]" width="1400" height="788" src="https://www.youtube.com/embed/9rrqInUeG8U?feature=oembed" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe></div>
</div>
</figure>
<p>The standard way of adding two lists is to use the <code>+</code> operator like this:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># METHOD 1: ADD +
lst = ['Alice', 'Bob', 'Ann']
lst_new = lst + [42, 21]
print(lst)
print(lst_new)</pre>
<p>While the <code>+</code> operator is the most readable one (especially for beginner coders), it’s not the best choice in most scenarios. The reason is that it creates a new list each time you call it. This can become very slow and I’ve seen many practical code snippets where the list data structure used with the <code>+</code> operator is the bottleneck of the whole <a href="https://academy.finxter.com/" target="_blank" rel="noreferrer noopener">algorithm</a>.</p>
<p>In the above code snippet, you create two <a href="https://blog.finxter.com/python-list-methods-cheat-sheet-instant-pdf-download/" target="_blank" rel="noreferrer noopener">list objects</a> in memory—even though your goal is probably just to update the existing list <code>['Alice', 'Bob', 'Ann']</code>.</p>
<p>This can be nicely demonstrated in the code visualization tool:</p>
<p> <iframe width="800" height="500" frameborder="0" src="https://pythontutor.com/iframe-embed.html#code=%23%20METHOD%201%3A%20ADD%20%2B%0Alst%20%3D%20%5B'Alice',%20'Bob',%20'Ann'%5D%0Alst_new%20%3D%20lst%20%2B%20%5B42,%2021%5D%0Aprint%28lst%29%0Aprint%28lst_new%29&codeDivHeight=400&codeDivWidth=350&cumulative=false&curInstr=0&heapPrimitives=nevernest&origin=opt-frontend.js&py=3&rawInputLstJSON=%5B%5D&textReferences=false"> </iframe> </p>
<p>Just keep clicking “Next” until the second list appears in memory.</p>
<h2>Method 2: INPLACE Add (+=)</h2>
<p>The <code>+= </code>operator is not well understood by the <a href="https://www.python.org/" target="_blank" rel="noreferrer noopener">Python </a>community. Many of my students (<a rel="noreferrer noopener" href="https://blog.finxter.com/subscribe/" target="_blank">join us for free</a>) believe the add operation <code>lst += [3, 4]</code> is just short for <code>lst = lst + [3, 4]</code>. This is wrong and I’ll demonstrate it in the following example:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># METHOD 2: INPLACE ADD +=
lst = ['Alice', 'Bob', 'Ann']
lst_old = lst
lst += [42, 21]
print(lst)
print(lst_old)</pre>
<p>Again, you can visualize the memory objects with the following interactive tool (click “Next”):</p>
<p> <iframe width="800" height="500" frameborder="0" src="https://pythontutor.com/iframe-embed.html#code=%23%20METHOD%202%3A%20INPLACE%20ADD%20%2B%3D%0Alst%20%3D%20%5B'Alice',%20'Bob',%20'Ann'%5D%0Alst_old%20%3D%20lst%0Alst%20%2B%3D%20%5B42,%2021%5D%0Aprint%28lst%29%0Aprint%28lst_old%29&codeDivHeight=400&codeDivWidth=350&cumulative=false&curInstr=0&heapPrimitives=nevernest&origin=opt-frontend.js&py=3&rawInputLstJSON=%5B%5D&textReferences=false"> </iframe> </p>
<p>The takeaway is that the <code>+=</code> operation performs INPLACE add. It changes an existing list object rather than creating a new one. This makes it more efficient in the majority of cases. Only if you absolutely need to create a new list, you should use the <code>+</code> operator. In all other cases, you should use the <code>+=</code> operator or the <a href="https://blog.finxter.com/python-list-append-vs-extend/" target="_blank" rel="noreferrer noopener"><code>extend()</code> method</a>. </p>
<p>Speaking of which…</p>
<h2>Method 3: Extend()</h2>
<p>Like the previous method <code>+=</code>, the <code>list.extend(iterable)</code> method adds a number of elements to the end of a list. The <a href="https://blog.finxter.com/python-list-methods/" target="_blank" rel="noreferrer noopener">method </a>operators in-place so no new list object is created.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># METHOD 3: EXTEND()
lst = ['Alice', 'Bob', 'Ann']
lst_old = lst
lst.extend([42, 21])
print(lst)
print(lst_old)</pre>
<p>Here’s the interactive memory visualization:</p>
<p> <iframe width="800" height="500" frameborder="0" src="https://pythontutor.com/iframe-embed.html#code=%23%20METHOD%203%3A%20EXTEND%28%29%0Alst%20%3D%20%5B'Alice',%20'Bob',%20'Ann'%5D%0Alst_old%20%3D%20lst%0Alst.extend%28%5B42,%2021%5D%29%0Aprint%28lst%29%0Aprint%28lst_old%29&codeDivHeight=400&codeDivWidth=350&cumulative=false&curInstr=0&heapPrimitives=nevernest&origin=opt-frontend.js&py=3&rawInputLstJSON=%5B%5D&textReferences=false"> </iframe> </p>
<p>Click “Next” and explore how the memory allocation “unfolds” as the execution proceeds.</p>
<h2>Speed Comparison Benchmark</h2>
<p>Having understood the differences of the three methods <code>+</code> vs <code>+=</code> vs <code>extend()</code>, you may ask: what’s the <a href="https://blog.finxter.com/python-cprofile-a-helpful-guide-with-prime-example/" target="_blank" rel="noreferrer noopener">fastest</a>?</p>
<p>To help you understand why it’s important to choose the best method, I’ve performed a detailed <a rel="noreferrer noopener" href="https://blog.finxter.com/python-profilers-how-to-speed-up-your-python-app/" target="_blank">speed</a> benchmark on my Intel i7 (8th Gen) Notebook (8GB RAM) concatenating lists with increasing sizes using the three methods described previously.</p>
<p>Here’s the result:</p>
<figure class="wp-block-image size-large"><img src="https://blog.finxter.com/wp-content/uploads/2020/05/speed.jpg" alt="" class="wp-image-8685" srcset="https://blog.finxter.com/wp-content/uploads/2020/05/speed.jpg 640w, https://blog.finxter.com/wp-content/uplo...00x225.jpg 300w" sizes="(max-width: 640px) 100vw, 640px" /></figure>
<p>The plot shows that with increasing list size, the runtime difference between the <code>+</code> method (Method 1), and the <code>+=</code> and <code>extend()</code> methods (Methods 2 and 3) becomes increasingly evident. The former creates a new list for each concatenation operation—and this slows it down. </p>
<p><strong>Result</strong>: Thus, both INPLACE methods <code>+=</code> and <code>extend()</code> are more than 30% faster than the <code>+</code> method for list concatenation.</p>
<p>You can reproduce the result with the following code snippet:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import time # Compare runtime of three methods
list_sizes = [i * 300000 for i in range(40)]
runtimes_1 = [] # Method 1: + Operator
runtimes_2 = [] # Method 2: += Operator
runtimes_3 = [] # Method 3: extend() for size in list_sizes: to_add = list(range(size)) # Get time stamps time_0 = time.time() lst = [1] lst = lst + to_add time_1 = time.time() lst = [1] lst += to_add time_2 = time.time() lst = [1] lst.extend(to_add) time_3 = time.time() # Calculate runtimes runtimes_1.append((size, time_1 - time_0)) runtimes_2.append((size, time_2 - time_1)) runtimes_3.append((size, time_3 - time_2)) # Plot everything
import matplotlib.pyplot as plt
import numpy as np runtimes_1 = np.array(runtimes_1)
runtimes_2 = np.array(runtimes_2)
runtimes_3 = np.array(runtimes_3) print(runtimes_1)
print(runtimes_2)
print(runtimes_3) plt.plot(runtimes_1[:,0], runtimes_1[:,1], label='Method 1: +')
plt.plot(runtimes_2[:,0], runtimes_2[:,1], label='Method 2: +=')
plt.plot(runtimes_3[:,0], runtimes_3[:,1], label='Method 3: extend()') plt.xlabel('list size')
plt.ylabel('runtime (seconds)') plt.legend()
plt.savefig('speed.jpg')
plt.show()</pre>
<p>If you liked this tutorial, <a href="https://blog.finxter.com/subscribe/">join my free email list</a> where I’ll send you the most comprehensive FREE Python email academy right in your INBOX. </p>
<p><a href="https://blog.finxter.com/subscribe/" target="_blank" rel="noreferrer noopener">Join the Finxter Community Now!</a></p>
<h2>Where to Go From Here?</h2>
<p>Enough theory, let’s get some practice!</p>
<p>To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?</p>
<p><strong>Practice projects is how you sharpen your saw in coding!</strong></p>
<p>Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?</p>
<p>Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.</p>
<p>Join my free webinar <a rel="noreferrer noopener" href="https://blog.finxter.com/webinar-freelancer/" target="_blank">“How to Build Your High-Income Skill Python”</a> and watch how I grew my coding business online and how you can, too—from the comfort of your own home.</p>
<p><a href="https://blog.finxter.com/webinar-freelancer/" target="_blank" rel="noreferrer noopener">Join the free webinar now!</a></p></p>
</div>


https://www.sickgaming.net/blog/2020/05/...vs-extend/
Reply



Forum Jump:


Users browsing this thread:
2 Guest(s)

Forum software by © MyBB Theme © iAndrew 2016