03-10-2020, 09:22 AM
How to Concatenate Lists in Python? [Interactive Guide]
<div><p>So you have two or more lists and you want to glue them together. This is called <strong>list concatenation</strong>. How can you do that?</p>
<figure class="wp-block-embed-youtube wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio">
<div class="wp-block-embed__wrapper">
<div class="ast-oembed-container"><iframe title="How to Concatenate Lists in Python? [Interactive Guide]" width="1400" height="788" src="https://www.youtube.com/embed/9rrqInUeG8U?feature=oembed" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe></div>
</div>
</figure>
<p>These are six ways of concatenating lists:</p>
<ol>
<li>List concatenation operator <code>+</code></li>
<li>List <code>append()</code> method</li>
<li>List <code>extend()</code> method</li>
<li>Asterisk operator <code>*</code></li>
<li><code>Itertools.chain()</code></li>
<li>List comprehension</li>
</ol>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">a = [1, 2, 3]
b = [4, 5, 6] # 1. List concatenation operator +
l_1 = a + b # 2. List append() method
l_2 = [] for el in a: l_2.append(el) for el in b: l_2.append(el) # 3. List extend() method
l_3 = []
l_3.extend(a)
l_3.extend(b) # 4. Asterisk operator *
l_4 = [*a, *b] # 5. Itertools.chain()
import itertools
l_5 = list(itertools.chain(a, b)) # 6. List comprehension
l_6 = [el for lst in (a, b) for el in lst]</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">'''
l_1 --> [1, 2, 3, 4, 5, 6]
l_2 --> [1, 2, 3, 4, 5, 6]
l_3 --> [1, 2, 3, 4, 5, 6]
l_4 --> [1, 2, 3, 4, 5, 6]
l_5 --> [1, 2, 3, 4, 5, 6]
l_6 --> [1, 2, 3, 4, 5, 6] '''</pre>
<p><strong>What’s the best way to concatenate two lists?</strong></p>
<p>If you’re busy, you may want to know the best answer immediately. Here it is:</p>
<p><strong>To concatenate two lists <code>l1</code>, <code>l2</code>, use the <code>l1.extend(l2)</code> method which is the fastest and the most readable. </strong></p>
<p><strong>To concatenate more than two lists, use the unpacking (asterisk) operator <code>[*l1, *l2, ..., *ln]</code>.</strong></p>
<p>You’ll find all of those six ways to concatenate lists in practical code projects so it’s important that you know them very well. Let’s dive into each of them and discuss the pros and cons.</p>
<h2>1. List Concatenation with + Operator</h2>
<p>If you use the + operator on two integers, you’ll get the sum of those integers. But if you use the + operator on two lists, you’ll get a new list that is the concatenation of those lists.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">l1 = [1, 2, 3]
l2 = [4, 5, 6]
l3 = l1 + l2
print(l3)</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">[1, 2, 3, 4, 5, 6]</pre>
<p>The problem with the + operator for list concatenation is that it creates a new list for each list concatenation operation. This can be very inefficient if you use the + operator multiple times in a loop.</p>
<p><strong>Performance:</strong></p>
<p>How fast is the + operator really? Here’s a common scenario how people use it to add new elements to a list in a loop. This is very inefficient:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import time start = time.time() l = []
for i in range(100000): l = l + [i] stop = time.time() print("Elapsed time: " + str(stop - start))</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">Elapsed time: 14.438847541809082</pre>
<p>The experiments were performed on my notebook with an Intel® Core™ i7-8565U 1.8GHz processor (with Turbo Boost up to 4.6 GHz) and 8 GB of RAM.</p>
<p>I measured the start and stop timestamps to calculate the total elapsed time for adding 100,000 elements to a list.</p>
<p>The result shows that it takes 14 seconds to perform this operation.</p>
<p>This seems slow (it is!). So let’s investigate some other methods to concatenate and their performance:</p>
<h2>2. List Concatenation with append()</h2>
<p>The <code>list.append(x)</code> method—as the name suggests—appends element <code>x</code> to the end of the <code>list</code>. <a rel="noreferrer noopener" href="https://blog.finxter.com/python-list-append/" target="_blank">You can read my full blog tutorial about it here.</a></p>
<p>Here’s a similar example that shows how you can use the <code>append()</code> method to concatenate two lists <code>l1</code> and <code>l2</code> and store the result in the list <code>l1</code>.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">l1 = [1, 2, 3]
l2 = [4, 5, 6]
for element in l2: l1.append(element)
print(l1)</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">[1, 2, 3, 4, 5, 6]</pre>
<p>It seems a bit odd to concatenate two lists by iterating over each element in one list. You’ll learn about an alternative in the next section but let’s first check the performance!</p>
<p><strong>Performance:</strong></p>
<p>I performed a similar experiment as before.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import time start = time.time() l = []
for i in range(100000): l.append(i) stop = time.time() print("Elapsed time: " + str(stop - start))
</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">Elapsed time: 0.006505012512207031</pre>
<p>I measured the start and stop timestamps to calculate the total elapsed time for adding 100,000 elements to a list.</p>
<p>The result shows that it takes only 0.006 seconds to perform this operation. This is a more than 2,000X improvement compared to the 14 seconds when using the + operator for the same problem.</p>
<p>While this is readable and performant, let’s investigate some other methods to concatenate lists.</p>
<h2>3. List Concatenation with extend()</h2>
<p>You’ve studied the <code>append()</code> method in the previous paragraphs. A major problem with it is that if you want to concatenate two lists, you have to iterate over all elements of the lists and append them one by one. This is complicated and there surely must be a better way. Is there?</p>
<p>You bet there is!</p>
<p>The method <code>list.extend(iter)</code> adds all elements in <code>iter</code> to the end of the <code>list</code>.</p>
<p>The difference between <code>append()</code> and <code>extend()</code> is that the former adds only one element and the latter adds a collection of elements to the list.</p>
<p>Here’s a similar example that shows how you can use the <code>extend()</code> method to concatenate two lists <code>l1</code> and <code>l2</code>.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">l1 = [1, 2, 3]
l2 = [4, 5, 6]
l1.extend(l2)
print(l1)</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">[1, 2, 3, 4, 5, 6]</pre>
<p>The code shows that the <code>extend()</code> method is more readable and concise than iteratively calling the <code>append()</code> method as done before. </p>
<p>But is it also fast? Let’s check the performance!</p>
<p><strong>Performance:</strong></p>
<p>I performed a similar experiment as before for the <code>append()</code> method.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import time start = time.time() l = []
l.extend(range(100000)) stop = time.time() print("Elapsed time: " + str(stop - start))
</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">Elapsed time: 0.0</pre>
<p>I measured the start and stop timestamps to calculate the total elapsed time for adding 100,000 elements to a list.</p>
<p>The result shows that it takes negligible time to run the code (0.0 seconds compared to 0.006 seconds for the <code>append()</code> operation above).</p>
<p><strong>The <code>extend()</code> method is the most concise and fastest way to concatenate lists.</strong></p>
<p>But there’s still more…</p>
<h2>4. List Concatenation with Asterisk Operator *</h2>
<p>There are many applications of the asterisk operator (<a rel="noreferrer noopener" href="https://blog.finxter.com/what-is-asterisk-in-python/" target="_blank">read this blog article</a> to learn about all of them). But one nice trick is to use it as an unpacking operator that “unpacks” the contents of a container data structure such as a list or a dictionary into another one. </p>
<p>Here’s a similar example that shows how you can use the asterisk operator to concatenate two lists <code>l1</code> and <code>l2</code>.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">l1 = [1, 2, 3]
l2 = [4, 5, 6]
l3 = [*l1, *l2]
print(l3)
</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">[1, 2, 3, 4, 5, 6]</pre>
<p>The code shows that the asterisk operator is short, concise, and readable (well, for Python pros at least). And the nice thing is that you can also use it to concatenate more than two lists (example: <code>l4 = [*l1, *l2, *l3]</code>). </p>
<p>But how about its performance?</p>
<p><strong>Performance:</strong></p>
<p>You’ve seen before that <code>extend()</code> is very fast. How does the asterisk operator compare against the <code>extend()</code> method? Let’s find out!</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import time l1 = list(range(0,1000000))
l2 = list(range(1000000,2000000)) start = time.time()
l1.extend(l2)
stop = time.time() print("Elapsed time extend(): " + str(stop - start))
# Elapsed time extend(): 0.015623092651367188 start = time.time()
l3 = [*l1, *l2]
stop = time.time() print("Elapsed time asterisk: " + str(stop - start))
# Elapsed time asterisk: 0.022125720977783203</pre>
<p>I measured the start and stop timestamps to calculate the total elapsed time for concatenating two lists of 1,000,000 elements each.</p>
<p>The result shows that the extend method is 32% faster than the asterisk operator for the given experiment. </p>
<p><strong>To merge two lists, use the extend() method which is more readable and faster compared to unpacking.</strong></p>
<p>Okay, let’s get really nerdy—what about external libraries?</p>
<h2>5. List Concatenation with itertools.chain()</h2>
<p>Python’s <a rel="noreferrer noopener" href="https://docs.python.org/3/library/itertools.html" target="_blank">itertools </a>module provides many tools to manipulate iterables such as <a rel="noreferrer noopener" href="https://blog.finxter.com/python-list-methods/" target="_blank">Python lists</a>. Interestingly, they also have a way of concatenating two iterables. After converting the resulting iterable to a list, you can also reach the same result as before.</p>
<p>Here’s a similar example that shows how you can use the <code>itertools.chain()</code> method to concatenate two lists <code>l1</code> and <code>l2</code>.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import itertools l1 = [1, 2, 3]
l2 = [4, 5, 6]
l3 = list(itertools.chain(l1, l2))
print(l3)</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">[1, 2, 3, 4, 5, 6]</pre>
<p>The result is the same. As readability is worse than using extend(), append(), or even the asterisk operator, you may wonder:</p>
<p>Is it any faster?</p>
<p><strong>Performance:</strong></p>
<p>You’ve seen before that <code>extend()</code> is very fast. How does the asterisk operator compare against the <code>extend()</code> method? Let’s find out!</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import time
import itertools l1 = list(range(2000000,3000000))
l2 = list(range(3000000,4000000)) start = time.time()
l1.extend(l2)
stop = time.time() print("Elapsed time extend(): " + str(stop - start))
# Elapsed time extend(): 0.015620946884155273 start = time.time()
l3 = list(itertools.chain(l1, l2))
stop = time.time() print("Elapsed time chain(): " + str(stop - start))
# Elapsed time chain(): 0.08469700813293457
</pre>
<p>I measured the start and stop timestamps to calculate the total elapsed time for concatenating two lists of 1,000,000 elements each.</p>
<p>The result shows that the <code>extend()</code> method is 5.6 times faster than the <code>itertools.chain()</code> method for the given experiment. </p>
<p>Here’s my recommendation:</p>
<p><strong>Never use <code>itertools.chain()</code> because it’s not only less readable, it’s also slower than the <code>extend()</code> method or even the built-in unpacking method for more than two lists.</strong></p>
<p>So let’s go back to more Pythonic solutions: everybody loves list comprehension…</p>
<h2>6. List Concatenation with List Comprehension</h2>
<p>List comprehension is a compact way of creating lists. The simple formula is <code>[ expression + context ]</code>.</p>
<ul>
<li>Expression: What to do with each list element?</li>
<li>Context: What list elements to select? It consists of an arbitrary number of for and if statements.</li>
</ul>
<p>The example <code>[x for x in range(3)]</code> creates the list <code>[0, 1, 2]</code>.</p>
<p><a href="https://blog.finxter.com/list-comprehension/" target="_blank" rel="noreferrer noopener">Need a refresher on list comprehension? Read this blog tutorial.</a></p>
<p>Here’s a similar example that shows how you can use list comprehension to concatenate two lists <code>l1</code> and <code>l2</code>.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">l1 = [1, 2, 3]
l2 = [4, 5, 6]
l3 = [ x for lst in (l1, l2) for x in lst ]
print(l3)
</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">[1, 2, 3, 4, 5, 6]</pre>
<p>What a mess! You’d never write code like this in practice. And yet, tutorials like <a rel="noreferrer noopener" href="https://mkyong.com/python/how-to-concatenate-lists-in-python/" target="_blank">this</a> one propose this exact same method.</p>
<p>The idea is to first iterate over all lists you want to concatenate in the first part of the context <code>lst in (l1, l2)</code> and then iterate over all elements in the respective list in the second part of the context <code>for x in lst</code>. </p>
<p>Do we get a speed performance out of it?</p>
<p><strong>Performance:</strong></p>
<p>You’ve seen before that <code>extend()</code> is very fast. How does the asterisk operator compare against the <code>extend()</code> method? Let’s find out!</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import time l1 = list(range(4000000,5000000))
l2 = list(range(6000000,7000000)) start = time.time()
l1.extend(l2)
stop = time.time() print("Elapsed time extend(): " + str(stop - start))
# Elapsed time extend(): 0.015620946884155273 start = time.time()
l3 = [ x for lst in (l1, l2) for x in lst ]
stop = time.time() print("Elapsed time list comprehension: " + str(stop - start))
# Elapsed time list comprehension: 0.11590242385864258
</pre>
<p>I measured the start and stop timestamps to calculate the total elapsed time for concatenating two lists of 1,000,000 elements each.</p>
<p>The result shows that the <code>extend()</code> method is an order of magnitude faster than using list comprehension. </p>
<h2>Summary</h2>
<p><strong>To concatenate two lists l1, l2, use the <code>l1.extend(l2)</code> method which is the fastest and the most readable. </strong></p>
<p><strong>To concatenate more than two lists, use the unpacking operator <code>[*l1, *l2, ..., *ln]</code>.</strong></p>
<p>Want to use your skills to create your dream life coding from home? Join my <a href="https://blog.finxter.com/webinar-freelancer/" target="_blank" rel="noreferrer noopener">Python freelancer webinar</a> and learn how to create your home-based coding business.</p>
<p>Click: <a rel="noreferrer noopener" href="https://blog.finxter.com/webinar-freelancer/" target="_blank">How to Become a Python Freelancer?</a></p>
</div>
https://www.sickgaming.net/blog/2020/03/...ive-guide/
<div><p>So you have two or more lists and you want to glue them together. This is called <strong>list concatenation</strong>. How can you do that?</p>
<figure class="wp-block-embed-youtube wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio">
<div class="wp-block-embed__wrapper">
<div class="ast-oembed-container"><iframe title="How to Concatenate Lists in Python? [Interactive Guide]" width="1400" height="788" src="https://www.youtube.com/embed/9rrqInUeG8U?feature=oembed" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe></div>
</div>
</figure>
<p>These are six ways of concatenating lists:</p>
<ol>
<li>List concatenation operator <code>+</code></li>
<li>List <code>append()</code> method</li>
<li>List <code>extend()</code> method</li>
<li>Asterisk operator <code>*</code></li>
<li><code>Itertools.chain()</code></li>
<li>List comprehension</li>
</ol>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">a = [1, 2, 3]
b = [4, 5, 6] # 1. List concatenation operator +
l_1 = a + b # 2. List append() method
l_2 = [] for el in a: l_2.append(el) for el in b: l_2.append(el) # 3. List extend() method
l_3 = []
l_3.extend(a)
l_3.extend(b) # 4. Asterisk operator *
l_4 = [*a, *b] # 5. Itertools.chain()
import itertools
l_5 = list(itertools.chain(a, b)) # 6. List comprehension
l_6 = [el for lst in (a, b) for el in lst]</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">'''
l_1 --> [1, 2, 3, 4, 5, 6]
l_2 --> [1, 2, 3, 4, 5, 6]
l_3 --> [1, 2, 3, 4, 5, 6]
l_4 --> [1, 2, 3, 4, 5, 6]
l_5 --> [1, 2, 3, 4, 5, 6]
l_6 --> [1, 2, 3, 4, 5, 6] '''</pre>
<p><strong>What’s the best way to concatenate two lists?</strong></p>
<p>If you’re busy, you may want to know the best answer immediately. Here it is:</p>
<p><strong>To concatenate two lists <code>l1</code>, <code>l2</code>, use the <code>l1.extend(l2)</code> method which is the fastest and the most readable. </strong></p>
<p><strong>To concatenate more than two lists, use the unpacking (asterisk) operator <code>[*l1, *l2, ..., *ln]</code>.</strong></p>
<p>You’ll find all of those six ways to concatenate lists in practical code projects so it’s important that you know them very well. Let’s dive into each of them and discuss the pros and cons.</p>
<h2>1. List Concatenation with + Operator</h2>
<p>If you use the + operator on two integers, you’ll get the sum of those integers. But if you use the + operator on two lists, you’ll get a new list that is the concatenation of those lists.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">l1 = [1, 2, 3]
l2 = [4, 5, 6]
l3 = l1 + l2
print(l3)</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">[1, 2, 3, 4, 5, 6]</pre>
<p>The problem with the + operator for list concatenation is that it creates a new list for each list concatenation operation. This can be very inefficient if you use the + operator multiple times in a loop.</p>
<p><strong>Performance:</strong></p>
<p>How fast is the + operator really? Here’s a common scenario how people use it to add new elements to a list in a loop. This is very inefficient:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import time start = time.time() l = []
for i in range(100000): l = l + [i] stop = time.time() print("Elapsed time: " + str(stop - start))</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">Elapsed time: 14.438847541809082</pre>
<p>The experiments were performed on my notebook with an Intel® Core™ i7-8565U 1.8GHz processor (with Turbo Boost up to 4.6 GHz) and 8 GB of RAM.</p>
<p>I measured the start and stop timestamps to calculate the total elapsed time for adding 100,000 elements to a list.</p>
<p>The result shows that it takes 14 seconds to perform this operation.</p>
<p>This seems slow (it is!). So let’s investigate some other methods to concatenate and their performance:</p>
<h2>2. List Concatenation with append()</h2>
<p>The <code>list.append(x)</code> method—as the name suggests—appends element <code>x</code> to the end of the <code>list</code>. <a rel="noreferrer noopener" href="https://blog.finxter.com/python-list-append/" target="_blank">You can read my full blog tutorial about it here.</a></p>
<p>Here’s a similar example that shows how you can use the <code>append()</code> method to concatenate two lists <code>l1</code> and <code>l2</code> and store the result in the list <code>l1</code>.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">l1 = [1, 2, 3]
l2 = [4, 5, 6]
for element in l2: l1.append(element)
print(l1)</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">[1, 2, 3, 4, 5, 6]</pre>
<p>It seems a bit odd to concatenate two lists by iterating over each element in one list. You’ll learn about an alternative in the next section but let’s first check the performance!</p>
<p><strong>Performance:</strong></p>
<p>I performed a similar experiment as before.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import time start = time.time() l = []
for i in range(100000): l.append(i) stop = time.time() print("Elapsed time: " + str(stop - start))
</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">Elapsed time: 0.006505012512207031</pre>
<p>I measured the start and stop timestamps to calculate the total elapsed time for adding 100,000 elements to a list.</p>
<p>The result shows that it takes only 0.006 seconds to perform this operation. This is a more than 2,000X improvement compared to the 14 seconds when using the + operator for the same problem.</p>
<p>While this is readable and performant, let’s investigate some other methods to concatenate lists.</p>
<h2>3. List Concatenation with extend()</h2>
<p>You’ve studied the <code>append()</code> method in the previous paragraphs. A major problem with it is that if you want to concatenate two lists, you have to iterate over all elements of the lists and append them one by one. This is complicated and there surely must be a better way. Is there?</p>
<p>You bet there is!</p>
<p>The method <code>list.extend(iter)</code> adds all elements in <code>iter</code> to the end of the <code>list</code>.</p>
<p>The difference between <code>append()</code> and <code>extend()</code> is that the former adds only one element and the latter adds a collection of elements to the list.</p>
<p>Here’s a similar example that shows how you can use the <code>extend()</code> method to concatenate two lists <code>l1</code> and <code>l2</code>.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">l1 = [1, 2, 3]
l2 = [4, 5, 6]
l1.extend(l2)
print(l1)</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">[1, 2, 3, 4, 5, 6]</pre>
<p>The code shows that the <code>extend()</code> method is more readable and concise than iteratively calling the <code>append()</code> method as done before. </p>
<p>But is it also fast? Let’s check the performance!</p>
<p><strong>Performance:</strong></p>
<p>I performed a similar experiment as before for the <code>append()</code> method.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import time start = time.time() l = []
l.extend(range(100000)) stop = time.time() print("Elapsed time: " + str(stop - start))
</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">Elapsed time: 0.0</pre>
<p>I measured the start and stop timestamps to calculate the total elapsed time for adding 100,000 elements to a list.</p>
<p>The result shows that it takes negligible time to run the code (0.0 seconds compared to 0.006 seconds for the <code>append()</code> operation above).</p>
<p><strong>The <code>extend()</code> method is the most concise and fastest way to concatenate lists.</strong></p>
<p>But there’s still more…</p>
<h2>4. List Concatenation with Asterisk Operator *</h2>
<p>There are many applications of the asterisk operator (<a rel="noreferrer noopener" href="https://blog.finxter.com/what-is-asterisk-in-python/" target="_blank">read this blog article</a> to learn about all of them). But one nice trick is to use it as an unpacking operator that “unpacks” the contents of a container data structure such as a list or a dictionary into another one. </p>
<p>Here’s a similar example that shows how you can use the asterisk operator to concatenate two lists <code>l1</code> and <code>l2</code>.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">l1 = [1, 2, 3]
l2 = [4, 5, 6]
l3 = [*l1, *l2]
print(l3)
</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">[1, 2, 3, 4, 5, 6]</pre>
<p>The code shows that the asterisk operator is short, concise, and readable (well, for Python pros at least). And the nice thing is that you can also use it to concatenate more than two lists (example: <code>l4 = [*l1, *l2, *l3]</code>). </p>
<p>But how about its performance?</p>
<p><strong>Performance:</strong></p>
<p>You’ve seen before that <code>extend()</code> is very fast. How does the asterisk operator compare against the <code>extend()</code> method? Let’s find out!</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import time l1 = list(range(0,1000000))
l2 = list(range(1000000,2000000)) start = time.time()
l1.extend(l2)
stop = time.time() print("Elapsed time extend(): " + str(stop - start))
# Elapsed time extend(): 0.015623092651367188 start = time.time()
l3 = [*l1, *l2]
stop = time.time() print("Elapsed time asterisk: " + str(stop - start))
# Elapsed time asterisk: 0.022125720977783203</pre>
<p>I measured the start and stop timestamps to calculate the total elapsed time for concatenating two lists of 1,000,000 elements each.</p>
<p>The result shows that the extend method is 32% faster than the asterisk operator for the given experiment. </p>
<p><strong>To merge two lists, use the extend() method which is more readable and faster compared to unpacking.</strong></p>
<p>Okay, let’s get really nerdy—what about external libraries?</p>
<h2>5. List Concatenation with itertools.chain()</h2>
<p>Python’s <a rel="noreferrer noopener" href="https://docs.python.org/3/library/itertools.html" target="_blank">itertools </a>module provides many tools to manipulate iterables such as <a rel="noreferrer noopener" href="https://blog.finxter.com/python-list-methods/" target="_blank">Python lists</a>. Interestingly, they also have a way of concatenating two iterables. After converting the resulting iterable to a list, you can also reach the same result as before.</p>
<p>Here’s a similar example that shows how you can use the <code>itertools.chain()</code> method to concatenate two lists <code>l1</code> and <code>l2</code>.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import itertools l1 = [1, 2, 3]
l2 = [4, 5, 6]
l3 = list(itertools.chain(l1, l2))
print(l3)</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">[1, 2, 3, 4, 5, 6]</pre>
<p>The result is the same. As readability is worse than using extend(), append(), or even the asterisk operator, you may wonder:</p>
<p>Is it any faster?</p>
<p><strong>Performance:</strong></p>
<p>You’ve seen before that <code>extend()</code> is very fast. How does the asterisk operator compare against the <code>extend()</code> method? Let’s find out!</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import time
import itertools l1 = list(range(2000000,3000000))
l2 = list(range(3000000,4000000)) start = time.time()
l1.extend(l2)
stop = time.time() print("Elapsed time extend(): " + str(stop - start))
# Elapsed time extend(): 0.015620946884155273 start = time.time()
l3 = list(itertools.chain(l1, l2))
stop = time.time() print("Elapsed time chain(): " + str(stop - start))
# Elapsed time chain(): 0.08469700813293457
</pre>
<p>I measured the start and stop timestamps to calculate the total elapsed time for concatenating two lists of 1,000,000 elements each.</p>
<p>The result shows that the <code>extend()</code> method is 5.6 times faster than the <code>itertools.chain()</code> method for the given experiment. </p>
<p>Here’s my recommendation:</p>
<p><strong>Never use <code>itertools.chain()</code> because it’s not only less readable, it’s also slower than the <code>extend()</code> method or even the built-in unpacking method for more than two lists.</strong></p>
<p>So let’s go back to more Pythonic solutions: everybody loves list comprehension…</p>
<h2>6. List Concatenation with List Comprehension</h2>
<p>List comprehension is a compact way of creating lists. The simple formula is <code>[ expression + context ]</code>.</p>
<ul>
<li>Expression: What to do with each list element?</li>
<li>Context: What list elements to select? It consists of an arbitrary number of for and if statements.</li>
</ul>
<p>The example <code>[x for x in range(3)]</code> creates the list <code>[0, 1, 2]</code>.</p>
<p><a href="https://blog.finxter.com/list-comprehension/" target="_blank" rel="noreferrer noopener">Need a refresher on list comprehension? Read this blog tutorial.</a></p>
<p>Here’s a similar example that shows how you can use list comprehension to concatenate two lists <code>l1</code> and <code>l2</code>.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">l1 = [1, 2, 3]
l2 = [4, 5, 6]
l3 = [ x for lst in (l1, l2) for x in lst ]
print(l3)
</pre>
<p>Output:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">[1, 2, 3, 4, 5, 6]</pre>
<p>What a mess! You’d never write code like this in practice. And yet, tutorials like <a rel="noreferrer noopener" href="https://mkyong.com/python/how-to-concatenate-lists-in-python/" target="_blank">this</a> one propose this exact same method.</p>
<p>The idea is to first iterate over all lists you want to concatenate in the first part of the context <code>lst in (l1, l2)</code> and then iterate over all elements in the respective list in the second part of the context <code>for x in lst</code>. </p>
<p>Do we get a speed performance out of it?</p>
<p><strong>Performance:</strong></p>
<p>You’ve seen before that <code>extend()</code> is very fast. How does the asterisk operator compare against the <code>extend()</code> method? Let’s find out!</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import time l1 = list(range(4000000,5000000))
l2 = list(range(6000000,7000000)) start = time.time()
l1.extend(l2)
stop = time.time() print("Elapsed time extend(): " + str(stop - start))
# Elapsed time extend(): 0.015620946884155273 start = time.time()
l3 = [ x for lst in (l1, l2) for x in lst ]
stop = time.time() print("Elapsed time list comprehension: " + str(stop - start))
# Elapsed time list comprehension: 0.11590242385864258
</pre>
<p>I measured the start and stop timestamps to calculate the total elapsed time for concatenating two lists of 1,000,000 elements each.</p>
<p>The result shows that the <code>extend()</code> method is an order of magnitude faster than using list comprehension. </p>
<h2>Summary</h2>
<p><strong>To concatenate two lists l1, l2, use the <code>l1.extend(l2)</code> method which is the fastest and the most readable. </strong></p>
<p><strong>To concatenate more than two lists, use the unpacking operator <code>[*l1, *l2, ..., *ln]</code>.</strong></p>
<p>Want to use your skills to create your dream life coding from home? Join my <a href="https://blog.finxter.com/webinar-freelancer/" target="_blank" rel="noreferrer noopener">Python freelancer webinar</a> and learn how to create your home-based coding business.</p>
<p>Click: <a rel="noreferrer noopener" href="https://blog.finxter.com/webinar-freelancer/" target="_blank">How to Become a Python Freelancer?</a></p>
</div>
https://www.sickgaming.net/blog/2020/03/...ive-guide/