{"id":113173,"date":"2020-05-20T08:14:18","date_gmt":"2020-05-20T08:14:18","guid":{"rendered":"https:\/\/blog.finxter.com\/?p=8676"},"modified":"2020-05-20T08:14:18","modified_gmt":"2020-05-20T08:14:18","slug":"python-list-concatenation-add-vs-inplace-add-vs-extend","status":"publish","type":"post","link":"https:\/\/sickgaming.net\/blog\/2020\/05\/20\/python-list-concatenation-add-vs-inplace-add-vs-extend\/","title":{"rendered":"Python List Concatenation: Add (+) vs INPLACE Add (+=) vs extend()"},"content":{"rendered":"<p>A wildly popular operation you&#8217;ll find in any (non-trivial) code base is to <a rel=\"noreferrer noopener\" href=\"https:\/\/blog.finxter.com\/concatenate-lists-in-python\/\" target=\"_blank\">concatenate lists<\/a>&#8212;but there are multiple methods to accomplish this. Master coders will always choose the right method for the right problem. <\/p>\n<p>This tutorial shows you the difference between three methods to concatenate <a href=\"https:\/\/blog.finxter.com\/python-lists\/\" target=\"_blank\" rel=\"noreferrer noopener\">lists<\/a>:<\/p>\n<ul>\n<li><strong>Concatenate two lists with the <code>+<\/code> operator. <\/strong>For example, the expression <code>[1, 2, 3] + [4, 5]<\/code> results in a new list <code>[1, 2, 3, 4, 5]<\/code>. <a rel=\"noreferrer noopener\" href=\"https:\/\/blog.finxter.com\/concatenate-lists-in-python\/\" target=\"_blank\">More here.<\/a><\/li>\n<li><strong>Concatenate two lists with the <code>+=<\/code> operator.<\/strong> This operation is inplace which means that you don&#8217;t create a new list and the result of the expression <code>lst += [4, 5]<\/code> is to add the elements on the right to the existing list object <code>lst<\/code>. <a rel=\"noreferrer noopener\" href=\"https:\/\/blog.finxter.com\/concatenate-lists-in-python\/\" target=\"_blank\">More here.<\/a><\/li>\n<li><strong>Concatenate two lists with the <code>extend()<\/code> method of Python lists. <\/strong>Like <code>+=<\/code>, this method modifies an existing list in place. So the result of <code>lst.extend([4, 5])<\/code> adds the elements <code>4<\/code> and <code>5<\/code> to the list <code>lst<\/code>. <a href=\"https:\/\/blog.finxter.com\/python-list-extend\/\" target=\"_blank\" rel=\"noreferrer noopener\">More here.<\/a><\/li>\n<\/ul>\n<p><strong>To summarize: the difference between the <code>+<\/code> method and the <code>+=<\/code> and <code>extend()<\/code> methods is that the former creates a new list and the latter modify an existing list object in-place.<\/strong><\/p>\n<p>You can quickly compare those three methods in the following interactive code shell:<\/p>\n<figure><iframe loading=\"lazy\" src=\"https:\/\/repl.it\/@finxter\/listinplaceaddvsaddvsextend?lite=true\" allowfullscreen=\"true\" width=\"100%\" height=\"1000px\"><\/iframe><\/figure>\n<p><strong>Puzzle<\/strong>: Can you already figure out the outputs of this code snippet?<\/p>\n<p>Fear not if you can&#8217;t! I&#8217;ll explain you each detailed example next.<\/p>\n<h2>Method 1: Add (+)<\/h2>\n<figure class=\"wp-block-embed-youtube wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\">\n<div class=\"wp-block-embed__wrapper\">\n<div class=\"ast-oembed-container\"><iframe loading=\"lazy\" title=\"How to Concatenate Lists in Python? [Interactive Guide]\" width=\"1400\" height=\"788\" src=\"https:\/\/www.youtube.com\/embed\/9rrqInUeG8U?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe><\/div>\n<\/div>\n<\/figure>\n<p>The standard way of adding two lists is to use the <code>+<\/code> operator like this:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"># METHOD 1: ADD +\nlst = ['Alice', 'Bob', 'Ann']\nlst_new = lst + [42, 21]\nprint(lst)\nprint(lst_new)<\/pre>\n<p>While the <code>+<\/code> operator is the most readable one (especially for beginner coders), it&#8217;s not the best choice in most scenarios. The reason is that it creates a new list each time you call it. This can become very slow and I&#8217;ve seen many practical code snippets where the list data structure used with the <code>+<\/code> operator is the bottleneck of the whole <a href=\"https:\/\/academy.finxter.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">algorithm<\/a>.<\/p>\n<p>In the above code snippet, you create two <a href=\"https:\/\/blog.finxter.com\/python-list-methods-cheat-sheet-instant-pdf-download\/\" target=\"_blank\" rel=\"noreferrer noopener\">list objects<\/a> in memory&#8212;even though your goal is probably just to update the existing list <code>['Alice', 'Bob', 'Ann']<\/code>.<\/p>\n<p>This can be nicely demonstrated in the code visualization tool:<\/p>\n<p> <iframe loading=\"lazy\" width=\"800\" height=\"500\" frameborder=\"0\" src=\"https:\/\/pythontutor.com\/iframe-embed.html#code=%23%20METHOD%201%3A%20ADD%20%2B%0Alst%20%3D%20%5B'Alice',%20'Bob',%20'Ann'%5D%0Alst_new%20%3D%20lst%20%2B%20%5B42,%2021%5D%0Aprint%28lst%29%0Aprint%28lst_new%29&#038;codeDivHeight=400&#038;codeDivWidth=350&#038;cumulative=false&#038;curInstr=0&#038;heapPrimitives=nevernest&#038;origin=opt-frontend.js&#038;py=3&#038;rawInputLstJSON=%5B%5D&#038;textReferences=false\"> <\/iframe> <\/p>\n<p>Just keep clicking &#8220;Next&#8221; until the second list appears in memory.<\/p>\n<h2>Method 2: INPLACE Add (+=)<\/h2>\n<p>The <code>+= <\/code>operator is not well understood by the <a href=\"https:\/\/www.python.org\/\" target=\"_blank\" rel=\"noreferrer noopener\">Python <\/a>community. Many of my students (<a rel=\"noreferrer noopener\" href=\"https:\/\/blog.finxter.com\/subscribe\/\" target=\"_blank\">join us for free<\/a>) believe the add operation <code>lst += [3, 4]<\/code> is just short for <code>lst = lst + [3, 4]<\/code>. This is wrong and I&#8217;ll demonstrate it in the following example:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"># METHOD 2: INPLACE ADD +=\nlst = ['Alice', 'Bob', 'Ann']\nlst_old = lst\nlst += [42, 21]\nprint(lst)\nprint(lst_old)<\/pre>\n<p>Again, you can visualize the memory objects with the following interactive tool (click &#8220;Next&#8221;):<\/p>\n<p> <iframe loading=\"lazy\" width=\"800\" height=\"500\" frameborder=\"0\" src=\"https:\/\/pythontutor.com\/iframe-embed.html#code=%23%20METHOD%202%3A%20INPLACE%20ADD%20%2B%3D%0Alst%20%3D%20%5B'Alice',%20'Bob',%20'Ann'%5D%0Alst_old%20%3D%20lst%0Alst%20%2B%3D%20%5B42,%2021%5D%0Aprint%28lst%29%0Aprint%28lst_old%29&#038;codeDivHeight=400&#038;codeDivWidth=350&#038;cumulative=false&#038;curInstr=0&#038;heapPrimitives=nevernest&#038;origin=opt-frontend.js&#038;py=3&#038;rawInputLstJSON=%5B%5D&#038;textReferences=false\"> <\/iframe> <\/p>\n<p>The takeaway is that the <code>+=<\/code> operation performs INPLACE add. It changes an existing list object rather than creating a new one. This makes it more efficient in the majority of cases. Only if you absolutely need to create a new list, you should use the <code>+<\/code> operator. In all other cases, you should use the <code>+=<\/code> operator or the <a href=\"https:\/\/blog.finxter.com\/python-list-append-vs-extend\/\" target=\"_blank\" rel=\"noreferrer noopener\"><code>extend()<\/code> method<\/a>. <\/p>\n<p>Speaking of which&#8230;<\/p>\n<h2>Method 3: Extend()<\/h2>\n<p>Like the previous method <code>+=<\/code>, the <code>list.extend(iterable)<\/code> method adds a number of elements to the end of a list. The <a href=\"https:\/\/blog.finxter.com\/python-list-methods\/\" target=\"_blank\" rel=\"noreferrer noopener\">method <\/a>operators in-place so no new list object is created.<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"># METHOD 3: EXTEND()\nlst = ['Alice', 'Bob', 'Ann']\nlst_old = lst\nlst.extend([42, 21])\nprint(lst)\nprint(lst_old)<\/pre>\n<p>Here&#8217;s the interactive memory visualization:<\/p>\n<p> <iframe loading=\"lazy\" width=\"800\" height=\"500\" frameborder=\"0\" src=\"https:\/\/pythontutor.com\/iframe-embed.html#code=%23%20METHOD%203%3A%20EXTEND%28%29%0Alst%20%3D%20%5B'Alice',%20'Bob',%20'Ann'%5D%0Alst_old%20%3D%20lst%0Alst.extend%28%5B42,%2021%5D%29%0Aprint%28lst%29%0Aprint%28lst_old%29&#038;codeDivHeight=400&#038;codeDivWidth=350&#038;cumulative=false&#038;curInstr=0&#038;heapPrimitives=nevernest&#038;origin=opt-frontend.js&#038;py=3&#038;rawInputLstJSON=%5B%5D&#038;textReferences=false\"> <\/iframe> <\/p>\n<p>Click &#8220;Next&#8221; and explore how the memory allocation &#8220;unfolds&#8221; as the execution proceeds.<\/p>\n<h2>Speed Comparison Benchmark<\/h2>\n<p>Having understood the differences of the three methods <code>+<\/code> vs <code>+=<\/code> vs <code>extend()<\/code>, you may ask: what&#8217;s the <a href=\"https:\/\/blog.finxter.com\/python-cprofile-a-helpful-guide-with-prime-example\/\" target=\"_blank\" rel=\"noreferrer noopener\">fastest<\/a>?<\/p>\n<p>To help you understand why it&#8217;s important to choose the best method, I&#8217;ve performed a detailed <a rel=\"noreferrer noopener\" href=\"https:\/\/blog.finxter.com\/python-profilers-how-to-speed-up-your-python-app\/\" target=\"_blank\">speed<\/a> benchmark on my Intel i7 (8th Gen) Notebook (8GB RAM) concatenating lists with increasing sizes using the three methods described previously.<\/p>\n<p>Here&#8217;s the result:<\/p>\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/blog.finxter.com\/wp-content\/uploads\/2020\/05\/speed.jpg\" alt=\"\" class=\"wp-image-8685\" srcset=\"https:\/\/blog.finxter.com\/wp-content\/uploads\/2020\/05\/speed.jpg 640w, https:\/\/blog.finxter.com\/wp-content\/uploads\/2020\/05\/speed-300x225.jpg 300w\" sizes=\"(max-width: 640px) 100vw, 640px\" \/><\/figure>\n<p>The plot shows that with increasing list size, the runtime difference between the <code>+<\/code> method (Method 1), and the <code>+=<\/code> and <code>extend()<\/code> methods (Methods 2 and 3) becomes increasingly evident. The former creates a new list for each concatenation operation&#8212;and this slows it down. <\/p>\n<p><strong>Result<\/strong>: Thus, both INPLACE methods <code>+=<\/code> and <code>extend()<\/code> are more than 30% faster than the <code>+<\/code> method for list concatenation.<\/p>\n<p>You can reproduce the result with the following code snippet:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">import time # Compare runtime of three methods\nlist_sizes = [i * 300000 for i in range(40)]\nruntimes_1 = [] # Method 1: + Operator\nruntimes_2 = [] # Method 2: += Operator\nruntimes_3 = [] # Method 3: extend() for size in list_sizes: to_add = list(range(size)) # Get time stamps time_0 = time.time() lst = [1] lst = lst + to_add time_1 = time.time() lst = [1] lst += to_add time_2 = time.time() lst = [1] lst.extend(to_add) time_3 = time.time() # Calculate runtimes runtimes_1.append((size, time_1 - time_0)) runtimes_2.append((size, time_2 - time_1)) runtimes_3.append((size, time_3 - time_2)) # Plot everything\nimport matplotlib.pyplot as plt\nimport numpy as np runtimes_1 = np.array(runtimes_1)\nruntimes_2 = np.array(runtimes_2)\nruntimes_3 = np.array(runtimes_3) print(runtimes_1)\nprint(runtimes_2)\nprint(runtimes_3) plt.plot(runtimes_1[:,0], runtimes_1[:,1], label='Method 1: +')\nplt.plot(runtimes_2[:,0], runtimes_2[:,1], label='Method 2: +=')\nplt.plot(runtimes_3[:,0], runtimes_3[:,1], label='Method 3: extend()') plt.xlabel('list size')\nplt.ylabel('runtime (seconds)') plt.legend()\nplt.savefig('speed.jpg')\nplt.show()<\/pre>\n<p>If you liked this tutorial, <a href=\"https:\/\/blog.finxter.com\/subscribe\/\">join my free email list<\/a> where I&#8217;ll send you the most comprehensive FREE Python email academy right in your INBOX. <\/p>\n<p><a href=\"https:\/\/blog.finxter.com\/subscribe\/\" target=\"_blank\" rel=\"noreferrer noopener\">Join the Finxter Community Now!<\/a><\/p>\n<h2>Where to Go From Here?<\/h2>\n<p>Enough theory, let\u2019s get some practice!<\/p>\n<p>To become successful in coding, you need to get out there and solve real problems for real people. That\u2019s how you can become a six-figure earner easily. And that\u2019s how you polish the skills you really need in practice. After all, what\u2019s the use of learning theory that nobody ever needs?<\/p>\n<p><strong>Practice projects is how you sharpen your saw in coding!<\/strong><\/p>\n<p>Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?<\/p>\n<p>Then become a Python freelance developer! It\u2019s the best way of approaching the task of improving your Python skills\u2014even if you are a complete beginner.<\/p>\n<p>Join my free webinar <a rel=\"noreferrer noopener\" href=\"https:\/\/blog.finxter.com\/webinar-freelancer\/\" target=\"_blank\">\u201cHow to Build Your High-Income Skill Python\u201d<\/a> and watch how I grew my coding business online and how you can, too\u2014from the comfort of your own home.<\/p>\n<p><a href=\"https:\/\/blog.finxter.com\/webinar-freelancer\/\" target=\"_blank\" rel=\"noreferrer noopener\">Join the free webinar now!<\/a><\/p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>A wildly popular operation you&#8217;ll find in any (non-trivial) code base is to concatenate lists&#8212;but there are multiple methods to accomplish this. Master coders will always choose the right method for the right problem. This tutorial shows you the difference between three methods to concatenate lists: Concatenate two lists with the + operator. For example, [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[857],"tags":[73,468,528],"class_list":["post-113173","post","type-post","status-publish","format-standard","hentry","category-python-tut","tag-programming","tag-python","tag-tutorial"],"_links":{"self":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts\/113173","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/comments?post=113173"}],"version-history":[{"count":0,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts\/113173\/revisions"}],"wp:attachment":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/media?parent=113173"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/categories?post=113173"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/tags?post=113173"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}