06-27-2020, 11:51 AM
List Difference | The Most Pythonic Way
<div><p class="has-luminous-vivid-amber-background-color has-background"><strong>Short answer</strong>: The most Pythonic way to compute the difference between two lists <code>l1</code> and <code>l2</code> is the list comprehension statement <code>[x for x in l1 if x not in set(l2)]</code>. This works even if you have duplicate list entries, it maintains the original list ordering, and it’s efficient due to the constant runtime complexity of the set membership operation.</p>
<figure class="wp-block-image size-large is-resized"><img src="https://blog.finxter.com/wp-content/uploads/2020/06/listDifference-1024x576.jpg" alt="" class="wp-image-10140" width="768" height="432" srcset="https://blog.finxter.com/wp-content/uploads/2020/06/listDifference-scaled.jpg 1024w, https://blog.finxter.com/wp-content/uplo...00x169.jpg 300w, https://blog.finxter.com/wp-content/uplo...68x432.jpg 768w" sizes="(max-width: 768px) 100vw, 768px" /></figure>
<p>What’s the best way to compute the difference between two lists in Python?</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">a = [5, 4, 3, 2, 1]
b = [4, 5, 6, 7] # a - b == [3, 2, 1]
# b - a == [6, 7]</pre>
<p>In Python, you always have multiple ways to solve the same (or a similar) problem. Let’s have an overview in the following interactive code shell:</p>
<p> <iframe src="https://trinket.io/embed/python/41397b54a0" width="100%" height="500" frameborder="0" marginwidth="0" marginheight="0" allowfullscreen></iframe> </p>
<p><em><strong>Exercise</strong>: Run the code and think about your preferred way!</em></p>
<p>Let’s dive into each of the methods to find the most Pythonic one for your particular scenario.</p>
<h2>Method 1: Set Difference</h2>
<p>The naive approach to solve this problem is to <a href="https://blog.finxter.com/python-list-to-set/" target="_blank" rel="noreferrer noopener" title="Python List to Set Conversion [Interactive Guide]">convert both lists into sets</a> and use the <a href="https://blog.finxter.com/sets-in-python/" target="_blank" rel="noreferrer noopener" title="The Ultimate Guide to Python Sets – with Harry Potter Examples">set minus</a> (or set difference) operation. </p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Method 1: Set Difference
print(set(a) - set(b))
# {1, 2, 3}
print(set(b) - set(a))
# {6, 7}</pre>
<p>This approach is elegant because it’s readable, efficient, and concise. </p>
<p>However, there are some unique properties to this method which you should be aware of:</p>
<ul>
<li>The result is a <a href="https://blog.finxter.com/sets-in-python/" title="The Ultimate Guide to Python Sets – with Harry Potter Examples" target="_blank" rel="noreferrer noopener">set </a>and not a <a href="https://blog.finxter.com/python-lists/" title="The Ultimate Guide to Python Lists" target="_blank" rel="noreferrer noopener">list</a>. You can <a href="https://blog.finxter.com/python-set-to-list/" title="Python Convert Set to List [Interactive Guide]" target="_blank" rel="noreferrer noopener">convert </a>it back to a list by using the <code>list(...)</code> constructor.</li>
<li>All <a href="https://blog.finxter.com/how-to-remove-duplicates-from-a-python-list-of-lists/" target="_blank" rel="noreferrer noopener" title="How to Remove Duplicates From a Python List of Lists?">duplicated </a>list entries are removed in the process because sets cannot have duplicated elements. </li>
<li>The order of the original list is lost because sets do not maintain the ordering of the elements. </li>
</ul>
<p>If all three properties are acceptable to you, this is by far the most <a href="https://blog.finxter.com/python-cprofile-a-helpful-guide-with-prime-example/" title="Python cProfile – A Helpful Guide with Prime Example" target="_blank" rel="noreferrer noopener">efficient</a> approach as evaluated later in this article!</p>
<p>However, how can you maintain the order of the original list elements while also allow duplicates? Let’s dive into the <em><strong>list comprehension</strong></em> alternative!</p>
<h2>Method 2: List Comprehension</h2>
<p><a href="https://blog.finxter.com/list-comprehension/" target="_blank" rel="noreferrer noopener" title="List Comprehension in Python — A Helpful Illustrated Guide">List comprehension</a> is a compact way of creating lists. The simple formula is <code>[expression + context]</code>.</p>
<ul>
<li><strong>Expression</strong>: What to do with each list element?</li>
<li><strong>Context</strong>: What elements to select? The context consists of an arbitrary number of <code>for</code> and <code>if</code> statements.</li>
</ul>
<p>You can use list comprehension to go over all elements in the first list but ignore them if they are in the second list:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Method 2: List Comprehension
print([x for x in a if x not in set(b)])
# [3, 2, 1]</pre>
<p>We used a small but effective optimization of converting the second list <code>b</code> to a set first. The reason is that checking membership <code>x in b</code> is much<a href="https://blog.finxter.com/complexity-of-python-operations/" target="_blank" rel="noreferrer noopener" title="Complexity of Python Operations"> faster for sets than for lists</a>. However, semantically, both variants are identical. </p>
<p>Here are the distinctive properties of this approach:</p>
<ul>
<li>The result of the list comprehension statement is a list.</li>
<li>The order of the original list is maintained.</li>
<li>Duplicate elements are maintained. </li>
</ul>
<p>If you rely on these more powerful guarantees, use the list comprehension approach because it’s the most Pythonic one.</p>
<h2>Method 3: Simple For Loop</h2>
<p>Surprisingly, some online tutorials recommend using a nested for loop (e.g.,<a href="https://kite.com/python/answers/how-to-get-the-difference-between-two-list-in-python" target="_blank" rel="noreferrer noopener" title="https://kite.com/python/answers/how-to-get-the-difference-between-two-list-in-python"> those guys</a>):</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Method 3: Nested For Loop
d = []
for x in a: if x not in b: d.append(x)
print(d)
# [3, 2, 1]</pre>
<p>In my opinion, this approach would only be used by absolute beginners or coders who come from other programming languages such as C++ or Java and don’t know essential Python features like <em>list comprehension</em>. You can optimize this method by converting the list <code>b</code> to a set first to accelerate the check <code>if x not in b</code> by a significant margin. </p>
<h2>Performance Evaluation</h2>
<p>Want to know the most performant one? In the following, I tested three different approaches:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import timeit init = 'l1 = list(range(100)); l2 = list(range(50))' # 1. Set Conversion
print(timeit.timeit('list(set(l1) - set(l2))', init, number = 10000)) # 2. List Comprehension
print(timeit.timeit('[x for x in l1 if x not in l2]', init, number = 10000)) # 3. List Comprehension + set
print(timeit.timeit('s = set(l2);[x for x in l1 if x not in s]', init, number = 10000)) '''
0.1620231000000001
0.5186101000000001
0.057180300000000184 '''</pre>
<p>You can run the code in our interactive Python shell:</p>
<p> <iframe height="700px" width="100%" src="https://repl.it/@finxter/listdifferencespeed?lite=true" scrolling="no" frameborder="no" allowtransparency="true" allowfullscreen="true" sandbox="allow-forms allow-pointer-lock allow-popups allow-same-origin allow-scripts allow-modals"></iframe> </p>
<p><em><strong>Exercise</strong>: Run the code. Which is fastest and why?</em></p>
<p>Although the first approach seems to be fastest, you now know that it has some disadvantages, too. (Loses duplicate info, loses ordering info.) From the two list comprehension approaches, the second one kills the first one in terms of <a href="https://blog.finxter.com/runtime-complexity-of-python-list-methods-easy-table-lookup/" target="_blank" rel="noreferrer noopener" title="Runtime Complexity of Python List Methods [Easy Table Lookup]">runtime complexity and performance!</a></p>
<h2>Where to Go From Here?</h2>
<p>Enough theory, let’s get some practice!</p>
<p>To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?</p>
<p><strong>Practice projects is how you sharpen your saw in coding!</strong></p>
<p>Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?</p>
<p>Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.</p>
<p>Join my free webinar <a rel="noreferrer noopener" href="https://blog.finxter.com/webinar-freelancer/" target="_blank">“How to Build Your High-Income Skill Python”</a> and watch how I grew my coding business online and how you can, too—from the comfort of your own home.</p>
<p><a href="https://blog.finxter.com/webinar-freelancer/" target="_blank" rel="noreferrer noopener">Join the free webinar now!</a></p>
</div>
https://www.sickgaming.net/blog/2020/06/...honic-way/
<div><p class="has-luminous-vivid-amber-background-color has-background"><strong>Short answer</strong>: The most Pythonic way to compute the difference between two lists <code>l1</code> and <code>l2</code> is the list comprehension statement <code>[x for x in l1 if x not in set(l2)]</code>. This works even if you have duplicate list entries, it maintains the original list ordering, and it’s efficient due to the constant runtime complexity of the set membership operation.</p>
<figure class="wp-block-image size-large is-resized"><img src="https://blog.finxter.com/wp-content/uploads/2020/06/listDifference-1024x576.jpg" alt="" class="wp-image-10140" width="768" height="432" srcset="https://blog.finxter.com/wp-content/uploads/2020/06/listDifference-scaled.jpg 1024w, https://blog.finxter.com/wp-content/uplo...00x169.jpg 300w, https://blog.finxter.com/wp-content/uplo...68x432.jpg 768w" sizes="(max-width: 768px) 100vw, 768px" /></figure>
<p>What’s the best way to compute the difference between two lists in Python?</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">a = [5, 4, 3, 2, 1]
b = [4, 5, 6, 7] # a - b == [3, 2, 1]
# b - a == [6, 7]</pre>
<p>In Python, you always have multiple ways to solve the same (or a similar) problem. Let’s have an overview in the following interactive code shell:</p>
<p> <iframe src="https://trinket.io/embed/python/41397b54a0" width="100%" height="500" frameborder="0" marginwidth="0" marginheight="0" allowfullscreen></iframe> </p>
<p><em><strong>Exercise</strong>: Run the code and think about your preferred way!</em></p>
<p>Let’s dive into each of the methods to find the most Pythonic one for your particular scenario.</p>
<h2>Method 1: Set Difference</h2>
<p>The naive approach to solve this problem is to <a href="https://blog.finxter.com/python-list-to-set/" target="_blank" rel="noreferrer noopener" title="Python List to Set Conversion [Interactive Guide]">convert both lists into sets</a> and use the <a href="https://blog.finxter.com/sets-in-python/" target="_blank" rel="noreferrer noopener" title="The Ultimate Guide to Python Sets – with Harry Potter Examples">set minus</a> (or set difference) operation. </p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Method 1: Set Difference
print(set(a) - set(b))
# {1, 2, 3}
print(set(b) - set(a))
# {6, 7}</pre>
<p>This approach is elegant because it’s readable, efficient, and concise. </p>
<p>However, there are some unique properties to this method which you should be aware of:</p>
<ul>
<li>The result is a <a href="https://blog.finxter.com/sets-in-python/" title="The Ultimate Guide to Python Sets – with Harry Potter Examples" target="_blank" rel="noreferrer noopener">set </a>and not a <a href="https://blog.finxter.com/python-lists/" title="The Ultimate Guide to Python Lists" target="_blank" rel="noreferrer noopener">list</a>. You can <a href="https://blog.finxter.com/python-set-to-list/" title="Python Convert Set to List [Interactive Guide]" target="_blank" rel="noreferrer noopener">convert </a>it back to a list by using the <code>list(...)</code> constructor.</li>
<li>All <a href="https://blog.finxter.com/how-to-remove-duplicates-from-a-python-list-of-lists/" target="_blank" rel="noreferrer noopener" title="How to Remove Duplicates From a Python List of Lists?">duplicated </a>list entries are removed in the process because sets cannot have duplicated elements. </li>
<li>The order of the original list is lost because sets do not maintain the ordering of the elements. </li>
</ul>
<p>If all three properties are acceptable to you, this is by far the most <a href="https://blog.finxter.com/python-cprofile-a-helpful-guide-with-prime-example/" title="Python cProfile – A Helpful Guide with Prime Example" target="_blank" rel="noreferrer noopener">efficient</a> approach as evaluated later in this article!</p>
<p>However, how can you maintain the order of the original list elements while also allow duplicates? Let’s dive into the <em><strong>list comprehension</strong></em> alternative!</p>
<h2>Method 2: List Comprehension</h2>
<p><a href="https://blog.finxter.com/list-comprehension/" target="_blank" rel="noreferrer noopener" title="List Comprehension in Python — A Helpful Illustrated Guide">List comprehension</a> is a compact way of creating lists. The simple formula is <code>[expression + context]</code>.</p>
<ul>
<li><strong>Expression</strong>: What to do with each list element?</li>
<li><strong>Context</strong>: What elements to select? The context consists of an arbitrary number of <code>for</code> and <code>if</code> statements.</li>
</ul>
<p>You can use list comprehension to go over all elements in the first list but ignore them if they are in the second list:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Method 2: List Comprehension
print([x for x in a if x not in set(b)])
# [3, 2, 1]</pre>
<p>We used a small but effective optimization of converting the second list <code>b</code> to a set first. The reason is that checking membership <code>x in b</code> is much<a href="https://blog.finxter.com/complexity-of-python-operations/" target="_blank" rel="noreferrer noopener" title="Complexity of Python Operations"> faster for sets than for lists</a>. However, semantically, both variants are identical. </p>
<p>Here are the distinctive properties of this approach:</p>
<ul>
<li>The result of the list comprehension statement is a list.</li>
<li>The order of the original list is maintained.</li>
<li>Duplicate elements are maintained. </li>
</ul>
<p>If you rely on these more powerful guarantees, use the list comprehension approach because it’s the most Pythonic one.</p>
<h2>Method 3: Simple For Loop</h2>
<p>Surprisingly, some online tutorials recommend using a nested for loop (e.g.,<a href="https://kite.com/python/answers/how-to-get-the-difference-between-two-list-in-python" target="_blank" rel="noreferrer noopener" title="https://kite.com/python/answers/how-to-get-the-difference-between-two-list-in-python"> those guys</a>):</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Method 3: Nested For Loop
d = []
for x in a: if x not in b: d.append(x)
print(d)
# [3, 2, 1]</pre>
<p>In my opinion, this approach would only be used by absolute beginners or coders who come from other programming languages such as C++ or Java and don’t know essential Python features like <em>list comprehension</em>. You can optimize this method by converting the list <code>b</code> to a set first to accelerate the check <code>if x not in b</code> by a significant margin. </p>
<h2>Performance Evaluation</h2>
<p>Want to know the most performant one? In the following, I tested three different approaches:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import timeit init = 'l1 = list(range(100)); l2 = list(range(50))' # 1. Set Conversion
print(timeit.timeit('list(set(l1) - set(l2))', init, number = 10000)) # 2. List Comprehension
print(timeit.timeit('[x for x in l1 if x not in l2]', init, number = 10000)) # 3. List Comprehension + set
print(timeit.timeit('s = set(l2);[x for x in l1 if x not in s]', init, number = 10000)) '''
0.1620231000000001
0.5186101000000001
0.057180300000000184 '''</pre>
<p>You can run the code in our interactive Python shell:</p>
<p> <iframe height="700px" width="100%" src="https://repl.it/@finxter/listdifferencespeed?lite=true" scrolling="no" frameborder="no" allowtransparency="true" allowfullscreen="true" sandbox="allow-forms allow-pointer-lock allow-popups allow-same-origin allow-scripts allow-modals"></iframe> </p>
<p><em><strong>Exercise</strong>: Run the code. Which is fastest and why?</em></p>
<p>Although the first approach seems to be fastest, you now know that it has some disadvantages, too. (Loses duplicate info, loses ordering info.) From the two list comprehension approaches, the second one kills the first one in terms of <a href="https://blog.finxter.com/runtime-complexity-of-python-list-methods-easy-table-lookup/" target="_blank" rel="noreferrer noopener" title="Runtime Complexity of Python List Methods [Easy Table Lookup]">runtime complexity and performance!</a></p>
<h2>Where to Go From Here?</h2>
<p>Enough theory, let’s get some practice!</p>
<p>To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?</p>
<p><strong>Practice projects is how you sharpen your saw in coding!</strong></p>
<p>Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?</p>
<p>Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.</p>
<p>Join my free webinar <a rel="noreferrer noopener" href="https://blog.finxter.com/webinar-freelancer/" target="_blank">“How to Build Your High-Income Skill Python”</a> and watch how I grew my coding business online and how you can, too—from the comfort of your own home.</p>
<p><a href="https://blog.finxter.com/webinar-freelancer/" target="_blank" rel="noreferrer noopener">Join the free webinar now!</a></p>
</div>
https://www.sickgaming.net/blog/2020/06/...honic-way/