[Tut] How to Combine Two Python Lists and Remove Duplicates in Second List? - Printable Version +- Sick Gaming (https://www.sickgaming.net) +-- Forum: Programming (https://www.sickgaming.net/forum-76.html) +--- Forum: Python (https://www.sickgaming.net/forum-83.html) +--- Thread: [Tut] How to Combine Two Python Lists and Remove Duplicates in Second List? (/thread-95619.html) |
[Tut] How to Combine Two Python Lists and Remove Duplicates in Second List? - xSicKxBot - 06-11-2020 How to Combine Two Python Lists and Remove Duplicates in Second List? <div><p><strong>Problem</strong>: Given two <a href="https://blog.finxter.com/python-lists/" target="_blank" rel="noreferrer noopener">lists </a><code>[1, 2, 2, 4]</code> and <code>[2, 5, 5, 5, 6]</code>. How do you <a href="https://blog.finxter.com/python-join-list/" target="_blank" rel="noreferrer noopener">combine </a>those lists to the new list <code>[1, 2, 2, 4, 5, 6]</code> by <a href="https://blog.finxter.com/how-to-remove-duplicates-from-a-python-list/" target="_blank" rel="noreferrer noopener">removing the duplicates</a> in the second list?</p> <p><em><strong>Note</strong>: You want to remove all duplicates in the second list and the elements in the second list that are already in the first list.</em></p> <p><strong>Solution</strong>: Use the following three steps to combine two lists and remove the duplicates in the second list:</p> <ul> <li>Convert the first and second lists to a set using the <code><a href="https://blog.finxter.com/sets-in-python/">set(...)</a></code> constructor.</li> <li>Use the <a href="https://blog.finxter.com/sets-in-python/" target="_blank" rel="noreferrer noopener">set minus</a> operation to get all elements that are in the second list but not in the first list. </li> <li>Create a new list by <a href="https://blog.finxter.com/concatenate-lists-in-python/" target="_blank" rel="noreferrer noopener">concatenating </a>those elements to the first list.</li> </ul> <p>Here’s the code:</p> <pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Create the two lists l1 = [1, 2, 2, 4] l2 = [2, 5, 5, 5, 6] # Find elements that are in second but not in first new = set(l2) - set(l1) # Create the new list using list concatenation l = l1 + list(new) print(l) # [1, 2, 2, 4, 5, 6]</pre> <p>Try it yourself in our interactive Python shell:</p> <p> <iframe src="https://trinket.io/embed/python/d6d3eca8e3" width="100%" height="356" frameborder="0" marginwidth="0" marginheight="0" allowfullscreen></iframe> </p> <p><em><strong>Exercise</strong>: Can you rewrite this in a single line of Python code (<a href="https://blog.finxter.com/python-one-liners-the-ultimate-collection/" target="_blank" rel="noreferrer noopener">Python One-Liner</a>)?</em></p> <p>Let’s dive into the more concise one-liner to accomplish the same thing:</p> <pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">l = l1 + list(set(l2) - set(l1))</pre> <p>If you want to learn about the most Pythonic way to <a rel="noreferrer noopener" href="https://blog.finxter.com/how-to-remove-duplicates-from-a-python-list/" target="_blank">remove ALL duplicates from a Python</a> list, read on:</p> <h2>How to Remove Duplicates From a Python List?</h2> <figure class="wp-block-embed-youtube wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio"> <div class="wp-block-embed__wrapper"> <div class="ast-oembed-container"><iframe title="How to Remove Duplicates From a Python List?" width="1400" height="788" src="https://www.youtube.com/embed/GXL23jfNk1Y?feature=oembed" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe></div> </div> </figure> <p><strong>Naive Method</strong>: Go over each element and check whether this element already exists in the list. If so, remove it. However, this takes a few lines of code. </p> <p><strong>Efficient Method: </strong>A shorter and more concise way is to create a dictionary out of the elements in the list to remove all duplicates and convert the dictionary back to a list. This preserves the order of the original list elements.</p> <pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">lst = ['Alice', 'Bob', 'Bob', 1, 1, 1, 2, 3, 3] print(list(dict.fromkeys(lst))) # ['Alice', 'Bob', 1, 2, 3]</pre> <ol> <li>Convert the list to a dictionary with <code>dict.fromkeys(lst)</code>. </li> <li>Convert the dictionary into a list with <code>list(dict)</code>.</li> </ol> <p>Each list element becomes a new key to the dictionary. For example, the list <code>[1, 2, 3]</code> becomes the dictionary <code>{1:None, 2:None, 3:None}</code>. All elements that occur multiple times will be assigned to the same key. Thus, the dictionary contains only unique keys—there cannot be multiple equal keys.</p> <figure class="wp-block-image size-large is-resized"><img src="https://blog.finxter.com/wp-content/uploads/2020/04/removeDupsPython-1024x576.jpg" alt="" class="wp-image-7701" width="512" height="288" srcset="https://blog.finxter.com/wp-content/uploads/2020/04/removeDupsPython-scaled.jpg 1024w, https://blog.finxter.com/wp-content/uploads/2020/04/removeDupsPython-300x169.jpg 300w, https://blog.finxter.com/wp-content/uploads/2020/04/removeDupsPython-768x432.jpg 768w" sizes="(max-width: 512px) 100vw, 512px" /></figure> <p>As dictionary values, you take dummy values (per default).</p> <p>Then, you convert the dictionary back to a list, throwing away the dummy values. </p> <p>Here’s the code:</p> <pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">>>> lst = [1, 1, 1, 3, 2, 5, 5, 2] >>> dic = dict.fromkeys(lst) >>> dic {1: None, 3: None, 2: None, 5: None} >>> duplicate_free = list(dic) >>> duplicate_free [1, 3, 2, 5]</pre> <p><strong>Related blog articles:</strong></p> <ul> <li><a href="https://blog.finxter.com/how-to-remove-duplicates-from-a-python-list-of-lists/" target="_blank" rel="noreferrer noopener">Python Remove Duplicates From List of Lists</a></li> <li><a rel="noreferrer noopener" href="https://blog.finxter.com/python-list-remove/" target="_blank">Python List Remove</a></li> <li><a rel="noreferrer noopener" href="https://blog.finxter.com/python-dictionary/" target="_blank">The Ultimate Guide to Python Dictionaries!</a></li> </ul> <h2>Where to Go From Here?</h2> <p>Enough theory, let’s get some practice!</p> <p>To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?</p> <p><strong>Practice projects is how you sharpen your saw in coding!</strong></p> <p>Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?</p> <p>Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.</p> <p>Join my free webinar <a rel="noreferrer noopener" href="https://blog.finxter.com/webinar-freelancer/" target="_blank">“How to Build Your High-Income Skill Python”</a> and watch how I grew my coding business online and how you can, too—from the comfort of your own home.</p> <p><a href="https://blog.finxter.com/webinar-freelancer/" target="_blank" rel="noreferrer noopener">Join the free webinar now!</a></p> </div> https://www.sickgaming.net/blog/2020/06/08/how-to-combine-two-python-lists-and-remove-duplicates-in-second-list/ |