{"id":113918,"date":"2020-06-08T08:14:54","date_gmt":"2020-06-08T08:14:54","guid":{"rendered":"https:\/\/blog.finxter.com\/?p=9385"},"modified":"2020-06-08T08:14:54","modified_gmt":"2020-06-08T08:14:54","slug":"how-to-combine-two-python-lists-and-remove-duplicates-in-second-list","status":"publish","type":"post","link":"https:\/\/sickgaming.net\/blog\/2020\/06\/08\/how-to-combine-two-python-lists-and-remove-duplicates-in-second-list\/","title":{"rendered":"How to Combine Two Python Lists and Remove Duplicates in Second List?"},"content":{"rendered":"<p><strong>Problem<\/strong>: Given two <a href=\"https:\/\/blog.finxter.com\/python-lists\/\" target=\"_blank\" rel=\"noreferrer noopener\">lists <\/a><code>[1, 2, 2, 4]<\/code> and <code>[2, 5, 5, 5, 6]<\/code>. How do you <a href=\"https:\/\/blog.finxter.com\/python-join-list\/\" target=\"_blank\" rel=\"noreferrer noopener\">combine <\/a>those lists to the new list <code>[1, 2, 2, 4, 5, 6]<\/code> by <a href=\"https:\/\/blog.finxter.com\/how-to-remove-duplicates-from-a-python-list\/\" target=\"_blank\" rel=\"noreferrer noopener\">removing the duplicates<\/a> in the second list?<\/p>\n<p><em><strong>Note<\/strong>: You want to remove all duplicates in the second list and the elements in the second list that are already in the first list.<\/em><\/p>\n<p><strong>Solution<\/strong>: Use the following three steps to combine two lists and remove the duplicates in the second list:<\/p>\n<ul>\n<li>Convert the first and second lists to a set using the <code><a href=\"https:\/\/blog.finxter.com\/sets-in-python\/\">set(...)<\/a><\/code> constructor.<\/li>\n<li>Use the <a href=\"https:\/\/blog.finxter.com\/sets-in-python\/\" target=\"_blank\" rel=\"noreferrer noopener\">set minus<\/a> operation to get all elements that are in the second list but not in the first list. <\/li>\n<li>Create a new list by <a href=\"https:\/\/blog.finxter.com\/concatenate-lists-in-python\/\" target=\"_blank\" rel=\"noreferrer noopener\">concatenating <\/a>those elements to the first list.<\/li>\n<\/ul>\n<p>Here&#8217;s the code:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"># Create the two lists\nl1 = [1, 2, 2, 4]\nl2 = [2, 5, 5, 5, 6] # Find elements that are in second but not in first\nnew = set(l2) - set(l1) # Create the new list using list concatenation\nl = l1 + list(new)\nprint(l)\n# [1, 2, 2, 4, 5, 6]<\/pre>\n<p>Try it yourself in our interactive Python shell:<\/p>\n<p> <iframe loading=\"lazy\" src=\"https:\/\/trinket.io\/embed\/python\/d6d3eca8e3\" width=\"100%\" height=\"356\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" allowfullscreen><\/iframe> <\/p>\n<p><em><strong>Exercise<\/strong>: Can you rewrite this in a single line of Python code (<a href=\"https:\/\/blog.finxter.com\/python-one-liners-the-ultimate-collection\/\" target=\"_blank\" rel=\"noreferrer noopener\">Python One-Liner<\/a>)?<\/em><\/p>\n<p>Let&#8217;s dive into the more concise one-liner to accomplish the same thing:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">l = l1 + list(set(l2) - set(l1))<\/pre>\n<p>If you want to learn about the most Pythonic way to <a rel=\"noreferrer noopener\" href=\"https:\/\/blog.finxter.com\/how-to-remove-duplicates-from-a-python-list\/\" target=\"_blank\">remove ALL duplicates from a Python<\/a> list, read on:<\/p>\n<h2>How to Remove Duplicates From a Python List?<\/h2>\n<figure class=\"wp-block-embed-youtube wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\">\n<div class=\"wp-block-embed__wrapper\">\n<div class=\"ast-oembed-container\"><iframe loading=\"lazy\" title=\"How to Remove Duplicates From a Python List?\" width=\"1400\" height=\"788\" src=\"https:\/\/www.youtube.com\/embed\/GXL23jfNk1Y?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe><\/div>\n<\/div>\n<\/figure>\n<p><strong>Naive Method<\/strong>: Go over each element and check whether this element already exists in the list. If so, remove it. However, this takes a few lines of code. <\/p>\n<p><strong>Efficient Method: <\/strong>A shorter and more concise way is to create a dictionary out of the elements in the list to remove all duplicates and convert the dictionary back to a list. This preserves the order of the original list elements.<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">lst = ['Alice', 'Bob', 'Bob', 1, 1, 1, 2, 3, 3]\nprint(list(dict.fromkeys(lst)))\n# ['Alice', 'Bob', 1, 2, 3]<\/pre>\n<ol>\n<li>Convert the list to a dictionary with <code>dict.fromkeys(lst)<\/code>. <\/li>\n<li>Convert the dictionary into a list with <code>list(dict)<\/code>.<\/li>\n<\/ol>\n<p>Each list element becomes a new key to the dictionary. For example, the list <code>[1, 2, 3]<\/code> becomes the dictionary <code>{1:None, 2:None, 3:None}<\/code>. All elements that occur multiple times will be assigned to the same key. Thus, the dictionary contains only unique keys&#8212;there cannot be multiple equal keys.<\/p>\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/blog.finxter.com\/wp-content\/uploads\/2020\/04\/removeDupsPython-1024x576.jpg\" alt=\"\" class=\"wp-image-7701\" width=\"512\" height=\"288\" srcset=\"https:\/\/blog.finxter.com\/wp-content\/uploads\/2020\/04\/removeDupsPython-scaled.jpg 1024w, https:\/\/blog.finxter.com\/wp-content\/uploads\/2020\/04\/removeDupsPython-300x169.jpg 300w, https:\/\/blog.finxter.com\/wp-content\/uploads\/2020\/04\/removeDupsPython-768x432.jpg 768w\" sizes=\"auto, (max-width: 512px) 100vw, 512px\" \/><\/figure>\n<p>As dictionary values, you take dummy values (per default).<\/p>\n<p>Then, you convert the dictionary back to a list, throwing away the dummy values. <\/p>\n<p>Here&#8217;s the code:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">>>> lst = [1, 1, 1, 3, 2, 5, 5, 2]\n>>> dic = dict.fromkeys(lst)\n>>> dic\n{1: None, 3: None, 2: None, 5: None}\n>>> duplicate_free = list(dic)\n>>> duplicate_free\n[1, 3, 2, 5]<\/pre>\n<p><strong>Related blog articles:<\/strong><\/p>\n<ul>\n<li><a href=\"https:\/\/blog.finxter.com\/how-to-remove-duplicates-from-a-python-list-of-lists\/\" target=\"_blank\" rel=\"noreferrer noopener\">Python Remove Duplicates From List of Lists<\/a><\/li>\n<li><a rel=\"noreferrer noopener\" href=\"https:\/\/blog.finxter.com\/python-list-remove\/\" target=\"_blank\">Python List Remove<\/a><\/li>\n<li><a rel=\"noreferrer noopener\" href=\"https:\/\/blog.finxter.com\/python-dictionary\/\" target=\"_blank\">The Ultimate Guide to Python Dictionaries!<\/a><\/li>\n<\/ul>\n<h2>Where to Go From Here?<\/h2>\n<p>Enough theory, let\u2019s get some practice!<\/p>\n<p>To become successful in coding, you need to get out there and solve real problems for real people. That\u2019s how you can become a six-figure earner easily. And that\u2019s how you polish the skills you really need in practice. After all, what\u2019s the use of learning theory that nobody ever needs?<\/p>\n<p><strong>Practice projects is how you sharpen your saw in coding!<\/strong><\/p>\n<p>Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?<\/p>\n<p>Then become a Python freelance developer! It\u2019s the best way of approaching the task of improving your Python skills\u2014even if you are a complete beginner.<\/p>\n<p>Join my free webinar <a rel=\"noreferrer noopener\" href=\"https:\/\/blog.finxter.com\/webinar-freelancer\/\" target=\"_blank\">\u201cHow to Build Your High-Income Skill Python\u201d<\/a> and watch how I grew my coding business online and how you can, too\u2014from the comfort of your own home.<\/p>\n<p><a href=\"https:\/\/blog.finxter.com\/webinar-freelancer\/\" target=\"_blank\" rel=\"noreferrer noopener\">Join the free webinar now!<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Problem: Given two lists [1, 2, 2, 4] and [2, 5, 5, 5, 6]. How do you combine those lists to the new list [1, 2, 2, 4, 5, 6] by removing the duplicates in the second list? Note: You want to remove all duplicates in the second list and the elements in the second [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[857],"tags":[73,468,528],"class_list":["post-113918","post","type-post","status-publish","format-standard","hentry","category-python-tut","tag-programming","tag-python","tag-tutorial"],"_links":{"self":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts\/113918","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/comments?post=113918"}],"version-history":[{"count":0,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts\/113918\/revisions"}],"wp:attachment":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/media?parent=113918"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/categories?post=113918"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/tags?post=113918"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}