![]() |
|
[Tut] How to Remove Duplicates From a Python List While Preserving Order? - Printable Version +- Sick Gaming (https://sickgaming.net) +-- Forum: Programming (https://sickgaming.net/forum-76.html) +--- Forum: Python (https://sickgaming.net/forum-83.html) +--- Thread: [Tut] How to Remove Duplicates From a Python List While Preserving Order? (/thread-97523.html) |
[Tut] How to Remove Duplicates From a Python List While Preserving Order? - xSicKxBot - 09-30-2020 How to Remove Duplicates From a Python List While Preserving Order? To remove duplicates from a Python list while preserving the order of the elements, use the code Code: list(dict.fromkeys(list))Code: dict.fromkeys()Code: NoneCode: list()Problem: How to remove duplicates from a Python list while keeping the order of the list elements preserved? You may find this question a little awkward. What has removing duplicatesto do with preserving the order of the elements? The reason is simple: a well-known and efficient way to remove duplicates from a list is to convert the list to a set—which is duplicated-free—and converting it back to a list. Here’s what you may find everywhere: Code: lst = [42, 42, 'Alice', 'Alice', 1]
dup_free = list(set(lst))
print(dup_free)
# ['Alice', 42, 1]The back-and-forth conversion Code: list(set(lst))Code: 'Alice'Code: 42So, how to remove duplicates while preserving the order of the elements? ![]() The most Pythonic and blazingly fast approach is to use a dictionary: Code: lst = [3, 3, 22, 22, 1]
result = list(dict.fromkeys(lst))
print(result)
# [3, 22, 1]The Code: dict.fromkeys()This is the most Pythonic way to remove duplicates from a Python list while preserving the order. Is this method fast? Like sets, dictionaries use hash tables, which means they are extremely fast. Do you want to develop the skills of a well-rounded Python professional—while getting paid in the process? Become a Python freelancer and order your book Leaving the Rat Race with Python on Amazon (Kindle/Print)! ![]() Do Python Dictionaries Preserve the Ordering of the Keys? Surprisingly, the dictionary keys in Python preserve the order of the elements. So, yes, the order of the elements is preserved. (source) Countless online resources like this argue that the order of dictionary keys is not preserved. They assume that the underlying implementation of the dictionary key iterables uses sets—and sets are well-known to be agnostic to the ordering of elements. But this assumption is wrong. The built-in Python dictionary implementation in cPython preserves the order. Here’s another example: Code: lst = ['Alice', 'Bob', 'Bob', 1, 1, 1, 2, 3, 3]
dic = dict.fromkeys(lst)
print(dic)
# {'Alice': None, 'Bob': None, 1: None, 2: None, 3: None}You see that the order of elements is preserved so when converting it back, the original ordering of the list elements is still preserved: Code: print(list(dic))
# ['Alice', 'Bob', 1, 2, 3]However, you cannot rely on it because any Python implementation could, theoretically, decide not to preserve the order (notice the “COULD” here is 100% theoretical and does not apply to the default cPython implementation). If you need to be certain that the order is preserved, you can use the ordered dictionary library. In cPython, this is just a wrapper for the default dict implementation. Source Article: How to Remove Duplicates From a Python List? Removing Duplicates From Ordered Lists For Older Versions Dictionaries only became ordered in all Python implementations when Python 3.7 was released (this was also an implementation detail of CPython 3.6). So, if you’re using an older version of Python, you will need to import the Code: OrderedDictCode: from collections import OrderedDict lst = [1, 1, 9, 1, 9, 6, 9, 7] result = list(OrderedDict.fromkeys(lst))The output is the following duplicate-free list with the order of the elements preserved: Code: print(result) # [1, 9, 6, 7]Let’s try this method in our interactive Python shell: https://repl.it/repls/GrandioseRemarkableBackups?lite=true Exercise: Run the code. Does it work? You can find more ways to remove duplicates while preserving the order in this detailed blog article: Related tutorial: Python List: Remove Duplicates and Keep the Order Where to Go From Here? Enough theory, let’s get some practice! To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs? Practice projects is how you sharpen your saw in coding! Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people? Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner. Join my free webinar “How to Build Your High-Income Skill Python” and watch how I grew my coding business online and how you can, too—from the comfort of your own home. Join the free webinar now! The post How to Remove Duplicates From a Python List While Preserving Order? first appeared on Finxter. |