---
description: A list is a container that contains different Python objects, which could be integers, words, values, etc. It is the equivalent of an array in other programming languages. So here will go through diff
title: How to Remove Duplicates from a List in Python
image: https://www.guru99.com/images/remove-duplicates-from-a-list-in-python.png
---

[Skip to content](#main)

**⚡ Smart Summary**

Removing duplicates from a Python list can be done with several built-in approaches, including set(), dict.fromkeys(), loops, list comprehensions, and library methods from NumPy and Pandas, each balancing order preservation against speed.

- 🔘 **Set method:** set() returns distinct elements instantly but does not preserve the original list order.
- ☑️ **Order preserved:** dict.fromkeys() and OrderedDict remove duplicates while keeping the first-seen insertion order.
- ✅ **Manual control:** A for-loop or list comprehension with a membership check keeps order and stays readable.
- 🧪 **Library methods:** NumPy unique() and Pandas unique() deduplicate data and return list output through the tolist() method.
- 🛠️ **Hashability limit:** Set and dict keys require hashable items, so lists or dictionaries need a loop-based approach.
- 🤖 **AI workflows:** Machine learning pipelines deduplicate training data with Pandas drop\_duplicates() to prevent bias and data leakage.

Read More

![Remove Duplicates from a List in Python](https://www.guru99.com/images/remove-duplicates-from-a-list-in-python.png)

## Python remove Duplicates from a List

A list is a container that contains different Python objects, which could be integers, words, values, etc. It is the equivalent of an array in other programming languages.

So here we will go through different ways in which we can remove duplicates from a given list in Python.

## Method 1) Remove duplicates from list using Set

To remove the duplicates from a list, you can make use of the built-in function set(). The specialty of the set() method is that it returns distinct elements.

We have a list : \[1,1,2,3,2,2,4,5,6,2,1]. The list has many duplicates which we need to remove and get back only the distinct elements. The list is given to the set() built-in function. Later the final list is displayed using the list() built-in function.

The output that we get is distinct elements where all the duplicate elements are eliminated.

**Example**

```
my_list = [1,1,2,3,2,2,4,5,6,2,1]
my_final_list = set(my_list)
print(list(my_final_list))
```

**Output:**

```
[1, 2, 3, 4, 5, 6]
```

## Method 2) Using the Temporary List

To remove duplicates from a given list, you can make use of an empty temporary list. For that first, you will have to loop through the list having duplicates and add the unique items to the temporary list. Later the temporary list is assigned to the main list.

**Example**

Here is a working example using a temporary list.

```
my_list = [1, 2, 3, 1, 2, 4, 5, 4 ,6, 2]
print("List Before ", my_list)
temp_list = []

for i in my_list:
    if i not in temp_list:
        temp_list.append(i)

my_list = temp_list

print("List After removing duplicates ", my_list)
```

**Output:**

```
List Before  [1, 2, 3, 1, 2, 4, 5, 4, 6, 2]
List After removing duplicates  [1, 2, 3, 4, 5, 6]
```

## Method 3) Using Dict

We can remove duplicates from the given list by importing OrderedDict from collections. It is available from python2.7 onwards. OrderedDict takes care of returning you the distinct elements in an order in which the key is present.

Let us make use of a list and use the fromkeys() method available in OrderedDict to get the unique elements from the list.

To make use of the OrderedDict.fromkeys() method, you have to import OrderedDict from collections, as shown below:

```
from collections import OrderedDict
```

Here is an example to remove duplicates using the OrderedDict.fromkeys() method.

**Example**

```
from collections import OrderedDict

my_list = ['a','x','a','y','a','b','b','c']

my_final_list = OrderedDict.fromkeys(my_list)

print(list(my_final_list))
```

**Output:**

```
['a', 'x', 'y', 'b', 'c']
```

From Python 3.5+ onwards, we can make use of the regular dict.fromkeys() to get the distinct elements from the list. The dict.fromkeys() method returns keys that are unique and helps to get rid of the duplicate values.

An example that shows the working of dict.fromkeys() on a list to give the unique items is as follows:

**Example**

```
my_list = ['a','x','a','y','a','b','b','c']
my_final_list = dict.fromkeys(my_list)
print(list(my_final_list))
```

**Output:**

```
['a', 'x', 'y', 'b', 'c']
```

**Don't Miss:**

- [PyQt5 Tutorial with Examples: Create Python GUIs with Qt](https://www.guru99.com/pyqt-tutorial.html)
- [Python abs() Function: Absolute Value Examples](https://www.guru99.com/abs-in-python.html)
- [type() and isinstance() in Python with Examples](https://www.guru99.com/type-isinstance-python.html)
- [How to reverse a String in Python (5 Methods)](https://www.guru99.com/reverse-a-string-in-python.html)

## Method 4) Using for-loop

Using [for-loop](https://www.guru99.com/python-loops-while-for-break-continue-enumerate.html), we will traverse the list of items to remove duplicates.

First initialize the array to empty i.e myFinallist = \[]. Inside the for-loop, add a check if the items in the list exist in the array myFinallist. If the items do not exist, add the item to the array myFinallist using the append() method.

So whenever the duplicate item is encountered it will be already present in the array myFinallist and will not be inserted. Let us now check the same in the example below:

**Example**

```
my_list = [1,2,2,3,1,4,5,1,2,6]
myFinallist = []
for i in my_list:
    if i not in myFinallist:
myFinallist.append(i)
print(list(myFinallist))
```

**Output:**

```
[1, 2, 3, 4, 5, 6]
```

## Method 5) Using list comprehension

List comprehensions are Python functions that are used for creating new sequences (such as lists, dictionaries, etc.) using sequences that have already been created. This helps you to reduce longer loops and make your code easier to read and maintain.

Let us make use of list comprehension to remove duplicates from the list given.

**Example**

```
my_list = [1,2,2,3,1,4,5,1,2,6]
my_finallist = []
[my_finallist.append(n) for n in my_list if n not in my_finallist]
print(my_finallist)
```

**Output:**

```
[1, 2, 3, 4, 5, 6]
```

## Method 6) Using NumPy unique() method

The method unique() from the NumPy module can help us remove duplicates from the list given.

To work with NumPy, first import the numpy module:

**Step 1)** Import NumPy module

```
import numpy as np
```

**Step 2)** Use your list with duplicates inside the unique() method. The output is converted back to a list using the tolist() method.

```
myFinalList = np.unique(my_list).tolist()
```

**Step 3)** Finally print the list:

```
print(myFinalList)
```

The final code with output is as follows:

```
import numpy as np
my_list = [1,2,2,3,1,4,5,1,2,6]
myFinalList = np.unique(my_list).tolist()
print(myFinalList)
```

**Output:**

```
[1, 2, 3, 4, 5, 6]
```

## Method 7) Using Pandas methods

The Pandas module has a unique() method that will give us the unique elements from the list given.

To work with the Pandas module:

**Step 1)** Import Pandas module

```
import pandas as pd
```

**Step 2)** Use your list with duplicates inside the unique() method:

```
myFinalList = pd.unique(my_list).tolist()
```

**Step 3)** Print the list:

```
print(myFinalList)
```

The final code with output is as follows:

```
import pandas as pd

my_list = [1,2,2,3,1,4,5,1,2,6]
myFinalList = pd.unique(my_list).tolist()
print(myFinalList)
```

**Output:**

```
[1, 2, 3, 4, 5, 6]
```

## Method 8) Using enumerate() and list comprehension

Here we use the combination of list comprehension and enumerate() to remove the duplicate elements. Enumerate returns an object with a counter to each element in the list. For example (0,1), (1,2) etc. Here the first value is the index, and the second value is the list item.

Each element is checked to see if it exists in the list, and if it does, it is removed from the list.

**Example**

```
my_list = [1,2,2,3,1,4,5,1,2,6]
my_finallist = [i for j, i in enumerate(my_list) if i not in my_list[:j]]
print(list(my_finallist))
```

**Output:**

```
[1, 2, 3, 4, 5, 6]
```

## FAQs

🔄 Does converting a list to a set preserve the original order?

No. A set is an unordered collection, so set() may return elements in a different order. To remove duplicates while preserving order, use dict.fromkeys() or a loop that appends only items not already seen.

🚀 What is the fastest way to remove duplicates from a large list?

For large lists, set() is fastest when order does not matter because membership checks are O(1). When order matters, list(dict.fromkeys(my\_list)) is almost as fast and keeps first-seen order in one line.

🔑 How do you remove duplicates from a list of dictionaries or unhashable items?

set() and dict.fromkeys() need hashable elements, so they fail on lists or dictionaries. Loop through the list and append items not already seen, or convert each item to a tuple or JSON string as a temporary key.

🧭 How can you remove duplicates and keep order in one line?

Use list(dict.fromkeys(my\_list)). Since Python 3.7 dictionaries preserve insertion order, so the keys stay in first-seen order. On earlier versions, use collections.OrderedDict.fromkeys() for the same result.

🔁 Does removing duplicates modify the original list?

No. Methods such as set(), dict.fromkeys(), comprehensions, and NumPy or Pandas unique() build a new list and leave the source unchanged. Reassign the result to the same variable if you want to overwrite it.

📊 When should you use NumPy or Pandas to remove duplicates?

Use numpy.unique() or pandas.unique() when the data already lives in arrays or a Series, or during data analysis. Note numpy.unique() also sorts the values, while pandas.unique() keeps first-seen order. Call tolist() for a plain list.

🤖 How is duplicate removal used in AI and machine learning?

Machine learning pipelines drop duplicate rows before training so repeated samples do not bias the model or leak between train and test sets. Pandas drop\_duplicates() is the common tool for cleaning tabular datasets.

✨ Can GitHub Copilot or agentic AI tools write deduplication code?

Yes. GitHub Copilot and agentic AI assistants generate set(), dict.fromkeys(), or Pandas deduplication code from a comment, suggest the order-preserving option, and refactor loops, though you should still verify edge cases like unhashable items.

#### Summarize this post with:

ChatGPTPerplexityGrokGoogle AI

**Stay Updated on AI** **Get Weekly AI Skills, Trends, Actionable Advice.**

##### Sign up for the newsletter

Subscribe for Free

You have successfully subscribed.  
Please check your inbox.

![AI-Newsletter]() Chosen by over **350,000+** professionals

[Scroll to top](#wrapper)

×

Search for:

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://www.guru99.com/#organization","name":"Guru99","sameAs":["https://www.facebook.com/Guru99Official","https://twitter.com/guru99com"],"logo":{"@type":"ImageObject","@id":"https://www.guru99.com/#logo","url":"https://www.guru99.com/images/guru99-logo-v1-150x59.png","contentUrl":"https://www.guru99.com/images/guru99-logo-v1-150x59.png","caption":"Guru99","inLanguage":"en-US"}},{"@type":"WebSite","@id":"https://www.guru99.com/#website","url":"https://www.guru99.com","name":"Guru99","publisher":{"@id":"https://www.guru99.com/#organization"},"inLanguage":"en-US"},{"@type":"ImageObject","@id":"https://www.guru99.com/images/remove-duplicates-from-a-list-in-python.png","url":"https://www.guru99.com/images/remove-duplicates-from-a-list-in-python.png","width":"700","height":"250","caption":"Remove Duplicates from a List in Python","inLanguage":"en-US"},{"@type":"BreadcrumbList","@id":"https://www.guru99.com/python-howto-remove-duplicates.html#breadcrumb","itemListElement":[{"@type":"ListItem","position":"1","item":{"@id":"https://www.guru99.com","name":"Home"}},{"@type":"ListItem","position":"2","item":{"@id":"https://www.guru99.com/python","name":"Python"}},{"@type":"ListItem","position":"3","item":{"@id":"https://www.guru99.com/python-howto-remove-duplicates.html","name":"How to Remove Duplicates from a List in Python"}}]},{"@type":"WebPage","@id":"https://www.guru99.com/python-howto-remove-duplicates.html#webpage","url":"https://www.guru99.com/python-howto-remove-duplicates.html","name":"How to Remove Duplicates from a List in Python","dateModified":"2026-10-07T14:48:47+05:30","isPartOf":{"@id":"https://www.guru99.com/#website"},"primaryImageOfPage":{"@id":"https://www.guru99.com/images/remove-duplicates-from-a-list-in-python.png"},"inLanguage":"en-US","breadcrumb":{"@id":"https://www.guru99.com/python-howto-remove-duplicates.html#breadcrumb"}},{"@type":"Person","@id":"https://www.guru99.com/author/editorial","name":"Editorial Team","url":"https://www.guru99.com/author/editorial","image":{"@type":"ImageObject","@id":"https://www.guru99.com/images/favicon-new-logo.png","url":"https://www.guru99.com/images/favicon-new-logo.png","caption":"Editorial Team","inLanguage":"en-US"},"sameAs":["https://www.guru99.com/editorial-team"],"worksFor":{"@id":"https://www.guru99.com/#organization"}},{"articleSection":"Python","headline":"How to Remove Duplicates from a List in Python","description":"A list is a container that contains different Python objects, which could be integers, words, values, etc. It is the equivalent of an array in other programming languages. So here will go through diff","keywords":"python","speakable":{"@type":"SpeakableSpecification","cssSelector":[".entry-title",".summary"]},"@type":"Article","author":{"@id":"https://www.guru99.com/author/editorial","name":"Editorial Team"},"dateModified":"2026-10-07T14:48:47+05:30","image":{"@id":"https://www.guru99.com/images/remove-duplicates-from-a-list-in-python.png"},"copyrightYear":"2026","name":"How to Remove Duplicates from a List in Python","subjectOf":[{"@type":"HowTo","name":"How to Remove Duplicates from a List in Python","description":"So here will go through different ways in which we can remove duplicates from a given list in Python.","step":[{"@type":"HowToStep","text":"Using Set method, To remove the duplicates from a list, you can make use of the built-in function set(). The specialty of set() method is that it returns distinct elements.","name":"Method 1) Remove duplicates from list using Set","url":"https://www.guru99.com/python-howto-remove-duplicates.html#step1"},{"@type":"HowToStep","text":"Next Method, To remove duplicates from a given list, you can make use of an empty temporary list.","name":"Method 2) Using the Temporary List","url":"https://www.guru99.com/python-howto-remove-duplicates.html#step2"},{"@type":"HowToStep","text":"Next, We can remove duplicates from the given list by importing OrderedDict from collections.","name":"Method 3) Using Dict","url":"https://www.guru99.com/python-howto-remove-duplicates.html#step3"},{"@type":"HowToStep","text":"Using for-loop, we will traverse the list of items to remove duplicates.","name":"Method 4) Using for-loop","url":"https://www.guru99.com/python-howto-remove-duplicates.html#step4"},{"@type":"HowToStep","text":"Next method, List comprehensions are Python functions that are used for creating new sequences (such as lists, dictionaries, etc.)","name":"Method 5) Using list comprehension","url":"https://www.guru99.com/python-howto-remove-duplicates.html#step5"},{"@type":"HowToStep","text":"Next step, The method unique() from Numpy module can help us remove duplicate from the list given.","name":"Method 6) Using Numpy unique() method","url":"https://www.guru99.com/python-howto-remove-duplicates.html#step6"},{"@type":"HowToStep","text":"The Pandas module has a unique() method that will give us the unique elements from the list given.","name":"Method 7) Using Pandas methods","url":"https://www.guru99.com/python-howto-remove-duplicates.html#step7"},{"@type":"HowToStep","text":"Last, Here the combination of list comprehension and enumerate to remove the duplicate elements.","name":"Method 8) Using enumerate() and list comprehension","url":"https://www.guru99.com/python-howto-remove-duplicates.html#step8"}]},{"@type":"FAQPage","mainEntity":[{"@type":"Question","name":"Does converting a list to a set preserve the original order?","acceptedAnswer":{"@type":"Answer","text":"No. A set is an unordered collection, so set() may return elements in a different order. To remove duplicates while preserving order, use dict.fromkeys() or a loop that appends only items not already seen."}},{"@type":"Question","name":"What is the fastest way to remove duplicates from a large list?","acceptedAnswer":{"@type":"Answer","text":"For large lists, set() is fastest when order does not matter because membership checks are O(1). When order matters, list(dict.fromkeys(my_list)) is almost as fast and keeps first-seen order in one line."}},{"@type":"Question","name":"How do you remove duplicates from a list of dictionaries or unhashable items?","acceptedAnswer":{"@type":"Answer","text":"set() and dict.fromkeys() need hashable elements, so they fail on lists or dictionaries. Loop through the list and append items not already seen, or convert each item to a tuple or JSON string as a temporary key."}},{"@type":"Question","name":"How can you remove duplicates and keep order in one line?","acceptedAnswer":{"@type":"Answer","text":"Use list(dict.fromkeys(my_list)). Since Python 3.7 dictionaries preserve insertion order, so the keys stay in first-seen order. On earlier versions, use collections.OrderedDict.fromkeys() for the same result."}},{"@type":"Question","name":"Does removing duplicates modify the original list?","acceptedAnswer":{"@type":"Answer","text":"No. Methods such as set(), dict.fromkeys(), comprehensions, and NumPy or Pandas unique() build a new list and leave the source unchanged. Reassign the result to the same variable if you want to overwrite it."}},{"@type":"Question","name":"When should you use NumPy or Pandas to remove duplicates?","acceptedAnswer":{"@type":"Answer","text":"Use numpy.unique() or pandas.unique() when the data already lives in arrays or a Series, or during data analysis. Note numpy.unique() also sorts the values, while pandas.unique() keeps first-seen order. Call tolist() for a plain list."}},{"@type":"Question","name":"How is duplicate removal used in AI and machine learning?","acceptedAnswer":{"@type":"Answer","text":"Machine learning pipelines drop duplicate rows before training so repeated samples do not bias the model or leak between train and test sets. Pandas drop_duplicates() is the common tool for cleaning tabular datasets."}},{"@type":"Question","name":"Can GitHub Copilot or agentic AI tools write deduplication code?","acceptedAnswer":{"@type":"Answer","text":"Yes. GitHub Copilot and agentic AI assistants generate set(), dict.fromkeys(), or Pandas deduplication code from a comment, suggest the order-preserving option, and refactor loops, though you should still verify edge cases like unhashable items."}}]}],"@id":"https://www.guru99.com/python-howto-remove-duplicates.html#schema-357774","isPartOf":{"@id":"https://www.guru99.com/python-howto-remove-duplicates.html#webpage"},"publisher":{"@id":"https://www.guru99.com/#organization"},"inLanguage":"en-US","mainEntityOfPage":{"@id":"https://www.guru99.com/python-howto-remove-duplicates.html#webpage"}}]}
```