How do you remove duplicates from a list?

See the Python Cookbook for a long discussion of many ways to do this:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/52560

If you don’t mind reordering the list, and all items can be used as dictionary keys (i.e. they are all hashable), you can simply do:

L = list(set(L))

or, for early Python versions:

d = {}
for x in L:
    d[x] = None
L = d.keys()

For unhashable objects, you can sort the sequence and then scan from the end of the list, deleting duplicates as you go:

if L:
   L.sort()
   last = L[-1]
   for i in range(len(L)-2, -1, -1):
       if last == L[i]:
           del L[i]
       else:
           last = L[i]

CATEGORY: programming