删除序列相同的元素并保持顺序

1. 可哈希的情况下

def dedupe(items):
    seen = set()
    for item in items:
        if item not in seen:
            yield item
            seen.add(item)
a = [1, 5, 2, 1, 9, 1, 5, 10]
print(list(dedupe(a))) ## [1, 5, 2, 9, 10]

2. 不可哈希，比如是字典的情况下

def dedupe(items, key=None):
    seen = set()
    for item in items:
        val = item if key is None else key(item)
        if val not in seen:
            yield item
            seen.add(val)
a = [ {'x':1, 'y':2}, {'x':1, 'y':3}, {'x':1, 'y':2}, {'x':2, 'y':4}]
print(list(dedupe(a,key=lambda d:(d['x'],d['y']))))  ##[{'x': 1, 'y': 2}, {'x': 1, 'y': 3}, {'x': 2, 'y': 4}]

3. 仅仅只是想消除重复元素，不考虑顺序的话，就可以用集合

a=[1, 5, 2, 1, 9, 1, 5, 10]
set(a) ##{1, 2, 5, 9, 10}