List集合之元素和对象去重

目录

1 List元素去重

1.1 移除List中指定某一元素

1.1.1 For循环移除

1.1.1.1 For移除不彻底问题

假如去除List中的Morning元素

 @Test
    public void testRemoveDuplicate(){
        List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
        List<String> l1= new ArrayList<>(strings);
       for (int i = 0; i < l1.size(); i++) {
            if("Morning".equals(l1.get(i))){
                l1.remove(i);
            }
        }
        System.out.println(l1);
    }
执行结果：
[Midday, Evening, Night, Morning, Midday]

查看执行结果发现还有元素Morning没有去掉

1.1.1.2 用 i-- 解决问题

产生的原因就是：ArrayList 是一个数组元素的集合当删掉第一个元素Morning后，集合后面的元素会往前移动，但是此时 i 又指向下一个元素
解决办法：在 list.remove(i) 后 i--
修改后：

 @Test
    public void testRemoveDuplicate(){
        List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
        List<String> l1= new ArrayList<>(strings);
       for (int i = 0; i < l1.size(); i++) {
            if("Morning".equals(l1.get(i))){
                l1.remove(i);
                i--;
            }
        }
        System.out.println(l1);
    }
执行结果：
[Midday, Evening, Night, Midday]

1.1.1.3 倒序遍历移除元素

 @Test
    public void testRemoveDuplicate(){
        List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
        List<String> l1= new ArrayList<>(strings);
       for (int i = l1.size()-1; i >=0 ; i--) {
            if("Morning".equals(l1.get(i))){
                l1.remove(i);
            }
        }
        System.out.println(l1);
    }
执行结果：
[Midday, Evening, Night, Midday]

1.1.2 ForEach移除

1.1.2.1 ConcurrentModificationException异常

@Test
    public void testRemoveDuplicate(){
        List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
        List<String> l1= new ArrayList<>(strings);
       for (String str:l1) {
            if("Morning".equals(str:l1)){
                l1.remove(i);
            }
        }
        System.out.println(l1);
    }
执行结果：
[Midday, Evening, Night, Midday]

会抛出 ConcurrentModificationException 异常
ConcurrentModificationException：当方法检测到对象的并发修改，但不允许这种修改时，抛出此异常。
原因：ArraayList 迭代器有个Itr，其内部有个属性expectedModCount。而集合有个fail-fast 快速失败检测机制，当进行remove()操作时，会比对expectedModCount是否与modCount相等，而前者一般不会改变，但 remove 操作会导致 modCount 发生改变。一旦两者不等，就会抛出ConcurrentModificationException异常。

解决办法：使用迭代器remove()方法

1.1.2.2 iterator遍历

@Test
    public void testRemoveDuplicate(){
        List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
        List<String> l1= new ArrayList<>(strings);
        Iterator<String> iterator = l1.iterator();
        while (iterator.hasNext()){
            String next = iterator.next();
            if("Morning".equals(next)){
                iterator.remove();
            }
        }
        System.out.println(l1);
    }
执行结果：
[Midday, Evening, Night, Midday]

1.2 移除List中重复元素

1.2.1 ForEach添加去重

这个是创建一个空的List，添加前判断下存在与否，不存在才添加，这样就抱着了元素不重复

@Test
    public void testRemoveDuplicate(){
        List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
        List<String> l1= new ArrayList<>(strings);
         List<String> l2 = new ArrayList<>();
        for(String str:l1){
            if(!l2.contains(str)){
                l2.add(str);
            }
        }
        System.out.println(l2);
    }  
执行结果：
[Midday, Evening, Night, Midday]

1.2.2 For双循环去重

从头和尾一起遍历，判断是否有相等，再移除

@Test
    public void testRemoveDuplicate(){
        List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
        List<String> l1= new ArrayList<>(strings);
        for (int i = 0; i < l1.size()-1; i++) {
            for(int j=l1.size()-1;j>i;j--){
                if(l1.get(j).equals(l1.get(i))){
                    l1.remove(j);
                    System.out.println("i=:"+i+",j=:"+j+l1);
                }
            }
        }
        
        System.out.println(l1);
    }
执行结果：
i=:0,j=:6[Morning, Midday, Evening, Night, Morning, Morning, Midday]
i=:0,j=:5[Morning, Midday, Evening, Night, Morning, Midday]
i=:0,j=:4[Morning, Midday, Evening, Night, Midday]
i=:1,j=:4[Morning, Midday, Evening, Night]
[Morning, Midday, Evening, Night]

1.2.3 ForEach循环重复坐标去重

这个方式需要先复制出一个List2，再循环遍历List2，判断List中的元素首尾出现的坐标位置是否一致，若一致，则说明没有重复的，否则重复，并移除重复位置的元素

@Test
    public void testRemoveDuplicate(){
        List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
        List<String> l1= new ArrayList<>(strings);
        List<String> l2= new ArrayList<>(strings);
        for(String s2:l2){
            if(l1.indexOf(s2)!=l1.lastIndexOf(s2)){
                l1.remove(l1.lastIndexOf(s2));
            }
        }     
        System.out.println(l1);
    }
执行结果：
[Morning, Midday, Evening, Night]

1.2.4 Set去重

@Test
    public void testRemoveDuplicate(){
        List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
        List<String> l1= new ArrayList<>(strings);
        List<String> l2= new ArrayList<>(new HashSet(l1)); 
        System.out.println(l2);
    }
执行结果：
[Evening, Night, Morning, Midday]

这个方式去重重复元素最简单，但是不能保证顺序，可以把HashSet替换成LinkedHashSet，就可以保证原来的顺序了

@Test
    public void testRemoveDuplicate(){
        List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
        List<String> l1= new ArrayList<>(strings);
        List<String> l2= new ArrayList<>(new LinkedHashSet(l1)); 
        System.out.println(l2);
    }
执行结果：
[Morning, Midday, Evening, Night]

1.2.5 Stream去重

@Test
    public void testRemoveDuplicate(){
        List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
        List<String> l1= new ArrayList<>(strings);
        List<String> collect = l1.stream().distinct().collect(Collectors.toList());
        System.out.println(collect);
    }
执行结果：
[Morning, Midday, Evening, Night]

2 List对象去重

2.1 使用的实体类

@Data
@NoArgsConstructor
@AllArgsConstructor
@String
public class User(){
	private String userId;
	private String userName;
	private Integer userAge;
}

2.2 利用Collectors.toMap去重

2.2.1 toMap去重说明

List<User> userList = new ArrayList<>();
        userList.add(new User("a", "xiaoming",12));
        userList.add(new User("b", "xiaoming",13));
        userList.add(new User("d", "xiaoming",15));
        userList.add(new User("a", "xiaoming",14));
       
        System.out.println("利用Collectors.toMap去重:");
        //利用Collectors.toMap去重
        userList.stream()
        		//或者这样写 Collectors.toMap(m -> m.getUserId(), 
                .collect(Collectors.toMap(User::getUserId, 
                			Function.identity(), (oldValue, newValue) -> oldValue))
                .values()
                .stream()
                .forEach(System.out::println); //打印

输出结果：

[{"userAge":12,"userId":"a","userName":"xiaoming"},
{"userAge":13,"userId":"b","userName":"xiaoming"},
{"userAge":15,"userId":"d","userName":"xiaoming"}]

其中，Collectors.toMap需要使用三个参数的版本，前两个参数一个是keyMapper函数一个是valueMapper函数的，用过toMap的都知道，去重的关键在于第三个参数BinaryOperator函数接口。

这个BinaryOperator函数接收两个参数，如上面代码，一个oldValue，一个newValue，字面意思，第一个旧值，第二个是新值。当stream构造Map时，会先调用Map的get方法获取该key对应节点的旧值，如果该值为null，则不会调用BinaryOperator函数，如果不为null，则会把获取的旧值与新值作为参数传给函数执行，然后把函数的返回值作为新值put到Map中。如果看不懂，请看源码吧

2.2.2 Funcion.identity()解释

Java 8允许在接口中加入具体方法。接口中的具体方法有两种，default方法和static方法，identity()就是Function接口的一个静态方法。
Function.identity()返回一个输出跟输入一样的Lambda表达式对象，等价于形如t -> t形式的Lambda表达式。
identity() 方法JDK源码如下：

static  Function identity() {
    return t -> t;
}

下面的代码中，Task::getTitle需要一个task并产生一个仅有一个标题的key
task -> task是一个用来返回自己的lambda表达式，上例中返回一个task

private static Map<String, Task> taskMap(List<Task> tasks) {
  return tasks.stream().collect(toMap(Task::getTitle, task -> task));
}

可以使用Function接口中的默认方法identity来让上面的代码代码变得更简洁明了、传递开发者意图时更加直接，下面是采用identity函数的代码。

import static java.util.function.Function.identity;
private static Map<String, Task> taskMap(List<Task> tasks) {
  return tasks.stream().collect(toMap(Task::getTitle, identity()));
}

2.3 利用Collectors.toCollection和TreeSet去重

List<User> userList = new ArrayList<>();
        userList.add(new User("a", "xiaoming",12));
        userList.add(new User("b", "xiaoming",13));
        userList.add(new User("d", "xiaoming",15));
        userList.add(new User("a", "xiaoming",14));
       
        System.out.println("利用Collectors.toMap去重:");
        //利用Collectors.toMap去重
        userList.stream()
                .collect(Collectors.toCollection(() ->
                	 new TreeSet<>(Comparator.comparing(User::getUserId))))
                .values()
                .stream()
                .forEach(System.out::println); //打印

输出结果：

[{"userAge":12,"userId":"a","userName":"xiaoming"},
{"userAge":13,"userId":"b","userName":"xiaoming"},
{"userAge":15,"userId":"d","userName":"xiaoming"}]

利用TreeSet原理去重，TreeSet内部使用的是TreeMap，使用指定Comparator比较元素，如果元素相同，则新元素代替旧元素，
TreeMap的put方法来放入元素的，有兴趣可以自己找源码
如果不想要返回TreeSet类型，那也可以使用Collectors.collectingAndThen转换成ArrayList，也可以用new ArrayList(set)，原理一样，如下：

List<User> distinctList = userList.stream()
                .collect(Collectors.collectingAndThen(Collectors.toCollection(() 
                	-> new TreeSet<>(Comparator.comparing(User::getUserId))), ArrayList::new));

注意：如果想根据两个或三个字段去重，可以在上述三个方法中的Comparator.comparing(User::getUserId))修改为Comparator.comparing(u -> u.getUserId() +"#" + u.getUserName() )) 这样就是根据两个字段去重，中间的#作用就是为了增加辨识度，也可以不加这个#，无论多少个字段去重只用在这里用+连接就可以了

posted @ 2023-04-04 11:58  上善若泪  阅读(238)  评论(0)    收藏  举报