Thursday, August 19, 2021

How to remove duplicates elements from ArrayList in Java - Example

You can remove duplicates or repeated elements from ArrayList in Java by converting ArrayList into HashSet in Java. but before doing that just keep in mind that the set doesn't preserver insertion order which is guaranteed by List, in fact, that’s the main difference between List and Set in Java. So when you convert ArrayList to HashSet all duplicates elements will be removed but the insertion order will be lost. Let’s see this in action by writing a Java program to remove duplicates from ArrayList in Java.

In this Java collection tutorial, we will see both approaches of deleting duplicates from ArrayList e.g using HashSet and LinkedHashSet and compare the order of elements in the final ArrayList which contains no duplicates.

If you are not very familiar with What is an ArrayList and HashSet in Java collection framework, I suggest reading Java ArrayList Example and 10 HashSet Example in Java. These articles contain a good introduction to most commonly used collection in Java i.e. ArrayList and HashSet.


Java program to delete duplicates from ArrayList

How to remove duplicate object from ArrayList in JavaHere is a quick example of removing duplicates from ArrayList in Java. Suppose you have an ArrayList of String which contains 4 Strings out of those ones is duplicate and we want to remove that duplicate:




//ArrayList with duplicates String
List<String> duplicateList = (List<String>) Arrays.asList("Android" , "Android", "iOS", "Windows mobile");
System.out.println("size of Arraylist with duplicates: " + duplicateList.size()); //should print 4 becaues of duplicates Android

System.out.println(duplicateList);
     
//Converting ArrayList to HashSet to remove duplicates
HashSet<String> listToSet = new HashSet<String>(duplicateList);
     
//Creating Arraylist without duplicate values
List<String> listWithoutDuplicates = new ArrayList<String>(listToSet);
System.out.println("size of ArrayList without duplicates: " + listToSet.size()); //should print 3 becaues of duplicates Android removed

System.out.println(listWithoutDuplicates);

Output:
size of Arraylist with duplicates: 4
[Android, Android, iOS, Windows mobile]
size of ArrayList without duplicates: 3
[Android, Windows mobile, iOS]


Now if you have noticed here duplicate entry "Android" has been removed from ArrayList but the order of ArrayList is not the same

Since we have converted ArrayList to HashSet we have lost the insertion order of elements. but don't worry there is another way of removing duplicates from ArrayList without losing the order of elements, that instead of HashSet we need to use LinkedHashSet, Which guarantees insertion order. 

Just remember checking whether ArrayList contains duplicates or not is completely different than removing it, which is what we are doing here. Here is another example of removing duplicate entries from ArrayList without losing insertion order or entries:

import java.util.ArrayList;
import java.util.Arrays;
import java.util.LinkedHashSet;
import java.util.List;

public class RemoveDuplicatesFromArrayList {

    public static void main(String args[]) {
   
        //ArrayList with duplicates String
        List<String> duplicateList = (List<String>) Arrays.asList("Android" , "Android", "iOS", "Windows mobile");
       
        //should print 4 becaues of duplicates Android
        System.out.println("size of Arraylist with duplicates: " + duplicateList.size());      
        System.out.println("ArrayList with duplicates: " + duplicateList);
     
        //Converting ArrayList to HashSet to remove duplicates
        LinkedHashSet<String> listToSet = new LinkedHashSet<String>(duplicateList);
     
        //Creating Arraylist without duplicate values
        List<String> listWithoutDuplicates = new ArrayList<String>(listToSet);

        //should print 3 because of duplicates Android removed
        System.out.println("size of ArrayList without duplicates: " + listToSet.size());
        System.out.println("ArrayList after removing duplicates in same order: " + listWithoutDuplicates);
     
     
    }
 
}

Output:
size of Arraylist with duplicates: 4
ArrayList with duplicates: [Android, Android, iOS, Windows mobile]
size of ArrayList without duplicates: 3
ArrayList after removing duplicates in the same order: [Android, iOS, Windows mobile]


So now we know how to remove duplicates from ArrayList in Java and also know how to preserve the order of elements while removing duplicates from  ArrayList. If you don't prefer converting List to Set than you can still go with copying data from one ArrayList to other ArrayList and removing duplicates by checking with ArrayList.contains() method.


Related Java ArrayList tutorials from this Blog

12 comments :

Rohan said...

If you don't need duplicates, then why not just used HashSet or LinkedHashSet from start instead of using ArrayList? How about any builtin method for deleting all duplicates from Arraylist ?

Unknown said...

I just wanted to ask very same question..what is the point of doing all this by hand when you have it already implemented and optimized by Java itself...

javin paul said...

Hi @Rohan and @Tomasz, There are many times when you are presented with a legacy code and you don't write everything by scratch. Sometime with gap in requirement or change of business requirement this kind of things creeps into code. I am firm believer to fix root cause but it's not always possible so work around come in place. If you have a ArrayList which can contain duplicates but you need to operate with only unique elements with same order, then you need to remove those duplicates.

Avinash said...

How to write a method which accepts the list and removes duplicates from the same list without changing the order,without using temp list and without using set?

Anonymous said...

Hi javin

Can you please tell me how to create custom ArrayList with simple example.

Thanks

Unknown said...

@Avinash, to remove duplicates from the list w/o using another list, use an iterator and a temporary set.
Set seen = HashSet();
Iterator it = myObject.iterator();
while (it.hasNext()) {
final MyObject o = it.getNext();
if (seen.contains(o)) {
it.remove(o);
} else {
seen.add(o);
}
}

Anonymous said...

Perhaps look at the implementation of:

https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#distinct--

?

Anyway even when interfacing with legacy code, I'd still expect it to be a List not some concrete implementation being passed around. Also are you sure the cast is needed with '(List) Arrays.asList(..)' ?

Anonymous said...

How do you remove duplicate values form ArrayList without using HashSet in Java?

javin paul said...

@Anonymous, then you need to employ the same algorithm we have used to remove duplicate values from array without using collection class in Java. Basically, you need to iterate and mark duplicate element dirty and then remove it by copying unique elements to another array.

Mubarak Sheik said...

@Rick Ryker, your code will entirely remove the element itself, instead of just removing the duplicates[iOS, Windows mobile] will be your result

hnrami said...

List duplicateList = (List) Arrays.asList("Android" , "Android", "iOS", "Windows mobile");

duplicateList.stream().distinct().forEach(System.out::println);

Sudhakar Sir said...

A Small Program for All Array List & Removing Duplicates, CASE_INSENSITIVE_ORDER
import java.util.ArrayList;
import java.util.Arrays;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.TreeSet;
import java.io.*;

public class RemoveDuplicateFromArrayList
{
public static void main(String args[])
{
List duplicateList=(List) Arrays.asList("Android","Android","Windows","iOS","Beta","Java","Java","iOs","Normal");
System.out.println("Size of Arraylist with Duplicates: "+duplicateList.size());
System.out.println("ArrayList with Duplicates: "+duplicateList);
LinkedHashSet listToSet=new LinkedHashSet(duplicateList);
List listWithoutDuplicates=new ArrayList(listToSet);

System.out.println("Size of the ArrayList without Duplicates: "+listToSet.size());
System.out.println("ArrayList after Removing Duplicates in Same Order: "+listWithoutDuplicates);
//CaseInsensitive Problem
System.out.println("Case Insensitive Model");
Set s1=new TreeSet(String.CASE_INSENSITIVE_ORDER);
s1.addAll(duplicateList);
System.out.println("Size of the ArrayList without Duplicates CASE_INSENSITIVE: "+s1.size());
System.out.println("ArrayList after Removing Duplicates in Same Order CASE_INSENSITIVE: "+s1);

}
}

Post a Comment