How to remove duplicates from a list in Java

In Java programming, it is necessary to eliminate duplicate elements from a list to prevent data repetition and code-related problems. Various techniques are available for removing duplicates from a list, including the following:

  • Making iterations in the list

  • Using the stream.distinct () method

  • Using a LinkedHashset

We will implement the three techniques separately to remove duplicates from the ArrayList, which is defined as:
(Arrays.asList(14, 10, 14, 72, 72, 13, 13, 10, 13, 42, 56, 56));

Making iterations in the list

This method makes iterations in the array to get rid of the duplicate. However, it is not time-efficient when it comes to handling large data sets.

Below is the code implementation.

import java.util.*;
public class main {
    public static <T> ArrayList<T> removeDupe(ArrayList<T> list){
          ArrayList<T> newList = new ArrayList<T>();
          for (T element : list) {
             if (!newList.contains(element)) {
                  newList.add(element);
            }
        }
          return newList;
    }
      public static void main(String args[]){
          ArrayList<Integer>  list = new ArrayList<>(
                Arrays.asList(14, 10, 14, 72, 72, 13, 13, 10, 13, 42, 56, 56));
          System.out.println("ArrayList with duplicates: "+ list);
          ArrayList<Integer> newList = removeDupe(list);
         System.out.println("ArrayList with duplicates removed: "
                           + newList);
    }
}

 Let’s take a look at the code explanation above:

  • Line 1: We import all the classes in the java.util package, which contains utility classes like ArrayList and Arrays.

  • Line 2: We define the main class of the program, which is called main. This is where the program begins execution.

  • Line 3: We define a generic method removeDupe()that takes an ArrayList of type T as an argument and returns an ArrayList of type T. The method will remove all duplicate elements from the input list and return a new list without duplicates.

  • Line 4: We create a new ArrayList called newList that will hold the unique elements.

  • Line 5: This starts a for loop that iterates through each element of the input list.

  • Lines 6–7: We check if the element is already in the newList using the contains() method of the ArrayList. If the element is not in the newList, it is added to the newList using the add() method.

  • Line 10: This returns the newList with all the duplicates removed.

  • Line 12: We define the main method of the program in this line, which is called when the program is executed.

  • Line 13: We create a new ArrayList called list and initialize it with a list of integers containing duplicates.

  • Line 15: We print out the original ArrayList that had duplicate elements.

  • Line 16: We call the removeDupe () method to remove duplicates from the list and assign the new list to a new ArrayList called newList.

  • Line 17: This prints out the new ArrayList with duplicates removed.

Using the stream.distinct() method

In this section, we employ the distinct() operation on the initial list to generate a fresh stream of unique elements. By distinct elements, we mean those elements from the array that appear only once. This means that duplicate elements from the original list are eliminated, resulting in a stream exclusively containing distinct elements.

import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;
public class main {
    public static void main(String args[]) {
        List<Integer> withDupes = Arrays.asList(14, 10, 14, 72, 72, 13, 13, 10, 13, 42, 56, 56);
        System.out.println(" List with duplicates: " + withDupes);
        List<Integer> withoutDupes = withDupes.stream().distinct() .collect(Collectors.toList());
        System.out.println("\n List without duplicates: " + withoutDupes);
    }
}

Explanation

  • Lines 1–3: We import all we need to work with lists, arrays and the stream.distinct() method.

  • Line 4: We call the main() method of the main class.

  • Line 5: In this line, we use the following keywords:

    • public: This is an access modifier.

    • static: This means the method can be called without an instance of the class void, which is the return type of the method.

    • main: This is the name of the method.

    • String args[]: This is the parameter of the method.

  • Line 6: We create a new List called withDupes and initialize it with a set of integer values that include duplicates using the Arrays.asList() method.

  • Line 7: Here, we print out the original List using the println() method.

  • Line 8: We first create a new List called withoutDupes. Then use the distinct() method to remove any duplicates in the initial list, withDupes. The resulting List is collected using the collect() method and is added to the withoutDupes ArrayList using the toList() method.

  • Line 9: Finally, we print out the withoutDupes list using the println() method. The \n character adds a new line before the output.

Using the LinkedHashSet on integers

In this method, we utilize sets to get rid of duplicates. As we know, sets do not have room for duplicate elements. All that happens is that we convert our array into a set, a step that removes the duplicates, and then we convert it back to an array.

Below is a code sample.

import java.util.ArrayList;
import java.util.Arrays;
import java.util.LinkedHashSet;
import java.util.Set;
class main {
  public static void main(String[] args) {
    ArrayList<Integer> numbers;
    numbers = new ArrayList<>(Arrays.asList(14, 10, 14, 72, 72, 13, 13, 10, 13, 42, 56, 56));
    System.out.println("ArrayList with duplicate elements: " + numbers);
    Set<Integer> set = new LinkedHashSet<>();
    set.addAll(numbers);
    numbers.clear();
    numbers.addAll(set);
    System.out.println("\n ArrayList without duplicate elements: " + numbers);
  }
}

Explanation

  • Lines 1–4: We import all we need to work with Arrays, ArrayLists, Sets , and LinkedHashset.

  • Line 5: We call the main() method of the main class.

  • Line 6: In this line, we use the keywords:

    • public: This is an access modifier.

    • static: This means the method can be called without an instance of the class void, which is the return type of the method.

    • main: This is the name of the method.

    • String args[]: This is the parameter of the method.

  • Line 7: Here, ArrayList <Integer> numbers means we declare a new ArrayList of type Integer with the name of numbers.

  • Line 8: We initialize the numbers ArrayList with a list of integers using the Arrays.asList() method.

  • Line 9: We print the numbers ArrayList that contains duplicate elements.

  • Line 10: We create a new LinkedHashSet named set that stores Integer objects.

  • Line 11: We then add all the elements of numbers ArrayList to the set LinkedHashSet. This will remove any duplicates from the set.

  • Line 12: In this line, we clear all the contents of numbers ArrayList.

  • Line 13: We then add all the elements from set LinkedHashSet back to the numbers ArrayList. This will add only the unique elements from the original numbers ArrayList.

  • Line 14: Finally, we print the numbers ArrayList that now contains only unique elements.

Free Resources

License: Creative Commons-Attribution NonCommercial-ShareAlike 4.0 (CC-BY-NC-SA 4.0)