Apriori Algorithm for Finding Frequent Itemsets with PySpark

Let’s say we run a grocery store and have a good amount of data from the point of sale. We want the sets of items frequently bought together to be placed on shelves near each other to boost sales and increase customer convenience. To achieve this, we can use the Apriori algorithm. It’s much faster than its brute-force variant and can be implemented in a distributed computing scenario.

We’ll first write the Python code for the parallel processing of dataset partitions at the worker nodes. We’ll then write the final central itemset frequency check by the master node. The code we’ll write can be run on a compute cluster for a full flavor of distributed computing.