What is the MLlib Architecture in Apache Spark?

MLlib

MLlib is the library offered by the Apache Spark framework to facilitate Machine Learning.

Features

MLlib is advertised as easy to use and offers high performance and compatibility to other services.

Ease of Use

Since it was developed for use with Spark, MLlib works well with all of its other APIs. It also works with NumPy in Python and R libraries, which makes it much easier to incorporate into well-known languages.

Performance

Apache claims that MLlib is much faster than its competitors. A comparison run by Apache with Hadoop’s MapReduce showed that MLlib was 100 times faster. According to Apache, MLlib also offers better results.

Compatibility

Spark can be run on many different services such as Amazon EC2, Hadoop YARN, Mesos, etc. It can even be run in its standalone cluster mode. Spark is also able to take data from numerous sources including other Apache services.

Functionality

MLlib comes with numerous built-in algorithms for classification, recommendation, regression, clustering, topic modeling, and other machine learning functions. Other utilities include feature transformations, ML pipelines, model evaluation, ML persistence, distributed linear algebra, and statistics.