Trusted answers to developer questions
Trusted Answers to Developer Questions

Related Tags

spark
big data
machine learning
mlib

What is the MLlib Architecture in Apache Spark?

Educative Answers Team

Grokking Modern System Design Interview for Engineers & Managers

Ace your System Design Interview and take your career to the next level. Learn to handle the design of applications like Netflix, Quora, Facebook, Uber, and many more in a 45-min interview. Learn the RESHADED framework for architecting web-scale applications by determining requirements, constraints, and assumptions before diving into a step-by-step design process.

MLlib

MLlib is the library offered by the Apache Spark framework to facilitate Machine Learning.

widget

Features

MLlib is advertised as easy to use and offers high performance and compatibility to other services.

Ease of Use

Since it was developed for use with Spark, MLlib works well with all of its other APIs. It also works with NumPy in Python and R libraries, which makes it much easier to incorporate into well-known languages.

Performance

Apache claims that MLlib is much faster than its competitors. A comparison run by Apache with Hadoop’s MapReduce showed that MLlib was 100 times faster. According to Apache, MLlib also offers better results.

Compatibility

Spark can be run on many different services such as Amazon EC2, Hadoop YARN, Mesos, etc. It can even be run in its standalone cluster mode. Spark is also able to take data from numerous sources including other Apache services.

Functionality

MLlib comes with numerous built-in algorithms for classification, recommendation, regression, clustering, topic modeling, and other machine learning functions. Other utilities include feature transformations, ML pipelines, model evaluation, ML persistence, distributed linear algebra, and statistics.

RELATED TAGS

spark
big data
machine learning
mlib
Copyright ©2022 Educative, Inc. All rights reserved

Grokking Modern System Design Interview for Engineers & Managers

Ace your System Design Interview and take your career to the next level. Learn to handle the design of applications like Netflix, Quora, Facebook, Uber, and many more in a 45-min interview. Learn the RESHADED framework for architecting web-scale applications by determining requirements, constraints, and assumptions before diving into a step-by-step design process.

Keep Exploring