Performance and Capacity Considerations
Explore how to assess and optimize performance and capacity considerations in machine learning systems. Learn to analyze training, evaluation, and sample complexities to choose appropriate models, balance system constraints, and apply layered funnel approaches for scalable, efficient ML deployments.
As we work on a machine learning-based system, our goal is generally to improve our metrics (engagement rate, etc.) while ensuring that we meet the capacity and performance requirements.
Major performance and capacity discussions come in during the following two phases of building a machine learning system:
-
Training time: How much training data and capacity is needed to build our predictor?
-
Evaluation time: What are the Service level agreement(SLA) that we have to meet while serving the model and capacity needs?
We need to consider the performance and capacity along with optimization for the ML task at hand, i.e., measure the complexity of the ML system at the training and evaluation time and use it in the decision process of building our ML system architecture as well as in the selection of the ML modeling technique.
Complexities consideration for an ML system
Machine learning algorithms have three different types of complexities:
-
Training complexity
The training complexity of a machine learning algorithm is the time taken by it to train the model for a given task.
-
Evaluation complexity
The ...