Performance and Capacity Considerations

As we work on a machine learning-based system, our goal is generally to improve our metrics (engagement rate, etc.) while ensuring that we meet the capacity and performance requirements.

Major performance and capacity discussions come in during the following two phases of building a machine learning system:

1. Training time: How much training data and capacity is needed to build our predictor?

2. Evaluation time: What are the Service level agreement(SLA) that we have to meet while serving the model and capacity needs?

We need to consider the performance and capacity along with optimization for the ML task at hand, i.e., measure the complexity of the ML system at the training and evaluation time and use it in the decision process of building our ML system architecture as well as in the selection of the ML modeling technique.

Complexities consideration for an ML system

Machine learning algorithms have three different types of complexities:

• Training complexity

  The training complexity of a machine learning algorithm is the time taken by it to train the model for a given task.

• Evaluation complexity

  The evaluation complexity of a machine learning algorithm is the time taken by it to evaluate the input at testing time.

• Sample complexity

  The sample complexity of a machine learning algorithm is the total number of training samples required to learn a target function successfully.

  📝 Sample complexity changes if the model capacity changes. For example, for a deep neural network, the number of training examples has to be considerably larger than decision trees and linear regression.

Comparison of training and evaluation complexities

You can see how the training and evaluation complexities can be used to evaluate which model will be best for a given task and resources.

Assume that

• $n$ is the number of the training samples
• $f$ is the number of features
• $n_{trees}$ is the number of trees (for tree-based algorithms)
• $n_{l_i}$

...