Behavioral Testing
Learn about behavioral testing, including consistency, invariance, and directional expectation tests.
We'll cover the following
Overview
The machine learning world is full of randomness, inaccuracy, and uncertainty. Often, we cannot test the exact values of something (like in regular deterministic software). Still, there are things or basic logic that remain unchanged in the behavior of our models and auxiliary functions for them.
Behavioral tests are a crucial part of the machine learning process because they help ensure our models are functioning as expected and producing reliable results. These tests examine the model and auxiliary functions under different conditions and input variations to identify potential issues or inconsistencies.
Behavioral testing involves various techniques and methods, such as consistency, invariance, negation, and directional expectation tests:

Consistency tests involve checking whether different data transformations are consistent with each other.

Invariance tests check whether particular properties of the model (called invariances) remain unchanged when given input that has been transformed or distorted in some way.

Negation tests involve checking whether the model produces the opposite output when given the opposite of the expected input.

Directional expectation tests involve checking whether the model changes its output in the expected direction when given input that deviates from the initial input in a specific way.
Consistency tests
Consistency tests ensure that the behavior of functions related to each other is consistent. In other words, they check that any transformations supposed to invert each other do so.
For example, in data preprocessing, we often use functions to convert things back and forth (e.g., converting from radians to degrees and vice versa, np.rad2deg
and np.deg2rad
).
In the world of computer vision, popular conversions are:
 different units (degrees to radians)
 different representations (quaternions vs. rotation matrices).
 different formats (
torch
vs.numpy
,uint
0…255 vs.float
0…1).
Consistency using unit tests
We may require consistency even in a simple unit test.
Get handson with 1400+ tech skills courses.