Is the Model Fair?

Now, we move to the exciting part. There is a model, and we want to know if it is fair. Before we learn how to do this systematically, we will perform some experiments:

• Examine regression coefficients. Logistic regression provides a straightforward method to understand internal logic. Each feature is associated with a numerical value. The bigger the value, the greater the contribution toward a positive score. Negative values push predictions to a negative class. By examining the values of the coefficients, we can understand which features are the most important and check if any of these are sensitive.

• For each observation where the value of gender_female is set to 1, we will set it to 0 and set the value of gender_male to 1. We will perform the same experiment as we did previously but do that in reverse.

Disclaimer: This content may include terms and phrases that are sensitive or potentially offensive; these are used strictly for educational or illustrative purposes. Educative does not endorse or encourage the expression of such sentiments in any way. Our intent is to shed light on these issues, promoting awareness and understanding, rather than to cause harm or discomfort. Viewer discretion is advised.

This is not enough to have a whole picture, but it will give us a better intuition in later parts of the course.

Experiments lead us to a few interesting conclusions. First, we know that the model actually uses gender information for prediction. We know it because related coefficients are non-zero. The value related to males is greater than one related to females. The model is more likely to assign good credit scores to males if all other features are the same.

But the effect could be marginal. To verify this, we try replacing values of gender attributes to see if it changes the prediction. We find that this is the case! The model likely has a fairness issue, but right now, we don’t know how to measure if it is severe.