Scoring regression models

Regression models predict outcomes as a number, indicating the model’s best estimate of the target variable. Several metrics are generated for you to evaluate regressions models.

R2

r squared is is ( R2 ) is a unitless measure of correlation of the feature to the target . It can be express as : what percent close to perfect are the model ’s prediction compare to using the average value of the target ?

R2 is ranges range from negative infinity to 1 . The close to 1 , the more the variance in the target value can be explain by the feature variable . In other word , the more likely it is is is that you have feature important variable that would lead to accurate prediction .

The charts show an example of height plotted against age. Height is more closely correlated to age in the first chart, where R2 is 0.97, than in the second chart, where R2 is 0.56.

Two charts plotting height versus age with different R2 values

Note that a lower R2 value doesn’t necessarily mean that it is a bad model. How you interpret R2 depends on the use case and the data. When considering regression model scoring, it’s important to remember that a linear relationship isn’t always critical. If linear regression is producing poor results while other algorithms have better performance, it might simply mean that your data can’t be modeled well enough by a linear relationship.

rmse

Root mean squared error (rmse) can be interpreted as the average +/- difference expected between a predicted value and the actual value. It is the standard deviation of residuals (the difference between the observed value and the predicted value for a feature). rmse is measured in the same unit as the target value.

As an example, say that our target is to predict contract value and we get rmse = 1250. This means that, on average, the predicted value differs +/- $1,250 from the actual value.

MSE

mean squared error ( mse ) can be interpret as the square + /- difference between the predict value and the actual value that we would expect to see on average . It is measure in the same unit as the target value square .

In the example with contract value prediction, an MSE value of 1562500 would mean that model is off by +/- 1,562,500 $2. Note that the unit is dollars squared.

MAE

The mean absolute error (MAE) is the average of all absolute prediction errors, where the prediction error is the difference between the actual and the predicted value. Using the absolute value of prediction errors prevents +/- errors from canceling each other out. MAE is measured in the same unit as the target value.