Part 2 Models and model families
When someone talks about supervised learning, they might say something like: “I fit K Nearest Neighbors on the dataset and got 57 percent accuracy.” This communicates the gist of what they did, but this isn’t enough information to reproduce what they actually did: we don’t know what value of \(k\) or the distance metric they used.
When someone says that they used K Nearest Neighbors, they are referring to a family of supervised learning models, that is, KNN with \(k \in \{1, 2, ...\}\) and some distance metric from a list of metrics, say, the \(\ell_p\) metrics. Together all the possible values of \(k\) combined with all the \(\ell_p\) metrics form a hyperparameter space, and we have to hope that our friend has selected hyperparameters in some reasonable way.
Once our friend has selected hyperparameters, say \(k=13\) and Euclidean distance, they are now speaking about a specific, unambiguous model. With these hyperparameters, we have enough information to fit exactly the same model ourselves.
These objects form the basis for our grammar:
model family=modelling technique+hyperparameter spacemodel=modelling technique+specific values of hyperparameters
As a concrete example, consider the glmnet package.
glmnetobjects fit with a specific value of \(\lambda\) correspond tomodelobjectscv.glmnetobjects correspond to amodel familycontaining performance information for various hyperparameter values
Once fit, a model family is a set of fit models. There are important differences between model family objects and model objects. For example, we typically perform inference on models rather than model families.