We simulate data as in Michel et al. 2012 : a linear model with a random design matrix X:
We provide a black-box function to create the data in the example script.
We can now run different estimators and look at their prediction score, as well as the feature maps that they recover. Namely, we will use
Note that the RidgeCV and the ElasticNetCV have names ending in CV that stands for cross-validation: in the list of possible alpha values that they are given, they choose the best by cross-validation.
As the estimators expose a fairly consistent API, we can all fit them in a for loop: they all have a fit method for fitting the data, a score method to retrieve the prediction score, and because they are all linear models, a coef_ attribute that stores the coefficients w estimated (see the code of the simulation).
All parameters estimated from the data end with an underscore
Use recursive feature elimination (RFE) with the SVM:
>>> from sklearn.feature_selection import RFE
Read the object’s documentation to find out how to use RFE.
Performance tip: increase the step parameter, or it will be very slow.
Source code to run the simulation
The full file to run the simulation can be found in Example of pattern recognition on simulated data