.. _manual_pipeline: ============================================================ Building your own neuroimaging machine-learning pipeline ============================================================ Nilearn comes with code to simplify the use of scikit-learn when dealing with neuroimaging data. For the moment, nilearn is focused on functional MRI data. Before using a machine learning tool, we may need to apply the following steps: 1. :ref:`data_loading` : load Nifti files and check consistency of data 2. :ref:`Masking data ` : if a mask is not provided, one is computed automatically 3. :ref:`resampling`: optionally data could be resampled to a different resolution 4. :ref:`temporal_filtering`: detrending, regressing out confounds, normalization .. _data_loading: Data loading and preprocessing =============================== Downloading the data ---------------------- To run demos, data are retrieved using a function provided by nilearn which downloads a dataset and returns a bunch of paths to the dataset files (more details in :ref:`loading_data`). We can then proceed loading them as if they were just any other files on our disk. For example, we can download the data from the `Haxby 2001 paper `_ : .. code-block:: default from nilearn import datasets dataset = datasets.fetch_haxby() ``dataset.func`` contains filenames referring to dataset files on the disk: .. code-block:: default list(sorted(dataset.keys())) # ['anat', 'description', 'func', 'mask', 'mask_face', 'mask_face_little', 'mask_house', 'mask_house_little', 'mask_vt', 'session_target'] dataset.func # ['.../haxby2001/subj2/bold.nii.gz'] Access supplementary information on the dataset: .. code-block:: default print(haxby_dataset['description']) The complete list of the data-downloading functions can be found in the :ref:`reference documentation for the datasets `. Loading non image data: experiment description ----------------------------------------------- An experiment may need additional information about subjects, sessions or experiments. In the Haxby experiment, :term:`fMRI` data are acquired while presenting different category of pictures to the subject (face, cat, ...) and the goal of this experiment is to predict which category is presented to the subjects from the brain activation. These conditions are presented as string into a CSV file. The `pandas `__ function ``read_csv`` is very useful to load this kind of data. .. literalinclude:: ../../examples/00_tutorials/plot_decoding_tutorial.py :start-after: # We use pandas to load them in an array. :end-before: # %% .. seealso:: * `pandas `_ is a very useful Python library to load CSV files and process their data For example, we will now consider only the conditions *cat* and *face* from our dataset. This can be done as follows: .. literalinclude:: ../../examples/00_tutorials/plot_decoding_tutorial.py :start-after: # (i.e. :term:`fMRI` signal is shorter): :end-before: # %% .. note:: If you are not comfortable with this kind of data processing, do not worry: there are plenty of examples in nilearn that allows you to easily load data from provided datasets. Do not hesitate to copy/paste the code and adapt it to your own data format if needed. More information can be found in the :ref:`data manipulation ` section. .. _masking: Masking the data: from 4D image to 2D array --------------------------------------------- While functional neuroimaging data consist in 4D images, positioned in a coordinate space (which we will call :ref:`Niimgs `). For use with the scikit-learn, they need to be converted into 2D arrays of samples and features. .. |niimgs| image:: ../images/niimgs.jpg :scale: 50% .. |arrays| image:: ../images/feature_array.jpg :scale: 35% .. |arrow| raw:: html → .. centered:: |niimgs| |arrow| |arrays| .. currentmodule:: nilearn.maskers We use masking to convert 4D data (i.e. 3D volume over time) into 2D data (i.e. voxels over time). For this purpose, we use the :class:`NiftiMasker` object, a very powerful data loading tool. Applying a mask ................ .. figure:: ../auto_examples/00_tutorials/images/sphx_glr_plot_decoding_tutorial_001.png :target: ../auto_examples/00_tutorials/plot_decoding_tutorial.html :align: right :scale: 30% If your dataset provides a mask, the :class:`NiftiMasker` can apply it automatically. All you have to do is to pass your mask as a parameter when creating your masker. Here we use the mask of the ventral stream, provided with the Haxby dataset. The :class:`NiftiMasker` can be seen as a *tube* that transforms data from 4D images to 2D arrays, but first it needs to 'fit' this data in order to learn simple parameters from it, such as its shape: .. include:: ../../examples/masker_note.rst .. code-block:: python # We first create a masker, giving it the options that we care # about. Here we use standardizing of the data, as it is often important # for decoding from nilearn.maskers import NiftiMasker masker = NiftiMasker(mask_img=mask_filename, standardize=True) # We give the masker a filename and retrieve a 2D array ready # for machine learning with scikit-learn fmri_masked = masker.fit_transform(fmri_filename) Note that you can call ``nifti_masker.transform(dataset.func[1])`` on new data to mask it in a similar way as the data that was used during the fit. Automatically computing a mask ............................... If your dataset does not provide a mask, the Nifti masker will compute one for you in the ``fit`` step. The generated mask can be accessed via the ``mask_img_`` attribute. Detailed information on automatic mask computation can be found in: :ref:`extracting_data`. Applying a scikit-learn machine learning method ================================================ Now that we have a 2D array, we can apply any estimator from the scikit-learn, using its ``fit``, ``predict`` or ``transform`` methods. Here, we use scikit-learn Support Vector Classification to learn how to predict the category of picture seen by the subject: .. code-block:: python svc.fit(fmri_masked, conditions) ########################################################################### # We can then predict the labels from the data prediction = svc.predict(fmri_masked) print(prediction) ########################################################################### We will not detail it here since there is a very good documentation about it in the `scikit-learn documentation `__ Unmasking (inverse_transform) ============================= Unmasking data is as easy as masking it! This can be done by using method ``inverse_transform`` on your processed data. As you may want to unmask several kinds of data (not only the data that you previously masked but also the results of an algorithm), the masker is clever and can take data of dimension 1D (resp. 2D) to convert it back to 3D (resp. 4D). .. code-block:: python coef_img = masker.inverse_transform(coef_) print(coef_img) Here we want to see the discriminating weights of some voxels. Visualizing results =================== Again the visualization code is simple. We can use an :term:`fMRI` slice as a background and plot the weights. Brighter points have a higher discriminating weight. .. code-block:: python from nilearn.plotting import plot_stat_map, show plot_stat_map( coef_img, bg_img=haxby_dataset.anat[0], title="SVM weights", display_mode="yx" ) show() .. figure:: ../auto_examples/02_decoding/images/sphx_glr_plot_haxby_anova_svm_001.png :target: ../auto_examples/plot_decoding_tutorial.html :align: center :scale: 50% Going further ============= .. currentmodule:: nilearn.maskers The :class:`NiftiMasker` is a very powerful object and we have only scratched the surface of its possibilities. It is described in more details in the section :ref:`nifti_masker`. Also, simple functions that can be used to perform elementary operations such as masking or filtering are described in :ref:`preprocessing_functions`.