.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/temporal-response-functions/partitions.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_temporal-response-functions_partitions.py: .. _exa-data_split: .. currentmodule:: eelbrain Data partitions for boosting ============================ The boosting algorithm can use two different forms of cross-validation: cross-validation as stopping criterion (always on), and cross-validation for model evaluation (optional). This requires paertitioning the data into different segments. The :func:`eelbrain.plot.preview_partitions` function is for exploring the effect of different parameters on the way the data is split. .. contents:: Sections :local: :backlinks: top Validation set -------------- During boosting, every training step consists in modifying one element of the kernel/TRF. After every such step, the new TRF is evaluated against the validation data. For continuous data (without :class:`Case` dimension), the default is to split the data into 10 equal-length segments, and perform 10 model fits, each using one of the segments as validation set. In the plots below, each "Split" shown on the y-axis corresponds to a separate run of the boosting algorithm. The results returned by the :func:`boosting` function would be based on to the average TRF of those 10 runs. .. GENERATED FROM PYTHON SOURCE LINES 18-24 .. code-block:: Python # sphinx_gallery_thumbnail_number = 6 from eelbrain import * p = plot.preview_partitions() .. image-sg:: /auto_examples/temporal-response-functions/images/sphx_glr_partitions_001.png :alt: partitions :srcset: /auto_examples/temporal-response-functions/images/sphx_glr_partitions_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 25-26 The number of partitions can be controlled with the `partitions` parameter: .. GENERATED FROM PYTHON SOURCE LINES 26-30 .. code-block:: Python p = plot.preview_partitions(partitions=5) p = plot.preview_partitions(partitions=7) .. rst-class:: sphx-glr-horizontal * .. image-sg:: /auto_examples/temporal-response-functions/images/sphx_glr_partitions_002.png :alt: partitions :srcset: /auto_examples/temporal-response-functions/images/sphx_glr_partitions_002.png :class: sphx-glr-multi-img * .. image-sg:: /auto_examples/temporal-response-functions/images/sphx_glr_partitions_003.png :alt: partitions :srcset: /auto_examples/temporal-response-functions/images/sphx_glr_partitions_003.png :class: sphx-glr-multi-img .. GENERATED FROM PYTHON SOURCE LINES 31-32 For data with multiple trials (data with a :class:`Case` dimension), the function attempts to use trials evenly across time: .. GENERATED FROM PYTHON SOURCE LINES 32-36 .. code-block:: Python p = plot.preview_partitions(20, partitions=5) p = plot.preview_partitions(20, partitions=2) .. rst-class:: sphx-glr-horizontal * .. image-sg:: /auto_examples/temporal-response-functions/images/sphx_glr_partitions_004.png :alt: partitions :srcset: /auto_examples/temporal-response-functions/images/sphx_glr_partitions_004.png :class: sphx-glr-multi-img * .. image-sg:: /auto_examples/temporal-response-functions/images/sphx_glr_partitions_005.png :alt: partitions :srcset: /auto_examples/temporal-response-functions/images/sphx_glr_partitions_005.png :class: sphx-glr-multi-img .. GENERATED FROM PYTHON SOURCE LINES 37-40 Validation and testing sets --------------------------- Testing the result of a model fit with cross-validation requires data that was never used during training. Testing with cross-validation is enabled in the :func:`boosting` function by setting ``test=True``. When testing is enabled, each data segment is used in turn as testing segment. For each testing segment, the remaining segment are used in different runs as training and validation data. The results of those runs are then averaged to predict responses in the testing data. This nested loop means that the number of boosting runs can get large quickly when using many partitions, so the default is to use just four partitions: .. GENERATED FROM PYTHON SOURCE LINES 40-43 .. code-block:: Python p = plot.preview_partitions(test=True) p = plot.preview_partitions(20, partitions=5, test=True) .. rst-class:: sphx-glr-horizontal * .. image-sg:: /auto_examples/temporal-response-functions/images/sphx_glr_partitions_006.png :alt: partitions :srcset: /auto_examples/temporal-response-functions/images/sphx_glr_partitions_006.png :class: sphx-glr-multi-img * .. image-sg:: /auto_examples/temporal-response-functions/images/sphx_glr_partitions_007.png :alt: partitions :srcset: /auto_examples/temporal-response-functions/images/sphx_glr_partitions_007.png :class: sphx-glr-multi-img .. _sphx_glr_download_auto_examples_temporal-response-functions_partitions.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: partitions.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: partitions.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: partitions.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_