.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/datasets/group-level-analysis.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_datasets_group-level-analysis.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_datasets_group-level-analysis.py:


.. currentmodule:: eelbrain


Group level analysis
====================

:class:`Datasets <Dataset>` provide a means to collect data for statistical
analysis.
A :class:`Dataset` is similar to a dataframe in R or :mod:`pandas`,
but can hold mass-univariate measurements in :class:`NDVars <NDVar>`.

This example illustrates how to construct a :class:`Datasets <Dataset>`
by first collecting the cases, or rows, of the desired data table,
and then combining them using :meth:`Dataset.from_caselist`.

.. contents:: Contents
   :local:

.. GENERATED FROM PYTHON SOURCE LINES 23-31

Simulating subject data
-----------------------

This example uses simulated EEG data to illustrate that.

We will use the simulated EEG data to derive a general input structure for
creating group level datasets:
A list of condition labels and corresponding (simulated) EEG responses.

.. GENERATED FROM PYTHON SOURCE LINES 31-38

.. code-block:: Python


    from eelbrain import *


    data = datasets.simulate_erp(seed=1)
    data.head()






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <figure><table border="1" frame="hsides" rules="none" cellpadding="2"><tr>
    <td style="text-align:left">#</td>
    <td style="text-align:left">cloze</td>
    <td style="text-align:left">predictability</td>
    <td style="text-align:left">n_chars</td>
    </tr>
    <tr>
    <td style="text-align:left">0</td>
    <td style="text-align:left">0.0054865</td>
    <td style="text-align:left">low</td>
    <td style="text-align:left">6</td>
    </tr>
    <tr>
    <td style="text-align:left">1</td>
    <td style="text-align:left">0.20557</td>
    <td style="text-align:left">low</td>
    <td style="text-align:left">5</td>
    </tr>
    <tr>
    <td style="text-align:left">2</td>
    <td style="text-align:left">0.95786</td>
    <td style="text-align:left">high</td>
    <td style="text-align:left">4</td>
    </tr>
    <tr>
    <td style="text-align:left">3</td>
    <td style="text-align:left">0.20756</td>
    <td style="text-align:left">low</td>
    <td style="text-align:left">3</td>
    </tr>
    <tr>
    <td style="text-align:left">4</td>
    <td style="text-align:left">0.042116</td>
    <td style="text-align:left">low</td>
    <td style="text-align:left">3</td>
    </tr>
    <tr>
    <td style="text-align:left">5</td>
    <td style="text-align:left">0.2161</td>
    <td style="text-align:left">low</td>
    <td style="text-align:left">3</td>
    </tr>
    <tr>
    <td style="text-align:left">6</td>
    <td style="text-align:left">0.82065</td>
    <td style="text-align:left">high</td>
    <td style="text-align:left">3</td>
    </tr>
    <tr>
    <td style="text-align:left">7</td>
    <td style="text-align:left">0.050949</td>
    <td style="text-align:left">low</td>
    <td style="text-align:left">4</td>
    </tr>
    <tr>
    <td style="text-align:left">8</td>
    <td style="text-align:left">0.025513</td>
    <td style="text-align:left">low</td>
    <td style="text-align:left">6</td>
    </tr>
    <tr>
    <td style="text-align:left">9</td>
    <td style="text-align:left">0.12511</td>
    <td style="text-align:left">low</td>
    <td style="text-align:left">5</td>
    </tr></table>
    <figcaption>NDVars: eeg</figcaption></figure>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 39-40

Average the data by condition to get two condition averages per subject:

.. GENERATED FROM PYTHON SOURCE LINES 40-44

.. code-block:: Python


    data_by_condition = data.aggregate('predictability')
    data_by_condition






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <figure><table border="1" frame="hsides" rules="none" cellpadding="2"><tr>
    <td style="text-align:left">#</td>
    <td style="text-align:left">n</td>
    <td style="text-align:left">cloze</td>
    <td style="text-align:left">predictability</td>
    <td style="text-align:left">n_chars</td>
    </tr>
    <tr>
    <td style="text-align:left">0</td>
    <td style="text-align:left">40</td>
    <td style="text-align:left">0.89466</td>
    <td style="text-align:left">high</td>
    <td style="text-align:left">4.95</td>
    </tr>
    <tr>
    <td style="text-align:left">1</td>
    <td style="text-align:left">40</td>
    <td style="text-align:left">0.13778</td>
    <td style="text-align:left">low</td>
    <td style="text-align:left">4.975</td>
    </tr></table>
    <figcaption>NDVars: eeg</figcaption></figure>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 45-46

Turn this into the general label/brain response structure:

.. GENERATED FROM PYTHON SOURCE LINES 46-50

.. code-block:: Python


    subject_data = list(data_by_condition.zip('predictability', 'eeg'))
    subject_data





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    [('high', <NDVar 'eeg': 140 time, 65 sensor>), ('low', <NDVar 'eeg': 140 time, 65 sensor>)]



.. GENERATED FROM PYTHON SOURCE LINES 51-59

Construct group level data
--------------------------

Use the procedure described above to simulate a group level dataset.

We collect the labels (subject and condition labels) and brain responses in
a list (``cases``).
Each entry in this list corresponds to one row of the desired :class:`Dataset`:

.. GENERATED FROM PYTHON SOURCE LINES 59-69

.. code-block:: Python


    cases = []  # list of rows
    for subject in range(10):
        data = datasets.simulate_erp(seed=subject)
        data_by_condition = data.aggregate('predictability')
        for predictability, eeg in data_by_condition.zip('predictability', 'eeg'):
            cases.append([str(subject), predictability, eeg])

    cases





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    [['0', 'high', <NDVar 'eeg': 140 time, 65 sensor>], ['0', 'low', <NDVar 'eeg': 140 time, 65 sensor>], ['1', 'high', <NDVar 'eeg': 140 time, 65 sensor>], ['1', 'low', <NDVar 'eeg': 140 time, 65 sensor>], ['2', 'high', <NDVar 'eeg': 140 time, 65 sensor>], ['2', 'low', <NDVar 'eeg': 140 time, 65 sensor>], ['3', 'high', <NDVar 'eeg': 140 time, 65 sensor>], ['3', 'low', <NDVar 'eeg': 140 time, 65 sensor>], ['4', 'high', <NDVar 'eeg': 140 time, 65 sensor>], ['4', 'low', <NDVar 'eeg': 140 time, 65 sensor>], ['5', 'high', <NDVar 'eeg': 140 time, 65 sensor>], ['5', 'low', <NDVar 'eeg': 140 time, 65 sensor>], ['6', 'high', <NDVar 'eeg': 140 time, 65 sensor>], ['6', 'low', <NDVar 'eeg': 140 time, 65 sensor>], ['7', 'high', <NDVar 'eeg': 140 time, 65 sensor>], ['7', 'low', <NDVar 'eeg': 140 time, 65 sensor>], ['8', 'high', <NDVar 'eeg': 140 time, 65 sensor>], ['8', 'low', <NDVar 'eeg': 140 time, 65 sensor>], ['9', 'high', <NDVar 'eeg': 140 time, 65 sensor>], ['9', 'low', <NDVar 'eeg': 140 time, 65 sensor>]]



.. GENERATED FROM PYTHON SOURCE LINES 70-71

This list can now be turned into a :class:`Dataset`:

.. GENERATED FROM PYTHON SOURCE LINES 71-75

.. code-block:: Python


    data = Dataset.from_caselist(['subject', 'predictability', 'eeg'], cases, random='subject')
    data.head()






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <figure><table border="1" frame="hsides" rules="none" cellpadding="2"><tr>
    <td style="text-align:left">#</td>
    <td style="text-align:left">subject</td>
    <td style="text-align:left">predictability</td>
    </tr>
    <tr>
    <td style="text-align:left">0</td>
    <td style="text-align:left">0</td>
    <td style="text-align:left">high</td>
    </tr>
    <tr>
    <td style="text-align:left">1</td>
    <td style="text-align:left">0</td>
    <td style="text-align:left">low</td>
    </tr>
    <tr>
    <td style="text-align:left">2</td>
    <td style="text-align:left">1</td>
    <td style="text-align:left">high</td>
    </tr>
    <tr>
    <td style="text-align:left">3</td>
    <td style="text-align:left">1</td>
    <td style="text-align:left">low</td>
    </tr>
    <tr>
    <td style="text-align:left">4</td>
    <td style="text-align:left">2</td>
    <td style="text-align:left">high</td>
    </tr>
    <tr>
    <td style="text-align:left">5</td>
    <td style="text-align:left">2</td>
    <td style="text-align:left">low</td>
    </tr>
    <tr>
    <td style="text-align:left">6</td>
    <td style="text-align:left">3</td>
    <td style="text-align:left">high</td>
    </tr>
    <tr>
    <td style="text-align:left">7</td>
    <td style="text-align:left">3</td>
    <td style="text-align:left">low</td>
    </tr>
    <tr>
    <td style="text-align:left">8</td>
    <td style="text-align:left">4</td>
    <td style="text-align:left">high</td>
    </tr>
    <tr>
    <td style="text-align:left">9</td>
    <td style="text-align:left">4</td>
    <td style="text-align:left">low</td>
    </tr></table>
    <figcaption>NDVars: eeg</figcaption></figure>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 76-80

Averaging by condition
----------------------

In a dataset that contains condition labels, these labels can be used to derive averages by condition:

.. GENERATED FROM PYTHON SOURCE LINES 80-84

.. code-block:: Python


    data_by_condition = data.aggregate('predictability', drop_bad=True)
    data_by_condition






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <figure><table border="1" frame="hsides" rules="none" cellpadding="2"><tr>
    <td style="text-align:left">#</td>
    <td style="text-align:left">n</td>
    <td style="text-align:left">predictability</td>
    </tr>
    <tr>
    <td style="text-align:left">0</td>
    <td style="text-align:left">10</td>
    <td style="text-align:left">high</td>
    </tr>
    <tr>
    <td style="text-align:left">1</td>
    <td style="text-align:left">10</td>
    <td style="text-align:left">low</td>
    </tr></table>
    <figcaption>NDVars: eeg</figcaption></figure>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 85-86

This could be used to retrieve those average responses:

.. GENERATED FROM PYTHON SOURCE LINES 86-89

.. code-block:: Python


    data_by_condition[0, 'eeg']





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    <NDVar 'eeg': 140 time, 65 sensor>



.. GENERATED FROM PYTHON SOURCE LINES 90-92

The grand average could be derived by aggregating without a model, resulting
in a single row:

.. GENERATED FROM PYTHON SOURCE LINES 92-95

.. code-block:: Python


    data.aggregate(drop_bad=True)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <figure><table border="1" frame="hsides" rules="none" cellpadding="2"><tr>
    <td style="text-align:left">#</td>
    <td style="text-align:left">n</td>
    </tr>
    <tr>
    <td style="text-align:left">0</td>
    <td style="text-align:left">20</td>
    </tr></table>
    <figcaption>NDVars: eeg</figcaption></figure>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 96-97

Many functions automatically average across cases ...

.. GENERATED FROM PYTHON SOURCE LINES 97-100

.. code-block:: Python


    p = plot.TopoButterfly('eeg', data=data)




.. image-sg:: /auto_examples/datasets/images/sphx_glr_group-level-analysis_001.png
   :alt: group level analysis
   :srcset: /auto_examples/datasets/images/sphx_glr_group-level-analysis_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 101-102

... and directly accept a parameter for averaging by condition:

.. GENERATED FROM PYTHON SOURCE LINES 102-105

.. code-block:: Python


    p = plot.TopoButterfly('eeg', 'predictability', data=data)




.. image-sg:: /auto_examples/datasets/images/sphx_glr_group-level-analysis_002.png
   :alt: group level analysis
   :srcset: /auto_examples/datasets/images/sphx_glr_group-level-analysis_002.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 106-107

Models can similarly be used to define conditions in statistical tests:

.. GENERATED FROM PYTHON SOURCE LINES 107-110

.. code-block:: Python


    result = testnd.TTestRelated('eeg', 'predictability', match='subject', data=data)
    result




.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    <TTestRelated 'eeg', 'predictability', 'high', 'low', 'subject' (n=10), samples=1023, p = .009>




.. _sphx_glr_download_auto_examples_datasets_group-level-analysis.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: group-level-analysis.ipynb <group-level-analysis.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: group-level-analysis.py <group-level-analysis.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: group-level-analysis.zip <group-level-analysis.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_