EEG speech envelope TRF

Analyze continuous speech data from the mTRF dataset 1: use the boosting algorithm for estimating temporal response functions (TRFs) to the acoustic envelope.

# Author: Christian Brodbeck <christianbrodbeck@nyu.edu>
# sphinx_gallery_thumbnail_number = 4
import os

from scipy.io import loadmat
import mne
from eelbrain import *

# Load the mTRF speech dataset and convert data to NDVars
root = mne.datasets.mtrf.data_path()
speech_path = os.path.join(root, 'speech_data.mat')
mdata = loadmat(speech_path)

# Time axis
tstep = 1. / mdata['Fs'][0, 0]
n_times = mdata['envelope'].shape[0]
time = UTS(0, tstep, n_times)
# Load the EEG sensor coordinates (drop fiducials coordinates, which are stored
# after sensor 128)
sensor = Sensor.from_montage('biosemi128')[:128]
# Frequency dimension for the spectrogram
band = Scalar('frequency', range(16))
# Create variables
envelope = NDVar(mdata['envelope'][:, 0], (time,), name='envelope')
eeg = NDVar(mdata['EEG'], (time, sensor), name='EEG', info={'unit': 'µV'})
spectrogram = NDVar(mdata['spectrogram'], (time, band), name='spectrogram')
# Exclude a bad channel
eeg = eeg[sensor.index(exclude='A13')]

Data

Plot the spectrogram of the speech stimulus:

plot.Array(spectrogram, xlim=5, w=6, h=2)

Out:

<Array: spectrogram>

Plot the envelope used as stimulus representation for deconvolution:

plot.UTS(envelope, xlim=5, w=6, h=2)

Out:

<UTS: envelope>

Plot the corresponding EEG data:

p = plot.TopoButterfly(eeg, xlim=5, w=7, h=2)
p.set_time(1.200)

Deconvolution

TRF for the envelope using boosting:

TRF from -100 to 400 ms
Basis of 100 ms Hamming windows
Use 4 partitionings of the data for cross-validation based early stopping

res = boosting(eeg, envelope, -0.100, 0.400, basis=0.100, partitions=4)
p = plot.TopoButterfly(res.h_scaled, w=6, h=2)
p.set_time(.180)

Out:

Fitting models:   0%|          | 0/508 [00:00<?, ?it/s]
Fitting models:   0%|          | 1/508 [00:00<01:45,  4.79it/s]
Fitting models:   2%|2         | 11/508 [00:00<00:12, 41.25it/s]
Fitting models:   4%|3         | 18/508 [00:00<00:09, 51.10it/s]
Fitting models:   5%|5         | 27/508 [00:00<00:07, 63.23it/s]
Fitting models:   7%|6         | 35/508 [00:00<00:07, 65.62it/s]
Fitting models:   8%|8         | 43/508 [00:00<00:06, 68.45it/s]
Fitting models:  10%|#         | 52/508 [00:00<00:06, 72.40it/s]
Fitting models:  12%|#1        | 60/508 [00:00<00:06, 70.43it/s]
Fitting models:  13%|#3        | 68/508 [00:01<00:06, 68.53it/s]
Fitting models:  15%|#4        | 75/508 [00:01<00:06, 65.87it/s]
Fitting models:  16%|#6        | 82/508 [00:01<00:06, 66.76it/s]
Fitting models:  18%|#7        | 90/508 [00:01<00:06, 69.48it/s]
Fitting models:  19%|#9        | 99/508 [00:01<00:05, 74.47it/s]
Fitting models:  21%|##1       | 107/508 [00:01<00:05, 72.77it/s]
Fitting models:  23%|##2       | 115/508 [00:01<00:05, 66.87it/s]
Fitting models:  24%|##4       | 122/508 [00:01<00:05, 66.19it/s]
Fitting models:  26%|##5       | 130/508 [00:02<00:05, 68.18it/s]
Fitting models:  27%|##7       | 139/508 [00:02<00:05, 70.03it/s]
Fitting models:  29%|##8       | 147/508 [00:02<00:05, 67.62it/s]
Fitting models:  31%|###       | 155/508 [00:02<00:05, 69.01it/s]
Fitting models:  32%|###2      | 164/508 [00:02<00:04, 74.02it/s]
Fitting models:  34%|###4      | 173/508 [00:02<00:04, 75.12it/s]
Fitting models:  36%|###6      | 183/508 [00:02<00:04, 78.02it/s]
Fitting models:  38%|###7      | 193/508 [00:02<00:03, 79.82it/s]
Fitting models:  40%|###9      | 202/508 [00:02<00:03, 81.51it/s]
Fitting models:  42%|####1     | 211/508 [00:03<00:03, 79.52it/s]
Fitting models:  44%|####3     | 221/508 [00:03<00:03, 81.42it/s]
Fitting models:  45%|####5     | 230/508 [00:03<00:03, 79.84it/s]
Fitting models:  47%|####6     | 238/508 [00:03<00:03, 77.70it/s]
Fitting models:  48%|####8     | 246/508 [00:03<00:03, 76.02it/s]
Fitting models:  50%|#####     | 255/508 [00:03<00:03, 79.13it/s]
Fitting models:  52%|#####1    | 263/508 [00:03<00:03, 75.89it/s]
Fitting models:  53%|#####3    | 271/508 [00:03<00:03, 76.04it/s]
Fitting models:  55%|#####4    | 279/508 [00:03<00:03, 75.27it/s]
Fitting models:  56%|#####6    | 287/508 [00:04<00:02, 74.64it/s]
Fitting models:  58%|#####8    | 295/508 [00:04<00:02, 75.83it/s]
Fitting models:  60%|#####9    | 303/508 [00:04<00:02, 72.41it/s]
Fitting models:  61%|######1   | 311/508 [00:04<00:02, 70.16it/s]
Fitting models:  63%|######2   | 319/508 [00:04<00:02, 65.85it/s]
Fitting models:  64%|######4   | 326/508 [00:04<00:02, 64.74it/s]
Fitting models:  66%|######5   | 333/508 [00:04<00:02, 61.26it/s]
Fitting models:  67%|######7   | 342/508 [00:04<00:02, 66.42it/s]
Fitting models:  69%|######8   | 350/508 [00:04<00:02, 69.74it/s]
Fitting models:  70%|#######   | 358/508 [00:05<00:02, 67.54it/s]
Fitting models:  72%|#######1  | 365/508 [00:05<00:02, 67.25it/s]
Fitting models:  73%|#######3  | 372/508 [00:05<00:02, 64.78it/s]
Fitting models:  75%|#######4  | 379/508 [00:05<00:02, 57.01it/s]
Fitting models:  76%|#######6  | 388/508 [00:05<00:01, 62.63it/s]
Fitting models:  78%|#######7  | 396/508 [00:05<00:01, 65.90it/s]
Fitting models:  79%|#######9  | 403/508 [00:05<00:01, 60.86it/s]
Fitting models:  81%|########  | 410/508 [00:05<00:01, 59.50it/s]
Fitting models:  82%|########2 | 417/508 [00:06<00:01, 55.88it/s]
Fitting models:  83%|########3 | 423/508 [00:06<00:01, 54.70it/s]
Fitting models:  85%|########4 | 430/508 [00:06<00:01, 56.02it/s]
Fitting models:  86%|########5 | 436/508 [00:06<00:01, 56.75it/s]
Fitting models:  87%|########7 | 444/508 [00:06<00:01, 62.30it/s]
Fitting models:  89%|########8 | 451/508 [00:06<00:00, 63.47it/s]
Fitting models:  90%|######### | 458/508 [00:06<00:00, 62.04it/s]
Fitting models:  92%|#########1| 465/508 [00:06<00:00, 57.71it/s]
Fitting models:  93%|#########2| 471/508 [00:07<00:00, 54.68it/s]
Fitting models:  94%|#########4| 480/508 [00:07<00:00, 60.66it/s]
Fitting models:  96%|#########5| 487/508 [00:07<00:00, 60.21it/s]
Fitting models:  97%|#########7| 495/508 [00:07<00:00, 63.63it/s]
Fitting models:  99%|#########8| 502/508 [00:07<00:00, 63.50it/s]

Multiple predictors

Multiple predictors additively explain the signal:

# Derive acoustic onsets from the envelope
onset = envelope.diff('time', name='onset').clip(0)
onset *= envelope.max() / onset.max()
plot.UTS([[envelope, onset]], xlim=5, w=6, h=2)

Out:

<UTS: envelope, onset>

res_onset = boosting(eeg, [onset, envelope], -0.100, 0.400, basis=0.100, partitions=4)
p = plot.TopoButterfly(res_onset.h_scaled, w=6, h=3)
p.set_time(.150)

Out:

Fitting models:   0%|          | 0/508 [00:00<?, ?it/s]
Fitting models:   0%|          | 1/508 [00:00<01:53,  4.47it/s]
Fitting models:   1%|1         | 7/508 [00:00<00:20, 23.98it/s]
Fitting models:   3%|2         | 13/508 [00:00<00:15, 32.23it/s]
Fitting models:   4%|3         | 18/508 [00:00<00:13, 35.33it/s]
Fitting models:   5%|4         | 23/508 [00:00<00:13, 35.93it/s]
Fitting models:   6%|5         | 28/508 [00:00<00:12, 39.00it/s]
Fitting models:   6%|6         | 33/508 [00:00<00:12, 38.45it/s]
Fitting models:   8%|7         | 39/508 [00:01<00:11, 40.27it/s]
Fitting models:   9%|8         | 45/508 [00:01<00:11, 42.04it/s]
Fitting models:  10%|9         | 50/508 [00:01<00:10, 43.92it/s]
Fitting models:  11%|#         | 55/508 [00:01<00:10, 41.54it/s]
Fitting models:  12%|#1        | 60/508 [00:01<00:11, 39.74it/s]
Fitting models:  13%|#2        | 65/508 [00:01<00:11, 39.07it/s]
Fitting models:  14%|#3        | 69/508 [00:01<00:11, 39.27it/s]
Fitting models:  14%|#4        | 73/508 [00:01<00:11, 38.68it/s]
Fitting models:  15%|#5        | 77/508 [00:02<00:11, 38.79it/s]
Fitting models:  16%|#6        | 82/508 [00:02<00:10, 39.57it/s]
Fitting models:  17%|#6        | 86/508 [00:02<00:10, 39.29it/s]
Fitting models:  18%|#8        | 92/508 [00:02<00:10, 40.91it/s]
Fitting models:  19%|#9        | 97/508 [00:02<00:09, 41.67it/s]
Fitting models:  20%|##        | 102/508 [00:02<00:09, 40.73it/s]
Fitting models:  21%|##1       | 107/508 [00:02<00:09, 40.32it/s]
Fitting models:  22%|##2       | 112/508 [00:02<00:09, 40.59it/s]
Fitting models:  23%|##3       | 117/508 [00:03<00:10, 37.68it/s]
Fitting models:  24%|##4       | 123/508 [00:03<00:09, 39.16it/s]
Fitting models:  25%|##5       | 129/508 [00:03<00:09, 40.32it/s]
Fitting models:  26%|##6       | 134/508 [00:03<00:09, 41.33it/s]
Fitting models:  27%|##7       | 139/508 [00:03<00:09, 39.38it/s]
Fitting models:  28%|##8       | 143/508 [00:03<00:09, 37.04it/s]
Fitting models:  29%|##8       | 147/508 [00:03<00:09, 37.42it/s]
Fitting models:  30%|##9       | 151/508 [00:03<00:09, 36.11it/s]
Fitting models:  31%|###       | 157/508 [00:04<00:08, 39.13it/s]
Fitting models:  32%|###2      | 163/508 [00:04<00:08, 41.25it/s]
Fitting models:  33%|###3      | 168/508 [00:04<00:08, 42.47it/s]
Fitting models:  34%|###4      | 173/508 [00:04<00:08, 41.31it/s]
Fitting models:  35%|###5      | 178/508 [00:04<00:08, 41.19it/s]
Fitting models:  36%|###6      | 183/508 [00:04<00:07, 40.70it/s]
Fitting models:  37%|###7      | 189/508 [00:04<00:07, 41.62it/s]
Fitting models:  38%|###8      | 194/508 [00:04<00:07, 43.25it/s]
Fitting models:  39%|###9      | 199/508 [00:05<00:07, 39.01it/s]
Fitting models:  40%|###9      | 203/508 [00:05<00:07, 38.91it/s]
Fitting models:  41%|####      | 208/508 [00:05<00:07, 40.80it/s]
Fitting models:  42%|####1     | 213/508 [00:05<00:07, 40.08it/s]
Fitting models:  43%|####2     | 218/508 [00:05<00:06, 41.87it/s]
Fitting models:  44%|####3     | 223/508 [00:05<00:06, 42.16it/s]
Fitting models:  45%|####4     | 228/508 [00:05<00:06, 42.55it/s]
Fitting models:  46%|####5     | 233/508 [00:05<00:06, 42.73it/s]
Fitting models:  47%|####6     | 238/508 [00:06<00:06, 40.32it/s]
Fitting models:  48%|####7     | 243/508 [00:06<00:06, 42.08it/s]
Fitting models:  49%|####8     | 248/508 [00:06<00:06, 41.43it/s]
Fitting models:  50%|####9     | 253/508 [00:06<00:06, 40.97it/s]
Fitting models:  51%|#####     | 258/508 [00:06<00:06, 39.78it/s]
Fitting models:  52%|#####1    | 263/508 [00:06<00:05, 42.06it/s]
Fitting models:  53%|#####2    | 268/508 [00:06<00:05, 41.21it/s]
Fitting models:  54%|#####3    | 273/508 [00:06<00:05, 42.12it/s]
Fitting models:  55%|#####4    | 278/508 [00:07<00:05, 38.51it/s]
Fitting models:  56%|#####5    | 282/508 [00:07<00:05, 38.13it/s]
Fitting models:  56%|#####6    | 286/508 [00:07<00:05, 38.24it/s]
Fitting models:  57%|#####7    | 291/508 [00:07<00:05, 40.95it/s]
Fitting models:  58%|#####8    | 296/508 [00:07<00:05, 39.91it/s]
Fitting models:  59%|#####9    | 302/508 [00:07<00:04, 41.42it/s]
Fitting models:  60%|######    | 307/508 [00:07<00:04, 40.32it/s]
Fitting models:  62%|######1   | 313/508 [00:07<00:04, 41.58it/s]
Fitting models:  63%|######2   | 318/508 [00:07<00:04, 42.79it/s]
Fitting models:  64%|######3   | 323/508 [00:08<00:04, 37.01it/s]
Fitting models:  64%|######4   | 327/508 [00:08<00:05, 36.19it/s]
Fitting models:  65%|######5   | 331/508 [00:08<00:04, 35.89it/s]
Fitting models:  66%|######6   | 336/508 [00:08<00:04, 36.34it/s]
Fitting models:  67%|######6   | 340/508 [00:08<00:04, 36.33it/s]
Fitting models:  68%|######7   | 344/508 [00:08<00:04, 36.02it/s]
Fitting models:  69%|######8   | 348/508 [00:08<00:04, 34.87it/s]
Fitting models:  69%|######9   | 353/508 [00:09<00:04, 36.59it/s]
Fitting models:  70%|#######   | 358/508 [00:09<00:04, 36.82it/s]
Fitting models:  71%|#######1  | 363/508 [00:09<00:03, 39.38it/s]
Fitting models:  72%|#######2  | 367/508 [00:09<00:04, 33.98it/s]
Fitting models:  73%|#######3  | 372/508 [00:09<00:03, 37.84it/s]
Fitting models:  74%|#######4  | 376/508 [00:09<00:03, 35.47it/s]
Fitting models:  75%|#######4  | 380/508 [00:09<00:03, 33.63it/s]
Fitting models:  76%|#######5  | 385/508 [00:09<00:03, 33.11it/s]
Fitting models:  77%|#######6  | 389/508 [00:10<00:03, 33.85it/s]
Fitting models:  77%|#######7  | 393/508 [00:10<00:03, 30.99it/s]
Fitting models:  78%|#######8  | 397/508 [00:10<00:03, 29.55it/s]
Fitting models:  79%|#######8  | 401/508 [00:10<00:03, 27.55it/s]
Fitting models:  80%|#######9  | 404/508 [00:10<00:04, 25.16it/s]
Fitting models:  80%|########  | 407/508 [00:10<00:03, 25.39it/s]
Fitting models:  81%|########  | 411/508 [00:10<00:04, 24.03it/s]
Fitting models:  81%|########1 | 414/508 [00:11<00:03, 25.09it/s]
Fitting models:  82%|########2 | 418/508 [00:11<00:03, 26.33it/s]
Fitting models:  83%|########3 | 422/508 [00:11<00:03, 26.28it/s]
Fitting models:  84%|########3 | 425/508 [00:11<00:03, 26.70it/s]
Fitting models:  84%|########4 | 428/508 [00:11<00:03, 24.27it/s]
Fitting models:  85%|########5 | 432/508 [00:11<00:02, 27.44it/s]
Fitting models:  86%|########6 | 437/508 [00:11<00:02, 30.26it/s]
Fitting models:  87%|########7 | 442/508 [00:11<00:01, 34.93it/s]
Fitting models:  88%|########7 | 446/508 [00:12<00:01, 32.47it/s]
Fitting models:  89%|########8 | 451/508 [00:12<00:01, 33.22it/s]
Fitting models:  90%|########9 | 455/508 [00:12<00:01, 34.48it/s]
Fitting models:  90%|######### | 459/508 [00:12<00:01, 31.94it/s]
Fitting models:  91%|#########1| 463/508 [00:12<00:01, 33.61it/s]
Fitting models:  92%|#########1| 467/508 [00:12<00:01, 32.55it/s]
Fitting models:  93%|#########2| 471/508 [00:12<00:01, 30.43it/s]
Fitting models:  94%|#########3| 475/508 [00:13<00:01, 31.90it/s]
Fitting models:  94%|#########4| 479/508 [00:13<00:00, 33.84it/s]
Fitting models:  95%|#########5| 483/508 [00:13<00:00, 32.39it/s]
Fitting models:  96%|#########5| 487/508 [00:13<00:00, 31.87it/s]
Fitting models:  97%|#########6| 491/508 [00:13<00:00, 31.64it/s]
Fitting models:  98%|#########7| 496/508 [00:13<00:00, 34.81it/s]
Fitting models:  99%|#########8| 501/508 [00:13<00:00, 35.39it/s]
Fitting models:  99%|#########9| 505/508 [00:13<00:00, 34.13it/s]

Compare models

Compare model quality through the correlation between measured and predicted responses:

plot.Topomap([res.r, res_onset.r], w=4, h=2, ncol=2, axtitle=['envelope', 'envelope + onset'])

Out:

<Topomap: Correlation>

References

1: Crosse, M. J., Liberto, D., M, G., Bednar, A., & Lalor, E. C. (2016). The Multivariate Temporal Response Function (mTRF) Toolbox: A MATLAB Toolbox for Relating Neural Signals to Continuous Stimuli. Frontiers in Human Neuroscience, 10. https://doi.org/10.3389/fnhum.2016.00604

Total running time of the script: ( 0 minutes 26.366 seconds)

Gallery generated by Sphinx-Gallery