Multiple regression

Multiple regression designs apply to at least two scenarios.

• Analyzing a single subject’s data when trials are associated with one or multiple continuous variables.

• Analyzing group data when subjects are associated with one or multiple individual difference variables.

Here the analysis is illustrated for a simulated dataset from a single subject. Group analysis works analogously, except that each case in the datatset would represent a different subject rather than a different trial.

For such group analysis, it is necessary to reduce each subject’s data to a single case first because multiple regression assumes a fixed effects model. Such designs are described under Two-stage test.

Simulated data

# sphinx_gallery_thumbnail_number = 2
from eelbrain import *


data = datasets.simulate_erp(snr=0.5)
data.head()

cloze	predictability	n_chars
0.026139	low	4
0.99529	high	5
0.93062	high	3
0.82404	high	7
0.17041	low	4
0.18508	low	4
0.23345	low	4
0.80784	high	3
0.24979	low	7
0.26753	low	4

NDVars: eeg

The data represents 80 trials from a simulated word reading paradigm, where each word is associated with a word length (n_chars) and predictability (cloze).

p = plot.Scatter('cloze', 'n_chars', data=data, h=3)

Plot the average of each level on n_chars to illustrate the linear increase of the response around 130 ms with n_char.

p = plot.TopoButterfly('eeg', 'n_chars', data=data, t=.130, axh=2, w=6)

Multiple regression

Fit a multiple regression model. Estimate $p$-values using a cluster-based permuatation test, using a cluster-forming threshold of uncorrected $p$=.05 and 10,000 permutations (the default).

lm = testnd.LM('eeg', 'n_chars + cloze', data=data, tstart=0.050, tstop=0.500, pmin=0.05)

Permutation test:   0%|          | 0/10000 [00:00<?, ? permutations/s]
Permutation test:   0%|          | 9/10000 [00:00<01:51, 89.58 permutations/s]
Permutation test:   0%|          | 21/10000 [00:00<01:34, 105.38 permutations/s]
Permutation test:   0%|          | 34/10000 [00:00<01:27, 114.29 permutations/s]
Permutation test:   0%|          | 46/10000 [00:00<01:33, 106.08 permutations/s]
Permutation test:   1%|          | 57/10000 [00:00<01:37, 102.35 permutations/s]
Permutation test:   1%|          | 68/10000 [00:00<01:35, 103.46 permutations/s]
Permutation test:   1%|          | 79/10000 [00:00<01:35, 103.92 permutations/s]
Permutation test:   1%|          | 90/10000 [00:00<01:37, 102.10 permutations/s]
Permutation test:   1%|          | 101/10000 [00:00<01:37, 101.28 permutations/s]
Permutation test:   1%|          | 112/10000 [00:01<01:36, 102.29 permutations/s]
Permutation test:   1%|          | 123/10000 [00:01<01:35, 103.29 permutations/s]
Permutation test:   1%|▏         | 134/10000 [00:01<01:36, 102.76 permutations/s]
Permutation test:   1%|▏         | 145/10000 [00:01<01:34, 104.76 permutations/s]
Permutation test:   2%|▏         | 156/10000 [00:01<01:33, 105.23 permutations/s]
Permutation test:   2%|▏         | 167/10000 [00:01<01:34, 104.43 permutations/s]
Permutation test:   2%|▏         | 178/10000 [00:01<01:33, 105.52 permutations/s]
Permutation test:   2%|▏         | 189/10000 [00:01<01:34, 103.51 permutations/s]
Permutation test:   2%|▏         | 200/10000 [00:01<01:34, 103.22 permutations/s]
Permutation test:   2%|▏         | 212/10000 [00:02<01:33, 104.16 permutations/s]
Permutation test:   2%|▏         | 223/10000 [00:02<01:34, 103.44 permutations/s]
Permutation test:   2%|▏         | 234/10000 [00:02<01:36, 101.09 permutations/s]
Permutation test:   2%|▏         | 245/10000 [00:02<01:36, 101.61 permutations/s]
Permutation test:   3%|▎         | 259/10000 [00:02<01:29, 108.93 permutations/s]
Permutation test:   3%|▎         | 271/10000 [00:02<01:27, 111.15 permutations/s]
Permutation test:   3%|▎         | 283/10000 [00:02<01:26, 112.63 permutations/s]
Permutation test:   3%|▎         | 295/10000 [00:02<01:24, 114.42 permutations/s]
Permutation test:   3%|▎         | 308/10000 [00:02<01:22, 117.28 permutations/s]
Permutation test:   3%|▎         | 320/10000 [00:03<01:23, 115.34 permutations/s]
Permutation test:   3%|▎         | 333/10000 [00:03<01:23, 116.06 permutations/s]
Permutation test:   3%|▎         | 345/10000 [00:03<01:22, 117.07 permutations/s]
Permutation test:   4%|▎         | 358/10000 [00:03<01:20, 119.32 permutations/s]
Permutation test:   4%|▎         | 370/10000 [00:03<01:20, 118.93 permutations/s]
Permutation test:   4%|▍         | 382/10000 [00:03<01:21, 117.99 permutations/s]
Permutation test:   4%|▍         | 394/10000 [00:03<01:21, 117.60 permutations/s]
Permutation test:   4%|▍         | 406/10000 [00:03<01:21, 117.96 permutations/s]
Permutation test:   4%|▍         | 418/10000 [00:03<01:21, 117.58 permutations/s]
Permutation test:   4%|▍         | 430/10000 [00:03<01:21, 117.54 permutations/s]
Permutation test:   4%|▍         | 442/10000 [00:04<01:21, 116.70 permutations/s]
Permutation test:   5%|▍         | 454/10000 [00:04<01:22, 116.10 permutations/s]
Permutation test:   5%|▍         | 466/10000 [00:04<01:24, 112.87 permutations/s]
Permutation test:   5%|▍         | 478/10000 [00:04<01:29, 106.77 permutations/s]
Permutation test:   5%|▍         | 489/10000 [00:04<01:30, 105.17 permutations/s]
Permutation test:   5%|▌         | 500/10000 [00:04<01:30, 105.03 permutations/s]
Permutation test:   5%|▌         | 511/10000 [00:04<01:30, 105.32 permutations/s]
Permutation test:   5%|▌         | 522/10000 [00:04<01:30, 104.57 permutations/s]
Permutation test:   5%|▌         | 533/10000 [00:04<01:31, 103.70 permutations/s]
Permutation test:   5%|▌         | 544/10000 [00:05<01:32, 102.35 permutations/s]
Permutation test:   6%|▌         | 556/10000 [00:05<01:31, 103.58 permutations/s]
Permutation test:   6%|▌         | 567/10000 [00:05<01:29, 105.35 permutations/s]
Permutation test:   6%|▌         | 578/10000 [00:05<01:30, 104.61 permutations/s]
Permutation test:   6%|▌         | 589/10000 [00:05<01:30, 104.54 permutations/s]
Permutation test:   6%|▌         | 600/10000 [00:05<01:30, 103.69 permutations/s]
Permutation test:   6%|▌         | 611/10000 [00:05<01:30, 104.21 permutations/s]
Permutation test:   6%|▌         | 622/10000 [00:05<01:32, 101.52 permutations/s]
Permutation test:   6%|▋         | 633/10000 [00:05<01:30, 103.54 permutations/s]
Permutation test:   6%|▋         | 644/10000 [00:05<01:30, 103.13 permutations/s]
Permutation test:   7%|▋         | 655/10000 [00:06<01:30, 102.83 permutations/s]
Permutation test:   7%|▋         | 666/10000 [00:06<01:31, 101.93 permutations/s]
Permutation test:   7%|▋         | 677/10000 [00:06<01:29, 103.79 permutations/s]
Permutation test:   7%|▋         | 689/10000 [00:06<01:26, 107.94 permutations/s]
Permutation test:   7%|▋         | 702/10000 [00:06<01:23, 111.44 permutations/s]
Permutation test:   7%|▋         | 715/10000 [00:06<01:20, 115.70 permutations/s]
Permutation test:   7%|▋         | 727/10000 [00:06<01:19, 116.41 permutations/s]
Permutation test:   7%|▋         | 740/10000 [00:06<01:18, 117.74 permutations/s]
Permutation test:   8%|▊         | 753/10000 [00:06<01:16, 120.50 permutations/s]
Permutation test:   8%|▊         | 766/10000 [00:07<01:17, 119.55 permutations/s]
Permutation test:   8%|▊         | 779/10000 [00:07<01:15, 121.75 permutations/s]
Permutation test:   8%|▊         | 792/10000 [00:07<01:14, 123.31 permutations/s]
Permutation test:   8%|▊         | 805/10000 [00:07<01:14, 123.53 permutations/s]
Permutation test:   8%|▊         | 818/10000 [00:07<01:13, 124.14 permutations/s]
Permutation test:   8%|▊         | 831/10000 [00:07<01:14, 122.62 permutations/s]
Permutation test:   8%|▊         | 844/10000 [00:07<01:13, 124.45 permutations/s]
Permutation test:   9%|▊         | 857/10000 [00:07<01:13, 124.23 permutations/s]
Permutation test:   9%|▊         | 870/10000 [00:07<01:13, 124.00 permutations/s]
Permutation test:   9%|▉         | 883/10000 [00:07<01:13, 124.06 permutations/s]
Permutation test:   9%|▉         | 896/10000 [00:08<01:13, 123.35 permutations/s]
Permutation test:   9%|▉         | 909/10000 [00:08<01:15, 120.64 permutations/s]
Permutation test:   9%|▉         | 922/10000 [00:08<01:16, 118.99 permutations/s]
Permutation test:   9%|▉         | 935/10000 [00:08<01:15, 120.14 permutations/s]
Permutation test:   9%|▉         | 948/10000 [00:08<01:14, 122.14 permutations/s]
Permutation test:  10%|▉         | 961/10000 [00:08<01:13, 122.31 permutations/s]
Permutation test:  10%|▉         | 974/10000 [00:08<01:14, 121.41 permutations/s]
Permutation test:  10%|▉         | 987/10000 [00:08<01:13, 123.40 permutations/s]
Permutation test:  10%|█         | 1000/10000 [00:08<01:13, 123.16 permutations/s]
Permutation test:  10%|█         | 1013/10000 [00:09<01:13, 122.70 permutations/s]
Permutation test:  10%|█         | 1026/10000 [00:09<01:13, 122.27 permutations/s]
Permutation test:  10%|█         | 1039/10000 [00:09<01:12, 124.10 permutations/s]
Permutation test:  11%|█         | 1052/10000 [00:09<01:12, 123.60 permutations/s]
Permutation test:  11%|█         | 1065/10000 [00:09<01:11, 125.05 permutations/s]
Permutation test:  11%|█         | 1078/10000 [00:09<01:12, 122.42 permutations/s]
Permutation test:  11%|█         | 1091/10000 [00:09<01:12, 123.67 permutations/s]
Permutation test:  11%|█         | 1104/10000 [00:09<01:12, 123.32 permutations/s]
Permutation test:  11%|█         | 1117/10000 [00:09<01:12, 122.94 permutations/s]
Permutation test:  11%|█▏        | 1130/10000 [00:09<01:12, 122.69 permutations/s]
Permutation test:  11%|█▏        | 1143/10000 [00:10<01:11, 123.33 permutations/s]
Permutation test:  12%|█▏        | 1156/10000 [00:10<01:14, 119.08 permutations/s]
Permutation test:  12%|█▏        | 1168/10000 [00:10<01:14, 118.63 permutations/s]
Permutation test:  12%|█▏        | 1181/10000 [00:10<01:14, 119.05 permutations/s]
Permutation test:  12%|█▏        | 1194/10000 [00:10<01:12, 121.13 permutations/s]
Permutation test:  12%|█▏        | 1207/10000 [00:10<01:12, 121.01 permutations/s]
Permutation test:  12%|█▏        | 1220/10000 [00:10<01:12, 120.78 permutations/s]
Permutation test:  12%|█▏        | 1233/10000 [00:10<01:12, 121.40 permutations/s]
Permutation test:  12%|█▏        | 1246/10000 [00:10<01:11, 122.29 permutations/s]
Permutation test:  13%|█▎        | 1259/10000 [00:11<01:12, 120.91 permutations/s]
Permutation test:  13%|█▎        | 1272/10000 [00:11<01:11, 122.74 permutations/s]
Permutation test:  13%|█▎        | 1285/10000 [00:11<01:11, 122.60 permutations/s]
Permutation test:  13%|█▎        | 1298/10000 [00:11<01:10, 124.18 permutations/s]
Permutation test:  13%|█▎        | 1311/10000 [00:11<01:10, 123.96 permutations/s]
Permutation test:  13%|█▎        | 1324/10000 [00:11<01:11, 122.00 permutations/s]
Permutation test:  13%|█▎        | 1337/10000 [00:11<01:10, 122.12 permutations/s]
Permutation test:  14%|█▎        | 1350/10000 [00:11<01:10, 122.77 permutations/s]
Permutation test:  14%|█▎        | 1363/10000 [00:11<01:10, 122.91 permutations/s]
Permutation test:  14%|█▍        | 1376/10000 [00:12<01:10, 122.21 permutations/s]
Permutation test:  14%|█▍        | 1389/10000 [00:12<01:10, 122.83 permutations/s]
Permutation test:  14%|█▍        | 1402/10000 [00:12<01:14, 115.05 permutations/s]
Permutation test:  14%|█▍        | 1415/10000 [00:12<01:12, 119.12 permutations/s]
Permutation test:  14%|█▍        | 1428/10000 [00:12<01:11, 120.46 permutations/s]
Permutation test:  14%|█▍        | 1441/10000 [00:12<01:10, 121.03 permutations/s]
Permutation test:  15%|█▍        | 1454/10000 [00:12<01:09, 122.12 permutations/s]
Permutation test:  15%|█▍        | 1467/10000 [00:12<01:10, 120.42 permutations/s]
Permutation test:  15%|█▍        | 1480/10000 [00:12<01:09, 121.84 permutations/s]
Permutation test:  15%|█▍        | 1493/10000 [00:12<01:09, 122.47 permutations/s]
Permutation test:  15%|█▌        | 1506/10000 [00:13<01:09, 121.72 permutations/s]
Permutation test:  15%|█▌        | 1519/10000 [00:13<01:08, 123.51 permutations/s]
Permutation test:  15%|█▌        | 1532/10000 [00:13<01:07, 124.64 permutations/s]
Permutation test:  15%|█▌        | 1545/10000 [00:13<01:07, 124.66 permutations/s]
Permutation test:  16%|█▌        | 1558/10000 [00:13<01:07, 125.22 permutations/s]
Permutation test:  16%|█▌        | 1571/10000 [00:13<01:08, 123.01 permutations/s]
Permutation test:  16%|█▌        | 1584/10000 [00:13<01:07, 124.29 permutations/s]
Permutation test:  16%|█▌        | 1597/10000 [00:13<01:07, 123.98 permutations/s]
Permutation test:  16%|█▌        | 1610/10000 [00:13<01:07, 123.54 permutations/s]
Permutation test:  16%|█▌        | 1623/10000 [00:14<01:07, 123.80 permutations/s]
Permutation test:  16%|█▋        | 1636/10000 [00:14<01:08, 122.15 permutations/s]
Permutation test:  16%|█▋        | 1649/10000 [00:14<01:10, 119.16 permutations/s]
Permutation test:  17%|█▋        | 1662/10000 [00:14<01:08, 121.51 permutations/s]
Permutation test:  17%|█▋        | 1675/10000 [00:14<01:08, 121.82 permutations/s]
Permutation test:  17%|█▋        | 1688/10000 [00:14<01:07, 123.48 permutations/s]
Permutation test:  17%|█▋        | 1701/10000 [00:14<01:07, 122.96 permutations/s]
Permutation test:  17%|█▋        | 1714/10000 [00:14<01:08, 121.81 permutations/s]
Permutation test:  17%|█▋        | 1727/10000 [00:14<01:07, 122.67 permutations/s]
Permutation test:  17%|█▋        | 1740/10000 [00:14<01:07, 121.86 permutations/s]
Permutation test:  18%|█▊        | 1753/10000 [00:15<01:07, 122.07 permutations/s]
Permutation test:  18%|█▊        | 1766/10000 [00:15<01:06, 123.56 permutations/s]
Permutation test:  18%|█▊        | 1779/10000 [00:15<01:06, 124.32 permutations/s]
Permutation test:  18%|█▊        | 1792/10000 [00:15<01:05, 124.73 permutations/s]
Permutation test:  18%|█▊        | 1805/10000 [00:15<01:05, 125.62 permutations/s]
Permutation test:  18%|█▊        | 1818/10000 [00:15<01:06, 123.32 permutations/s]
Permutation test:  18%|█▊        | 1831/10000 [00:15<01:05, 124.72 permutations/s]
Permutation test:  18%|█▊        | 1844/10000 [00:15<01:06, 123.49 permutations/s]
Permutation test:  19%|█▊        | 1857/10000 [00:15<01:06, 123.29 permutations/s]
Permutation test:  19%|█▊        | 1870/10000 [00:16<01:06, 123.08 permutations/s]
Permutation test:  19%|█▉        | 1883/10000 [00:16<01:07, 121.14 permutations/s]
Permutation test:  19%|█▉        | 1896/10000 [00:16<01:08, 117.60 permutations/s]
Permutation test:  19%|█▉        | 1909/10000 [00:16<01:07, 119.37 permutations/s]
Permutation test:  19%|█▉        | 1922/10000 [00:16<01:07, 118.98 permutations/s]
Permutation test:  19%|█▉        | 1935/10000 [00:16<01:06, 121.56 permutations/s]
Permutation test:  19%|█▉        | 1948/10000 [00:16<01:05, 123.32 permutations/s]
Permutation test:  20%|█▉        | 1961/10000 [00:16<01:05, 122.07 permutations/s]
Permutation test:  20%|█▉        | 1974/10000 [00:16<01:04, 124.22 permutations/s]
Permutation test:  20%|█▉        | 1987/10000 [00:17<01:04, 124.71 permutations/s]
Permutation test:  20%|██        | 2000/10000 [00:17<01:04, 124.30 permutations/s]
Permutation test:  20%|██        | 2013/10000 [00:17<01:04, 124.34 permutations/s]
Permutation test:  20%|██        | 2026/10000 [00:17<01:04, 124.36 permutations/s]
Permutation test:  20%|██        | 2039/10000 [00:17<01:03, 125.75 permutations/s]
Permutation test:  21%|██        | 2052/10000 [00:17<01:02, 126.18 permutations/s]
Permutation test:  21%|██        | 2065/10000 [00:17<01:03, 124.52 permutations/s]
Permutation test:  21%|██        | 2078/10000 [00:17<01:03, 124.00 permutations/s]
Permutation test:  21%|██        | 2091/10000 [00:17<01:04, 122.95 permutations/s]
Permutation test:  21%|██        | 2104/10000 [00:17<01:04, 123.17 permutations/s]
Permutation test:  21%|██        | 2117/10000 [00:18<01:04, 121.76 permutations/s]
Permutation test:  21%|██▏       | 2130/10000 [00:18<01:04, 121.22 permutations/s]
Permutation test:  21%|██▏       | 2143/10000 [00:18<01:06, 117.82 permutations/s]
Permutation test:  22%|██▏       | 2156/10000 [00:18<01:05, 119.61 permutations/s]
Permutation test:  22%|██▏       | 2169/10000 [00:18<01:05, 120.07 permutations/s]
Permutation test:  22%|██▏       | 2182/10000 [00:18<01:03, 122.39 permutations/s]
Permutation test:  22%|██▏       | 2195/10000 [00:18<01:03, 121.96 permutations/s]
Permutation test:  22%|██▏       | 2208/10000 [00:18<01:04, 121.45 permutations/s]
Permutation test:  22%|██▏       | 2221/10000 [00:18<01:03, 122.79 permutations/s]
Permutation test:  22%|██▏       | 2234/10000 [00:19<01:03, 122.76 permutations/s]
Permutation test:  22%|██▏       | 2247/10000 [00:19<01:03, 122.41 permutations/s]
Permutation test:  23%|██▎       | 2260/10000 [00:19<01:02, 123.66 permutations/s]
Permutation test:  23%|██▎       | 2273/10000 [00:19<01:02, 123.88 permutations/s]
Permutation test:  23%|██▎       | 2286/10000 [00:19<01:02, 123.87 permutations/s]
Permutation test:  23%|██▎       | 2299/10000 [00:19<01:02, 124.17 permutations/s]
Permutation test:  23%|██▎       | 2312/10000 [00:19<01:03, 121.67 permutations/s]
Permutation test:  23%|██▎       | 2325/10000 [00:19<01:03, 121.80 permutations/s]
Permutation test:  23%|██▎       | 2338/10000 [00:19<01:03, 120.46 permutations/s]
Permutation test:  24%|██▎       | 2351/10000 [00:19<01:02, 121.46 permutations/s]
Permutation test:  24%|██▎       | 2364/10000 [00:20<01:02, 122.21 permutations/s]
Permutation test:  24%|██▍       | 2377/10000 [00:20<01:03, 120.08 permutations/s]
Permutation test:  24%|██▍       | 2390/10000 [00:20<01:04, 117.40 permutations/s]
Permutation test:  24%|██▍       | 2403/10000 [00:20<01:04, 118.25 permutations/s]
Permutation test:  24%|██▍       | 2415/10000 [00:20<01:03, 118.68 permutations/s]
Permutation test:  24%|██▍       | 2427/10000 [00:20<01:03, 118.35 permutations/s]
Permutation test:  24%|██▍       | 2440/10000 [00:20<01:04, 117.87 permutations/s]
Permutation test:  25%|██▍       | 2453/10000 [00:20<01:03, 119.38 permutations/s]
Permutation test:  25%|██▍       | 2466/10000 [00:20<01:02, 120.25 permutations/s]
Permutation test:  25%|██▍       | 2479/10000 [00:21<01:03, 118.59 permutations/s]
Permutation test:  25%|██▍       | 2492/10000 [00:21<01:02, 120.11 permutations/s]
Permutation test:  25%|██▌       | 2505/10000 [00:21<01:01, 120.99 permutations/s]
Permutation test:  25%|██▌       | 2518/10000 [00:21<01:01, 121.40 permutations/s]
Permutation test:  25%|██▌       | 2531/10000 [00:21<01:01, 121.86 permutations/s]
Permutation test:  25%|██▌       | 2544/10000 [00:21<01:02, 119.31 permutations/s]
Permutation test:  26%|██▌       | 2557/10000 [00:21<01:02, 119.85 permutations/s]
Permutation test:  26%|██▌       | 2569/10000 [00:21<01:02, 119.59 permutations/s]
Permutation test:  26%|██▌       | 2581/10000 [00:21<01:03, 117.58 permutations/s]
Permutation test:  26%|██▌       | 2594/10000 [00:22<01:02, 119.31 permutations/s]
Permutation test:  26%|██▌       | 2606/10000 [00:22<01:02, 119.02 permutations/s]
Permutation test:  26%|██▌       | 2618/10000 [00:22<01:04, 114.48 permutations/s]
Permutation test:  26%|██▋       | 2631/10000 [00:22<01:03, 116.81 permutations/s]
Permutation test:  26%|██▋       | 2643/10000 [00:22<01:03, 116.75 permutations/s]
Permutation test:  27%|██▋       | 2656/10000 [00:22<01:02, 116.86 permutations/s]
Permutation test:  27%|██▋       | 2668/10000 [00:22<01:02, 117.43 permutations/s]
Permutation test:  27%|██▋       | 2680/10000 [00:22<01:03, 115.70 permutations/s]
Permutation test:  27%|██▋       | 2692/10000 [00:22<01:02, 116.80 permutations/s]
Permutation test:  27%|██▋       | 2705/10000 [00:22<01:01, 118.37 permutations/s]
Permutation test:  27%|██▋       | 2718/10000 [00:23<01:01, 118.28 permutations/s]
Permutation test:  27%|██▋       | 2731/10000 [00:23<01:00, 120.48 permutations/s]
Permutation test:  27%|██▋       | 2744/10000 [00:23<01:00, 120.14 permutations/s]
Permutation test:  28%|██▊       | 2757/10000 [00:23<00:59, 120.88 permutations/s]
Permutation test:  28%|██▊       | 2770/10000 [00:23<01:00, 119.01 permutations/s]
Permutation test:  28%|██▊       | 2782/10000 [00:23<01:01, 116.65 permutations/s]
Permutation test:  28%|██▊       | 2794/10000 [00:23<01:01, 116.39 permutations/s]
Permutation test:  28%|██▊       | 2806/10000 [00:23<01:01, 117.24 permutations/s]
Permutation test:  28%|██▊       | 2818/10000 [00:23<01:02, 115.30 permutations/s]
Permutation test:  28%|██▊       | 2831/10000 [00:24<01:02, 115.05 permutations/s]
Permutation test:  28%|██▊       | 2843/10000 [00:24<01:01, 116.36 permutations/s]
Permutation test:  29%|██▊       | 2856/10000 [00:24<01:02, 114.94 permutations/s]
Permutation test:  29%|██▊       | 2868/10000 [00:24<01:05, 108.94 permutations/s]
Permutation test:  29%|██▉       | 2879/10000 [00:24<01:07, 105.69 permutations/s]
Permutation test:  29%|██▉       | 2890/10000 [00:24<01:07, 105.02 permutations/s]
Permutation test:  29%|██▉       | 2901/10000 [00:24<01:07, 104.47 permutations/s]
Permutation test:  29%|██▉       | 2912/10000 [00:24<01:09, 102.66 permutations/s]
Permutation test:  29%|██▉       | 2923/10000 [00:24<01:11, 99.24 permutations/s]
Permutation test:  29%|██▉       | 2934/10000 [00:25<01:11, 99.19 permutations/s]
Permutation test:  29%|██▉       | 2945/10000 [00:25<01:10, 100.76 permutations/s]
Permutation test:  30%|██▉       | 2956/10000 [00:25<01:11, 99.06 permutations/s]
Permutation test:  30%|██▉       | 2967/10000 [00:25<01:11, 98.73 permutations/s]
Permutation test:  30%|██▉       | 2978/10000 [00:25<01:09, 100.65 permutations/s]
Permutation test:  30%|██▉       | 2989/10000 [00:25<01:11, 97.74 permutations/s]
Permutation test:  30%|███       | 3000/10000 [00:25<01:10, 99.99 permutations/s]
Permutation test:  30%|███       | 3011/10000 [00:25<01:10, 99.20 permutations/s]
Permutation test:  30%|███       | 3021/10000 [00:25<01:11, 97.64 permutations/s]
Permutation test:  30%|███       | 3032/10000 [00:26<01:08, 101.06 permutations/s]
Permutation test:  30%|███       | 3043/10000 [00:26<01:11, 97.66 permutations/s]
Permutation test:  31%|███       | 3053/10000 [00:26<01:11, 96.86 permutations/s]
Permutation test:  31%|███       | 3065/10000 [00:26<01:07, 102.10 permutations/s]
Permutation test:  31%|███       | 3078/10000 [00:26<01:03, 108.71 permutations/s]
Permutation test:  31%|███       | 3091/10000 [00:26<01:01, 113.05 permutations/s]
Permutation test:  31%|███       | 3105/10000 [00:26<00:58, 118.54 permutations/s]
Permutation test:  31%|███       | 3117/10000 [00:26<00:57, 118.77 permutations/s]
Permutation test:  31%|███▏      | 3131/10000 [00:26<00:56, 122.52 permutations/s]
Permutation test:  31%|███▏      | 3144/10000 [00:26<00:55, 123.18 permutations/s]
Permutation test:  32%|███▏      | 3157/10000 [00:27<00:55, 123.83 permutations/s]
Permutation test:  32%|███▏      | 3170/10000 [00:27<00:55, 123.26 permutations/s]
Permutation test:  32%|███▏      | 3184/10000 [00:27<00:54, 125.78 permutations/s]
Permutation test:  32%|███▏      | 3198/10000 [00:27<00:53, 127.17 permutations/s]
Permutation test:  32%|███▏      | 3211/10000 [00:27<00:53, 127.94 permutations/s]
Permutation test:  32%|███▏      | 3224/10000 [00:27<00:54, 125.37 permutations/s]
Permutation test:  32%|███▏      | 3237/10000 [00:27<00:53, 126.24 permutations/s]
Permutation test:  32%|███▎      | 3250/10000 [00:27<00:53, 125.19 permutations/s]
Permutation test:  33%|███▎      | 3263/10000 [00:27<00:53, 126.25 permutations/s]
Permutation test:  33%|███▎      | 3276/10000 [00:28<00:52, 127.15 permutations/s]
Permutation test:  33%|███▎      | 3289/10000 [00:28<00:52, 126.82 permutations/s]
Permutation test:  33%|███▎      | 3302/10000 [00:28<00:55, 120.97 permutations/s]
Permutation test:  33%|███▎      | 3315/10000 [00:28<00:56, 117.67 permutations/s]
Permutation test:  33%|███▎      | 3327/10000 [00:28<00:59, 111.75 permutations/s]
Permutation test:  33%|███▎      | 3339/10000 [00:28<01:00, 110.63 permutations/s]
Permutation test:  34%|███▎      | 3351/10000 [00:28<01:03, 105.04 permutations/s]
Permutation test:  34%|███▎      | 3362/10000 [00:28<01:04, 103.41 permutations/s]
Permutation test:  34%|███▎      | 3373/10000 [00:28<01:04, 102.45 permutations/s]
Permutation test:  34%|███▍      | 3384/10000 [00:29<01:04, 102.17 permutations/s]
Permutation test:  34%|███▍      | 3395/10000 [00:29<01:04, 102.57 permutations/s]
Permutation test:  34%|███▍      | 3406/10000 [00:29<01:04, 101.83 permutations/s]
Permutation test:  34%|███▍      | 3417/10000 [00:29<01:03, 103.94 permutations/s]
Permutation test:  34%|███▍      | 3428/10000 [00:29<01:04, 102.17 permutations/s]
Permutation test:  34%|███▍      | 3439/10000 [00:29<01:05, 100.56 permutations/s]
Permutation test:  34%|███▍      | 3450/10000 [00:29<01:04, 101.05 permutations/s]
Permutation test:  35%|███▍      | 3461/10000 [00:29<01:06, 98.76 permutations/s]
Permutation test:  35%|███▍      | 3472/10000 [00:29<01:06, 97.88 permutations/s]
Permutation test:  35%|███▍      | 3483/10000 [00:30<01:05, 99.63 permutations/s]
Permutation test:  35%|███▍      | 3493/10000 [00:30<01:06, 97.86 permutations/s]
Permutation test:  35%|███▌      | 3503/10000 [00:30<01:06, 97.21 permutations/s]
Permutation test:  35%|███▌      | 3514/10000 [00:30<01:06, 97.82 permutations/s]
Permutation test:  35%|███▌      | 3525/10000 [00:30<01:06, 98.02 permutations/s]
Permutation test:  35%|███▌      | 3536/10000 [00:30<01:05, 99.14 permutations/s]
Permutation test:  35%|███▌      | 3546/10000 [00:30<01:05, 99.18 permutations/s]
Permutation test:  36%|███▌      | 3557/10000 [00:30<01:03, 101.18 permutations/s]
Permutation test:  36%|███▌      | 3568/10000 [00:30<01:04, 99.78 permutations/s]
Permutation test:  36%|███▌      | 3578/10000 [00:31<01:04, 99.24 permutations/s]
Permutation test:  36%|███▌      | 3588/10000 [00:31<01:05, 98.33 permutations/s]
Permutation test:  36%|███▌      | 3599/10000 [00:31<01:03, 100.34 permutations/s]
Permutation test:  36%|███▌      | 3610/10000 [00:31<01:03, 100.12 permutations/s]
Permutation test:  36%|███▌      | 3621/10000 [00:31<01:04, 99.34 permutations/s]
Permutation test:  36%|███▋      | 3632/10000 [00:31<01:03, 100.19 permutations/s]
Permutation test:  36%|███▋      | 3643/10000 [00:31<01:03, 100.46 permutations/s]
Permutation test:  37%|███▋      | 3654/10000 [00:31<01:03, 99.82 permutations/s]
Permutation test:  37%|███▋      | 3665/10000 [00:31<01:02, 101.27 permutations/s]
Permutation test:  37%|███▋      | 3676/10000 [00:31<01:03, 98.97 permutations/s]
Permutation test:  37%|███▋      | 3686/10000 [00:32<01:03, 99.06 permutations/s]
Permutation test:  37%|███▋      | 3697/10000 [00:32<01:03, 99.54 permutations/s]
Permutation test:  37%|███▋      | 3708/10000 [00:32<01:01, 101.70 permutations/s]
Permutation test:  37%|███▋      | 3719/10000 [00:32<01:02, 99.79 permutations/s]
Permutation test:  37%|███▋      | 3730/10000 [00:32<01:02, 100.92 permutations/s]
Permutation test:  37%|███▋      | 3741/10000 [00:32<01:01, 102.45 permutations/s]
Permutation test:  38%|███▊      | 3752/10000 [00:32<01:01, 101.00 permutations/s]
Permutation test:  38%|███▊      | 3763/10000 [00:32<01:01, 102.04 permutations/s]
Permutation test:  38%|███▊      | 3774/10000 [00:32<01:02, 99.03 permutations/s]
Permutation test:  38%|███▊      | 3784/10000 [00:33<01:03, 98.40 permutations/s]
Permutation test:  38%|███▊      | 3794/10000 [00:33<01:03, 98.44 permutations/s]
Permutation test:  38%|███▊      | 3805/10000 [00:33<01:02, 98.63 permutations/s]
Permutation test:  38%|███▊      | 3816/10000 [00:33<01:01, 100.60 permutations/s]
Permutation test:  38%|███▊      | 3827/10000 [00:33<01:00, 102.12 permutations/s]
Permutation test:  38%|███▊      | 3838/10000 [00:33<01:00, 101.47 permutations/s]
Permutation test:  38%|███▊      | 3849/10000 [00:33<01:01, 100.39 permutations/s]
Permutation test:  39%|███▊      | 3860/10000 [00:33<00:59, 102.59 permutations/s]
Permutation test:  39%|███▊      | 3871/10000 [00:33<01:01, 100.17 permutations/s]
Permutation test:  39%|███▉      | 3882/10000 [00:34<01:00, 101.88 permutations/s]
Permutation test:  39%|███▉      | 3893/10000 [00:34<01:00, 101.42 permutations/s]
Permutation test:  39%|███▉      | 3904/10000 [00:34<01:01, 99.43 permutations/s]
Permutation test:  39%|███▉      | 3914/10000 [00:34<01:02, 97.70 permutations/s]
Permutation test:  39%|███▉      | 3925/10000 [00:34<01:01, 98.93 permutations/s]
Permutation test:  39%|███▉      | 3936/10000 [00:34<01:01, 98.75 permutations/s]
Permutation test:  39%|███▉      | 3946/10000 [00:34<01:01, 99.08 permutations/s]
Permutation test:  40%|███▉      | 3957/10000 [00:34<01:00, 99.73 permutations/s]
Permutation test:  40%|███▉      | 3967/10000 [00:34<01:00, 99.30 permutations/s]
Permutation test:  40%|███▉      | 3978/10000 [00:35<01:00, 99.63 permutations/s]
Permutation test:  40%|███▉      | 3988/10000 [00:35<01:02, 96.40 permutations/s]
Permutation test:  40%|███▉      | 3999/10000 [00:35<00:59, 100.13 permutations/s]
Permutation test:  40%|████      | 4010/10000 [00:35<00:59, 100.79 permutations/s]
Permutation test:  40%|████      | 4021/10000 [00:35<01:00, 99.46 permutations/s]
Permutation test:  40%|████      | 4032/10000 [00:35<00:59, 100.78 permutations/s]
Permutation test:  40%|████      | 4043/10000 [00:35<00:59, 99.84 permutations/s]
Permutation test:  41%|████      | 4054/10000 [00:35<00:59, 100.51 permutations/s]
Permutation test:  41%|████      | 4065/10000 [00:35<01:00, 98.64 permutations/s]
Permutation test:  41%|████      | 4076/10000 [00:35<00:58, 100.76 permutations/s]
Permutation test:  41%|████      | 4087/10000 [00:36<01:00, 98.40 permutations/s]
Permutation test:  41%|████      | 4097/10000 [00:36<01:00, 97.51 permutations/s]
Permutation test:  41%|████      | 4107/10000 [00:36<01:01, 96.56 permutations/s]
Permutation test:  41%|████      | 4118/10000 [00:36<00:59, 99.22 permutations/s]
Permutation test:  41%|████▏     | 4128/10000 [00:36<00:59, 98.74 permutations/s]
Permutation test:  41%|████▏     | 4138/10000 [00:36<00:59, 98.52 permutations/s]
Permutation test:  41%|████▏     | 4149/10000 [00:36<00:59, 97.68 permutations/s]
Permutation test:  42%|████▏     | 4160/10000 [00:36<01:00, 96.91 permutations/s]
Permutation test:  42%|████▏     | 4171/10000 [00:36<00:58, 100.26 permutations/s]
Permutation test:  42%|████▏     | 4182/10000 [00:37<00:59, 97.77 permutations/s]
Permutation test:  42%|████▏     | 4192/10000 [00:37<00:59, 98.35 permutations/s]
Permutation test:  42%|████▏     | 4203/10000 [00:37<00:57, 100.65 permutations/s]
Permutation test:  42%|████▏     | 4214/10000 [00:37<00:56, 101.68 permutations/s]
Permutation test:  42%|████▏     | 4225/10000 [00:37<00:57, 100.94 permutations/s]
Permutation test:  42%|████▏     | 4236/10000 [00:37<00:57, 99.81 permutations/s]
Permutation test:  42%|████▏     | 4246/10000 [00:37<00:58, 97.98 permutations/s]
Permutation test:  43%|████▎     | 4257/10000 [00:37<00:57, 100.14 permutations/s]
Permutation test:  43%|████▎     | 4268/10000 [00:37<00:58, 98.62 permutations/s]
Permutation test:  43%|████▎     | 4278/10000 [00:38<00:57, 98.93 permutations/s]
Permutation test:  43%|████▎     | 4288/10000 [00:38<00:57, 99.05 permutations/s]
Permutation test:  43%|████▎     | 4299/10000 [00:38<00:57, 99.91 permutations/s]
Permutation test:  43%|████▎     | 4310/10000 [00:38<00:55, 101.81 permutations/s]
Permutation test:  43%|████▎     | 4324/10000 [00:38<00:51, 110.95 permutations/s]
Permutation test:  43%|████▎     | 4337/10000 [00:38<00:49, 114.68 permutations/s]
Permutation test:  43%|████▎     | 4349/10000 [00:38<00:48, 115.77 permutations/s]
Permutation test:  44%|████▎     | 4363/10000 [00:38<00:46, 120.85 permutations/s]
Permutation test:  44%|████▍     | 4376/10000 [00:38<00:46, 120.65 permutations/s]
Permutation test:  44%|████▍     | 4389/10000 [00:38<00:46, 120.15 permutations/s]
Permutation test:  44%|████▍     | 4402/10000 [00:39<00:46, 120.54 permutations/s]
Permutation test:  44%|████▍     | 4415/10000 [00:39<00:46, 121.19 permutations/s]
Permutation test:  44%|████▍     | 4428/10000 [00:39<00:45, 122.28 permutations/s]
Permutation test:  44%|████▍     | 4441/10000 [00:39<00:44, 123.54 permutations/s]
Permutation test:  45%|████▍     | 4454/10000 [00:39<00:44, 123.94 permutations/s]
Permutation test:  45%|████▍     | 4467/10000 [00:39<00:45, 121.56 permutations/s]
Permutation test:  45%|████▍     | 4480/10000 [00:39<00:45, 122.24 permutations/s]
Permutation test:  45%|████▍     | 4493/10000 [00:39<00:45, 121.75 permutations/s]
Permutation test:  45%|████▌     | 4506/10000 [00:39<00:45, 121.53 permutations/s]
Permutation test:  45%|████▌     | 4519/10000 [00:40<00:44, 121.92 permutations/s]
Permutation test:  45%|████▌     | 4532/10000 [00:40<00:44, 121.65 permutations/s]
Permutation test:  45%|████▌     | 4545/10000 [00:40<00:46, 117.74 permutations/s]
Permutation test:  46%|████▌     | 4558/10000 [00:40<00:45, 120.46 permutations/s]
Permutation test:  46%|████▌     | 4571/10000 [00:40<00:44, 120.91 permutations/s]
Permutation test:  46%|████▌     | 4584/10000 [00:40<00:44, 122.06 permutations/s]
Permutation test:  46%|████▌     | 4597/10000 [00:40<00:44, 122.63 permutations/s]
Permutation test:  46%|████▌     | 4610/10000 [00:40<00:44, 120.25 permutations/s]
Permutation test:  46%|████▌     | 4623/10000 [00:40<00:44, 122.04 permutations/s]
Permutation test:  46%|████▋     | 4636/10000 [00:41<00:44, 121.83 permutations/s]
Permutation test:  46%|████▋     | 4649/10000 [00:41<00:43, 121.94 permutations/s]
Permutation test:  47%|████▋     | 4662/10000 [00:41<00:43, 122.19 permutations/s]
Permutation test:  47%|████▋     | 4675/10000 [00:41<00:43, 122.84 permutations/s]
Permutation test:  47%|████▋     | 4688/10000 [00:41<00:43, 123.09 permutations/s]
Permutation test:  47%|████▋     | 4701/10000 [00:41<00:42, 123.73 permutations/s]
Permutation test:  47%|████▋     | 4714/10000 [00:41<00:43, 121.46 permutations/s]
Permutation test:  47%|████▋     | 4727/10000 [00:41<00:42, 122.63 permutations/s]
Permutation test:  47%|████▋     | 4740/10000 [00:41<00:43, 120.94 permutations/s]
Permutation test:  48%|████▊     | 4753/10000 [00:41<00:43, 121.74 permutations/s]
Permutation test:  48%|████▊     | 4766/10000 [00:42<00:42, 122.46 permutations/s]
Permutation test:  48%|████▊     | 4779/10000 [00:42<00:43, 120.19 permutations/s]
Permutation test:  48%|████▊     | 4792/10000 [00:42<00:44, 117.60 permutations/s]
Permutation test:  48%|████▊     | 4805/10000 [00:42<00:44, 117.24 permutations/s]
Permutation test:  48%|████▊     | 4818/10000 [00:42<00:43, 120.34 permutations/s]
Permutation test:  48%|████▊     | 4831/10000 [00:42<00:42, 121.28 permutations/s]
Permutation test:  48%|████▊     | 4844/10000 [00:42<00:42, 122.34 permutations/s]
Permutation test:  49%|████▊     | 4857/10000 [00:42<00:42, 119.94 permutations/s]
Permutation test:  49%|████▊     | 4870/10000 [00:42<00:42, 119.86 permutations/s]
Permutation test:  49%|████▉     | 4883/10000 [00:43<00:42, 119.96 permutations/s]
Permutation test:  49%|████▉     | 4896/10000 [00:43<00:42, 119.07 permutations/s]
Permutation test:  49%|████▉     | 4909/10000 [00:43<00:42, 119.78 permutations/s]
Permutation test:  49%|████▉     | 4922/10000 [00:43<00:42, 120.48 permutations/s]
Permutation test:  49%|████▉     | 4935/10000 [00:43<00:41, 120.90 permutations/s]
Permutation test:  49%|████▉     | 4948/10000 [00:43<00:42, 119.24 permutations/s]
Permutation test:  50%|████▉     | 4961/10000 [00:43<00:41, 120.61 permutations/s]
Permutation test:  50%|████▉     | 4974/10000 [00:43<00:41, 121.84 permutations/s]
Permutation test:  50%|████▉     | 4987/10000 [00:43<00:41, 120.93 permutations/s]
Permutation test:  50%|█████     | 5000/10000 [00:44<00:41, 121.02 permutations/s]
Permutation test:  50%|█████     | 5013/10000 [00:44<00:41, 121.63 permutations/s]
Permutation test:  50%|█████     | 5026/10000 [00:44<00:42, 117.94 permutations/s]
Permutation test:  50%|█████     | 5038/10000 [00:44<00:42, 117.60 permutations/s]
Permutation test:  51%|█████     | 5051/10000 [00:44<00:41, 118.45 permutations/s]
Permutation test:  51%|█████     | 5064/10000 [00:44<00:41, 119.53 permutations/s]
Permutation test:  51%|█████     | 5077/10000 [00:44<00:41, 120.07 permutations/s]
Permutation test:  51%|█████     | 5090/10000 [00:44<00:41, 118.27 permutations/s]
Permutation test:  51%|█████     | 5103/10000 [00:44<00:40, 120.79 permutations/s]
Permutation test:  51%|█████     | 5116/10000 [00:45<00:40, 120.00 permutations/s]
Permutation test:  51%|█████▏    | 5129/10000 [00:45<00:40, 121.54 permutations/s]
Permutation test:  51%|█████▏    | 5142/10000 [00:45<00:39, 123.12 permutations/s]
Permutation test:  52%|█████▏    | 5155/10000 [00:45<00:39, 122.84 permutations/s]
Permutation test:  52%|█████▏    | 5168/10000 [00:45<00:39, 123.42 permutations/s]
Permutation test:  52%|█████▏    | 5181/10000 [00:45<00:38, 123.93 permutations/s]
Permutation test:  52%|█████▏    | 5194/10000 [00:45<00:39, 122.08 permutations/s]
Permutation test:  52%|█████▏    | 5207/10000 [00:45<00:39, 122.58 permutations/s]
Permutation test:  52%|█████▏    | 5220/10000 [00:45<00:39, 119.55 permutations/s]
Permutation test:  52%|█████▏    | 5233/10000 [00:45<00:39, 119.87 permutations/s]
Permutation test:  52%|█████▏    | 5246/10000 [00:46<00:39, 120.13 permutations/s]
Permutation test:  53%|█████▎    | 5259/10000 [00:46<00:39, 121.20 permutations/s]
Permutation test:  53%|█████▎    | 5272/10000 [00:46<00:40, 116.49 permutations/s]
Permutation test:  53%|█████▎    | 5285/10000 [00:46<00:39, 118.65 permutations/s]
Permutation test:  53%|█████▎    | 5298/10000 [00:46<00:39, 119.81 permutations/s]
Permutation test:  53%|█████▎    | 5311/10000 [00:46<00:39, 119.37 permutations/s]
Permutation test:  53%|█████▎    | 5323/10000 [00:46<00:39, 119.17 permutations/s]
Permutation test:  53%|█████▎    | 5335/10000 [00:46<00:39, 117.39 permutations/s]
Permutation test:  53%|█████▎    | 5347/10000 [00:46<00:39, 118.00 permutations/s]
Permutation test:  54%|█████▎    | 5359/10000 [00:47<00:39, 118.09 permutations/s]
Permutation test:  54%|█████▎    | 5371/10000 [00:47<00:39, 118.13 permutations/s]
Permutation test:  54%|█████▍    | 5383/10000 [00:47<00:39, 117.72 permutations/s]
Permutation test:  54%|█████▍    | 5396/10000 [00:47<00:38, 119.87 permutations/s]
Permutation test:  54%|█████▍    | 5408/10000 [00:47<00:38, 119.67 permutations/s]
Permutation test:  54%|█████▍    | 5421/10000 [00:47<00:38, 119.02 permutations/s]
Permutation test:  54%|█████▍    | 5433/10000 [00:47<00:38, 118.07 permutations/s]
Permutation test:  54%|█████▍    | 5446/10000 [00:47<00:38, 119.72 permutations/s]
Permutation test:  55%|█████▍    | 5459/10000 [00:47<00:38, 119.13 permutations/s]
Permutation test:  55%|█████▍    | 5471/10000 [00:47<00:38, 118.85 permutations/s]
Permutation test:  55%|█████▍    | 5483/10000 [00:48<00:38, 118.15 permutations/s]
Permutation test:  55%|█████▍    | 5495/10000 [00:48<00:38, 117.82 permutations/s]
Permutation test:  55%|█████▌    | 5507/10000 [00:48<00:38, 115.93 permutations/s]
Permutation test:  55%|█████▌    | 5519/10000 [00:48<00:38, 115.66 permutations/s]
Permutation test:  55%|█████▌    | 5531/10000 [00:48<00:38, 116.69 permutations/s]
Permutation test:  55%|█████▌    | 5544/10000 [00:48<00:37, 118.26 permutations/s]
Permutation test:  56%|█████▌    | 5557/10000 [00:48<00:37, 119.98 permutations/s]
Permutation test:  56%|█████▌    | 5569/10000 [00:48<00:37, 118.87 permutations/s]
Permutation test:  56%|█████▌    | 5582/10000 [00:48<00:36, 119.88 permutations/s]
Permutation test:  56%|█████▌    | 5594/10000 [00:49<00:36, 119.76 permutations/s]
Permutation test:  56%|█████▌    | 5607/10000 [00:49<00:36, 120.15 permutations/s]
Permutation test:  56%|█████▌    | 5620/10000 [00:49<00:36, 120.48 permutations/s]
Permutation test:  56%|█████▋    | 5633/10000 [00:49<00:35, 122.69 permutations/s]
Permutation test:  56%|█████▋    | 5646/10000 [00:49<00:35, 123.84 permutations/s]
Permutation test:  57%|█████▋    | 5659/10000 [00:49<00:35, 123.96 permutations/s]
Permutation test:  57%|█████▋    | 5672/10000 [00:49<00:35, 122.03 permutations/s]
Permutation test:  57%|█████▋    | 5685/10000 [00:49<00:35, 122.49 permutations/s]
Permutation test:  57%|█████▋    | 5698/10000 [00:49<00:35, 121.94 permutations/s]
Permutation test:  57%|█████▋    | 5711/10000 [00:49<00:35, 122.02 permutations/s]
Permutation test:  57%|█████▋    | 5724/10000 [00:50<00:35, 121.96 permutations/s]
Permutation test:  57%|█████▋    | 5737/10000 [00:50<00:34, 121.98 permutations/s]
Permutation test:  57%|█████▊    | 5750/10000 [00:50<00:36, 116.78 permutations/s]
Permutation test:  58%|█████▊    | 5763/10000 [00:50<00:35, 118.89 permutations/s]
Permutation test:  58%|█████▊    | 5776/10000 [00:50<00:35, 120.19 permutations/s]
Permutation test:  58%|█████▊    | 5789/10000 [00:50<00:34, 120.39 permutations/s]
Permutation test:  58%|█████▊    | 5802/10000 [00:50<00:34, 121.25 permutations/s]
Permutation test:  58%|█████▊    | 5815/10000 [00:50<00:34, 120.11 permutations/s]
Permutation test:  58%|█████▊    | 5828/10000 [00:50<00:34, 122.02 permutations/s]
Permutation test:  58%|█████▊    | 5841/10000 [00:51<00:33, 122.52 permutations/s]
Permutation test:  59%|█████▊    | 5854/10000 [00:51<00:33, 122.94 permutations/s]
Permutation test:  59%|█████▊    | 5867/10000 [00:51<00:33, 123.28 permutations/s]
Permutation test:  59%|█████▉    | 5880/10000 [00:51<00:33, 122.89 permutations/s]
Permutation test:  59%|█████▉    | 5893/10000 [00:51<00:33, 123.53 permutations/s]
Permutation test:  59%|█████▉    | 5906/10000 [00:51<00:33, 123.52 permutations/s]
Permutation test:  59%|█████▉    | 5919/10000 [00:51<00:33, 121.11 permutations/s]
Permutation test:  59%|█████▉    | 5932/10000 [00:51<00:33, 122.08 permutations/s]
Permutation test:  59%|█████▉    | 5945/10000 [00:51<00:33, 120.24 permutations/s]
Permutation test:  60%|█████▉    | 5958/10000 [00:51<00:33, 121.09 permutations/s]
Permutation test:  60%|█████▉    | 5971/10000 [00:52<00:33, 121.35 permutations/s]
Permutation test:  60%|█████▉    | 5984/10000 [00:52<00:33, 119.20 permutations/s]
Permutation test:  60%|█████▉    | 5996/10000 [00:52<00:34, 114.54 permutations/s]
Permutation test:  60%|██████    | 6010/10000 [00:52<00:33, 119.74 permutations/s]
Permutation test:  60%|██████    | 6023/10000 [00:52<00:33, 118.62 permutations/s]
Permutation test:  60%|██████    | 6036/10000 [00:52<00:33, 120.01 permutations/s]
Permutation test:  60%|██████    | 6049/10000 [00:52<00:32, 121.19 permutations/s]
Permutation test:  61%|██████    | 6062/10000 [00:52<00:33, 119.20 permutations/s]
Permutation test:  61%|██████    | 6075/10000 [00:52<00:32, 120.28 permutations/s]
Permutation test:  61%|██████    | 6088/10000 [00:53<00:32, 119.46 permutations/s]
Permutation test:  61%|██████    | 6101/10000 [00:53<00:32, 120.47 permutations/s]
Permutation test:  61%|██████    | 6114/10000 [00:53<00:31, 121.82 permutations/s]
Permutation test:  61%|██████▏   | 6127/10000 [00:53<00:31, 122.48 permutations/s]
Permutation test:  61%|██████▏   | 6140/10000 [00:53<00:31, 122.68 permutations/s]
Permutation test:  62%|██████▏   | 6153/10000 [00:53<00:31, 120.83 permutations/s]
Permutation test:  62%|██████▏   | 6166/10000 [00:53<00:31, 121.76 permutations/s]
Permutation test:  62%|██████▏   | 6179/10000 [00:53<00:31, 122.28 permutations/s]
Permutation test:  62%|██████▏   | 6192/10000 [00:53<00:31, 121.01 permutations/s]
Permutation test:  62%|██████▏   | 6205/10000 [00:54<00:31, 121.25 permutations/s]
Permutation test:  62%|██████▏   | 6218/10000 [00:54<00:31, 121.21 permutations/s]
Permutation test:  62%|██████▏   | 6231/10000 [00:54<00:32, 117.58 permutations/s]
Permutation test:  62%|██████▏   | 6243/10000 [00:54<00:32, 116.31 permutations/s]
Permutation test:  63%|██████▎   | 6255/10000 [00:54<00:32, 115.28 permutations/s]
Permutation test:  63%|██████▎   | 6267/10000 [00:54<00:32, 115.24 permutations/s]
Permutation test:  63%|██████▎   | 6279/10000 [00:54<00:31, 116.46 permutations/s]
Permutation test:  63%|██████▎   | 6291/10000 [00:54<00:31, 117.41 permutations/s]
Permutation test:  63%|██████▎   | 6303/10000 [00:54<00:31, 115.68 permutations/s]
Permutation test:  63%|██████▎   | 6316/10000 [00:55<00:31, 117.19 permutations/s]
Permutation test:  63%|██████▎   | 6328/10000 [00:55<00:31, 117.56 permutations/s]
Permutation test:  63%|██████▎   | 6341/10000 [00:55<00:30, 119.75 permutations/s]
Permutation test:  64%|██████▎   | 6354/10000 [00:55<00:30, 120.46 permutations/s]
Permutation test:  64%|██████▎   | 6367/10000 [00:55<00:29, 122.66 permutations/s]
Permutation test:  64%|██████▍   | 6380/10000 [00:55<00:29, 121.16 permutations/s]
Permutation test:  64%|██████▍   | 6393/10000 [00:55<00:30, 119.68 permutations/s]
Permutation test:  64%|██████▍   | 6406/10000 [00:55<00:30, 118.85 permutations/s]
Permutation test:  64%|██████▍   | 6418/10000 [00:55<00:30, 117.45 permutations/s]
Permutation test:  64%|██████▍   | 6431/10000 [00:55<00:29, 119.87 permutations/s]
Permutation test:  64%|██████▍   | 6443/10000 [00:56<00:29, 119.68 permutations/s]
Permutation test:  65%|██████▍   | 6456/10000 [00:56<00:29, 118.79 permutations/s]
Permutation test:  65%|██████▍   | 6468/10000 [00:56<00:30, 117.41 permutations/s]
Permutation test:  65%|██████▍   | 6480/10000 [00:56<00:30, 114.57 permutations/s]
Permutation test:  65%|██████▍   | 6493/10000 [00:56<00:30, 113.68 permutations/s]
Permutation test:  65%|██████▌   | 6505/10000 [00:56<00:30, 115.12 permutations/s]
Permutation test:  65%|██████▌   | 6518/10000 [00:56<00:29, 118.14 permutations/s]
Permutation test:  65%|██████▌   | 6531/10000 [00:56<00:29, 117.41 permutations/s]
Permutation test:  65%|██████▌   | 6544/10000 [00:56<00:28, 120.52 permutations/s]
Permutation test:  66%|██████▌   | 6557/10000 [00:57<00:28, 119.28 permutations/s]
Permutation test:  66%|██████▌   | 6570/10000 [00:57<00:28, 120.74 permutations/s]
Permutation test:  66%|██████▌   | 6583/10000 [00:57<00:28, 119.09 permutations/s]
Permutation test:  66%|██████▌   | 6596/10000 [00:57<00:28, 121.50 permutations/s]
Permutation test:  66%|██████▌   | 6609/10000 [00:57<00:27, 121.53 permutations/s]
Permutation test:  66%|██████▌   | 6622/10000 [00:57<00:27, 123.04 permutations/s]
Permutation test:  66%|██████▋   | 6635/10000 [00:57<00:28, 118.78 permutations/s]
Permutation test:  66%|██████▋   | 6647/10000 [00:57<00:28, 116.61 permutations/s]
Permutation test:  67%|██████▋   | 6659/10000 [00:57<00:28, 116.58 permutations/s]
Permutation test:  67%|██████▋   | 6671/10000 [00:57<00:28, 117.14 permutations/s]
Permutation test:  67%|██████▋   | 6684/10000 [00:58<00:27, 119.23 permutations/s]
Permutation test:  67%|██████▋   | 6696/10000 [00:58<00:28, 115.50 permutations/s]
Permutation test:  67%|██████▋   | 6708/10000 [00:58<00:28, 116.46 permutations/s]
Permutation test:  67%|██████▋   | 6722/10000 [00:58<00:26, 121.71 permutations/s]
Permutation test:  67%|██████▋   | 6735/10000 [00:58<00:27, 119.56 permutations/s]
Permutation test:  67%|██████▋   | 6748/10000 [00:58<00:26, 122.20 permutations/s]
Permutation test:  68%|██████▊   | 6761/10000 [00:58<00:26, 120.16 permutations/s]
Permutation test:  68%|██████▊   | 6774/10000 [00:58<00:26, 119.69 permutations/s]
Permutation test:  68%|██████▊   | 6787/10000 [00:58<00:26, 122.33 permutations/s]
Permutation test:  68%|██████▊   | 6800/10000 [00:59<00:26, 122.18 permutations/s]
Permutation test:  68%|██████▊   | 6813/10000 [00:59<00:26, 122.35 permutations/s]
Permutation test:  68%|██████▊   | 6826/10000 [00:59<00:25, 123.23 permutations/s]
Permutation test:  68%|██████▊   | 6839/10000 [00:59<00:25, 123.46 permutations/s]
Permutation test:  69%|██████▊   | 6852/10000 [00:59<00:25, 124.09 permutations/s]
Permutation test:  69%|██████▊   | 6865/10000 [00:59<00:25, 123.53 permutations/s]
Permutation test:  69%|██████▉   | 6878/10000 [00:59<00:25, 122.66 permutations/s]
Permutation test:  69%|██████▉   | 6891/10000 [00:59<00:25, 124.14 permutations/s]
Permutation test:  69%|██████▉   | 6904/10000 [00:59<00:25, 122.83 permutations/s]
Permutation test:  69%|██████▉   | 6917/10000 [01:00<00:24, 124.01 permutations/s]
Permutation test:  69%|██████▉   | 6930/10000 [01:00<00:24, 123.82 permutations/s]
Permutation test:  69%|██████▉   | 6943/10000 [01:00<00:25, 121.87 permutations/s]
Permutation test:  70%|██████▉   | 6956/10000 [01:00<00:25, 117.19 permutations/s]
Permutation test:  70%|██████▉   | 6969/10000 [01:00<00:25, 118.39 permutations/s]
Permutation test:  70%|██████▉   | 6982/10000 [01:00<00:25, 119.44 permutations/s]
Permutation test:  70%|██████▉   | 6995/10000 [01:00<00:25, 120.11 permutations/s]
Permutation test:  70%|███████   | 7008/10000 [01:00<00:24, 122.29 permutations/s]
Permutation test:  70%|███████   | 7021/10000 [01:00<00:24, 121.17 permutations/s]
Permutation test:  70%|███████   | 7034/10000 [01:00<00:24, 121.76 permutations/s]
Permutation test:  70%|███████   | 7047/10000 [01:01<00:24, 121.96 permutations/s]
Permutation test:  71%|███████   | 7060/10000 [01:01<00:24, 121.45 permutations/s]
Permutation test:  71%|███████   | 7073/10000 [01:01<00:23, 122.80 permutations/s]
Permutation test:  71%|███████   | 7086/10000 [01:01<00:23, 124.27 permutations/s]
Permutation test:  71%|███████   | 7099/10000 [01:01<00:23, 125.43 permutations/s]
Permutation test:  71%|███████   | 7112/10000 [01:01<00:23, 125.43 permutations/s]
Permutation test:  71%|███████▏  | 7125/10000 [01:01<00:23, 124.27 permutations/s]
Permutation test:  71%|███████▏  | 7138/10000 [01:01<00:22, 125.69 permutations/s]
Permutation test:  72%|███████▏  | 7151/10000 [01:01<00:23, 123.78 permutations/s]
Permutation test:  72%|███████▏  | 7164/10000 [01:02<00:22, 124.62 permutations/s]
Permutation test:  72%|███████▏  | 7177/10000 [01:02<00:22, 124.29 permutations/s]
Permutation test:  72%|███████▏  | 7190/10000 [01:02<00:23, 121.92 permutations/s]
Permutation test:  72%|███████▏  | 7203/10000 [01:02<00:23, 120.57 permutations/s]
Permutation test:  72%|███████▏  | 7216/10000 [01:02<00:24, 114.39 permutations/s]
Permutation test:  72%|███████▏  | 7229/10000 [01:02<00:23, 117.79 permutations/s]
Permutation test:  72%|███████▏  | 7241/10000 [01:02<00:23, 116.60 permutations/s]
Permutation test:  73%|███████▎  | 7253/10000 [01:02<00:23, 115.79 permutations/s]
Permutation test:  73%|███████▎  | 7265/10000 [01:02<00:23, 116.92 permutations/s]
Permutation test:  73%|███████▎  | 7278/10000 [01:02<00:22, 119.16 permutations/s]
Permutation test:  73%|███████▎  | 7290/10000 [01:03<00:22, 119.18 permutations/s]
Permutation test:  73%|███████▎  | 7303/10000 [01:03<00:22, 118.55 permutations/s]
Permutation test:  73%|███████▎  | 7316/10000 [01:03<00:22, 121.32 permutations/s]
Permutation test:  73%|███████▎  | 7329/10000 [01:03<00:22, 120.68 permutations/s]
Permutation test:  73%|███████▎  | 7342/10000 [01:03<00:21, 122.57 permutations/s]
Permutation test:  74%|███████▎  | 7355/10000 [01:03<00:21, 121.25 permutations/s]
Permutation test:  74%|███████▎  | 7368/10000 [01:03<00:21, 121.73 permutations/s]
Permutation test:  74%|███████▍  | 7381/10000 [01:03<00:21, 119.51 permutations/s]
Permutation test:  74%|███████▍  | 7395/10000 [01:03<00:21, 121.87 permutations/s]
Permutation test:  74%|███████▍  | 7408/10000 [01:04<00:21, 123.13 permutations/s]
Permutation test:  74%|███████▍  | 7421/10000 [01:04<00:21, 121.16 permutations/s]
Permutation test:  74%|███████▍  | 7434/10000 [01:04<00:21, 117.68 permutations/s]
Permutation test:  74%|███████▍  | 7446/10000 [01:04<00:22, 114.07 permutations/s]
Permutation test:  75%|███████▍  | 7458/10000 [01:04<00:22, 112.97 permutations/s]
Permutation test:  75%|███████▍  | 7470/10000 [01:04<00:22, 112.59 permutations/s]
Permutation test:  75%|███████▍  | 7483/10000 [01:04<00:22, 113.99 permutations/s]
Permutation test:  75%|███████▍  | 7495/10000 [01:04<00:21, 114.39 permutations/s]
Permutation test:  75%|███████▌  | 7507/10000 [01:04<00:21, 115.66 permutations/s]
Permutation test:  75%|███████▌  | 7520/10000 [01:05<00:20, 119.34 permutations/s]
Permutation test:  75%|███████▌  | 7533/10000 [01:05<00:20, 119.53 permutations/s]
Permutation test:  75%|███████▌  | 7546/10000 [01:05<00:20, 121.76 permutations/s]
Permutation test:  76%|███████▌  | 7559/10000 [01:05<00:20, 120.89 permutations/s]
Permutation test:  76%|███████▌  | 7572/10000 [01:05<00:19, 122.97 permutations/s]
Permutation test:  76%|███████▌  | 7585/10000 [01:05<00:19, 122.65 permutations/s]
Permutation test:  76%|███████▌  | 7598/10000 [01:05<00:20, 119.44 permutations/s]
Permutation test:  76%|███████▌  | 7611/10000 [01:05<00:20, 118.54 permutations/s]
Permutation test:  76%|███████▌  | 7624/10000 [01:05<00:19, 121.70 permutations/s]
Permutation test:  76%|███████▋  | 7637/10000 [01:06<00:19, 120.49 permutations/s]
Permutation test:  76%|███████▋  | 7650/10000 [01:06<00:19, 121.39 permutations/s]
Permutation test:  77%|███████▋  | 7663/10000 [01:06<00:19, 118.01 permutations/s]
Permutation test:  77%|███████▋  | 7675/10000 [01:06<00:19, 116.78 permutations/s]
Permutation test:  77%|███████▋  | 7688/10000 [01:06<00:19, 115.74 permutations/s]
Permutation test:  77%|███████▋  | 7700/10000 [01:06<00:19, 116.63 permutations/s]
Permutation test:  77%|███████▋  | 7713/10000 [01:06<00:19, 119.79 permutations/s]
Permutation test:  77%|███████▋  | 7726/10000 [01:06<00:19, 119.18 permutations/s]
Permutation test:  77%|███████▋  | 7738/10000 [01:06<00:19, 118.16 permutations/s]
Permutation test:  78%|███████▊  | 7751/10000 [01:06<00:18, 120.23 permutations/s]
Permutation test:  78%|███████▊  | 7764/10000 [01:07<00:18, 118.24 permutations/s]
Permutation test:  78%|███████▊  | 7776/10000 [01:07<00:18, 118.70 permutations/s]
Permutation test:  78%|███████▊  | 7790/10000 [01:07<00:18, 121.47 permutations/s]
Permutation test:  78%|███████▊  | 7803/10000 [01:07<00:17, 122.62 permutations/s]
Permutation test:  78%|███████▊  | 7816/10000 [01:07<00:17, 122.06 permutations/s]
Permutation test:  78%|███████▊  | 7830/10000 [01:07<00:17, 123.41 permutations/s]
Permutation test:  78%|███████▊  | 7843/10000 [01:07<00:17, 124.07 permutations/s]
Permutation test:  79%|███████▊  | 7856/10000 [01:07<00:17, 121.27 permutations/s]
Permutation test:  79%|███████▊  | 7869/10000 [01:07<00:17, 123.57 permutations/s]
Permutation test:  79%|███████▉  | 7882/10000 [01:08<00:17, 122.39 permutations/s]
Permutation test:  79%|███████▉  | 7895/10000 [01:08<00:17, 122.57 permutations/s]
Permutation test:  79%|███████▉  | 7908/10000 [01:08<00:17, 121.27 permutations/s]
Permutation test:  79%|███████▉  | 7921/10000 [01:08<00:17, 117.60 permutations/s]
Permutation test:  79%|███████▉  | 7934/10000 [01:08<00:17, 120.19 permutations/s]
Permutation test:  79%|███████▉  | 7947/10000 [01:08<00:16, 121.84 permutations/s]
Permutation test:  80%|███████▉  | 7960/10000 [01:08<00:16, 124.08 permutations/s]
Permutation test:  80%|███████▉  | 7974/10000 [01:08<00:16, 126.59 permutations/s]
Permutation test:  80%|███████▉  | 7987/10000 [01:08<00:16, 125.06 permutations/s]
Permutation test:  80%|████████  | 8000/10000 [01:08<00:15, 126.37 permutations/s]
Permutation test:  80%|████████  | 8013/10000 [01:09<00:15, 125.93 permutations/s]
Permutation test:  80%|████████  | 8026/10000 [01:09<00:15, 125.52 permutations/s]
Permutation test:  80%|████████  | 8039/10000 [01:09<00:15, 126.69 permutations/s]
Permutation test:  81%|████████  | 8052/10000 [01:09<00:15, 127.51 permutations/s]
Permutation test:  81%|████████  | 8066/10000 [01:09<00:15, 128.17 permutations/s]
Permutation test:  81%|████████  | 8079/10000 [01:09<00:15, 126.80 permutations/s]
Permutation test:  81%|████████  | 8092/10000 [01:09<00:15, 125.24 permutations/s]
Permutation test:  81%|████████  | 8105/10000 [01:09<00:15, 125.81 permutations/s]
Permutation test:  81%|████████  | 8118/10000 [01:09<00:15, 123.52 permutations/s]
Permutation test:  81%|████████▏ | 8131/10000 [01:10<00:15, 123.80 permutations/s]
Permutation test:  81%|████████▏ | 8144/10000 [01:10<00:15, 123.66 permutations/s]
Permutation test:  82%|████████▏ | 8157/10000 [01:10<00:15, 120.73 permutations/s]
Permutation test:  82%|████████▏ | 8170/10000 [01:10<00:15, 118.20 permutations/s]
Permutation test:  82%|████████▏ | 8183/10000 [01:10<00:15, 120.19 permutations/s]
Permutation test:  82%|████████▏ | 8196/10000 [01:10<00:15, 115.18 permutations/s]
Permutation test:  82%|████████▏ | 8208/10000 [01:10<00:16, 110.71 permutations/s]
Permutation test:  82%|████████▏ | 8220/10000 [01:10<00:16, 109.63 permutations/s]
Permutation test:  82%|████████▏ | 8232/10000 [01:10<00:16, 106.13 permutations/s]
Permutation test:  82%|████████▏ | 8243/10000 [01:11<00:16, 103.95 permutations/s]
Permutation test:  83%|████████▎ | 8254/10000 [01:11<00:17, 102.64 permutations/s]
Permutation test:  83%|████████▎ | 8265/10000 [01:11<00:16, 103.54 permutations/s]
Permutation test:  83%|████████▎ | 8276/10000 [01:11<00:16, 102.61 permutations/s]
Permutation test:  83%|████████▎ | 8287/10000 [01:11<00:16, 101.82 permutations/s]
Permutation test:  83%|████████▎ | 8298/10000 [01:11<00:16, 101.88 permutations/s]
Permutation test:  83%|████████▎ | 8309/10000 [01:11<00:16, 101.69 permutations/s]
Permutation test:  83%|████████▎ | 8320/10000 [01:11<00:16, 100.63 permutations/s]
Permutation test:  83%|████████▎ | 8331/10000 [01:11<00:16, 102.14 permutations/s]
Permutation test:  83%|████████▎ | 8342/10000 [01:12<00:16, 100.81 permutations/s]
Permutation test:  84%|████████▎ | 8353/10000 [01:12<00:15, 103.08 permutations/s]
Permutation test:  84%|████████▎ | 8364/10000 [01:12<00:16, 100.69 permutations/s]
Permutation test:  84%|████████▍ | 8375/10000 [01:12<00:17, 94.46 permutations/s]
Permutation test:  84%|████████▍ | 8385/10000 [01:12<00:16, 95.46 permutations/s]
Permutation test:  84%|████████▍ | 8395/10000 [01:12<00:16, 95.69 permutations/s]
Permutation test:  84%|████████▍ | 8406/10000 [01:12<00:16, 97.01 permutations/s]
Permutation test:  84%|████████▍ | 8416/10000 [01:12<00:16, 97.16 permutations/s]
Permutation test:  84%|████████▍ | 8426/10000 [01:12<00:16, 97.04 permutations/s]
Permutation test:  84%|████████▍ | 8436/10000 [01:13<00:16, 97.73 permutations/s]
Permutation test:  84%|████████▍ | 8447/10000 [01:13<00:15, 97.93 permutations/s]
Permutation test:  85%|████████▍ | 8458/10000 [01:13<00:15, 100.19 permutations/s]
Permutation test:  85%|████████▍ | 8469/10000 [01:13<00:15, 101.88 permutations/s]
Permutation test:  85%|████████▍ | 8480/10000 [01:13<00:15, 100.87 permutations/s]
Permutation test:  85%|████████▍ | 8491/10000 [01:13<00:15, 100.04 permutations/s]
Permutation test:  85%|████████▌ | 8502/10000 [01:13<00:15, 99.07 permutations/s]
Permutation test:  85%|████████▌ | 8514/10000 [01:13<00:14, 101.93 permutations/s]
Permutation test:  85%|████████▌ | 8525/10000 [01:13<00:14, 99.77 permutations/s]
Permutation test:  85%|████████▌ | 8535/10000 [01:13<00:14, 99.13 permutations/s]
Permutation test:  85%|████████▌ | 8546/10000 [01:14<00:14, 100.06 permutations/s]
Permutation test:  86%|████████▌ | 8557/10000 [01:14<00:14, 98.90 permutations/s]
Permutation test:  86%|████████▌ | 8567/10000 [01:14<00:14, 97.91 permutations/s]
Permutation test:  86%|████████▌ | 8578/10000 [01:14<00:14, 99.79 permutations/s]
Permutation test:  86%|████████▌ | 8588/10000 [01:14<00:14, 99.37 permutations/s]
Permutation test:  86%|████████▌ | 8599/10000 [01:14<00:13, 100.09 permutations/s]
Permutation test:  86%|████████▌ | 8610/10000 [01:14<00:13, 100.66 permutations/s]
Permutation test:  86%|████████▌ | 8621/10000 [01:14<00:13, 102.05 permutations/s]
Permutation test:  86%|████████▋ | 8632/10000 [01:14<00:13, 102.08 permutations/s]
Permutation test:  86%|████████▋ | 8643/10000 [01:15<00:13, 102.48 permutations/s]
Permutation test:  87%|████████▋ | 8654/10000 [01:15<00:13, 101.87 permutations/s]
Permutation test:  87%|████████▋ | 8665/10000 [01:15<00:13, 99.01 permutations/s]
Permutation test:  87%|████████▋ | 8676/10000 [01:15<00:13, 100.46 permutations/s]
Permutation test:  87%|████████▋ | 8687/10000 [01:15<00:13, 100.09 permutations/s]
Permutation test:  87%|████████▋ | 8698/10000 [01:15<00:12, 100.26 permutations/s]
Permutation test:  87%|████████▋ | 8709/10000 [01:15<00:12, 99.65 permutations/s]
Permutation test:  87%|████████▋ | 8720/10000 [01:15<00:12, 100.23 permutations/s]
Permutation test:  87%|████████▋ | 8731/10000 [01:15<00:12, 99.26 permutations/s]
Permutation test:  87%|████████▋ | 8742/10000 [01:16<00:12, 101.24 permutations/s]
Permutation test:  88%|████████▊ | 8753/10000 [01:16<00:12, 100.50 permutations/s]
Permutation test:  88%|████████▊ | 8764/10000 [01:16<00:12, 99.76 permutations/s]
Permutation test:  88%|████████▊ | 8774/10000 [01:16<00:12, 96.89 permutations/s]
Permutation test:  88%|████████▊ | 8784/10000 [01:16<00:12, 95.48 permutations/s]
Permutation test:  88%|████████▊ | 8795/10000 [01:16<00:12, 97.32 permutations/s]
Permutation test:  88%|████████▊ | 8806/10000 [01:16<00:12, 98.16 permutations/s]
Permutation test:  88%|████████▊ | 8817/10000 [01:16<00:11, 99.53 permutations/s]
Permutation test:  88%|████████▊ | 8827/10000 [01:16<00:11, 99.36 permutations/s]
Permutation test:  88%|████████▊ | 8838/10000 [01:17<00:11, 100.61 permutations/s]
Permutation test:  88%|████████▊ | 8849/10000 [01:17<00:11, 100.00 permutations/s]
Permutation test:  89%|████████▊ | 8860/10000 [01:17<00:11, 101.62 permutations/s]
Permutation test:  89%|████████▊ | 8871/10000 [01:17<00:11, 99.57 permutations/s]
Permutation test:  89%|████████▉ | 8881/10000 [01:17<00:11, 98.49 permutations/s]
Permutation test:  89%|████████▉ | 8892/10000 [01:17<00:11, 99.56 permutations/s]
Permutation test:  89%|████████▉ | 8902/10000 [01:17<00:11, 98.55 permutations/s]
Permutation test:  89%|████████▉ | 8913/10000 [01:17<00:10, 101.78 permutations/s]
Permutation test:  89%|████████▉ | 8924/10000 [01:17<00:10, 101.84 permutations/s]
Permutation test:  89%|████████▉ | 8935/10000 [01:17<00:10, 102.40 permutations/s]
Permutation test:  89%|████████▉ | 8946/10000 [01:18<00:10, 101.13 permutations/s]
Permutation test:  90%|████████▉ | 8957/10000 [01:18<00:10, 100.51 permutations/s]
Permutation test:  90%|████████▉ | 8969/10000 [01:18<00:10, 101.89 permutations/s]
Permutation test:  90%|████████▉ | 8980/10000 [01:18<00:10, 101.20 permutations/s]
Permutation test:  90%|████████▉ | 8991/10000 [01:18<00:09, 102.90 permutations/s]
Permutation test:  90%|█████████ | 9002/10000 [01:18<00:09, 101.43 permutations/s]
Permutation test:  90%|█████████ | 9013/10000 [01:18<00:09, 101.64 permutations/s]
Permutation test:  90%|█████████ | 9024/10000 [01:18<00:09, 101.70 permutations/s]
Permutation test:  90%|█████████ | 9035/10000 [01:18<00:09, 101.17 permutations/s]
Permutation test:  90%|█████████ | 9046/10000 [01:19<00:09, 99.77 permutations/s]
Permutation test:  91%|█████████ | 9057/10000 [01:19<00:09, 102.49 permutations/s]
Permutation test:  91%|█████████ | 9068/10000 [01:19<00:09, 100.20 permutations/s]
Permutation test:  91%|█████████ | 9079/10000 [01:19<00:09, 100.72 permutations/s]
Permutation test:  91%|█████████ | 9090/10000 [01:19<00:08, 102.87 permutations/s]
Permutation test:  91%|█████████ | 9101/10000 [01:19<00:08, 102.45 permutations/s]
Permutation test:  91%|█████████ | 9112/10000 [01:19<00:08, 101.71 permutations/s]
Permutation test:  91%|█████████ | 9123/10000 [01:19<00:08, 101.05 permutations/s]
Permutation test:  91%|█████████▏| 9134/10000 [01:19<00:08, 102.72 permutations/s]
Permutation test:  91%|█████████▏| 9145/10000 [01:20<00:08, 100.59 permutations/s]
Permutation test:  92%|█████████▏| 9156/10000 [01:20<00:08, 102.88 permutations/s]
Permutation test:  92%|█████████▏| 9167/10000 [01:20<00:08, 98.46 permutations/s]
Permutation test:  92%|█████████▏| 9178/10000 [01:20<00:08, 99.59 permutations/s]
Permutation test:  92%|█████████▏| 9189/10000 [01:20<00:08, 100.08 permutations/s]
Permutation test:  92%|█████████▏| 9200/10000 [01:20<00:07, 100.78 permutations/s]
Permutation test:  92%|█████████▏| 9211/10000 [01:20<00:07, 102.08 permutations/s]
Permutation test:  92%|█████████▏| 9222/10000 [01:20<00:07, 101.87 permutations/s]
Permutation test:  92%|█████████▏| 9233/10000 [01:20<00:07, 101.73 permutations/s]
Permutation test:  92%|█████████▏| 9244/10000 [01:21<00:07, 98.78 permutations/s]
Permutation test:  93%|█████████▎| 9255/10000 [01:21<00:07, 101.71 permutations/s]
Permutation test:  93%|█████████▎| 9266/10000 [01:21<00:07, 100.86 permutations/s]
Permutation test:  93%|█████████▎| 9277/10000 [01:21<00:07, 101.47 permutations/s]
Permutation test:  93%|█████████▎| 9289/10000 [01:21<00:06, 104.77 permutations/s]
Permutation test:  93%|█████████▎| 9300/10000 [01:21<00:06, 104.36 permutations/s]
Permutation test:  93%|█████████▎| 9311/10000 [01:21<00:06, 102.33 permutations/s]
Permutation test:  93%|█████████▎| 9322/10000 [01:21<00:06, 100.09 permutations/s]
Permutation test:  93%|█████████▎| 9333/10000 [01:21<00:06, 101.62 permutations/s]
Permutation test:  93%|█████████▎| 9344/10000 [01:22<00:06, 100.12 permutations/s]
Permutation test:  94%|█████████▎| 9355/10000 [01:22<00:06, 99.07 permutations/s]
Permutation test:  94%|█████████▎| 9365/10000 [01:22<00:06, 96.91 permutations/s]
Permutation test:  94%|█████████▍| 9377/10000 [01:22<00:06, 98.94 permutations/s]
Permutation test:  94%|█████████▍| 9388/10000 [01:22<00:06, 98.68 permutations/s]
Permutation test:  94%|█████████▍| 9399/10000 [01:22<00:06, 99.30 permutations/s]
Permutation test:  94%|█████████▍| 9409/10000 [01:22<00:05, 99.07 permutations/s]
Permutation test:  94%|█████████▍| 9420/10000 [01:22<00:05, 99.93 permutations/s]
Permutation test:  94%|█████████▍| 9431/10000 [01:22<00:05, 98.75 permutations/s]
Permutation test:  94%|█████████▍| 9441/10000 [01:23<00:05, 98.59 permutations/s]
Permutation test:  95%|█████████▍| 9452/10000 [01:23<00:05, 101.68 permutations/s]
Permutation test:  95%|█████████▍| 9463/10000 [01:23<00:05, 102.88 permutations/s]
Permutation test:  95%|█████████▍| 9474/10000 [01:23<00:05, 103.74 permutations/s]
Permutation test:  95%|█████████▍| 9485/10000 [01:23<00:05, 101.51 permutations/s]
Permutation test:  95%|█████████▍| 9496/10000 [01:23<00:05, 100.51 permutations/s]
Permutation test:  95%|█████████▌| 9507/10000 [01:23<00:04, 102.46 permutations/s]
Permutation test:  95%|█████████▌| 9518/10000 [01:23<00:04, 101.29 permutations/s]
Permutation test:  95%|█████████▌| 9529/10000 [01:23<00:04, 99.28 permutations/s]
Permutation test:  95%|█████████▌| 9539/10000 [01:23<00:04, 99.41 permutations/s]
Permutation test:  96%|█████████▌| 9550/10000 [01:24<00:04, 100.08 permutations/s]
Permutation test:  96%|█████████▌| 9561/10000 [01:24<00:04, 100.67 permutations/s]
Permutation test:  96%|█████████▌| 9572/10000 [01:24<00:04, 98.72 permutations/s]
Permutation test:  96%|█████████▌| 9582/10000 [01:24<00:04, 96.77 permutations/s]
Permutation test:  96%|█████████▌| 9592/10000 [01:24<00:04, 96.91 permutations/s]
Permutation test:  96%|█████████▌| 9603/10000 [01:24<00:03, 100.10 permutations/s]
Permutation test:  96%|█████████▌| 9614/10000 [01:24<00:03, 98.28 permutations/s]
Permutation test:  96%|█████████▋| 9625/10000 [01:24<00:03, 99.44 permutations/s]
Permutation test:  96%|█████████▋| 9636/10000 [01:24<00:03, 99.46 permutations/s]
Permutation test:  96%|█████████▋| 9646/10000 [01:25<00:03, 99.51 permutations/s]
Permutation test:  97%|█████████▋| 9657/10000 [01:25<00:03, 99.15 permutations/s]
Permutation test:  97%|█████████▋| 9668/10000 [01:25<00:03, 100.98 permutations/s]
Permutation test:  97%|█████████▋| 9679/10000 [01:25<00:03, 100.40 permutations/s]
Permutation test:  97%|█████████▋| 9690/10000 [01:25<00:03, 100.79 permutations/s]
Permutation test:  97%|█████████▋| 9701/10000 [01:25<00:02, 102.30 permutations/s]
Permutation test:  97%|█████████▋| 9712/10000 [01:25<00:02, 102.16 permutations/s]
Permutation test:  97%|█████████▋| 9723/10000 [01:25<00:02, 102.08 permutations/s]
Permutation test:  97%|█████████▋| 9734/10000 [01:25<00:02, 103.68 permutations/s]
Permutation test:  97%|█████████▋| 9745/10000 [01:26<00:02, 101.88 permutations/s]
Permutation test:  98%|█████████▊| 9756/10000 [01:26<00:02, 103.30 permutations/s]
Permutation test:  98%|█████████▊| 9767/10000 [01:26<00:02, 102.48 permutations/s]
Permutation test:  98%|█████████▊| 9778/10000 [01:26<00:02, 102.22 permutations/s]
Permutation test:  98%|█████████▊| 9790/10000 [01:26<00:01, 106.20 permutations/s]
Permutation test:  98%|█████████▊| 9802/10000 [01:26<00:01, 109.67 permutations/s]
Permutation test:  98%|█████████▊| 9815/10000 [01:26<00:01, 113.69 permutations/s]
Permutation test:  98%|█████████▊| 9828/10000 [01:26<00:01, 116.53 permutations/s]
Permutation test:  98%|█████████▊| 9840/10000 [01:26<00:01, 116.82 permutations/s]
Permutation test:  99%|█████████▊| 9853/10000 [01:26<00:01, 119.81 permutations/s]
Permutation test:  99%|█████████▊| 9865/10000 [01:27<00:01, 119.81 permutations/s]
Permutation test:  99%|█████████▉| 9878/10000 [01:27<00:01, 121.38 permutations/s]
Permutation test:  99%|█████████▉| 9891/10000 [01:27<00:00, 122.49 permutations/s]
Permutation test:  99%|█████████▉| 9904/10000 [01:27<00:00, 124.01 permutations/s]
Permutation test:  99%|█████████▉| 9918/10000 [01:27<00:00, 126.36 permutations/s]
Permutation test:  99%|█████████▉| 9931/10000 [01:27<00:00, 126.38 permutations/s]
Permutation test:  99%|█████████▉| 9944/10000 [01:27<00:00, 124.95 permutations/s]
Permutation test: 100%|█████████▉| 9958/10000 [01:27<00:00, 126.91 permutations/s]
Permutation test: 100%|█████████▉| 9971/10000 [01:27<00:00, 127.19 permutations/s]
Permutation test: 100%|█████████▉| 9984/10000 [01:28<00:00, 127.15 permutations/s]
Permutation test: 100%|█████████▉| 9997/10000 [01:28<00:00, 127.60 permutations/s]
Permutation test: 100%|██████████| 10000/10000 [01:28<00:00, 113.44 permutations/s]

A quick plot suggests an early effect related to n_chars around 130 ms, and a later effect of cloze around 400 ms:

p = plot.TopoButterfly(lm, t=0.130, axh=2, w=6)

Detailed plots of the two effects show that the cluster is quite variable due to noise in the data.

p = plot.TopoArray(lm.masked_parameter_map('n_chars'), t=[0.110, 0.130, 0.150], title='N Chars')
p = plot.TopoArray(lm.masked_parameter_map('cloze'), t=[0.380, 0.400, 0.420], title='Cloze')

• 
• 

To access some of the test results we need to know the index of different effects:

lm.effects

('intercept', 'n_chars', 'cloze')

Create a plot that shows the spatial cluster extent across time.

EFFECTS = [
    ('n_chars', (0.110, 0.150)),
    ('cloze', (0.380, 0.420)),
]

t_maps = []
for effect, time in EFFECTS:
    # t-maps are retrieved by effect name
    t = lm.t(effect).mean(time=time)
    # p-maps are stored in a list, so we need to know the index of te effect
    index = lm.effects.index(effect)
    # We are interested in the maximal spatial cluster extent, i.e., any sensor that is part of the cluster at any time
    p = lm.p[index].min(time=time)
    # Create a masked average t map
    t_av = t.mask(p > 0.05)
    t_maps.append(t_av)

p = plot.Topomap(t_maps, ncol=2)