Datasets / OPENNEURO:ds004752

OPENNEURO:ds004752

Not yet verified

Dataset of intracranial EEG, scalp EEG and beamforming sources from epilepsy patients performing a verbal working memory task

Human 12.8 GB 919 files
  • 7 reproduced
  • 3 partially reproduced
  • 10 not reproduced

Counted separately: 1 claim that also uses data from outside the dataset.

  • 1 blocked

Tested on dataset version 1.0.1

View on OPENNEURO Paper JSON How to read outcomes

Claims

From the paper 10.7554/elife.78677

Reproducing Dimakopoulos et al. (2022) from OpenNeuro ds004752 21

Notebook reproduce_dimakopoulos_2022.ipynb

Reproduced7

  • ReproducedResults, Task and behavior, paragraph 2; Figure 1b

    The mean correct response rate was 91% (for both IN and OUT trials) and decreased with set size, from 97% at set size 4 to 89% at 6 and 83% at 8.

    The mean correct response rate was 91% (both for IN and OUT trials). The rate of correct responses decreased with set size from a set size of 4 (97% correct responses) to set sizes of 6 (89%) and 8 (83%) (Figure 1b)

    Results, Task and behavior, paragraph 2; Figure 1b

    Under all three readings (pooled trials, mean of 68 sessions, mean of 15 participants) the four rates are within 10% and fall from set size 4 to 6 to 8 (pooled 91.3, 97.7, 90.3 and 84.5%). The statement's "both for IN and OUT" holds only roughly, IN 89.0% and OUT 93.5% pooled.

    How the test differed from the paper’s method

    • The 17 placeholder trials with set size 0 (no probe, no response, flagged as artifacts) are left out; counted as errors they lower the pooled rate to 90.8%.
  • ReproducedResults, Task and behavior, paragraph 2

    Across participants the memory capacity (Cowan's K) averaged 6.1.

    Across the participants, the memory capacity averaged 6.1 (Cowan’s K, [correct IN rate +correct OUT rate –1]×set size), which indicates that the participants were able to maintain at least six letters in memory

    Results, Task and behavior, paragraph 2

    Reading (a), rates over all trials times 8, mean over participants, gives 6.59 (range 5.56 to 7.62); the median is 6.63 and the same over sessions 6.57 and 6.67, all within 6.1 +- 0.61. K from the set size 8 rates alone is 5.43 (mean) and 5.33 (median) over participants, outside; over sessions its median is 5.61, within, and its mean 5.41, outside.

    How the test differed from the paper’s method

    • The set size and the averaging are open in the paper; the criterion's readings were all run and 5 of 8 match.
  • ReproducedResults, Task and behavior, paragraph 2; Figure 1c

    The response time of correct trials (3045 trials) was 1.1 +- 0.5 s and grew with set size (1.1, 1.2 and 1.3 s for 4, 6 and 8 letters, 53 ms per item); correct decisions were faster than incorrect ones (1.1 against 1.3 s).

    The mean response time (RT) for correct trials (3045 trials) was 1.1±0.5 s and increased with workload from set size 4 (1.1±0.5 s) to 6 (1.2±0.5 s) and 8 (1.3±0.6 s), 53 ms/item (Figure 1c). Correct IN/OUT decisions were made more rapidly than incorrect decisions (1.1±0.5 s vs 1.3±0.6 s)

    Results, Task and behavior, paragraph 2; Figure 1c

    The events hold 3045 correct trials, the paper's count. Under the median with the trials without the artifact flag (pool p2, 2903 correct and 282 incorrect) every value is within 10%, 1.13 s overall, 1.07, 1.14 and 1.21 s at set sizes 4, 6 and 8, 48.1 ms per item, 1.38 s for incorrect trials, and correct trials are faster. With all 3045 correct trials (pool p1, the paper's count) the slope is 47.69 ms per item, just outside 47.7 to 58.3, and the rest holds, so the pass rests on p2. The mean is outside for the overall (1.29 to 1.30 s) and incorrect (1.68 to 1.70 s) values. The spread is 0.58 to 0.60 s against 0.5.

    How the test differed from the paper’s method

    • The paper calls the central value a mean; only the median matches, a reading the criterion allows because the paper's code prints a median.
  • ReproducedResults, Power spectral density in cortical and hippocampal recordings, paragraph 2; Figure 2b and 2d

    In Participant 1, high alpha to low beta activity ([11 14] Hz) appeared toward the end of maintenance in temporal and occipital contacts.

    After the letters disappeared from the screen, activity occurred in the [11 14] Hz range (high alpha/low beta, Figure 2b) toward the end of the maintenance period in temporal and occipital contacts (permutation test p<0.05, Figure 2d)

    Results, Power spectral density in cortical and hippocampal recordings, paragraph 2; Figure 2b and 2d

    What the paper reports

    • permutation test, p < 0.05

    The largest [11 14] Hz increases in [-2 0] s are at the posterior contacts H3 (35%), H4 (26%, temporal), H2, F8 (occipital), H5 and G3 (occipital). The 5 contacts with a significant increase over fixation (p < 0.05, 200 permutations) are F8 (occipital), H4 and H5 (temporal), and H2 and H3 (no atlas label; the paper calls H3 occipital); with all trials G3 is added (6 contacts). The 64 tests are uncorrected; 3.2 significant contacts are expected by chance (P of 5 or more = 0.22) and none survives a Benjamini-Hochberg correction, so the evidence is weak. The paper reports no magnitude.

    How the test differed from the paper’s method

    • The paper does not describe the permutation test of Figure 2d; maintenance and fixation are swapped in a random half of the trials, 200 times, one-sided.
  • ReproducedResults, Directed functional coupling between hippocampus and averaged EEG sources, paragraph 1rests on an assumptionquestion to the reader

    A similar GC pattern appeared with the signals of left temporal scalp electrodes, and it disappeared with a Laplacian derivation.

    A similar GC pattern emerged when using the signals from left temporal scalp electrodes but was eliminated when using a Laplacian derivation

    Results, Directed functional coupling between hippocampus and averaged EEG sources, paragraph 1

    The derivative scalp channels are referenced to the averaged mastoids (A1 = -A2). Under h1, T5 (e2), d1 and the paired test (s1), encoding ΔGranger is below maintenance with the mastoid reference (medians 0.0035 and 0.0109, p = 0.020, 10 participants) and not after the spline Laplacian (p = 0.29) or the Hjorth derivation (p = 0.89). This holds in 1 of 36 cells (18 h, e and d readings, two tests), 1 of 9 d1 readings, at p = 0.02 without correction for the readings, and appeared only after the white-matter reference was moved (verifier round 1), and the encoding median is positive, so the flow toward the hippocampus in encoding is absent. Under h2 and h3 no d1 reading is significant with the mastoid reference. Under every d2 reading the test is significant with the mastoid reference (p = 0.005 to 0.015) and stays significant after both derivations (p = 0.005 to 0.025). The group test (s2) finds no reversal under any reading. 8 participants have T3 and 10 have T5 in the derivative files.

    How the test differed from the paper’s method

    • The scalp channels are those of the derivative files (the same trials as the sources); the spline Laplacian is MNE's on standard 10-20 positions, standing in for FieldTrip's.
    • The outcome rests on readings h1 and d1. h1 is adopted because the Methods describe the hippocampal LFP re-referenced to a white-matter contact, which h1 is (h2 is the code's bipolar derivation, and the Methods come first); d1 because the Results say ΔGranger was averaged over [4 8] Hz, while d2 makes the pattern by construction (fig4d-reversal-all). The criterion lets the electrode and the test vary (one electrode and test reading) and leaves h and d open, so the adoption of h1 and d1 decides which cells count; whether the criterion should require the sign pattern is asked of a reader. Before the white-matter reference was moved to the hemisphere of the hippocampal contact (verifier round 1), no reading met the criterion.
  • ReproducedResults, Directed functional coupling and the participants' performance, paragraph 2; Figure 4f; Table 1, columns Heschl ΔGranger incorrect trialsrests on an assumption

    On incorrect trials, the net flow between auditory cortex and hippocampus did not have the same direction in all participants and did not reverse (p = 0.37, paired permutation test).

    For incorrect trials, the net information flow ΔGranger from auditory cortex to hippocampus did not show the same directionality in all participants and did not reverse in direction (p=0.37, paired permutation test, Figure 4f)

    Results, Directed functional coupling and the participants' performance, paragraph 2; Figure 4f; Table 1, columns Heschl ΔGranger incorrect trials

    What the paper reports

    • paired permutation test, p = 0.37 (n = 15 participants)

    Each derivative virtual sensor is an exact linear combination of the file's scalp channels (R² = 1.000000 in all 150 fits), so each period's filter was recovered and applied to the raw EEG, re-referenced to the averaged mastoids. On the correct trials the recomputed AC source matches the deposited one (median correlation per participant 0.996 to 0.999, theta ΔGranger within 0.0001). On the 254 incorrect trials of set sizes 6 and 8 without the artifact flag (5 to 40 per participant), with d1 encoding and maintenance ΔGranger do not differ under any reading, p = 0.71 (h1), 0.19 (h2) and 1.00 (h3) with 200 sign flips (exact 0.66, 0.17 and 1.00), median differences -0.0001 to 0.014. Encoding is negative in 6 or 7 of 15 participants and the full pattern appears in 3 to 5, so the direction is not the same in all. The signs match Table 1's incorrect-trial values in 9 to 11 of 29 cells, below the 14.5 that chance gives. The result does not discriminate between correct and incorrect trials: under the same d1 reading the correct trials show no reversal either (fig4d-reversal-all, p = 0.13 to 0.92), so the paper's contrast between them is not reproduced, although this claim's criterion is met. With d2 the test is significant (p = 0.005) under every reading, as d2 gives the pattern by construction.

    How the test differed from the paper’s method

    • The auditory source of the incorrect trials is computed from the raw scalp EEG with the filters recovered from the correct-trial derivative files. The paper does not say whether it computed separate filters for the incorrect trials; an LCMV filter depends on the data covariance, so its own filters may differ. The outcome rests on this choice.
    • The derivative's sampling rates differ by subject; the recomputed source is formed at 500 Hz and downsampled to 40 Hz as the iEEG is.
  • ReproducedResults, Directed functional coupling and the participants' performance, paragraph 2; Figure 4d legendrests on an assumption

    With correct trials subsampled to the number of incorrect trials (10%), the reversal was equally present (p < 0.0005).

    We calculated the GC in a subset of correct trials (median of 200 permutations of a number of correct trials that equals the mean percentage of incorrect trials=10%); the effect was equally present for the subset of correct trials (p<0.0005)

    Results, Directed functional coupling and the participants' performance, paragraph 2; Figure 4d legend

    What the paper reports

    • paired permutation test, p < 5 × 10⁻⁴ (n = 15 participants)

    With 10% of the correct trials and 200 draws, 5 of the 12 t2 readings meet the criterion, all d2 (h1 with the median over draws, h2 and h3 with either aggregation), p = 0.005 to 0.010 (exact sign-flip p = 0.0001) and the sign pattern in 13 to 15 participants; h1 with the mean has it in 11. The d1 readings show no effect (1 to 4 participants, p = 0.38 to 0.92). The controls of fig4d-reversal-all apply, d2 produces the pattern without a directed effect; whether d2 should count is asked of a reader.

    How the test differed from the paper’s method

    • The outcome rests on the d2 reading; under the Results' averaging (d1) the claim is not reproduced.

Partially reproduced3

  • Partially reproducedResults, Task and behavior, paragraph 1; Table 1

    Fifteen participants (median age 29 years, range 18 to 56, 7 male) performed the modified Sternberg task in 71 sessions of 50 trials each.

    Fifteen participants (median age 29 years, range [18–56], 7 male, Table 1) performed a modified Sternberg WM task (71 sessions in total, 50 trials each)

    Results, Task and behavior, paragraph 1; Table 1

    15 participants, median age 29 years (range 18 to 56) and 7 male match the paper. The dataset holds 68 sessions, not 71, and 15 sessions have fewer than 50 trials (34 to 49), 3353 trials in all; the mean of 49.3 trials per session is given as measured. The count of correct trials (3045) equals the paper's, which suggests the trials the paper analysed are all there and the paper's 71 x 50 is a description, not a count.

  • Partially reproducedResults, Power spectral density in cortical and hippocampal recordings, paragraph 1; Figure 2a to 2cverifier disagreed at firstquestion to the reader

    In Participant 1, gamma power (above 40 Hz) rose by more than 100% over fixation during encoding in an occipital contact (H3) and a temporal contact over auditory cortex (C2), and gamma increased significantly only in temporal and occipital-parietal contacts.

    In an occipital recording of Participant 1 (grid contact H3), strong gamma activity (>40 Hz) in the relative PSD occurred while the participant viewed the letters during encoding (increase >100% with respect to fixation, Figure 2b). Similarly, encoding elicited gamma activity in a temporal recording over auditory cortex (increase >100%, grid contact C2, Figure 2c)

    Results, Power spectral density in cortical and hippocampal recordings, paragraph 1; Figure 2a to 2c

    What the paper reports

    • gamma increase on grid contacts during encoding: permutation test, z-score > 1.96, p < 0.05

    Participant 1 is sub-10 (participants.tsv). Gamma at H3 rose by 179% over [40 100] Hz and [-5 -3] s (183% with all set size 6 and 8 trials), above 100%. At C2 gamma [60 80] Hz in [-4 -3] s rose by 17% on average (14% with all trials; largest time-frequency point 61%), not above 100%. The code's z-score test marks 11 of 64 contacts (A2, B1, B2, E1 and H7 temporal; A3, C6 and D6 parietal; G4 occipital; H2 and H3 without an atlas label), all in temporal and occipital-parietal cortex. That statistic z-scores over frequency and takes the maximum; on one trial-permuted data set it exceeded 1.96 at 1 contact of 64. The outcome is partial because two of the criterion's three parts hold (H3 and the z-test) and C2 does not. The third part tests little on this grid. By the atlas labels the 64 contacts are 25 temporal, 24 parietal, 6 occipital, 8 unlabelled and 1 labelled hippocampus, so almost any set of significant contacts meets it; the partial outcome rests on H3 holding and C2 not.

    How the test differed from the paper’s method

    • The ECoG is re-referenced to a white-matter depth contact chosen by rule (the paper names none); the contact used is printed in the data check.
    • The PSD baseline leaves out fixation times whose 10-cycle window would start before the epoch (the epochs begin at -6 s).
    • Contacts are assigned to regions by the dataset's atlas labels; H3 has none and is taken as occipital, as the paper names it.
  • Partially reproducedTable 1, columns Heschl ΔGranger correct trialsrests on an assumption

    Table 1 reports, for each participant, a negative ΔGranger to Heschl gyrus in encoding and a positive one in maintenance on correct trials.

    Not quoted: The per-participant values are in Table 1 only; its legend names them and the Results state the sign pattern (fig4d-reversal-all).

    Participants are matched by the eLife_identifier column of participants.tsv, an assumption, since the pathologies of participants 5, 6 and 15 differ from Table 1 (5 and 6 look swapped). Under five d2 readings the signs match Table 1 in all 30 cells (measured values are from h2-t1-d2), and under four more (the three h1 d2 readings and h3-t2-median-d2) in 26 to 29, so the partial criterion (26 signs) holds; at most 5 of 30 values are within tolerance, short of half, and the values do not track Table 1 (r = -0.51 to 0.30). Under d1 the signs match in 12 to 18 cells. As fig4d-reversal-all shows, d2 makes this sign pattern without a directed effect, so the sign agreement is not evidence of the paper's values; whether d2 should count is asked of a reader.

    How the test differed from the paper’s method

    • The outcome rests on the d2 reading (the code's minimum and maximum over 4 to 8 Hz); under the Results' averaging (d1) the claim is not reproduced.
    • The paper's participant numbers are taken from the eLife_identifier column of participants.tsv, although three pathologies there differ from Table 1; this affects only the per-row comparisons.

Not reproduced10

  • Not reproducedResults, Power spectral density in cortical and hippocampal recordings, paragraph 4; Figure 2e, Figure 3c, 3h and 3m

    In the hippocampus of all four participants with ECoG, beta activity ([12 24] Hz) rose by more than 100% toward the end of maintenance.

    In the hippocampus of all four participants, we found elevated activity in the beta range ([12 24] Hz) toward the end of the maintenance period (increase >100%, Figure 2e, Figure 3c, h and m)

    Results, Power spectral density in cortical and hippocampal recordings, paragraph 4; Figure 2e, Figure 3c, 3h and 3m

    None of the 24 readings (hippocampal channel, band, window, mean or maximum) gives an increase over 100% in all four participants, nor in three. Band means stay below 100% everywhere (largest 94%, Participant 3). With the maximum over time-frequency points, Participant 3 (up to 250%) and Participant 4 (110% under h1, up to 197% under h3) exceed 100%; Participant 1 reaches 55% and Participant 2 at most 13%.

    How the test differed from the paper’s method

    • The hippocampal contact and reference are not given in the paper; readings h1 to h3 were run with the electrode and reference choices stated in wmflow.hippocampal_channels, the white-matter reference taken in the hemisphere of the hippocampal contact.
  • Not reproducedResults, Functional coupling between hippocampus and cortex, paragraph 1; Figure 2f and 2g

    In Participant 1, phase locking between hippocampus and auditory cortex was enhanced in the theta range during maintenance (PLV 0.4 at contact C3), and theta PLV in maintenance was highest to contacts over auditory cortex.

    Compared to encoding, maintenance showed enhanced PLV in the theta range between hippocampal LFP and cortical ECoG (PLV=0.4 in contact C3, permutation test p<0.05, Figure 2f). PLV in the [4 8] Hz theta range increased significantly with several contacts over auditory cortex (permutation test p<0.05, Figure 2g)

    Results, Functional coupling between hippocampus and cortex, paragraph 1; Figure 2f and 2g

    What the paper reports

    • PLV maintenance against the reference period, contact C3: cluster-based non-parametric permutation test, p < 0.05
    • theta PLV over grid contacts: permutation test, p < 0.05

    Theta PLV between the hippocampus and C3 in maintenance is 0.240 (h1), 0.164 (h2) and 0.195 (h3), below 0.36. It does not exceed fixation (p = 0.72, 0.49 and 0.35) and no contact shows a significant theta increase over fixation except 2 non-auditory contacts under h2. Against encoding, the variant, C3 is significant (p = 0.010 to 0.025) with 16 to 21 significant contacts, 4 to 6 of them auditory. The highest maintenance theta PLV is at A1 under every reading; the six highest include C3 and B3 under h1, B3 but not C3 under h2, and C3 and B3 but not A3 under h3.

    How the test differed from the paper’s method

    • The per-contact and per-frequency test is a trial-wise swap of the two periods' phasors, 200 times, against the 95th percentile; the cluster step of the code is reported as runs of significant frequencies.
  • Not reproducedResults, Directed functional coupling between hippocampus and ECoG, paragraph 1; Figure 2h and 2lrests on an assumptionquestion to the reader

    In Participant 1, information flowed from auditory cortex to hippocampus during encoding (net flow significant in [6 8] Hz) and reversed, from hippocampus to auditory cortex, during maintenance (significant in [5 8] Hz).

    The net information flow ΔGranger (GC hipp→cortex – GC cortex→hipp) during encoding was significant in the [6 8] Hz range (blue bar in Figure 2h, p<0.05 permutation test against a null distribution). During maintenance, the information flow in the theta frequency range was reversed

    Results, Directed functional coupling between hippocampus and ECoG, paragraph 1; Figure 2h and 2l

    What the paper reports

    • ΔGranger, encoding, [6 8] Hz: permutation test against a null distribution, p < 0.05
    • ΔGranger, maintenance, [5 8] Hz: permutation test against a null distribution, p < 0.05

    With clusters (c2) no reading shows the reversal between the hippocampus and C2. Under h1 encoding has significant clusters overlapping theta (7.55 to 9.05 Hz, p = 0.01) and maintenance none; under h2 and h3 neither period has a theta cluster. The per-frequency test (c1) shows the reversal under h1 only (encoding 6.75 to 7.2 and 7.55 to 9.05 Hz, maintenance 7.0 to 7.45 Hz). On shifted trial pairs of this participant (cortical trials shifted by 1 to 10 positions) c1 flags the reversal in 40% (h1) and 20% (h2) of shifts and theta in 30% to 70% of single tests, and c2 flags the reversal in 0% of shifts; the direction-swap null puts 8% to 16% of bins beyond its 5th or 95th percentile, against 5%. With each period's evoked response subtracted from both signals, c1 still flags the reversal in 20% to 50% of shifts in Participants 1 to 4 and theta in 20% to 83% of single tests, and c2 the reversal in none, so c1's excess does not come from the evoked responses; with a scrambled-trial null c1 flags theta in 20% to 60% of single tests. The scrambled-trial cluster test of the Figure 2h legend, run beside the criterion, flags theta in 0% to 20% of single shifted tests and the reversal in none; on the real pairing it finds an encoding cluster overlapping theta under h1 (7.45 to 9.05 Hz) and no maintenance cluster under any reading. The rates rest on 5 to 10 shifts per cell. The code's variant (AHL2-AHL3, trial-shuffle null, 1st and 99th percentiles) finds encoding at 5.9 to 6.15 Hz and nothing in maintenance.

    How the test differed from the paper’s method

    • The outcome rests on the cluster reading (c2). The per-frequency reading (c1), which the Methods describe, flags the reversal in 20% to 90% of shifted trial pairings in Participants 1 to 3, which have no trial-by-trial coupling but keep the stimulus-locked responses, and in 20% to 50% with those responses subtracted, far from the 5% of the paper's p < 0.05, so as run here it is not the paper's test; c2, the cluster version of the same test, flags it in 0% to 10%. The Figure 2h legend names a cluster test against scrambled trials, which the criterion does not; it was run beside the criterion and agrees with c2 (asked of a reader in questions/openneuro-ds004752.yaml). The electrode (AHL) and the white-matter reference contact are chosen by rule, not given by the paper.
    • GC is computed in Python with a Wilson factorisation following FieldTrip, checked on an analytic autoregressive process; signals are resampled with scipy's polyphase filter.
  • Not reproducedResults, Directed functional coupling between hippocampus and ECoG, paragraphs 1 and 2; Figure 2i to 2k

    In Participant 1, the strongest net flow in theta during encoding was from auditory cortex contacts to hippocampus; during maintenance theta flow from hippocampus was significant to auditory and occipital cortex, and this spatial pattern was significant against a null distribution.

    the maximal net information flow ΔGranger (GC hipp→cortex – GC cortex→hipp) during encoding occurred from auditory cortex to hippocampus (p<0.05, permutation test, Figure 2i). During maintenance, the theta ΔGranger was significant from hippocampus to both auditory cortex and occipital cortex (permutation test p<0.05, Figure 2j)

    Results, Directed functional coupling between hippocampus and ECoG, paragraphs 1 and 2; Figure 2i to 2k

    What the paper reports

    • encoding ΔGranger map: permutation test, p < 0.05
    • maintenance ΔGranger map: permutation test, p < 0.05

    In encoding the most negative theta ΔGranger is at G6, an occipital contact (h1 and h2; H5 under h3), not an auditory one; with h1, 22 contacts are significant in encoding, C1 and C3 among them. Of the 7 contacts with significant positive ΔGranger in maintenance (h1), none is auditory and H3 is not among them. The spatial test (maintenance, h1) is not significant, the true scalar products having a median of -0.0041 against a null median of -0.0012 and a null 95th percentile of 0.013 (0% of true values above it).

    How the test differed from the paper’s method

    • For the notebook's time limit (each spatial test takes about 6 minutes), the per-contact tests ran with h1 only (h2 and h3 values are shown without a test), and the spatial test ran on maintenance with h1 only.
    • In the spatial null each trial's window is drawn at random from fixation (1 s) or maintenance (2 s), as the Methods describe; the fixation cross-spectra are scaled by 2 so that the two window lengths enter on one scale under the same 20 s padding (a standard choice).
  • Not reproducedResults, Directed functional coupling between hippocampus and ECoG, paragraph 4; Figure 3d, 3e, 3i and 3jrests on an assumption

    The same reversal appeared in Participant 2 between auditory cortex and hippocampus (encoding [6 8] Hz, maintenance [6 10] Hz) and in Participant 3 between visual cortex and the right hippocampus (encoding 6 to 9 Hz, maintenance [6 8] Hz).

    Similarly in Participant 2, the time course of GC followed the same pattern between auditory cortex (anterior strip electrode contact in Figure 3a) and hippocampus (Figure 3d and e). ... The time course of information flow between visual cortex and hippocampus (Figure 3i and j) followed the same pattern as described for the auditory cortex above

    Results, Directed functional coupling between hippocampus and ECoG, paragraph 4; Figure 3d, 3e, 3i and 3j

    What the paper reports

    • cluster-based non-parametric permutation test against a null distribution with scrambled trials, p < 0.05

    With clusters (c2) neither participant shows the reversal under any reading. In Participant 2 (sub-11, most anterior contact TBAL1) maintenance has a positive theta cluster under h1 (4.0 to 5.65 Hz) and h2 (4.0 to 6.75 Hz) and encoding none. In Participant 3 (sub-12, most posterior contact PL1, right hippocampus AHR with the right white-matter reference AHR3) encoding has a negative cluster only at 15.5 to 17.15 Hz, outside theta, and maintenance none; with the left hippocampus maintenance has theta clusters and encoding none. The per-frequency test (c1) shows the reversal in Participant 3 under h1 and h2 and in Participant 2 under no reading; on shifted trial pairs c1 flags the reversal in 40% to 90% of shifts in these participants (20% to 40% with the evoked responses subtracted) and c2 in 0% to 10%. The scrambled-trial cluster test of the Figure 3 legend, run beside the criterion, agrees with c2: in Participant 2 maintenance has theta clusters under h1 to h3 and encoding none in theta, and in Participant 3 neither period has a cluster under any reading; on shifted pairs it flags the reversal in none.

    How the test differed from the paper’s method

    • The outcome rests on the cluster reading (c2), for the calibration given under fig2h-gc-reversal-p1; the Figure 3 legend and the claims file's statistics name a cluster-based test.
    • The most anterior and most posterior contacts are chosen by MNI y. In sub-11 the electrode named PHL lies in the right hemisphere by its coordinates and atlas labels (PHL1 at x = 33.6, Right Hippocampus); contacts are chosen by name.
  • Not reproducedResults, Directed functional coupling between hippocampus and ECoG, paragraph 4; Figure 3n and 3overifier disagreed at first

    In Participant 4, recordings from right parietal cortex showed no significant GC with the hippocampus during the task.

    However, in Participant 4, the recordings from the right cortical hemisphere (Figure 3k) did not show significant GC between LFP and ECoG during task performance (Figure 3n and o)

    Results, Directed functional coupling between hippocampus and ECoG, paragraph 4; Figure 3n and 3o

    Significant GC is found at every contact of Participant 4, so the absence the claim states does not hold. On the real pairing the per-frequency test of the criterion finds significant ΔGranger somewhere in [4 20] Hz in 24 of 24 contact and period tests with h1, 20 of 24 with h2 and 24 of 24 with h3 (in [4 8] Hz in 21, 13 and 23), and the cluster test in 16, 14 and 22 of 24; the largest absolute theta ΔGranger is 0.041, 0.101 and 0.039. On shifted trial pairs of Participant 4 (4 contacts, 5 shifts), with no trial-by-trial coupling, the per-frequency test flags 90% to 100% of tests and the cluster test 10% to 40%, against the nominal 5%, so these tests alone could not show an absence. The cluster test flags the real pairing more often than the shifted one in three of four reading and period cells (h1 encoding 58% against 40%, h1 maintenance 75% against 10%, h2 encoding 100% against 30%, h2 maintenance 17% against 30%). The scrambled-trial cluster test of the Figure 2h and 3 legends, run beside the criterion, holds its false-positive rate on shifted pairs of this participant (a cluster anywhere in [4 20] Hz in 17% of encoding and 0% of maintenance tests, a theta cluster in none) and on the real pairing flags all 12 contacts in at least one period under each of h1, h2 and h3 (under h1, 8 of 12 in encoding and 10 of 12 in maintenance), with an encoding theta cluster at 7, 12 and 12 of 12 contacts. The 12 ECoG contacts include STG and precentral contacts besides the postcentral and IPL ones; all are tested, and as every one is flagged, keeping only the parietal ones would not change the outcome. The outcome was blocked (methods_underspecified) until round 3 of verification.

  • Not reproducedResults, Directed functional coupling between hippocampus and averaged EEG sources, paragraph 1; Figure 4a

    Across the 15 participants, GC between hippocampal LFP and auditory cortex EEG sources showed net flow from auditory cortex to hippocampus during encoding ([6 10] Hz) and from hippocampus to auditory cortex during maintenance ([6 9] Hz and [13 15] Hz).

    During encoding, the net information flow was from auditory cortex to hippocampus (light blue curve – dark blue curve, blue bar, p<0.05, group cluster-based permutation test). During maintenance, the net information flow was reversed (dark red curve – light red curve, red bar, p<0.05, group cluster-based permutation test), i.e., from hippocampus to auditory cortex

    Results, Directed functional coupling between hippocampus and averaged EEG sources, paragraph 1; Figure 4a

    What the paper reports

    • encoding, [6 10] Hz: group cluster-based permutation t-test, p < 0.05, FDR-corrected (n = 15 participants)
    • maintenance, [6 9] Hz and [13 15] Hz: group cluster-based permutation t-test, p < 0.05, FDR-corrected (n = 15 participants)

    No frequency in [4 20] Hz differs between the two directions after FDR correction, in either period, under any hippocampal reading or either t-test (smallest uncorrected Monte Carlo p 0.003 to 0.035). The group-mean theta ΔGranger is positive in both periods (0.0012 to 0.0058).

    How the test differed from the paper’s method

    • The auditory source is the dataset's AC virtual sensor; the Heschl and superior temporal readings (a2, a3) cannot be run because the dataset holds only the five averaged sensors.
  • Not reproducedResults, Directed functional coupling between hippocampus and averaged EEG sources, paragraph 2; Figure 4b and 4c

    Theta net flow was strongest between hippocampus and auditory cortex, toward the hippocampus during encoding (ΔGranger -0.049, p = 0.0009) and toward the cortex during maintenance (ΔGranger 0.034, p = 0.001); other areas showed weaker flow in the same directions.

    During encoding, the mean information flow was strongest from auditory cortex to hippocampus (ΔGranger=−0.049, p=0.0009, Kruskal-Wallis test, Figure 4b). ... the strongest flow appeared from hippocampus to auditory cortex (ΔGranger=0.034, p=0.001, Kruskal-Wallis test)

    Results, Directed functional coupling between hippocampus and averaged EEG sources, paragraph 2; Figure 4b and 4c

    What the paper reports

    • encoding, across cortical areas: Kruskal-Wallis test, p = 9 × 10⁻⁴ (n = 15 participants)
    • maintenance, across cortical areas: Kruskal-Wallis test, p = 0.001 (n = 15 participants)

    Under no reading do all parts hold, and no reading puts the auditory value within tolerance in both periods. The measured values are from the closest reading (h1, 10% draws, mean over draws, d2, group mean); under it the Kruskal-Wallis test is not significant (p = 0.40 and 0.31) and DLPFC (encoding) and OFC (maintenance) are more extreme. The Kruskal-Wallis test reaches p < 0.05 in one period of one reading only (h2-t2-median-d2, encoding, p = 0.009, DLPFC most negative), the auditory source is never the most extreme area in both periods, and Dunn's tests find no pair with AC significant.

    How the test differed from the paper’s method

    • The areas are the five derivative sensors (AC, DLPFC, OFC, PPC, V1), not the code's set with Broca.
    • The 10% subsampling uses 200 draws for AC and 20 (the code's number) for the other areas.
  • Not reproducedResults, Directed functional coupling and the participants' performance, paragraph 1; Figure 4dquestion to the reader

    The reversal of theta net flow between hippocampus and auditory cortex appeared in each of the 15 participants, negative in encoding (median -0.049) and positive in maintenance (median 0.034), p = 4.1e-10 by a paired permutation test, correct trials only.

    The reversal of ΔGranger appeared in all 15 participants individually (Figure 4d). We averaged ΔGranger for each participant in the [4 8] Hz theta frequency range. The ΔGranger between hippocampus and auditory cortex, was negative during encoding and was positive during maintenance in the theta frequency range (p=4.1e-10, paired permutation test)

    Results, Directed functional coupling and the participants' performance, paragraph 1; Figure 4d

    What the paper reports

    • paired permutation test, p = 4.1 × 10⁻¹⁰ (n = 15 participants)

    No reading meets the criterion, because the medians are about half the paper's; the most negative encoding median is -0.027 (h1, 10% draws, mean over draws, d2, with maintenance 0.021) and the largest maintenance median 0.031 (h3, same). With the extreme value d2 the sign pattern holds in 11 to 15 participants (15 in five readings) and the paired test gives p = 0.005 to 0.010 with 200 permutations (exact sign-flip p = 0.0001). With the band mean d1, which the Results' "We averaged ΔGranger" describes, there is no effect, 1 to 5 participants show the pattern, medians within 0.007 of zero, p = 0.13 to 0.92. Controls show the d2 pattern comes from the statistic. Taking the opposite extremes (maximum in encoding, minimum in maintenance) gives the mirror pattern in 14 or 15 participants (p = 0.005), and d2 of the direction-swap null, a spectrum with no direction, is negative in encoding and positive in maintenance in 15 of 15. Pooling the three ECoG values gives the pattern in 16 of 18 under d2. Whether d2 should count as a reading is asked of a reader (questions/openneuro-ds004752.yaml).

    How the test differed from the paper’s method

    • Only the AC sensor is available as the auditory source (reading a1).
  • Not reproducedResults, Directed functional coupling and the participants' performance, paragraph 1; Figure 4erests on an assumption

    The directionality and its reversal were absent for other areas such as lateral prefrontal cortex, whose net flow had a lower median and higher variability (p = 0.16, paired permutation test, correct trials).

    The directionality and its reversal was missing for all other areas, e.g., lateral prefrontal cortex (p=0.16, paired permutation test, Figure 4e)

    Results, Directed functional coupling and the participants' performance, paragraph 1; Figure 4e

    What the paper reports

    • paired permutation test, p = 0.16 (n = 15 participants)

    By the paper's test on the DLPFC sensor (d1, all correct trials, 200 sign flips) maintenance ΔGranger exceeds encoding under h1 (p = 0.010, exact sign-flip p = 0.0009, median difference 0.0057) and h3 (p = 0.045, exact p = 0.020), and not under h2 (p = 0.27, exact p = 0.33). Under the d2 readings every area, DLPFC included, shows the reversal at p = 0.005 to 0.010, and under d1 the auditory source itself shows none (p = 0.14 to 0.92), so the contrast the paper draws between the auditory source and other areas does not hold under a common reading. Before the white-matter reference was moved to the hemisphere of the hippocampal contact (verifier round 1), h1 gave p = 0.065.

    How the test differed from the paper’s method

    • The lateral prefrontal source is the dataset's DLPFC sensor.
    • The outcome rests on reading h1, adopted because the Methods describe the hippocampal LFP re-referenced to a white-matter contact; under h2, the code's bipolar derivation, the claim is reproduced.

Blocked1

  • BlockedResults, Source reconstruction of the scalp EEG, paragraph 1; Table 1· Method cannot be run hereuses data from outside the dataset

    In each participant, more than 80% of the EEG sources that differed significantly from fixation lay in the left hemisphere, in encoding and in maintenance.

    In each participant, the proportion of significant sources in the left hemisphere exceeded 80% of all significant sources

    Results, Source reconstruction of the scalp EEG, paragraph 1; Table 1

    The test needs LCMV beamforming of the scalp EEG on an AAL grid and FieldTrip's cluster-based source statistics. FieldTrip runs in MATLAB, which cannot be installed in the locked Python environment, and the code aligns each participant's electrodes to the template head by hand (ft_electroderealign with cfg.method interactive). The dataset holds five averaged virtual sensors per participant, not the source grid, so the hemisphere shares cannot be read from it. An approximation in MNE on a template head with template 10-20 positions and an AAL atlas from outside the dataset was not attempted; 4 participants have 8 scalp channels in the derivative files (6 without the mastoids).

Findings about the data

What attempts found about the dataset itself, for its owner and the archive.

  • OtherThe dataset holds 68 sessions with 3353 trials (15 sessions with fewer than 50 trials), where the paper reports 71 sessions of 50 trials; the 3045 correct trials equal the paper's count. The README says each session includes about 50 trials and gives no session count.
  • Metadata errorIn every channels.tsv, low_cutoff is 1000 and high_cutoff is 0.5, the reverse of the 0.5 to 1000 Hz passband given in the ieeg.json and eeg.json sidecars and the paper.
  • Description missingThe EDF headers leave the physical dimension of every signal blank; only channels.tsv gives µV. The derivative virtual sensors carry no unit, and their sampling rate varies from 200 to 4096 Hz across subjects without description.
  • Metadata errorIn sub-11 the contacts named PHL lie at positive MNI x (right) and those named PHR at negative x, and the atlas labels follow the coordinates (PHL1 at x = 33.6, "Right Hippocampus"; PHR1 at x = -31.1), so the electrode names give the wrong hemisphere; 3 contacts of sub-12 have the same mismatch, and in sub-01 the atlas label names the hemisphere opposite to the x coordinate for 12 contacts (for example ECL1 at x = -24, "Right Amygdala").
  • Metadata errorparticipants.tsv gives a pathology different from Table 1 of the paper for eLife participants 5 (hippocampal sclerosis against brain contusion), 6 (brain contusion against hippocampal sclerosis) and 15 (dysembryoplastic neuroepithelial tumor against hippocampal sclerosis); 5 and 6 look swapped.
  • Metadata errorThe README lists the virtual sensors as temporal superior lobe, lateral prefrontal cortex, occipital cortex, posterior parietal cortex and Broca, but the derivative files hold DLPFC, OFC, PPC, AC and V1 (no Broca), plus the scalp channels, which the README does not mention.
  • Identifier missingThe derivative LCMV files carry no trial numbers. Their trials are the correct trials of set sizes 6 and 8 in session order (shown here by correlating their scalp channels with the raw EEG), without the artifact-flagged trials in 13 subjects and with them in sub-13 and sub-14.
  • Data missingThe derivative files hold sources for correct trials only, and the beamformer filters, electrode alignment and head model are not deposited, so the paper's source statistics (Table 1) cannot be repeated from the dataset. The incorrect-trial sources of Figure 4f are not deposited either; they can be approximated because each virtual sensor is an exact linear combination of the file's own scalp channels, which gives the correct-trial filters.
  • Other17 trials (16 in sub-14, 1 in sub-12) have SetSize 0, no Match, ResponseTime 0 and the artifact flag, apparently placeholders; ProbeLetter is n/a for every trial of sub-10 to sub-15. Neither is described in the events sidecar.
  • Published code disagreesIn calculateGranger_random_trial_sampling.m at commit 395ae13 (lines 31 and 34 to 37; and the group medians of MeanSourceGranger.m) the same row of FieldTrip's grangerspctrm is stored as cortex to hippocampus for encoding and as hippocampus to cortex for maintenance, although both periods use the same channel pair, so the sign of maintenance ΔGranger is flipped relative to encoding. Combined with the minimum (encoding) and maximum (maintenance) over 4 to 8 Hz taken in MeanSourceGranger.m and DeltaGranger_EEG.m, this yields negative encoding and positive maintenance values without a directed effect.
  • Paper inconsistentThe Results say ΔGranger was averaged over [4 8] Hz for each participant ("We averaged ΔGranger"), while the Figure 4d legend reports the "maximal ΔGranger" and the code takes the minimum in encoding and the maximum in maintenance. On this dataset the averaged value shows no effect and the extreme value shows the reported sign pattern, as it also does for every other area and for spectra with no direction.
  • Methods not availableQuestion for the authors. Which hippocampal contact (electrode, contact number, monopolar against which white-matter contact, or which bipolar pair) entered the GC, PSD and PLV analyses of each participant, and which contact was the ECoG reference? The Methods name a white-matter reference without saying which, and the code's bipolar pairs differ by participant. Readings tried here (contact 1 of the left anterior electrode against an unlabelled contact of the same hemisphere, a bipolar pair, the mean over electrodes) give different outcomes for fig4e-lpfc-no-reversal and scalp-temporal-laplacian, and the per-frequency GC test gives different bands for Participants 1 and 3.
  • Methods not availableQuestion for the authors. Is the per-participant ΔGranger of Figure 4 and Table 1 the mean over 4 to 8 Hz or the minimum (encoding) and maximum (maintenance), and were the 10% subsampled spectra aggregated by the median (Results) or the mean (Methods) over 20 (code) or 200 (Methods) draws?
  • Methods not availableQuestion for the authors. How was the direction-swap null of the per-frequency GC test built (Methods, Statistics, paragraph 1), and was significance corrected over frequencies? Built here by swapping the two channels in a random half of the trials, the null puts 4% to 26% of bins beyond its 5th or 95th percentile on shifted trial pairings, which have no trial-by-trial coupling but keep the stimulus-locked responses, and the per-frequency test flags a theta reversal in 20% to 90% of them (20% to 50% with those responses subtracted), far from the 5% of p < 0.05. The code's significance script uses a trial-shuffle null with the 1st and 99th percentiles instead, and the Figure 2h and 3 legends name a cluster test against scrambled trials.
  • Methods not availableQuestion for the authors. For the incorrect trials of Figure 4f, was the LCMV filter computed from the incorrect trials' own data or taken from the correct trials? The deposited files hold only the correct trials' sources, from which the correct-trial filters can be recovered exactly.

Questions

Where the author or a verifier believed the claims file misread the paper, they asked, citing the paper, and a reader who had not seen the data answered from the paper alone, amending the claims file where it was wrong. The author then ran the claim as answered. Attempts before protocol pilot-11 also asked here for method details the claims file did not give.

  • openasked 2026-10-28about fig4d-reversal-all

    Should reading d2 (the minimum of ΔGranger over 4 to 8 Hz in encoding and the maximum in maintenance) be kept as a reading that can decide fig4d-reversal-all, table1-heschl-dgranger-correct and balanced-correct-subset, and the d2 readings of fig4bc-auditory-strongest, scalp-temporal-laplacian and fig4e-lpfc-no-reversal? Taking the minimum in one period and the maximum in the other gives a negative encoding value and a positive maintenance value by construction, so the sign pattern these criteria test, and the paired test between the periods, hold under d2 whether or not there is a directed effect. The same question applies to the claim statements, which describe the result as averaged over the band.

    Passage. We averaged ΔGranger for each participant in the [4 8] Hz theta frequency range (Results, Directed functional coupling and the participants' performance, paragraph 1), against the "maximal ΔGranger" of the Figure 4d legend and the minimum and maximum taken in MeanSourceGranger.m and DeltaGranger_EEG.m.

    Why it matters. The verifier showed on synthetic uncoupled autoregressive data that d2 gives negative encoding and positive maintenance values while d1 stays near 0. In the notebook d2 of the direction-swap null, a spectrum with no direction, has the pattern in 15 of 15 participants, and taking the opposite extremes gives the mirror pattern at p = 0.005. table1-heschl-dgranger-correct (partially reproduced) and balanced-correct-subset (reproduced) pass under d2 only, and fail under d1 (12 to 18 matching signs; no reading with p below 0.05). If d2 is excluded, or the criteria require the pattern to exceed what d2 gives on the null, both become not reproduced.

    No reader has answered it yet.
  • openasked 2026-10-28about fig2h-gc-reversal-p1

    Should the test of fig2h-gc-reversal-p1 and fig3-gc-reversal-p2-p3 be the cluster-based test against a null with scrambled trials that the Figure 2h and Figure 3 legends name? The criterion makes the Methods' direction-swap null primary, per frequency (c1) or with clusters (c2), and runs a scrambled-trial null only as the code's variant, for Participant 1 only, at the 1st and 99th percentiles and not as a cluster test. The same question applies to fig3no-p4-right-no-gc, whose criterion takes the test of fig2h-gc-reversal-p1.

    Passage. The net information flow ΔGranger ... was significant in the [6 8] Hz range (blue bar in Figure 2h, p<0.05 permutation test against a null distribution) (Results, Directed functional coupling between hippocampus and ECoG, paragraph 1), against the Figure 2h and Figure 3 legends, which describe a cluster-based non-parametric permutation test against a null distribution with scrambled trials, and Methods, Statistics, paragraph 1, which describes switching the direction at random across trials.

    Why it matters. The verifier reported this as a place where the criterion departs from the figure legends. The attempt ran the legends' test beside the criterion. On shifted trial pairs it flags a theta cluster in 0% to 20% of single tests and the reversal in none, while the criterion's per-frequency test flags the reversal in 20% to 90%. On the real pairings it shows no reversal in Participants 1 to 3, as c2 does, and finds clusters at all 12 right parietal contacts of Participant 4, against fig3no-p4-right-no-gc.

    No reader has answered it yet.
  • openasked 2026-10-28about fig2bc-gamma-encoding-p1

    Does the third condition of the criterion, that the contacts with z above 1.96 lie in the temporal and occipital-parietal parts of the grid only, test anything on this grid? By the dataset's atlas labels every one of the 64 grid contacts of Participant 1 is temporal, parietal, occipital or unlabelled, apart from one labelled hippocampus, so almost any set of significant contacts meets it. Should the outcome then rest on the two contacts (H3 and C2) alone?

    Passage. gamma increased significantly only in temporal and occipital-parietal contacts (claim statement, from Results, Power spectral density in cortical and hippocampal recordings, paragraph 1, and Figure 2a).

    Why it matters. The verifier holds that the condition is near-empty and that the outcome should be not reproduced, as C2 rises by 17% against the required 100%. The attempt keeps partially reproduced because H3 rises by 179%, and the record notes the disagreement.

    No reader has answered it yet.
  • openasked 2026-10-28about scalp-temporal-laplacian

    Should the criterion require the sign pattern of the "similar GC pattern" (negative ΔGranger in encoding and positive in maintenance), or a correction over its 18 readings and two tests? As written it passes when, under one reading, encoding is below maintenance at p < 0.05 with the mastoid reference and not after the Laplacian.

    Passage. A similar GC pattern emerged when using the signals from left temporal scalp electrodes but was eliminated when using a Laplacian derivation (Results, Directed functional coupling between hippocampus and averaged EEG sources, paragraph 1).

    Why it matters. The claim passes on 1 of 36 cells (h1, T5, d1, paired test, p = 0.020 uncorrected) where the encoding median is positive (0.0035), so the flow toward the hippocampus in encoding that the paper's pattern has is absent. The verifier reported that the criterion does not require the sign pattern.

    No reader has answered it yet.

Notebooks

Every registered entry that uses this dataset. An entry can hold several notebooks, and can use more than one dataset.

Reproduction Information flow between hippocampus and auditory cortex in verbal working memory (Dimakopoulos et al. 2022)

Tests the findings of Dimakopoulos, Mégevand, Stieglitz, Imbach and Sarnthein (2022) on OpenNeuro ds004752, which holds hippocampal depth recordings, ECoG in four participants, scalp EEG and LCMV virtual sensors from 15 epilepsy patients performing a modified Sternberg task. It checks the behaviour, the power and phase locking in the ECoG participants, and the spectral Granger causality between the hippocampus and the cortex in encoding and maintenance, per participant and across the group. All files

  • Reproducing Dimakopoulos et al. (2022) from OpenNeuro ds004752
    reproduce_dimakopoulos_2022.ipynb
    Source

Verification

  • Round 1
    2026-10-28
    Agreed with 20 of 21 outcomes. Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook). Reviewed: Round 1. The executed notebook of commit 1ef70ec8 (author-answers) with the conclusion cells and the summary removed, the helper module and the claims file, with the paper. The verifier also read the archive file list and README and tested the helper functions on synthetic input. agrees compares its outcome with the outcome of the record it reviewed.
    • fig2bc-gamma-encoding-p1: the verifier’s outcome was “Not reproduced”. Cell 26, H3 rises 179% (passes), but C2 rises only 17% as a mean and 61% at its maximum, both below 100%.
  • Round 2
    2026-10-28
    Agreed with 19 of 21 outcomes. Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook). Reviewed: Round 2. The full executed notebook of commit fc3c318b (author-revise-1) with its conclusions and summary, the helper module, the README, this attempt record, the claims file and the round-1 report, with the paper. agrees compares its outcome with the outcome of the record it reviewed. Its other points: the README's run time (about 18 minutes, against about 23 from the printed timings), the code_mismatch finding, which it could not check from its materials, and the ECoG reference of sub-10 and sub-12 (AHL4, next to the hippocampal reference AHL3), which is documented and may feed a shared signal into the PLV and GC of Participants 1 and 3. In reply the run time is corrected (26 minutes with the added cells), the finding cites the code's lines (31 and 34 to 37 of calculateGranger_random_trial_sampling.m at 395ae13), and the ECoG reference is left as documented, as no other white-matter contact is available on another depth electrode in those subjects.
    • fig2bc-gamma-encoding-p1: the verifier’s outcome was “Not reproduced”. C2 rises 17% against the required 100% (cell 30). The part that holds is near-empty on this grid: every one of the 64 contacts is temporal, parietal, occipital or unlabelled, plus one labelled hippocampus (cell 28). Almost any set of significant contacts would satisfy it, so only H3 carries information.
    • fig3no-p4-right-no-gc: the verifier’s outcome was “Not reproduced”. The criterion's test was run and flags every contact (cell 56). Against the attempt's own shifted-pair baseline, the real pairing is flagged more often by the cluster test: h1 maintenance 75% against 10%, h2 encoding 100% against 30%. That is evidence of GC, against a claim of absence. The method was specified and run, so this is not a block.
  • Round 3
    2026-10-28
    Agreed with 1 of 1 outcomes. Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook). Reviewed: Round 3. The change of outcome proposed by author-revise-2 (commit 505073dd) for fig3no-p4-right-no-gc and the analysis behind it in the executed notebook (cells 47 and 57), with the claims file and the paper. Nothing else was reviewed. Its other points, that the claim and the Figure 3k legend say right parietal while Participant 4's contacts include STG and precentral ones, and that the criterion names the direction-swap test while the legend names a scrambled-trial cluster test, are reported without a change to the claim; the notes now say that all 12 contacts are tested.

Method differences

How the tests differed from the paper’s method or data, and the claims each applies to.

  • The 17 placeholder trials with set size 0 (no probe, no response, flagged as artifacts) are left out; counted as errors they lower the pooled rate to 90.8%.fig1b-accuracy
  • The set size and the averaging are open in the paper; the criterion's readings were all run and 5 of 8 match.capacity-cowan-k
  • The paper calls the central value a mean; only the median matches, a reading the criterion allows because the paper's code prints a median.fig1c-response-time
  • The ECoG is re-referenced to a white-matter depth contact chosen by rule (the paper names none); the contact used is printed in the data check.fig2bc-gamma-encoding-p1
  • The PSD baseline leaves out fixation times whose 10-cycle window would start before the epoch (the epochs begin at -6 s).fig2bc-gamma-encoding-p1
  • Contacts are assigned to regions by the dataset's atlas labels; H3 has none and is taken as occipital, as the paper names it.fig2bc-gamma-encoding-p1
  • The paper does not describe the permutation test of Figure 2d; maintenance and fixation are swapped in a random half of the trials, 200 times, one-sided.fig2d-cortical-beta-maintenance-p1
  • The hippocampal contact and reference are not given in the paper; readings h1 to h3 were run with the electrode and reference choices stated in wmflow.hippocampal_channels, the white-matter reference taken in the hemisphere of the hippocampal contact.hipp-beta-maintenance
  • The per-contact and per-frequency test is a trial-wise swap of the two periods' phasors, 200 times, against the 95th percentile; the cluster step of the code is reported as runs of significant frequencies.fig2fg-plv-theta-p1
  • The outcome rests on the cluster reading (c2). The per-frequency reading (c1), which the Methods describe, flags the reversal in 20% to 90% of shifted trial pairings in Participants 1 to 3, which have no trial-by-trial coupling but keep the stimulus-locked responses, and in 20% to 50% with those responses subtracted, far from the 5% of the paper's p < 0.05, so as run here it is not the paper's test; c2, the cluster version of the same test, flags it in 0% to 10%. The Figure 2h legend names a cluster test against scrambled trials, which the criterion does not; it was run beside the criterion and agrees with c2 (asked of a reader in questions/openneuro-ds004752.yaml). The electrode (AHL) and the white-matter reference contact are chosen by rule, not given by the paper.fig2h-gc-reversal-p1
  • GC is computed in Python with a Wilson factorisation following FieldTrip, checked on an analytic autoregressive process; signals are resampled with scipy's polyphase filter.fig2h-gc-reversal-p1
  • For the notebook's time limit (each spatial test takes about 6 minutes), the per-contact tests ran with h1 only (h2 and h3 values are shown without a test), and the spatial test ran on maintenance with h1 only.fig2ijk-spatial-spread-p1
  • In the spatial null each trial's window is drawn at random from fixation (1 s) or maintenance (2 s), as the Methods describe; the fixation cross-spectra are scaled by 2 so that the two window lengths enter on one scale under the same 20 s padding (a standard choice).fig2ijk-spatial-spread-p1
  • The outcome rests on the cluster reading (c2), for the calibration given under fig2h-gc-reversal-p1; the Figure 3 legend and the claims file's statistics name a cluster-based test.fig3-gc-reversal-p2-p3
  • The most anterior and most posterior contacts are chosen by MNI y. In sub-11 the electrode named PHL lies in the right hemisphere by its coordinates and atlas labels (PHL1 at x = 33.6, Right Hippocampus); contacts are chosen by name.fig3-gc-reversal-p2-p3
  • The auditory source is the dataset's AC virtual sensor; the Heschl and superior temporal readings (a2, a3) cannot be run because the dataset holds only the five averaged sensors.fig4a-group-gc-reversal
  • The scalp channels are those of the derivative files (the same trials as the sources); the spline Laplacian is MNE's on standard 10-20 positions, standing in for FieldTrip's.scalp-temporal-laplacian
  • The outcome rests on readings h1 and d1. h1 is adopted because the Methods describe the hippocampal LFP re-referenced to a white-matter contact, which h1 is (h2 is the code's bipolar derivation, and the Methods come first); d1 because the Results say ΔGranger was averaged over [4 8] Hz, while d2 makes the pattern by construction (fig4d-reversal-all). The criterion lets the electrode and the test vary (one electrode and test reading) and leaves h and d open, so the adoption of h1 and d1 decides which cells count; whether the criterion should require the sign pattern is asked of a reader. Before the white-matter reference was moved to the hemisphere of the hippocampal contact (verifier round 1), no reading met the criterion.scalp-temporal-laplacian
  • The areas are the five derivative sensors (AC, DLPFC, OFC, PPC, V1), not the code's set with Broca.fig4bc-auditory-strongest
  • The 10% subsampling uses 200 draws for AC and 20 (the code's number) for the other areas.fig4bc-auditory-strongest
  • Only the AC sensor is available as the auditory source (reading a1).fig4d-reversal-all
  • The outcome rests on the d2 reading (the code's minimum and maximum over 4 to 8 Hz); under the Results' averaging (d1) the claim is not reproduced.table1-heschl-dgranger-correct
  • The paper's participant numbers are taken from the eLife_identifier column of participants.tsv, although three pathologies there differ from Table 1; this affects only the per-row comparisons.table1-heschl-dgranger-correct
  • The lateral prefrontal source is the dataset's DLPFC sensor.fig4e-lpfc-no-reversal
  • The outcome rests on reading h1, adopted because the Methods describe the hippocampal LFP re-referenced to a white-matter contact; under h2, the code's bipolar derivation, the claim is reproduced.fig4e-lpfc-no-reversal
  • The auditory source of the incorrect trials is computed from the raw scalp EEG with the filters recovered from the correct-trial derivative files. The paper does not say whether it computed separate filters for the incorrect trials; an LCMV filter depends on the data covariance, so its own filters may differ. The outcome rests on this choice.fig4f-incorrect-no-reversal
  • The derivative's sampling rates differ by subject; the recomputed source is formed at 500 Hz and downsampled to 40 Hz as the iEEG is.fig4f-incorrect-no-reversal
  • The outcome rests on the d2 reading; under the Results' averaging (d1) the claim is not reproduced.balanced-correct-subset

Provenance

Attempt record
openneuro-ds004752-dimakopoulos-2022-01
Paper
10.7554/elife.78677
Date
2026-10-28
Made by
An agent, run by CatalystNeuro
Dataset
OPENNEURO:ds004752
Version
1.0.1 (doi:10.18112/openneuro.ds004752.v1.0.1)
Run on
GitHub Actions
Harness
Claude Code
Model
claude-opus-5-5
Protocol
pilot-12

Transcripts of the agent runs

Each run’s prompt, messages and tool calls, with long output cut and the text of the paper left out, and its final report.