Datasets / DANDI:001051

DANDI:001051

Not yet verified

Large-scale Neuropixels recordings through SHIELD implant during visual change detection task with dynamic gating of engagement

electrophysiological House mouse 1.8 TB 666 files
  • 10 reproduced
  • 5 partially reproduced
  • 4 not reproduced
  • 20 blocked

Counted separately: 6 claims that also use data from outside the dataset.

  • 3 reproduced
  • 1 not reproduced
  • 2 blocked

Counted separately: 1 check that the dataset is the data behind the paper.

  • 1 partially reproduced

Tested on dataset version 0.260922.1827

View on DANDI Paper JSON How to read outcomes

Claims

From the paper SHIELD: Skull-shaped hemispheric implants enabling large-scale electrophysiology datasets in the mouse brain

Reproducing Bennett et al. (2024) from dandiset 001051 30

Notebook reproduce_bennett_2024.ipynb

Reproduced13

  • ReproducedResults, High-quality multi-Neuropixels recordings, paragraph 2; Figure 4B

    A typical 6-probe session records 1,821 non-noise units (SD 341.9) from 28.1 areas (SD 5.1).

    we recorded activity from over 1,800 units spanning ... 25 brain areas (this includes all non-noise units, as described in STAR Methods; 1,821 ± 341.9 units; 28.1 ± 5.1 areas, mean ± SD

    Results, High-quality multi-Neuropixels recordings, paragraph 2; Figure 4B

    44 sessions of the 87 have in-area units on exactly six probes; they have 1821.0 non-noise units (SD 341.9) and 28.11 areas (SD 5.12), the paper's values; unchanged without the 6 sessions that differ from the code's table.

  • ReproducedDiscussion, paragraph 2

    Stable recordings were made from 25 mice in 87 experiments with 467 probe insertions.

    high-quality recordings from 25 mice, totaling 87 experiments and 467 probe insertions

    Discussion, paragraph 2

    Over the 87 sessions 25 mice, 87 experiments and 507 probes with units; 507 is within 467 +- 46.7. Of these 467 are in the code's units table; the other 40 probes carry 2004 units, 1873 on 'Track not annotated' channels, which the paper's tables leave out. Over all 99 sessions 27 mice and 578 probes with units.

    How the test differed from the paper’s method

    • The dandiset holds 40 probe recordings in the 87 sessions that the paper's tables leave out; nothing in the files marks them, so insertions are counted as probes with units, with the code's 467 shown.
  • ReproducedSTAR Methods, Neuropixels recordings, Probe insertion, paragraph 2

    On average 5.46 of the 6 intended probes per session were successfully inserted.

    Overall, we achieved a penetration success of 5.46 probes per session

    STAR Methods, Neuropixels recordings, Probe insertion, paragraph 2

    507 probes with units over 87 sessions give 5.83 per session (a), 578 over 99 give 5.84 (b), within 5.46 +- 0.546; with the code's 467 insertions, 5.37. The code's probes table lists 467 probes for the 87 sessions and 533 for the 99 (5.37 and 5.38 per session), every one marked inserted, so it records no failed attempt. No count gives 5.46.

  • ReproducedResults, High-quality multi-Neuropixels recordings, paragraph 4; Figure 4Duses data from outside the dataset

    Unit drift with SHIELD is only slightly higher than with the 5-mm window (median 25.1 against 22.82 micrometres).

    found it was only slightly higher than that observed in our previous recordings using the 5-mm insertion window method (SHIELD: 25.1 ± 0.1 mm, 5-mm window: 22.82 ± 0.05; median ± SEM

    Results, High-quality multi-Neuropixels recordings, paragraph 4; Figure 4D

    Median max_drift 25.05 um over 144,030 SHIELD non-noise units (paper 143,828) and 22.82 over 262,177 VBN non-noise units, within 25.1 +- 0.1 and 22.82 +- 0.05; 25.22 without the 6 differing sessions.

    How the test differed from the paper’s method

    • SHIELD units include 202 more non-noise units than the paper's count, from the 40 probe recordings the paper's tables leave out (143,828 non-noise units on the 467 insertions in the code's table, 202 on the other 40). The code's own table has 144,068 for the 87 sessions; outside the 6 sessions where it describes other units it and the files both give 135,334.
  • ReproducedResults, High-quality multi-Neuropixels recordings, paragraph 5uses data from outside the dataset

    The fraction of units passing the standard quality filter is very similar for SHIELD (41.1%) and the 5-mm window (40.7%).

    the percentage of units passing our standard quality metrics filter (see STAR Methods) was very similar for the SHIELD versus 5-mm window implants (% units passing quality metrics criteria: SHIELD: 41.1, 5-mm window: 40.7)

    Results, High-quality multi-Neuropixels recordings, paragraph 5

    41.09% of SHIELD and 40.69% of VBN non-noise units pass the QC filter.

  • ReproducedResults, High-quality multi-Neuropixels recordings, paragraph 6; Figure 4E

    About 300 units are recorded per probe on each day, and unit yield does not differ across the four recording days with SHIELD (p = 0.48).

    On each recording day, we measured the activity of roughly 300 units from each probe (Figure 4E). ... unit yield was not significantly different across days for SHIELD mice (p = 0.48, Kruskal-Wallis H test)

    Results, High-quality multi-Neuropixels recordings, paragraph 6; Figure 4E

    What the paper reports

    • Kruskal-Wallis H test, p = 0.48 (n = 129, 126, 121 and 91 insertions on days 1 to 4)

    Over all 500 insertions with non-noise units, medians 308, 302, 287.5 and 277 per day and Kruskal-Wallis p = 0.333; over the 467 insertions of the code's table (129, 126, 121 and 91, the figure's counts) medians 314, 305.5, 295 and 303 and p = 0.477. Every median is within 300 +- 30.

  • ReproducedResults, High-quality multi-Neuropixels recordings, paragraph 6; Figure 4Euses data from outside the dataset

    With the 5-mm window, unit yield decreased on the second recording day (p = 6e-22).

    In contrast to the 5-mm window preparation, for which unit yield decreased on the second recording day (p = 6e–22, Kruskal-Wallis H test)

    Results, High-quality multi-Neuropixels recordings, paragraph 6; Figure 4E

    What the paper reports

    • Kruskal-Wallis H test, p = 6 × 10⁻²² (n = 459 and 446 VBN insertions on days 1 and 2)

    VBN medians 321 (459 insertions) and 261 (446) on days 1 and 2; Kruskal-Wallis p = 1.2e-24 for day 1 against day 2 (a) and 1.2e-19 over the code's six groups (b).

  • ReproducedResults, High-quality multi-Neuropixels recordings, paragraph 6; Figure 4E

    The average firing rate of recorded units is similar across the four recording days (p = 0.21).

    Additionally, the average firing rate of recorded units was similar across days (p = 0.21 Kruskal-Wallis H test).

    Results, High-quality multi-Neuropixels recordings, paragraph 6; Figure 4E

    What the paper reports

    • Kruskal-Wallis H test, p = 0.21 (n = 23, 24, 22 and 18 sessions on days 1 to 4)

    Kruskal-Wallis p = 0.223 over 23, 24, 22 and 18 sessions (text 0.21, code 0.26).

  • ReproducedResults, High-quality multi-Neuropixels recordings, paragraph 7; Figures 4E and 4F

    A consistent fraction of VISp units has significant spatial receptive fields on each of the four recording days (p = 0.38).

    we measured a consistent fraction of units with well-defined spatial receptive fields (RFs) (Figure 4E and 4F; p = 0.38 Kruskal-Wallis H test)

    Results, High-quality multi-Neuropixels recordings, paragraph 7; Figures 4E and 4F

    What the paper reports

    • Kruskal-Wallis H test, p = 0.38 (n = 12, 11, 12 and 11 sessions on days 1 to 4)

    With RF p values recomputed here and p < 0.01, median fractions 0.900, 0.877, 0.913 and 0.900 over 12, 11, 12 and 11 sessions, Kruskal-Wallis p = 0.671; at 0.05, p = 0.368; with the code's p values, 0.864 and 0.813.

    How the test differed from the paper’s method

    • RF p values are recomputed with the AllenSDK chi-square bootstrap (1000 resamplings) with their own seed; they match the code's table (r = 0.9990) where units agree.
  • ReproducedResults, High-quality multi-Neuropixels recordings, paragraph 7; Figure 4G

    The population response of VISp units to an image is highly similar across the four recording days.

    In addition, the stimulusevoked temporal dynamics were highly similar across the 4 recording days (Figure 4G).

    Results, High-quality multi-Neuropixels recordings, paragraph 7; Figure 4G

    595, 482, 534 and 446 VISp units with significant RFs on days 1 to 4 (12, 11, 12 and 11 sessions and mice); Kruskal-Wallis p = 0.343 for the per-unit response 0 to 250 ms (a) and 0.70 for the time of peak (b); the day population responses correlate at r = 0.977 to 0.991.

  • ReproducedResults, SHIELD recordings reveal multi-regional subnetworks, paragraphs 3 and 4; Figure 7A

    VISp alpha events are rhythmic at 3 to 5 Hz, and MOp events occur in a slightly higher band, 5 to 7 Hz.

    we observed rhythmic 3–5 Hz waves of activity in VISp (Figure 7A, left) ... These MOp alpha-like events occurred in a slightly higher frequency band (5–7 Hz)

    Results, SHIELD recordings reveal multi-regional subnetworks, paragraphs 3 and 4; Figure 7A

    Under frequency reading (b), the autocorrelogram spectrum peak, VISp events at 4.0 Hz and MOp at 4.5 Hz (medians), MOp higher by the one-sided rank-sum test (p 5.3e-7 to 0.0016) in all 8 L, E and set combinations, which the criterion accepts. The 2 s autocorrelogram gives a 0.5 Hz grid on which the MOp median sits on the lower edge of the widened 4.5 to 7.7 Hz range (22.7% of L2 E2 MOp values are exactly 4.5 Hz); with the spectrum zero-padded to 0.049 Hz steps the medians are 4.05 to 4.10 Hz (VISp) and 4.64 to 4.74 Hz (MOp), inside the ranges, with MOp higher at p 7.1e-7 to 0.0038 in all 8. Under reading (a), the inverse of the median interval between maxima, VISp 4.17 to 4.21 Hz and MOp 4.35 to 4.55 Hz, and MOp is not significantly higher (p 0.078 to 0.76). 1469 to 1704 qualifying VISp events and 80 to 122 MOp events. The detector finds synthetic sinusoidal rates of 3 to 5 Hz only (none at 2.5 Hz or below or 5.5 Hz or above), so a 5 to 7 Hz MOp band cannot appear under either reading; this follows from the event rule of the claims file.

    How the test differed from the paper’s method

    • Run under the claims file's 16 combinations of area membership (L1, L2), event rule (E1, E2), random-time coherence (R1, R2) and session set (a, b); population windows are cut from one 10 ms grid per session. Events are detected on the smoothed rate, and the autocorrelogram threshold, CCGs and coherence use the unsmoothed rate, a detail taken from the code's printed counts quoted in the claims file's header (1653 VISp and 113 MOp qualifying events, code cell 12), which the unsmoothed autocorrelogram matches (1643 to 1704 and 119 to 122 under E2) and the smoothed one does not (3035 to 3426 and 152 to 170). The Benjamini-Hochberg family of the pair test holds brain areas only (not CP-RF, CP-noRF, VIS or unannotated labels); this changes significance for 4 of 734 pair results.
    • Reading (b) restricts the spectrum peak to 1 to 20 Hz, a standard choice; the zero-padded spectrum (2048 points) is a standard interpolation shown as a variant.
  • ReproducedResults, SHIELD recordings reveal multi-regional subnetworks, paragraph 5; Figure 7B, right

    VISp alpha events are often triggered by the offset of visual stimuli, while the timing of MOp events is not related to visual stimuli.

    The VISp events were often triggered by the offset of visual stimulation (Figure 7B) ... The timing of MOp events was not related to visual stimuli (Figure 7B).

    Results, SHIELD recordings reveal multi-regional subnetworks, paragraph 5; Figure 7B, right

    In all combinations VISp events peak 100 to 150 ms after stimulus offset (KS against uniform p 9.2e-233 or less; chi-square p printed as 0) and MOp events do not differ from uniform (chi-square p 0.41 to 0.53, KS p 0.28 to 0.62).

    How the test differed from the paper’s method

    • Run under the claims file's 16 combinations of area membership (L1, L2), event rule (E1, E2), random-time coherence (R1, R2) and session set (a, b); population windows are cut from one 10 ms grid per session. Events are detected on the smoothed rate, and the autocorrelogram threshold, CCGs and coherence use the unsmoothed rate, a detail taken from the code's printed counts quoted in the claims file's header (1653 VISp and 113 MOp qualifying events, code cell 12), which the unsmoothed autocorrelogram matches (1643 to 1704 and 119 to 122 under E2) and the smoothed one does not (3035 to 3426 and 152 to 170). The Benjamini-Hochberg family of the pair test holds brain areas only (not CP-RF, CP-noRF, VIS or unannotated labels); this changes significance for 4 of 734 pair results.
    • The stimulus before an event is the last active image presentation, as the code's stimulus-trials table holds active presentations.
  • ReproducedResults, Fine-scale alpha rhythm coupling between visual cortex and striatum, paragraph 4; Figure 8E

    Triggered on VISp alpha events, CP-RF firing rates are more modulated than CP-noRF rates, particularly in the troughs of the oscillation.

    We found that CP-RF firing rates were significantly more modulated, particularly during the troughs of the oscillation when the visual thalamo-cortical network is suppressed.

    Results, Fine-scale alpha rhythm coupling between visual cortex and striatum, paragraph 4; Figure 8E

    What the paper reports

    • Wilcoxon rank-sum test per time point, p < 0.05, Benjamini-Hochberg-corrected (n = 439 events)

    In every combination (387 to 493 events, 4 or 5 sessions) CP-RF and CP-noRF rates differ in 112 to 222 bins by the rank-sum test, 24 or 25 at the troughs of the VISp rate, and the CP-RF range over 0 to 1 s is larger (1.96 to 2.11 against 1.18 to 1.34 Hz; 0.50 to 0.56 against 0.30 to 0.32 normalised). The code's signed-rank variant gives 98 to 216 bins.

    How the test differed from the paper’s method

    • Run under the claims file's 16 combinations of area membership (L1, L2), event rule (E1, E2), random-time coherence (R1, R2) and session set (a, b); population windows are cut from one 10 ms grid per session. Events are detected on the smoothed rate, and the autocorrelogram threshold, CCGs and coherence use the unsmoothed rate, a detail taken from the code's printed counts quoted in the claims file's header (1653 VISp and 113 MOp qualifying events, code cell 12), which the unsmoothed autocorrelogram matches (1643 to 1704 and 119 to 122 under E2) and the smoothed one does not (3035 to 3426 and 152 to 170). The Benjamini-Hochberg family of the pair test holds brain areas only (not CP-RF, CP-noRF, VIS or unannotated labels); this changes significance for 4 of 734 pair results.

Partially reproduced6

  • Partially reproducedResults, High-quality multi-Neuropixels recordings, paragraph 2; Figures S4A and S4B

    Across recording days each mouse received about 20 probe insertions (SD 4.1) into about 10 implant holes (10.4, SD 2.3).

    we typically made ... probe insertions (20 ± 4.1, mean ± SD; Figure S4A) targeting ... implant holes (10.4 ± 2.3, mean ± SD; Figure S4B)

    Results, High-quality multi-Neuropixels recordings, paragraph 2; Figures S4A and S4B

    Insertions per mouse, mean 20.28 (SD 3.34) over the 87 sessions (reading a) and 21.41 (SD 3.33) over all sessions (b), within 20 +- 4.1; with only the 467 insertions of the code's table, 18.68. The implant holes per mouse (10.4 +- 2.3) could not be measured, as no file records the hole of an insertion; that part is untested, so the claim holds only in the part that could be tested.

    How the test differed from the paper’s method

    • Holes per mouse not tested (no hole per insertion in the dandiset or the code's tables).
  • Partially reproducedResults, High-quality multi-Neuropixels recordings, paragraph 8

    Task performance is stable across the four recording days despite repeated insertions (median rewards 59.5, 63.5, 56.5, 60.5; p = 0.96).

    we found that mouse behavioral performance on the visual task was stable across the 4 recording days despite repeated probe insertions (median rewards received: day 1, 59.5; day 2, 63.5; day 3, 56.5; day 4, 60.5; Kruskal-Wallis test: p = 0.96)

    Results, High-quality multi-Neuropixels recordings, paragraph 8

    What the paper reports

    • Kruskal-Wallis test, p = 0.96

    No reading finds a difference across days (Kruskal-Wallis p 0.731 to 0.817), so the claim of stability holds. The medians do not all match: over the 87 sessions (a) all rewards 73, 65, 61, 53.5 and (b) earned rewards 67, 59, 55, 48 (days 2 and 3 within tolerance in each); over all sessions (a) 68, 63.5, 60, 56.5 (days 2 to 4) and (b) 62.5, 57.5, 54, 50.5 (days 1 to 3). Medians fall from day 1 to day 4 in every reading. Hit trials exceed earned rewards because of the no-reward block. The paper's medians end in .5 on days 1 and 3, which needs an even number of sessions those days; the 87-session set has 23 on day 1 and the 99-session set 25 on day 3, so neither set is the paper's.

  • Partially reproducedResults, SHIELD recordings reveal multi-regional subnetworks, paragraph 5; Figure 7B

    VISp and MOp alpha events occur when the mouse is immobile and the pupil constricted, with pupil diameter about 25% smaller than at random session times.

    VISp and MOp events tended to occur during periods of lower arousal, when the animal was immobile and the pupil was constricted (during alpha events, pupil diameter was ... 25% smaller relative to randomly chosen session times)

    Results, SHIELD recordings reveal multi-regional subnetworks, paragraph 5; Figure 7B

    At qualifying events the pupil is smaller (Wilcoxon p at most 9.1e-15) and running lower (0.03 to 1.16 against 4.1 to 7.7 cm/s at random times, p at most 8.0e-6) for VISp and for MOp in every combination. The change is not 25%: VISp -12.9 to -15.4% (area, mean), -7.5 to -8.8% (diameter); MOp -30.8 to -36.5% (area), -17.3 to -20.9% (diameter); pooled -14.0 to -16.3% (area). No reading falls within -22.5 to -27.5%. The running and pupil streams share the stimulus clock as far as the data check shows: running slows after hits against misses from a median 0.225 s after the change, and the pupil constricts after white against black flashes from a median 0.35 s, with small contrasts before time 0. Alignment of at least one stream is not shown in 12 of the 99 sessions (2 without a running contrast, as they have no hit or no miss trials, 2 without eye tracking, 5 with a weak running contrast and 3 with a weak pupil contrast).

    How the test differed from the paper’s method

    • Run under the claims file's 16 combinations of area membership (L1, L2), event rule (E1, E2), random-time coherence (R1, R2) and session set (a, b); population windows are cut from one 10 ms grid per session. Events are detected on the smoothed rate, and the autocorrelogram threshold, CCGs and coherence use the unsmoothed rate, a detail taken from the code's printed counts quoted in the claims file's header (1653 VISp and 113 MOp qualifying events, code cell 12), which the unsmoothed autocorrelogram matches (1643 to 1704 and 119 to 122 under E2) and the smoothed one does not (3035 to 3426 and 152 to 170). The Benjamini-Hochberg family of the pair test holds brain areas only (not CP-RF, CP-noRF, VIS or unannotated labels); this changes significance for 4 of 734 pair results.
    • Diameter is the square root of window-mean area; two sessions without eye tracking contribute no pupil values.
  • Partially reproducedResults, Fine-scale alpha rhythm coupling between visual cortex and striatum, paragraph 2; Figures 8B and 8C

    CP units with significant spatial receptive fields lie mostly in the medial dorsal striatum, midway along the anterior-posterior extent of CP.

    These units were biased toward the medial dorsal striatum mid-way along the anteriorposterior extent of the CP

    Results, Fine-scale alpha rhythm coupling between visual cortex and striatum, paragraph 2; Figures 8B and 8C

    688 of 2376 QC CP units (52 sessions, 21 mice) have significant RFs. They are more dorsal than CP-noRF units (median y 3150 against 3500 um, one-sided p = 1.9e-58) but more lateral (median z 4075 against 4350 um, two-sided p = 1.8e-33). Their median AP coordinate, 5388 um, is in the posterior part of the CP sampled (3700 to 6500 um), where the RF fraction rises from 0.10 to 0.45 across AP terciles.

    How the test differed from the paper’s method

    • RF significance recomputed here (link-code-tables); the CCF z axis orientation is checked against VISp (3050) and midline structures (5300).
  • Partially reproducedResults, Fine-scale alpha rhythm coupling between visual cortex and striatum, paragraph 3; Figure 8Dverifier disagreed at firstquestion to the reader

    CP units with receptive fields respond more strongly to the task's full-field images than CP units without receptive fields.

    CP-RF units showed significantly larger visual responses to this stimulus compared with CPnoRF units (Figure 8D).

    Results, Fine-scale alpha rhythm coupling between visual cortex and striatum, paragraph 3; Figure 8D

    What the paper reports

    • Wilcoxon rank-sum test per time point, p < 0.05, Benjamini-Hochberg-corrected (n = 80 CP-RF and 205 CP-noRF units)

    In the five sessions CP-RF units respond more than CP-noRF units in 53 bins of 0 to 250 ms (first at 55 ms), CP-RF higher in all, none before onset, with the Methods' QC, the criterion's units. The unit counts there, 70, 162 and 218, miss the expected 80 +- 8 and 205 +- 20.5 for CP-RF and CP-noRF (VISp 218 is within 239 +- 23.9). With the code's filter (presence ratio above 0.9, noise units kept and split by their RF p values) the counts are 79, 206 and 239, within one unit of the paper's, with 78 significant bins, all CP-RF higher. The effect holds under both; the expected counts only under the variant, so the claim holds in part. Whether the criterion should take the code's filter is asked of a reader (questions/001051.yaml).

    How the test differed from the paper’s method

    • The unit counts match only with the code's filter (presence ratio above 0.9 and no noise filter), which the claims file names as a variant; the effect holds with both. Noise CP units now have RF p values, so the variant splits them by RF as the code does.

Not reproduced5

  • Not reproducedResults, High-quality multi-Neuropixels recordings, paragraph 5; Figure 4Duses data from outside the dataset

    The distributions of unit quality metrics (presence ratio, amplitude cutoff, ISI violations, SNR, isolation distance, drift, amplitude) are comparable for SHIELD and the 5-mm window.

    We found that the distribution of QC metrics was comparable with both surgical preparations (Figure 4D).

    Results, High-quality multi-Neuropixels recordings, paragraph 5; Figure 4D

    Reading (a), the code's Mann-Whitney test, finds all 7 metrics different (p 3.9e-292 to 3.9e-6), as the code's printed output does; reading (b), medians within 10%, holds for 5 of 7 (isi_violations ratio 0.79, amplitude_cutoff 0.89). Both readings give the same outcome. Effect sizes are small (rank-biserial at most 0.07 in absolute value) and the two metrics outside 10% are better for SHIELD.

  • Not reproducedSTAR Methods, Neuropixels recordings, Probe insertion, paragraph 1; Figure S4I

    Activity is transiently suppressed after probe insertion and recovers with a time constant of 25 to 30 minutes.

    we find that activity is transiently suppressed directly after probe insertion and gradually recovers with a time constant of 25-30 minutes (Figure S4I)

    STAR Methods, Neuropixels recordings, Probe insertion, paragraph 1; Figure S4I

    Activity recovers over the session (normalised rate 0.826 in minute 1, 0.999 at minute 30, 1.077 at the end, reading a), but the fitted time constant over the common 77.3 min from the first stimulus is 36.0 min (SE 2.3) under (a) and 35.4 min (SE 2.3) under (b), outside 22.5 to 33 min; per-session fits give a median of 34.8 min (IQR 23.1 to 52.5). Shorter windows give longer time constants, 41.2 to 42.2 min over the shortest active block (64 min), 45.9 to 47.1 at 60 min, 65.2 to 84.0 at 40 to 50 min, and no recovery at 30 min (tau unbounded); per-session fits over each session's active block give a median of 39.8 min (IQR 25.7 to 66.8), 26 of 87 sessions inside the range. No window gives a pooled tau inside the range, so the outcome does not rest on the window.

    How the test differed from the paper’s method

    • The fit (a - b exp(-t / tau) on 1-minute means, least squares) and the 87 sessions (the paper has 85) are not given by the paper.
    • The fit window is not given (the S4I legend plots the change over time from experiment start); the main fit uses the longest window all sessions share, 77.3 min from the first stimulus, which runs past the active block. Windows of 30 to 64 min and per-session active blocks are shown; all give tau above 33 min.
  • Not reproducedResults, SHIELD recordings reveal multi-regional subnetworks, paragraph 5; Figure S6

    Activity during post-stimulus VISp alpha events is weakly but significantly correlated with activity during the preceding stimulus (r = 0.77, p = 1.8e-6), as if events replay recent stimulation.

    Activity during post-stimulus alpha events was weakly but significantly correlated with activity during the preceding stimulus, indicating that alpha events tend to replay recent sensory stimulation (Figure S6; r = 0.77

    Results, SHIELD recordings reveal multi-regional subnetworks, paragraph 5; Figure S6

    What the paper reports

    • correlation, r = 0.77, p = 1.8 × 10⁻⁶

    Across sessions (7 under L1, as in the paper, 8 under L2; four points each) the correlation of evoked and event responses is r = 0.21 to 0.63 over 32 combinations of the readings and windows, below 0.693 in all; p < 0.05 in 18 of 32. Events are assigned to the last presentation before them, as the criterion says, which takes events that start during an image (473 of 1656 L2 E2 events over all sessions) and after the active block (64); keeping only events 0 to 0.5 s after the offset, the post-stimulus events of the S6 legend, gives r = 0.19 to 0.53, p < 0.05 in 14 of 32, none within tolerance. The per-unit variant gives r = 0.46 to 0.67 (0.52 to 0.66 with post-offset events only).

    How the test differed from the paper’s method

    • Run under the claims file's 16 combinations of area membership (L1, L2), event rule (E1, E2), random-time coherence (R1, R2) and session set (a, b); population windows are cut from one 10 ms grid per session. Events are detected on the smoothed rate, and the autocorrelogram threshold, CCGs and coherence use the unsmoothed rate, a detail taken from the code's printed counts quoted in the claims file's header (1653 VISp and 113 MOp qualifying events, code cell 12), which the unsmoothed autocorrelogram matches (1643 to 1704 and 119 to 122 under E2) and the smoothed one does not (3035 to 3426 and 152 to 170). The Benjamini-Hochberg family of the pair test holds brain areas only (not CP-RF, CP-noRF, VIS or unannotated labels); this changes significance for 4 of 734 pair results.
    • Rows are normalised by their maximum, a standard choice; sessions enter when each of the four images precedes a qualifying event.
  • Not reproducedResults, SHIELD recordings reveal multi-regional subnetworks, paragraph 7 (top of page 13); Figure 7Equestion to the reader

    In the 3 to 7 Hz band, time lags between VISp or MOp and their associated regions straddle zero; neither consistently leads or follows.

    Time lags between VISp and MOp and their associated regions in the 3–7 Hz band straddled zero lag, indicating that these areas did not consistently lead or follow activity in the other network nodes (Figure 7E).

    Results, SHIELD recordings reveal multi-regional subnetworks, paragraph 7 (top of page 13); Figure 7E

    On the code's test 94 of 102 pair results differ from zero after correction; only VISp to VISl under L1 does not. Median lags are -23 to +14 ms and the fraction of positive lags 0.12 (MOp to PO) to 0.97 (VISp to VISal), over 25 to 1143 events per pair. Whether the claim means that the median lag does not differ from zero (the criterion) or that the distribution of lags spans zero is asked of a reader (questions/001051.yaml).

    How the test differed from the paper’s method

    • Run under the claims file's 16 combinations of area membership (L1, L2), event rule (E1, E2), random-time coherence (R1, R2) and session set (a, b); population windows are cut from one 10 ms grid per session. Events are detected on the smoothed rate, and the autocorrelogram threshold, CCGs and coherence use the unsmoothed rate, a detail taken from the code's printed counts quoted in the claims file's header (1653 VISp and 113 MOp qualifying events, code cell 12), which the unsmoothed autocorrelogram matches (1643 to 1704 and 119 to 122 under E2) and the smoothed one does not (3035 to 3426 and 152 to 170). The Benjamini-Hochberg family of the pair test holds brain areas only (not CP-RF, CP-noRF, VIS or unannotated labels); this changes significance for 4 of 734 pair results.
  • Not reproducedResults, SHIELD recordings reveal multi-regional subnetworks, paragraph 4; Figures 7A and 7F

    Neither VISp nor MOp alpha events engage the hippocampus.

    Neither visual nor motor events appeared to engage the hippocampus.

    Results, SHIELD recordings reveal multi-regional subnetworks, paragraph 4; Figures 7A and 7F

    For VISp events DG is significantly and positively coherent in every combination (+0.012 to +0.032), with CA1, CA3, ProS, SUB or POST also under R2, and ProS under L1 E2 R1 and SUB under L2 E2 R1; for MOp events CA1 and DG under L2 E1 R2, and CA1 (+0.025, q = 0.044) under L1 E2 R2. 20 of 32 combinations and references have a significant positive hippocampal pair. The hippocampal effects are small next to the visual areas (+0.06 to +0.39).

    How the test differed from the paper’s method

    • Run under the claims file's 16 combinations of area membership (L1, L2), event rule (E1, E2), random-time coherence (R1, R2) and session set (a, b); population windows are cut from one 10 ms grid per session. Events are detected on the smoothed rate, and the autocorrelogram threshold, CCGs and coherence use the unsmoothed rate, a detail taken from the code's printed counts quoted in the claims file's header (1653 VISp and 113 MOp qualifying events, code cell 12), which the unsmoothed autocorrelogram matches (1643 to 1704 and 119 to 122 under E2) and the smoothed one does not (3035 to 3426 and 152 to 170). The Benjamini-Hochberg family of the pair test holds brain areas only (not CP-RF, CP-noRF, VIS or unannotated labels); this changes significance for 4 of 734 pair results.

Blocked6

  • BlockedResults, SHIELD recordings reveal multi-regional subnetworks, paragraph 6; Figures 7C and 7D· Methods underspecified

    During VISp events, VISp is tightly coupled to LP and VISl (sharp CCG peaks near zero lag and a 3 to 7 Hz coherence peak), but not to PO or SSp.

    During VISp events, VISp activity was tightly coupled to the secondary visual thalamus (LP) and higher-order lateral visual cortex (VISl), as evidenced by sharp peaks in the CCG at near-zero lag. VISp was not strongly coupled to somatosensory thalamus (PO) or SSp. Indeed, coherence analysis revealed a prominent peak in the 3–7 Hz band between VISp population spiking activity and LP and VISl, but not PO or SSp (Figure 7D)

    Results, SHIELD recordings reveal multi-regional subnetworks, paragraph 6; Figures 7C and 7D

    The CCG part holds in every combination (LP and VISl peak at 0 to 20 ms, heights 0.335 to 0.508, above PO and SSp at 0.055 to 0.092). The coherence part holds under R1 in all 8 combinations, but under R2 only with L2 E1: with L1 SSp is also significant (+0.008 to +0.011, q 0.0032 and 2.5e-5), with L2 E2 SSp is (+0.007, q 0.048 and 0.036; PO +0.011 at q 0.051). Every part is tested in every combination. The readings disagree, which the claims file's header makes methods underspecified. Open for the authors is how the random-time coherence is formed (the Methods' wording reads as R1, the code does R2).

    How the test differed from the paper’s method

    • Run under the claims file's 16 combinations of area membership (L1, L2), event rule (E1, E2), random-time coherence (R1, R2) and session set (a, b); population windows are cut from one 10 ms grid per session. Events are detected on the smoothed rate, and the autocorrelogram threshold, CCGs and coherence use the unsmoothed rate, a detail taken from the code's printed counts quoted in the claims file's header (1653 VISp and 113 MOp qualifying events, code cell 12), which the unsmoothed autocorrelogram matches (1643 to 1704 and 119 to 122 under E2) and the smoothed one does not (3035 to 3426 and 152 to 170). The Benjamini-Hochberg family of the pair test holds brain areas only (not CP-RF, CP-noRF, VIS or unannotated labels); this changes significance for 4 of 734 pair results.
  • BlockedResults, SHIELD recordings reveal multi-regional subnetworks, paragraph 6; Figures 7C and 7D· Methods underspecifiedverifier disagreed at first

    During MOp events, MOp is coupled to PO and SSp but not to LP or VISl.

    Conversely, MOp showed the opposite pattern: strong coupling to PO and SSp, but not to LP or VISl.

    Results, SHIELD recordings reveal multi-regional subnetworks, paragraph 6; Figures 7C and 7D

    The criterion needs MOp-PO to be significantly coherent. Under E1 (8 of 16 combinations) PO is untested (fewer than 25 events), so coherence_part is False there. Under E2 in all 8 combinations MOp is significantly coherent with PO (+0.15 to +0.20) and SSp (+0.19 to +0.25) and not with LP, and the CCGs of PO and SSp peak at 0 to 20 ms (0.20 to 0.33) against 0.03 to 0.06 for LP and VISl. VISl is untested under L1 and under L2 E1, and coherence_part counts an untested VISl as passing, so every part is tested and holds only under L2 E2 (4 combinations; VISl +0.005 and +0.027, q 0.89 and 0.34). The readings disagree, which the claims file's header makes methods underspecified. Changed from reproduced after verification round 3 agreed with the change proposed after round 2.

    How the test differed from the paper’s method

    • Run under the claims file's 16 combinations of area membership (L1, L2), event rule (E1, E2), random-time coherence (R1, R2) and session set (a, b); population windows are cut from one 10 ms grid per session. Events are detected on the smoothed rate, and the autocorrelogram threshold, CCGs and coherence use the unsmoothed rate, a detail taken from the code's printed counts quoted in the claims file's header (1653 VISp and 113 MOp qualifying events, code cell 12), which the unsmoothed autocorrelogram matches (1643 to 1704 and 119 to 122 under E2) and the smoothed one does not (3035 to 3426 and 152 to 170). The Benjamini-Hochberg family of the pair test holds brain areas only (not CP-RF, CP-noRF, VIS or unannotated labels); this changes significance for 4 of 734 pair results.
    • Under E1 the PO part, and under L1 and L2 E1 the VISl part, are untested (fewer than 25 events); all four parts are tested only under L2 E2 (4 of the 16 combinations).
  • BlockedResults, SHIELD recordings reveal multi-regional subnetworks, paragraph 8 (page 13; the text cites it as Figure 6F); Figure 7F· Methods underspecified

    Beyond visual cortex and thalamus, VISp alpha events enhance coherence of VISp with ACA, RSP and CP.

    In addition to visual corticothalamic regions, VISp alpha events enhanced coherence between VISp and medial cortical structures anterior cingulate area (ACA) and RSP, as well as CP

    Results, SHIELD recordings reveal multi-regional subnetworks, paragraph 8 (page 13; the text cites it as Figure 6F); Figure 7F

    What the paper reports

    • Wilcoxon signed-rank test, event against random coherence, p < 0.05, Benjamini/Hochberg-corrected (n = 34 CCF regions)

    ACA, RSP, CP, LGd, LP and VISl are each significant and positive in 14 of 16 combinations when any subdivision counts and 8 of 16 when all must (22 of 32 combination and mode rows); under L1 E2 R1 CP is not (q = 0.1), and with all subdivisions required ACAv or RSPagl or RSPd fail under R1. SSp has no subdivisions in these files; RSP has RSPagl, RSPd and RSPv, ACA has ACAd and ACAv, under both L1 and L2. The readings disagree.

    How the test differed from the paper’s method

    • Run under the claims file's 16 combinations of area membership (L1, L2), event rule (E1, E2), random-time coherence (R1, R2) and session set (a, b); population windows are cut from one 10 ms grid per session. Events are detected on the smoothed rate, and the autocorrelogram threshold, CCGs and coherence use the unsmoothed rate, a detail taken from the code's printed counts quoted in the claims file's header (1653 VISp and 113 MOp qualifying events, code cell 12), which the unsmoothed autocorrelogram matches (1643 to 1704 and 119 to 122 under E2) and the smoothed one does not (3035 to 3426 and 152 to 170). The Benjamini-Hochberg family of the pair test holds brain areas only (not CP-RF, CP-noRF, VIS or unannotated labels); this changes significance for 4 of 734 pair results.
  • BlockedResults, SHIELD recordings reveal multi-regional subnetworks, paragraph 8 (page 13; the text cites it as Figure 6F); Figure 7F· Methods underspecified

    MOp alpha events enhance coherence of MOp with SSp, PO, MRN and the deep layers of the superior colliculus (SCiw).

    Conversely, MOp events enhanced coherence between MOp and SSp/PO, as well as motor-related midbrain structures MRN and deep layers of the superior colliculus (SCiw).

    Results, SHIELD recordings reveal multi-regional subnetworks, paragraph 8 (page 13; the text cites it as Figure 6F); Figure 7F

    What the paper reports

    • Wilcoxon signed-rank test, event against random coherence, p < 0.05, Benjamini/Hochberg-corrected (n = 34 CCF regions)

    Under E2, SSp, PO, MRN and SCiw are significant and positive with L1 (R1 and R2) and L2 R2; with L2 R1 MRN is not (q = 0.11); under E1 PO, MRN and SCiw have too few events. MRN and SCiw each come from one session. The readings disagree.

    How the test differed from the paper’s method

    • Run under the claims file's 16 combinations of area membership (L1, L2), event rule (E1, E2), random-time coherence (R1, R2) and session set (a, b); population windows are cut from one 10 ms grid per session. Events are detected on the smoothed rate, and the autocorrelogram threshold, CCGs and coherence use the unsmoothed rate, a detail taken from the code's printed counts quoted in the claims file's header (1653 VISp and 113 MOp qualifying events, code cell 12), which the unsmoothed autocorrelogram matches (1643 to 1704 and 119 to 122 under E2) and the smoothed one does not (3035 to 3426 and 152 to 170). The Benjamini-Hochberg family of the pair test holds brain areas only (not CP-RF, CP-noRF, VIS or unannotated labels); this changes significance for 4 of 734 pair results.
  • BlockedResults, Fine-scale alpha rhythm coupling between visual cortex and striatum, paragraph 3; Figure 8D· Methods underspecifiedverifier disagreed at first

    CP-RF responses lag VISp by about 20 ms, and CP-noRF responses lag CP-RF by about another 20 ms.

    CP-RF lagged behind VISp responses by ... 20 ms, whereas CP-noRF had slower responses that lagged behind CP-RF units by an additional 20 ms

    Results, Fine-scale alpha rhythm coupling between visual cortex and striatum, paragraph 3; Figure 8D

    With latency as the first time the population mean reaches half its peak (b), CP-RF lags VISp by 16 ms and CP-noRF lags CP-RF by 20 ms, within 20 +- 5; with the time of peak (a) the lags are 34 and 9 ms, outside. Both filters (the Methods' QC and the code's) give the same values. The paper says only that CP-RF lagged VISp by about 20 ms and that CP-noRF had slower responses, and gives no latency definition; the claims file lists both readings without preferring either. The readings give different outcomes and nothing in the paper picks one, so the claim is methods underspecified. Half-peak latency had been adopted as an assumption after it was seen to match; changed from reproduced after verification round 3 agreed with the change proposed after round 2. Question for the authors - which latency measure was used.

    How the test differed from the paper’s method

    • Both latency readings of the claims file were run, half-peak time of the population mean (b) and time of the population peak (a); the paper gives no measure.
  • BlockedResults, Fine-scale alpha rhythm coupling between visual cortex and striatum, paragraph 4; Figure 8F· Methods underspecified

    During VISp alpha events, CP-RF units are more coherent with VISp in the 3 to 7 Hz band than CP-noRF units (p = 3.8e-14, 439 events).

    As predicted, CP-RF units displayed higher coherence with VISp in the 3–7 Hz band, indicating that alpha events coordinate functionally related cortico-striatal subnetworks.

    Results, Fine-scale alpha rhythm coupling between visual cortex and striatum, paragraph 4; Figure 8F

    What the paper reports

    • Wilcoxon signed-rank test, p = 3.8 × 10⁻¹⁴ (n = 439 events)

    Under R2 (the code) CP-RF coherence with VISp exceeds CP-noRF (0.096 to 0.110 against 0.042 to 0.052, paired Wilcoxon p 6.5e-11 to 1.2e-14); under R1 it does not (p 0.36 to 0.75). The readings disagree. The code's L2 E2 readings give 439 events, the paper's count, and p = 1.9e-10 under R2.

    How the test differed from the paper’s method

    • Run under the claims file's 16 combinations of area membership (L1, L2), event rule (E1, E2), random-time coherence (R1, R2) and session set (a, b); population windows are cut from one 10 ms grid per session. Events are detected on the smoothed rate, and the autocorrelogram threshold, CCGs and coherence use the unsmoothed rate, a detail taken from the code's printed counts quoted in the claims file's header (1653 VISp and 113 MOp qualifying events, code cell 12), which the unsmoothed autocorrelogram matches (1643 to 1704 and 119 to 122 under E2) and the smoothed one does not (3035 to 3426 and 152 to 170). The Benjamini-Hochberg family of the pair test holds brain areas only (not CP-RF, CP-noRF, VIS or unannotated labels); this changes significance for 4 of 734 pair results.

Not tested in a notebook 16

11 data missing from the dataset, 4 metadata insufficient, 1 methods underspecified

Blocked16

  • BlockedResults, Installation of SHIELD implant and QC evaluation, paragraph 3; Figure 3B· Data missing from the dataset

    Of 106 mice entering SHIELD surgery, 79 passed all quality checks, 14 passed after a flag that resolved and 13 failed, a final passing rate of 87.7%, on par with 90% (122 of 135) for the earlier 5-mm cranial window surgery.

    Of the 106 mice entering surgery for this study, 79 passed all quality checks with no issues, 14 passed with a flag that eventually resolved, and 13 mice failed ... The final passing rate of 87.7% is on par with the success rate for our previous 5 mm cranial window surgery

    Results, Installation of SHIELD implant and QC evaluation, paragraph 3; Figure 3B

    Surgery records and QC flags for the 106 mice entering surgery are not in the dandiset, which holds only the 27 recorded mice (99 sessions); neither the files, the dandiset metadata nor the code's tables hold them.

  • BlockedResults, Installation of SHIELD implant and QC evaluation, paragraph 4; Figure 3C· Data missing from the dataset

    Intrinsic signal imaging through the implant maps the retinotopic boundaries of VISp and many higher visual areas.

    We found that standard procedures successfully mapped retinotopic boundaries of VISp and many higher visual cortical areas (Figure 3C)

    Results, Installation of SHIELD implant and QC evaluation, paragraph 4; Figure 3C

    No intrinsic signal images or sign maps are in the dandiset; the files hold only an ISI-derived area label per unit (isi_label).

  • BlockedResults, Installation of SHIELD implant and QC evaluation, paragraph 5; Figures 3D and S3E· Data missing from the dataset

    The implanted left hemisphere shows little GFAP activation or loss of DAPI and NeuN staining compared with the unaltered right hemisphere (6 mice, 56 days after surgery).

    found little evidence of GFAP activation or cell loss, indicating that the implant did not compromise the underlying tissue

    Results, Installation of SHIELD implant and QC evaluation, paragraph 5; Figures 3D and S3E

    No histology (GFAP, DAPI, NeuN sections) is in the dandiset or the paper's repository.

  • BlockedResults, Installation of SHIELD implant and QC evaluation, paragraph 5· Data missing from the dataset

    Inserted probes rarely caused significant intracranial bleeding, in 3 of 547 tracks.

    We found that inserted probes rarely caused significant intracranial bleeding (3/547 tracks).

    Results, Installation of SHIELD implant and QC evaluation, paragraph 5

    Probe-track histology is not in the dandiset; the files give CCF positions of channels only, with no record of bleeding.

  • BlockedResults, Installation of SHIELD implant and QC evaluation, paragraph 6; Figures 3F to 3H· Data missing from the datasetuses data from outside the dataset

    In MRI, the 5-mm cranial window deforms the brain under the window, while the brain under SHIELD implants (250 and 400 micrometre inserts) is close to naive mice; the thicker insert trended toward less deformation.

    revealed clear brain deformation for the non-contoured 5-mm window over the visual cortex ... the area between surface contours for SHIELD mice was more similar to naive mice ... though the thicker design trended toward less deformation

    Results, Installation of SHIELD implant and QC evaluation, paragraph 6; Figures 3F to 3H

    What the paper reports

    • Wilcoxon rank-sum test, area between contours summed over the 4 most posterior sections, p < 0.05, Benjamini-Hochberg-corrected (n = naive 3, SHIELD 250 4, SHIELD 400 5, 5-mm window 5 mice)

    No MRI volumes or contours are in the dandiset, and the naive-mouse data of Qiu et al. 2018 would only be the comparison; the measurement itself is missing.

  • BlockedResults, Installation of SHIELD implant and QC evaluation, paragraph 7; Figure 3I· Data missing from the datasetuses data from outside the dataset

    SHIELD mice and 5-mm window mice learn the change detection task at similar rates (median sessions to criterion 12.5 against 14, p = 0.17).

    Mice with either implant had similar learning rates (Figure 3I; median number of sessions to criterion: SHIELD 12.5 ± 2; 5-mm window 14 ± 3; p = 0.17, Wilcoxon rank-sum test)

    Results, Installation of SHIELD implant and QC evaluation, paragraph 7; Figure 3I

    What the paper reports

    • Wilcoxon rank-sum test, p = 0.17

    Training histories (sessions to criterion) of SHIELD and 5-mm window mice are not in the dandiset or the paper's repository; the dandiset holds only the recording sessions.

  • BlockedResults, High-quality multi-Neuropixels recordings, paragraph 3; Figure 4B· Metadata insufficient

    Insertions along the same trajectory (same hole and probe orientation) enter the brain at similar CCF locations, with a median absolute deviation of 150 micrometres medial-lateral and 200 micrometres anterior-posterior over 15 trajectories.

    for 15 unique trajectories, we found the median absolute deviation in CCF coordinates was 150 mm in the medial-lateral axis and 200 mm in the anterior-posterior axis

    Results, High-quality multi-Neuropixels recordings, paragraph 3; Figure 4B

    The hole and probe orientation of each insertion are not recorded, so insertions cannot be grouped into trajectories; the files give CCF channel positions only.

  • BlockedResults, High-quality multi-Neuropixels recordings, paragraph 4; Figure 4C· Methods underspecified

    Discontinuous or abrupt shifts in unit depth in the drift map occurred in 5.1% of insertions (23 of 454).

    We rarely observed discontinuous or abrupt shifts in the depth of units in the drift map (5.1% of insertions; 23/ 454)

    Results, High-quality multi-Neuropixels recordings, paragraph 4; Figure 4C

    The paper judged drift maps by eye and gives no rule for an abrupt shift, and the files do not hold per-spike depths (only unit peak channels and spike amplitudes), from which drift maps are drawn.

  • BlockedResults, High-quality multi-Neuropixels recordings, paragraph 7; Figure 4F· Data missing from the dataset

    VISp receptive fields recorded through adjacent holes on successive days lie where the ISI-based retinotopic map predicts.

    the region of visual space preferred on each day matched the expected location indicated by the ISI-based retinotopic map

    Results, High-quality multi-Neuropixels recordings, paragraph 7; Figure 4F

    The ISI retinotopic maps and the insertion images needed to predict each insertion's retinotopic location are not in the dandiset.

  • BlockedFigure S4D and S4E legends; STAR Methods, Probe insertion, paragraph 1· Metadata insufficient

    Retracting probes 100 micrometres after insertion reduces unit drift (unit means 24.48 against 30.25 micrometres; session means 24.23 against 30.56).

    Units from sessions where probes were retracted had lower drift (mean = 24.48 μm) than those from sessions where probes were not retracted (mean = 30.25 μm; Kolmogrov-Smirnov test, p < .001)

    Figure S4D and S4E legends; STAR Methods, Probe insertion, paragraph 1

    What the paper reports

    • unit drift, retracted against not retracted sessions: Kolmogorov-Smirnov test, p < 0.001 (n = 25,896 and 76,851 units)

    No file, and none of the code's tables, records which sessions had probes retracted 100 um; the 23 retracted sessions cannot be identified. The drift and yield measures are in the files.

  • BlockedFigure S4F legend· Metadata insufficient

    Retracted and non-retracted sessions record similar numbers of good units (means 1125 against 1280, p = 0.318).

    Number of good units recorded per session is similar across retracted and non-retracted sessions. ... Distributions did not differ significantly (Kolmogrov-Smirnov test, p = 0.318).

    Figure S4F legend

    What the paper reports

    • Kolmogorov-Smirnov test, p = 0.318

    No file, and none of the code's tables, records which sessions had probes retracted 100 um; the 23 retracted sessions cannot be identified. The drift and yield measures are in the files.

  • BlockedFigure S4G and S4H legends· Metadata insufficient

    Sharpened probes show drift comparable to unsharpened probes and a lower insertion failure rate (p = 0.0004).

    Sharpened Neuropixels probes show comparable drift to unsharpened probes. ... Sharpened Neuropixels probes have a lower insertion fail rate than unsharpened probes (p = 0.0004, Fisher’s exact test).

    Figure S4G and S4H legends

    What the paper reports

    • insertion failure rate, sharpened against unsharpened: Fisher's exact test, p = 4 × 10⁻⁴ (n = 32 insertions before and 112 after sharpening)

    No file records which probes were sharpened or the failed insertion attempts (failed probes leave no record, or records of unannotated probes); the code's tables do not either.

  • BlockedResults, Dual-hemisphere SHIELD implant, paragraph 1 (end, page 10); Figures S3B and S3D· Data missing from the dataset

    Dura-Gel can stay on the implant for at least 2 weeks without visible brain deterioration or a drop in unit yield.

    Dura-Gel can remain in place for at least 2 weeks without visible signs of brain deterioration (Figure S3B) or a drop in unit yield (Figure S3D).

    Results, Dual-hemisphere SHIELD implant, paragraph 1 (end, page 10); Figures S3B and S3D

    No session records Dura-Gel use or days since its application, and the 24 Dura-Gel insertions cannot be identified; the paper used Dura-Gel for the optotagging and bilateral experiments, whose mice are not in the dandiset.

  • BlockedResults, Dual-hemisphere SHIELD implant, paragraph 1; Figures 5C and 5D· Data missing from the dataset

    Simultaneous Neuropixels recordings from both hemispheres are feasible with the dual-hemisphere implant.

    In preliminary experiments, we demonstrate the feasibility of simultaneous, bilateral Neuropixels recordings with the SHIELD method (Figures 5C and 5D).

    Results, Dual-hemisphere SHIELD implant, paragraph 1; Figures 5C and 5D

    No dual-hemisphere session is in the dandiset; no annotated unit lies more than 550 um right of the midline (largest z 6250 against midline 5700), and those that do are in midline structures (LSr, ACB, STR, TTd).

  • BlockedResults, SHIELD is compatible with optogenetics, paragraph 2; Figures 6B and 6C· Data missing from the dataset

    Blue light through the implant in a VGAT-ChR2 mouse silences visually evoked activity of VISp regular-spiking units at the recording site, with activity near baseline 2 mm away, and the effect depends on distance and power.

    Stimulusevoked activity of regular spiking (RS) units in VISp was silenced when a 2-mW laser spot was targeted to the recording location but returned to near baseline at a distance of 2 mm (Figure 6B). The dependence of this silencing effect on distance and laser power (Figure 6C)

    Results, SHIELD is compatible with optogenetics, paragraph 2; Figures 6B and 6C

    The VGAT-ChR2 mouse is not in the dandiset; its genotypes are Sst-IRES-Cre;Ai32, Vip-IRES-Cre;Ai32, Ai32 and wild type, and the optotagging tables hold the Sst and Vip ChR2 tagging at 0.95 to 2.05 power levels, not the cortical silencing protocol.

  • BlockedResults, SHIELD is compatible with optogenetics, paragraph 3; Figures 6E and 6F· Data missing from the dataset

    Red light through the implant evokes reliable short-latency spikes from ChrimsonR-tagged Vglut2 midbrain units, found up to 4 mm below the brain surface.

    Photo-stimulation with pulses (10 ms) of red light (638 nm) evoked reliable, short-latency spiking responses from units recorded in the midbrain (Figure 6E). Such optotagged units were found up to 4 mm below the surface of the brain

    Results, SHIELD is compatible with optogenetics, paragraph 3; Figures 6E and 6F

    The Vglut2-IRES-Cre mouse with ChrimsonR in MRN is not in the dandiset (see genotypes); no session has 638 nm, five-pulse 20 Hz trains.

Findings about the data

What attempts found about the dataset itself, for its owner and the archive.

  • Metadata errorIn 6 of the 87 sessions the paper analysed (1176989662, 1178460518, 1178693650, 1179911454, 1180266229, 1181096406) the units under each unit id differ from the paper's units table at the cited code commit (max_drift and firing_rate differ for 55 to 99.8% of units, quality for 14 to 38%, and receptive-field p values recomputed from the files are uncorrelated with the table's; the files' spike counts fit the files' firing rates but not the table's). The dandiset seems to hold a different spike sorting or unit assignment for these sessions than the one the paper used.
  • Data missingIn 5 of those sessions (1176989662, 1178693650, 1179911454, 1180266229, 1181096406) the paper's units table has 328 units, 133 of them in CP, that the session files do not hold. Each session's are on one probe of the table (probe_id 88, 46, 121, 133 and 139, with 38 to 114 of these units) whose probe_id is not among the file's and on which the file has no unit; none of their peak channels is a peak channel of a file's unit.
  • Selection not documentedThe 87 sessions passing ephys QC are listed only in the paper's code; the files and the dandiset metadata do not mark them, and the 12 other sessions are not explained. In the 87 sessions the files also hold 40 probe recordings (2004 units, 1873 on unannotated channels) that the paper's tables leave out, and nothing marks them.
  • Description missingNo file records the implant hole or probe orientation of an insertion, which probes were sharpened, which sessions had probes retracted 100 um, Dura-Gel use, or failed insertion attempts, so Figures 4B (trajectories), S4B, S4D to S4H and S3D cannot be tested.
  • Data missingThe dandiset holds none of the Figure 3 material (surgery records, ISI images, histology, MRI, training histories), the dual-hemisphere sessions of Figure 5, or the VGAT-ChR2 and Vglut2-Cre mice of Figure 6; its 27 mice are all single-hemisphere change-detection mice.
  • Data missingSessions 1187475832 and 1202438738 have no eye tracking. In the others the running and pupil streams follow the stimulus and trial clock (running slows after hits from a median 0.225 s after the change, the pupil constricts after white flashes from a median 0.35 s), but in 5 sessions for running and 3 for the pupil the contrasts are too weak to show it, and 2 sessions (1173189336, 1230732154) have no running contrast, as they have no hit or no miss trials; in all, alignment of at least one stream is not shown in 12 of the 99 sessions.
  • Paper inconsistentThe Methods describe 8 natural images in the change detection task; every session holds four image stimuli (three natural images and im115_r at 0.7 contrast) besides omitted flashes.
  • Methods not availableFor the alpha-event analyses (Figures 7, 8E, 8F and S6), please confirm (1) how the random-time coherence was formed, both areas at the random time (as the Methods read) or the reference area at the event against the other area at the random time (as the code does); (2) whether area membership used the ISI labels and exact CCF labels or the code's substring match (which puts POST units into PO and VISpm into VISp); (3) the event rule, four maxima (Methods) or three in a window with more than three (code). The outcomes of fig7cd-visp-coupling, fig7f-visp-network, fig7f-mop-network and fig8f-cp-rf-higher-coherence depend on these choices; the code's choices reproduce the 439 events of Figure 8F.
  • Methods not availablePlease state the latency measure of Figure 8D (half-peak time and time of peak give different lags, so the claim is blocked here), the fit, its window and the normalisation behind the 25 to 30 min recovery time constant of Figure S4I (fits here give 35 to 36 min over 77 min and longer time constants over shorter windows), and the windows and row normalisation of Figure S6F.

Questions

Where the author or a verifier believed the claims file misread the paper, they asked, citing the paper, and a reader who had not seen the data answered from the paper alone, amending the claims file where it was wrong. The author then ran the claim as answered. Attempts before protocol pilot-11 also asked here for method details the claims file did not give.

  • openasked 2026-11-01about fig7e-lags-straddle-zero

    The criterion judges "straddled zero" with the code's Wilcoxon signed-rank test of the per-event lags against zero, a pair counting as straddling zero only when the test is not significant. The paper names no test, and the Figure 7E legend shows a distribution of lags. Does the claim say that the distribution of lags spans zero (a descriptive reading, judged on the median lag or the spread of lags about zero), or that the median lag does not differ from zero?

    Passage. Time lags between VISp and MOp and their associated regions in the 3-7 Hz band straddled zero lag, indicating that these areas did not consistently lead or follow activity in the other network nodes (Figure 7E). (Results, paragraph 7 of the alpha section, top of page 13); (E) Distribution of time lags in the 3-7 Hz band during VISp-triggered (left) and MOp-triggered (right) events. (Figure 7 legend)

    Why it matters. With 25 to 1143 events per pair the signed-rank test is significant for 94 of 102 pair results (the code's own printed output is significant for all seven pairs), so the claim is not reproduced as the criterion has it. Under a descriptive reading the median lags are small, -23 to +14 ms against a 3 to 7 Hz cycle of 140 to 330 ms, but the fraction of positive lags reaches 0.97 for VISp-VISal and 0.12 for MOp-PO, so the outcome under that reading depends on the rule chosen.

    No reader has answered it yet.
  • openasked 2026-11-01about fig8d-cp-rf-larger-response

    The criterion takes the Methods' QC units (presence ratio above 0.95, isolation and contamination thresholds, noise units removed) and names the code's filter (presence ratio above 0.9, no noise filter) as a variant. The figure's unit counts (80 CP-RF, 205 CP-noRF, 239 VISp) are matched by the code's filter here (79, 206 and 239) and not by the Methods' QC (70, 162 and 218). Should the code's filter be the criterion's unit selection for Figure 8D, and the expected counts judged on it?

    Passage. CP RF 80 units, CP no RF 205, VISp 239 (Figure 8D panel labels); (D) Population visual response of CP units with significant RFs (red), CP units without RFs (blue), and VISp units to a full-field natural image presentation for 250 ms (Figure 8 legend); the code's tensor_utils.apply_good_units_filter keeps presence_ratio > 0.9 with no noise filter (claims file header).

    Why it matters. Under the Methods' QC the CP-RF and CP-noRF counts are outside tolerance (80 +- 8, 205 +- 20.5) and the claim is partially reproduced; under the code's filter all three counts are within one unit of the paper's and the response difference holds (78 significant bins, all CP-RF higher), which would make it reproduced. The same choice decides the units of fig8d-response-latencies.

    No reader has answered it yet.

Notebooks

Every registered entry that uses this dataset. An entry can hold several notebooks, and can use more than one dataset.

Reproduction SHIELD multi-Neuropixels recordings and alpha events (Bennett et al. 2024)

Tests the findings of Bennett, Ouellette, Ramirez et al. (2024) on dandiset 001051, which holds 99 change-detection sessions of 27 mice recorded with up to six Neuropixels probes through a SHIELD implant. It checks the paper's session list and unit table against the files, the unit yield, quality and stability across recording days, receptive fields and visual responses, and the alpha events of VISp and MOp with their coherence networks and the striatal subnetworks. The surgery, imaging, histology, MRI and optogenetics material of Figures 3, 5 and 6 is not in the dandiset. All files

Verification

  • Round 1
    2026-11-01
    Agreed with 43 of 46 outcomes. Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook). Reviewed: Round 1. The first executed notebook with the conclusion cells and the summary removed, the helper module and the claims file, with the paper. The verifier queried the archive metadata for the blocked claims and tested find_events and rf_pvalues on synthetic inputs. Its report had no attempt record and gave its own outcome for each claim; 43 of 46 agree with the record as it then stood.
    • fig7cd-mop-coupling: the verifier’s outcome was “Blocked”. Holds in all 8 E2 combinations, coherence and CCG. Under E1, PO has fewer than 25 events and is not tested (methods_underspecified). An area that is absent (fewer than 25 events) counts as not significant in the negative parts; MOp-VISl under L1 is never tested.
    • fig8d-cp-rf-larger-response: the verifier’s outcome was “Partially reproduced”. The response test holds: 53 significant bins, all with CP-RF higher. Under the Methods QC filter the counts are 70 and 162, outside 80 +- 8 and 205 +- 20.5; only the code's presence 0.9 variant gives 76, 209 and 239. Noise CP units have no RF p value, because rf_pvalues covers only good units, so they fall into CP-noRF in the variant.
    • fig8d-response-latencies: the verifier’s outcome was “Blocked”. Half-peak reading: lags 16 and 20 ms, within tolerance. Peak reading: 34 and 9 ms (QC) or 43 and 0 ms (variant), outside it (methods_underspecified).
  • Round 2
    2026-11-01
    Agreed with 44 of 46 outcomes. Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook). Reviewed: Round 2. The full executed notebook with the conclusion cells and the summary, the helper module, the README, the attempt record and the claims file, with the paper. The verifier agreed with 44 of the 46 outcomes and would block fig7cd-mop-coupling and fig8d-response-latencies as methods_underspecified.
    • fig7cd-mop-coupling: the verifier’s outcome was “Blocked”. Would block as methods_underspecified. The criterion's 'reproduced' condition needs MOp-PO to be significant. Cell 78 prints coherence_part False in all 8 E1 combinations, because PO is absent there; VISl is also untested in all L1 combinations and under L2 E1. All four parts are tested only under L2 E2, 4 of the 16 combinations. The claims file's rule is that every alpha claim is judged under each combination of readings, and the claim is methods_underspecified when they disagree; here the E1 readings do not reproduce the claim, so they disagree with E2. 'No tested part contradicts' is true, but the criterion does not use that test.
    • fig8d-response-latencies: the verifier’s outcome was “Blocked”. Would block as methods_underspecified. The half-peak assumption is stated, but it is weakly grounded. The paper says only that the 'response time course ... lagged' and that CP-noRF had 'slower responses', which does not pick an onset measure over a peak measure. The claims file offers both readings without preferring either. The half-peak reading was adopted after it turned out to be the one that matches: peak lags are 34 and 9 ms, half-peak lags 16 and 20 ms (cell 99).
  • Round 3
    2026-11-01
    Agreed with 2 of 2 outcomes. Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook). Reviewed: Round 3. The two changes of outcome proposed after round 2 (fig7cd-mop-coupling and fig8d-response-latencies) and the analysis behind them in the re-executed notebook, with the claims file and the paper. The verifier agreed with both.

Method differences

How the tests differed from the paper’s method or data, and the claims each applies to.

  • Run under the claims file's 16 combinations of area membership (L1, L2), event rule (E1, E2), random-time coherence (R1, R2) and session set (a, b); population windows are cut from one 10 ms grid per session. Events are detected on the smoothed rate, and the autocorrelogram threshold, CCGs and coherence use the unsmoothed rate, a detail taken from the code's printed counts quoted in the claims file's header (1653 VISp and 113 MOp qualifying events, code cell 12), which the unsmoothed autocorrelogram matches (1643 to 1704 and 119 to 122 under E2) and the smoothed one does not (3035 to 3426 and 152 to 170). The Benjamini-Hochberg family of the pair test holds brain areas only (not CP-RF, CP-noRF, VIS or unannotated labels); this changes significance for 4 of 734 pair results.fig7a-mop-higher-frequency fig7b-low-arousal fig7b-visp-offset-locked figs6-replay fig7cd-visp-coupling fig7cd-mop-coupling fig7e-lags-straddle-zero fig7f-visp-network fig7f-mop-network fig7-hippocampus-not-engaged fig8e-cp-rf-more-modulated fig8f-cp-rf-higher-coherence
  • A link claim added by the author; the code's notebook and tables are matched to the dandiset by session and unit identifiers.link-code-tables
  • Holes per mouse not tested (no hole per insertion in the dandiset or the code's tables).figs4ab-insertions-per-mouse
  • The dandiset holds 40 probe recordings in the 87 sessions that the paper's tables leave out; nothing in the files marks them, so insertions are counted as probes with units, with the code's 467 shown.scale-mice-sessions-insertions
  • SHIELD units include 202 more non-noise units than the paper's count, from the 40 probe recordings the paper's tables leave out (143,828 non-noise units on the 467 insertions in the code's table, 202 on the other 40). The code's own table has 144,068 for the 87 sessions; outside the 6 sessions where it describes other units it and the files both give 135,334.fig4d-unit-drift
  • RF p values are recomputed with the AllenSDK chi-square bootstrap (1000 resamplings) with their own seed; they match the code's table (r = 0.9990) where units agree.fig4ef-rf-fraction-stable
  • The fit (a - b exp(-t / tau) on 1-minute means, least squares) and the 87 sessions (the paper has 85) are not given by the paper.figs4i-insertion-suppression
  • The fit window is not given (the S4I legend plots the change over time from experiment start); the main fit uses the longest window all sessions share, 77.3 min from the first stimulus, which runs past the active block. Windows of 30 to 64 min and per-session active blocks are shown; all give tau above 33 min.figs4i-insertion-suppression
  • Reading (b) restricts the spectrum peak to 1 to 20 Hz, a standard choice; the zero-padded spectrum (2048 points) is a standard interpolation shown as a variant.fig7a-mop-higher-frequency
  • Diameter is the square root of window-mean area; two sessions without eye tracking contribute no pupil values.fig7b-low-arousal
  • The stimulus before an event is the last active image presentation, as the code's stimulus-trials table holds active presentations.fig7b-visp-offset-locked
  • Rows are normalised by their maximum, a standard choice; sessions enter when each of the four images precedes a qualifying event.figs6-replay
  • Under E1 the PO part, and under L1 and L2 E1 the VISl part, are untested (fewer than 25 events); all four parts are tested only under L2 E2 (4 of the 16 combinations).fig7cd-mop-coupling
  • RF significance recomputed here (link-code-tables); the CCF z axis orientation is checked against VISp (3050) and midline structures (5300).fig8bc-cp-rf-location
  • The unit counts match only with the code's filter (presence ratio above 0.9 and no noise filter), which the claims file names as a variant; the effect holds with both. Noise CP units now have RF p values, so the variant splits them by RF as the code does.fig8d-cp-rf-larger-response
  • Both latency readings of the claims file were run, half-peak time of the population mean (b) and time of the population peak (a); the paper gives no measure.fig8d-response-latencies

Provenance

Attempt record
001051-bennett-2024-01
Paper
SHIELD: Skull-shaped hemispheric implants enabling large-scale electrophysiology datasets in the mouse brain
Date
2026-11-01
Made by
An agent, run by CatalystNeuro
Dataset
DANDI:001051
Version
0.260922.1827 (doi:10.48324/dandi.001051/0.260922.1827)
Run on
GitHub Actions
Harness
Claude Code
Model
claude-opus-5-5
Protocol
pilot-13

Transcripts of the agent runs

Each run’s prompt, messages and tool calls, with long output cut and the text of the paper left out, and its final report.