Datasets / CRCNS:ac-2

Whole cell recordings from neurons in the primary auditory cortex of rat in response to pure tones

electrophysiological Norway rat 270 MB 3 files
  • 14 reproduced
  • 7 partially reproduced
  • 2 not reproduced

Tested on dataset version unversioned

View on CRCNS Paper JSON How to read outcomes

Claims

From the paper 10.1152/jn.00197.2004

Reproducing DeWeese and Zador (2004) from CRCNS ac-2 23

Notebook reproduce_deweese_2004.ipynb

Reproduced14

  • ReproducedResults, Trial-to-trial response variability, paragraph 1; Figure 3A and inset

    For the most reliable neuron, the responses to 20 presentations of a 32-kHz tone cluster tightly about the mean (q = 0.23 mV), and its variability averaged over the 26 tones with a mean PSP above 3 mV is 0.88 mV.

    The total variability of the responses shown in Fig. 3a was 0.23 mV. Averaging across tones, the total variability for this neuron was 0.88 mV

    Results, Trial-to-trial response variability, paragraph 1; Figure 3A and inset

    The neuron with the lowest <q>tones is 022801md01b, 0.876 mV over 26 tones, 20 repetitions; q is 0.229 mV for 31,118 Hz (reading a) and 0.648 mV for 34,448 Hz (reading b). The same neuron and values under both spike rules and both peak windows; with the variance over N 0.832 and 0.217 mV.

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
  • ReproducedResults, Trial-to-trial response variability, paragraphs 1 and 2; Figure 3Brests on an assumption

    Across the 33 neurons the tone-averaged variability index ranges over more than an order of magnitude, from below 1 to above 10 mV; about a fifth (7/33) have low variability (1 to 2 mV) and about half (18/33) have high variability (4 to 12 mV).

    Across the population, the variability index ranged over more than an order of magnitude, from �1 to �10 mV; about half (18/33) of the neurons showed a high variability index (4–12 mV; Fig. 3b).

    Results, Trial-to-trial response variability, paragraphs 1 and 2; Figure 3B

    <q>tones ranges 0.88 to 11.21 mV. 18 neurons lie between 4 and 12 mV. Between 1 and 2 mV strictly there are 6, outside 7 +- 0.7; under the criterion's second reading (q < 2 mV) there are 7, so both counts hold under that reading. With the claims file's spike rule the counts are 5 (6) and 20, which would give not reproduced; with the peak in the culling window (1b) 7 (8) and 15, which would give partially reproduced. With the -20 mV rule at any rise 5 (6) and 18, with spikethresh keeping the slow events 6 (7) and 20, each of which would give partially reproduced. In 2-mV bins 7, 8, 10, 4, 2 and 2 neurons, against 7, 8, 11, 3, 2 and 2 read from Figure 3B.

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • Spike trials are found with the data set's per-neuron spikethresh above the trace's baseline, an assumption (the paper gives no rule). The claims file's rule, a fast rise (above 10 mV/ms, a standard choice) to above -20 mV after the 12 mV junction potential correction, is shown as a reading; it misses slow depolarizations to about -10 mV (regenerative events or large PSPs), broad spikes that cross -20 mV with a rise below 10 mV/ms, and narrow spikes that peak below -20 mV. The -20 mV rule at any rise and spikethresh with the slow events kept (rise above 5 mV/ms) are shown as readings.
    • Open details as the claims file's header lists them; main readings 1a (peak searched over the whole trace after onset), 2a, 3 (N - 1), 4a, 5a, 7 (20 shuffles); the others shown.
    • The low count matches under the criterion's second reading, q < 2 mV.
  • ReproducedFigure 7A legend; Methods, LFP-based culling and PSP-based culling procedures, paragraph 3rests on an assumption

    The mean total variability index across neurons is 4.6 mV.

    total variability ��q�neurons �4.6 �0.5 mV, n �33 neurons

    Figure 7A legend; Methods, LFP-based culling and PSP-based culling procedures, paragraph 3

    <q>neurons = 4.58 +- 0.47 mV (SE, n = 33). With the claims file's spike rule 5.35 mV (outside 4.1 to 5.1); with the peak in the culling window 4.22, with the variance over N 4.29. With the -20 mV rule at any rise 4.43 mV (within), with spikethresh keeping the slow events 5.11 (just outside).

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • Spike trials are found with the data set's per-neuron spikethresh above the trace's baseline, an assumption (the paper gives no rule). The claims file's rule, a fast rise (above 10 mV/ms, a standard choice) to above -20 mV after the 12 mV junction potential correction, is shown as a reading; it misses slow depolarizations to about -10 mV (regenerative events or large PSPs), broad spikes that cross -20 mV with a rise below 10 mV/ms, and narrow spikes that peak below -20 mV. The -20 mV rule at any rise and spikethresh with the slow events kept (rise above 5 mV/ms) are shown as readings.
    • Open details as the claims file's header lists them; main readings 1a (peak searched over the whole trace after onset), 2a, 3 (N - 1), 4a, 5a, 7 (20 shuffles); the others shown.
  • ReproducedResults, Partitioning variability based on the LFP, paragraph 5rests on an assumption

    LFP-based culling reduces the mean variability index by 35%, from 4.8 to 3.1 mV, across the 29 neurons with an LFP.

    Across our sample, LFP-based culling reduced the mean variability by 35% (from 4.8 �0.5 to 3.1 �0.1 mV; n �29 neurons)

    Results, Partitioning variability based on the LFP, paragraph 5

    29 neurons carry the LFP flag. <q> is 4.80 +- 0.52 mV before and 3.19 +- 0.43 mV after LFP-based culling (a 33% fall), the second within tolerance by 0.01 mV. Reading 2b (3.12), reading 5b (3.04) and heights re-measured after culling (3.07) keep both within; the peak in the culling window (1b, 2.99) and the variance over N (2.94) put the second outside, which would not give reproduced. With the claims file's spike rule 5.41 and 3.75 mV, both outside; with the -20 mV rule at any rise 4.63 and 3.13 (both within); with spikethresh keeping the slow events 5.22 and 3.41 (the second outside).

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • Spike trials are found with the data set's per-neuron spikethresh above the trace's baseline, an assumption (the paper gives no rule). The claims file's rule, a fast rise (above 10 mV/ms, a standard choice) to above -20 mV after the 12 mV junction potential correction, is shown as a reading; it misses slow depolarizations to about -10 mV (regenerative events or large PSPs), broad spikes that cross -20 mV with a rise below 10 mV/ms, and narrow spikes that peak below -20 mV. The -20 mV rule at any rise and spikethresh with the slow events kept (rise above 5 mV/ms) are shown as readings.
    • Open details as the claims file's header lists them; main readings 1a (peak searched over the whole trace after onset), 2a, 3 (N - 1), 4a, 5a, 7 (20 shuffles); the others shown.
    • LFP-based culling follows PSP-based culling and is run on the trials it keeps, its mean LFP, threshold and scale factors recomputed over them (Figure 6 legend, "we repeat the above procedure after first performing the PSP-based culling procedure"; Figure 7B legend, "PSP-based culling followed by the LFP-based culling and estimation procedures"). Deciding both on the full ensemble is shown as a reading. Heights are measured once on the full ensemble and not re-measured after culling (a choice made here; the Methods say q is "recalculated"); heights taken again against the mean of the kept trials are shown as a reading.
    • The outcome also rests on detail 1, reading a (peak searched over the whole trace after onset).
  • ReproducedMethods, LFP-based estimation procedure, paragraph 1; Results, Partitioning variability based on the LFP, paragraph 5; Figure 5B

    After LFP-based culling, PSP height still correlates with LFP height across trials for many tones, as in the 60 unculled responses to one tone of the Figure 5 neuron.

    For many tones, we found that there was still a substantial correlation between the LFP and PSP for the ensemble of trials that remained after LFP-based culling (e.g., Fig. 5b)

    Methods, LFP-based estimation procedure, paragraph 1; Results, Partitioning variability based on the LFP, paragraph 5; Figure 5B

    Over 396 included tones of 29 neurons, after LFP-based culling, PSP height correlates positively with the LFP scale factor in 81% of tones and with p < 0.05 in 32% (5% expected by chance); with the LFP peak height 75% and 26%. The Figure 5 neuron (020701md01b, 63 repetitions) has two tones with 60 trials left, 31,118 and 5,527 Hz, r = 0.70 and 0.65. With the claims file's spike rule 405 tones, 79% and 31%. The -20 mV rule at any rise and spikethresh keeping the slow events give 401 tones, 80% and 31%, and 79% and 32%; heights re-measured after culling give the main values.

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • Spike trials are found with the data set's per-neuron spikethresh above the trace's baseline, an assumption (the paper gives no rule). The claims file's rule, a fast rise (above 10 mV/ms, a standard choice) to above -20 mV after the 12 mV junction potential correction, is shown as a reading; it misses slow depolarizations to about -10 mV (regenerative events or large PSPs), broad spikes that cross -20 mV with a rise below 10 mV/ms, and narrow spikes that peak below -20 mV. The -20 mV rule at any rise and spikethresh with the slow events kept (rise above 5 mV/ms) are shown as readings.
    • The Figure 5 neuron is the one with 63 repetitions; two of its tones have 60 trials left.
  • ReproducedResults, Partitioning variability based on PSP shape, paragraph 4; Figure 7B and 7Crests on an assumption

    PSP-based culling followed by LFP-based culling and estimation reduces variability by as much as 85%, to a best estimate of private variability of 1.6 mV across 28 neurons, comparable and low for all neurons.

    Combining both of these approaches resulted in an even greater reduction in variability (qPSP-culled,LFP-culled&estimation �1.6 �0.2 mV; Fig. 7, b and c)

    Results, Partitioning variability based on PSP shape, paragraph 4; Figure 7B and 7C

    <q PSP-culled,LFP-culled&estimation> = 1.48 +- 0.17 mV over 28 neurons; the largest reduction from total variability is 84.6%. Readings of details 1, 2, 4 and 5 give 1.42 to 1.78 mV and heights re-measured after culling 1.51 mV, all within; the variance over N (the claims file's variant of detail 3) gives 1.36, just outside. Deciding both cullings on the full ensemble gives 1.51 mV and 87.8%. With the claims file's spike rule 2.01 mV, outside; with the -20 mV rule at any rise 1.56 and with spikethresh keeping the slow events 1.72, both within. Values are 0.31 to 2.80 mV except 110900md05 (4.52 mV, three tones); SD 0.88 mV against 2.85 mV for total variability.

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • Spike trials are found with the data set's per-neuron spikethresh above the trace's baseline, an assumption (the paper gives no rule). The claims file's rule, a fast rise (above 10 mV/ms, a standard choice) to above -20 mV after the 12 mV junction potential correction, is shown as a reading; it misses slow depolarizations to about -10 mV (regenerative events or large PSPs), broad spikes that cross -20 mV with a rise below 10 mV/ms, and narrow spikes that peak below -20 mV. The -20 mV rule at any rise and spikethresh with the slow events kept (rise above 5 mV/ms) are shown as readings.
    • Open details as the claims file's header lists them; main readings 1a (peak searched over the whole trace after onset), 2a, 3 (N - 1), 4a, 5a, 7 (20 shuffles); the others shown.
    • LFP-based culling follows PSP-based culling and is run on the trials it keeps, its mean LFP, threshold and scale factors recomputed over them (Figure 6 legend, "we repeat the above procedure after first performing the PSP-based culling procedure"; Figure 7B legend, "PSP-based culling followed by the LFP-based culling and estimation procedures"). Deciding both on the full ensemble is shown as a reading. Heights are measured once on the full ensemble and not re-measured after culling (a choice made here; the Methods say q is "recalculated"); heights taken again against the mean of the kept trials are shown as a reading.
  • ReproducedResults, Variability of large mean responses, paragraph 1; Figure 7Drests on an assumption

    For tones with mean PSPs of at least 15 mV, total variability is lower (2.9 mV, 19 neurons) than for all tones, but the private variability is the same as for all tones (best estimate 1.5 mV, LFP-based bound 1.9 mV, 16 neurons).

    As shown in Fig. 7d, the total variability of these large mean responses (�q�neurons �2.9 �0.4 mV, n �19 neurons) was less than that of the full set of responses, yet they displayed the same level of private noise (�qPSP-culled,LFP-culled&estimation�neurons �1.5 � 0.2 mV; n �16 neurons) as the average tone.

    Results, Variability of large mean responses, paragraph 1; Figure 7D

    For tones with a mean PSP of at least 15 mV, total <q> = 2.92 +- 0.36 mV over 19 neurons, the LFP-based bound 1.62 +- 0.23 (within by 0.02 mV) and the best estimate 1.43 +- 0.20 mV, both from 14 neurons, not the paper's 16 (15 and 16 with the criteria applied after culling, 1.81 and 1.52 mV). The peak in the culling window (1b) gives an LFP-based bound of 1.57 and the variance over N 1.50, both outside, which would not give reproduced; 4b gives 1.61 and heights re-measured after culling 1.65, within. The claims file's spike rule gives 6.13 mV total over 21 neurons, and spikethresh keeping the slow events 5.50 over 21, either of which would give not reproduced; the -20 mV rule at any rise gives 3.00, 1.99 and 1.71 mV, the last 0.01 outside.

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • Spike trials are found with the data set's per-neuron spikethresh above the trace's baseline, an assumption (the paper gives no rule). The claims file's rule, a fast rise (above 10 mV/ms, a standard choice) to above -20 mV after the 12 mV junction potential correction, is shown as a reading; it misses slow depolarizations to about -10 mV (regenerative events or large PSPs), broad spikes that cross -20 mV with a rise below 10 mV/ms, and narrow spikes that peak below -20 mV. The -20 mV rule at any rise and spikethresh with the slow events kept (rise above 5 mV/ms) are shown as readings.
    • Open details as the claims file's header lists them; main readings 1a (peak searched over the whole trace after onset), 2a, 3 (N - 1), 4a, 5a, 7 (20 shuffles); the others shown.
    • LFP-based culling follows PSP-based culling and is run on the trials it keeps, its mean LFP, threshold and scale factors recomputed over them (Figure 6 legend, "we repeat the above procedure after first performing the PSP-based culling procedure"; Figure 7B legend, "PSP-based culling followed by the LFP-based culling and estimation procedures"). Deciding both on the full ensemble is shown as a reading. Heights are measured once on the full ensemble and not re-measured after culling (a choice made here; the Methods say q is "recalculated"); heights taken again against the mean of the kept trials are shown as a reading.
    • The outcome also rests on detail 1, reading a, and on the variance over N - 1 (detail 3).
  • ReproducedResults, Partitioning variability based on the LFP, paragraph 8

    The LFP recorded with the more variable neurons is not more variable than that recorded with the less variable neurons (CV 0.66 against 0.74, P = 0.5), so differences in total variability are not explained by gross state differences seen in the LFP.

    We therefore compared the CV of a simultaneously recorded LFP of the more variable neurons (CV �0.66 �0.06; n �14 neurons; mean �SE) and the less variable neurons (CV �0.74 �0.09; n �15 neurons). We found no significant difference (P �0.5; Student's t-test)

    Results, Partitioning variability based on the LFP, paragraph 8

    What the paper reports

    • Student's t-test, p = 0.5 (n = 14 more variable and 15 less variable neurons)

    Ranking the 29 LFP neurons by total <q>tones, the LFP height CV is 0.61 +- 0.05 (14 more variable) and 0.57 +- 0.06 (15 less variable); Student's t = 0.43, p = 0.674, difference 0.035. The less variable group is below the paper's 0.74 +- 0.09. Scale factor CV 0.89 and 0.77 (p = 0.350); the claims file's spike rule gives p = 0.428. The paper's slight opposite trend is not seen. The -20 mV rule at any rise gives 0.65 and 0.53 (p = 0.118), spikethresh keeping the slow events 0.56 and 0.61 (p = 0.511).

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • Spike trials are found with the data set's per-neuron spikethresh above the trace's baseline, an assumption (the paper gives no rule; the claims file's -20 mV rule, the -20 mV rule at any rise and spikethresh with the slow events kept are shown as readings).
    • LFP height is the signed extreme of the LFP in the 10-ms window around the mean LFP's largest deflection in the culling window.
  • ReproducedMethods, Surgery, paragraph 1rests on an assumption

    Total variability does not differ significantly between the three anesthesia protocols (pentobarbital 3.7, pentobarbital with diazepam 6.1, urethane 5.0 mV), and private variability differs slightly but significantly between pentobarbital alone (1.1 mV) and urethane (2.2 mV).

    We found no significant difference between the neuronal variability observed during these 3 protocols (pentobarbital alone ... and only a small, although statistically significant, difference in the private component of the variability for pentobarbital alone and urethane

    Methods, Surgery, paragraph 1

    Total <q> 3.69 +- 0.70 (pentobarbital, 13), 6.15 +- 2.81 (with diazepam, 3), 4.99 +- 0.58 mV (urethane, 17); ANOVA p = 0.251, Kruskal-Wallis p = 0.354. Private 1.01 +- 0.13 (12), 1.37 +- 0.33 (3), 1.93 +- 0.29 mV (13), the paper's group sizes, all within tolerance, the first and last by 0.01 and 0.03 mV; pentobarbital against urethane differ (t-test p = 0.0094, Mann-Whitney p = 0.0083). The margins are small: deciding both cullings on the full ensemble (pentobarbital private 0.98), the peak in the culling window (urethane private 1.82), the criteria after culling (1.84) and no negative scale factor rule in PSP culling (2.51) and the variance over N (pentobarbital 0.93, urethane 1.77) would each give partially reproduced; the claims file's spike rule gives a urethane total of 6.42 mV and an ANOVA p of 0.039, which would give not reproduced. The earlier record had partially reproduced, with both cullings decided on the full ensemble. Heights re-measured after culling give 1.01, 1.34 and 2.01 mV (t p = 0.0063) and the -20 mV rule at any rise keeps every value within with no total difference (ANOVA p = 0.199); spikethresh keeping the slow events gives a urethane total of 6.00 mV, outside.

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • Spike trials are found with the data set's per-neuron spikethresh above the trace's baseline, an assumption (the paper gives no rule). The claims file's rule, a fast rise (above 10 mV/ms, a standard choice) to above -20 mV after the 12 mV junction potential correction, is shown as a reading; it misses slow depolarizations to about -10 mV (regenerative events or large PSPs), broad spikes that cross -20 mV with a rise below 10 mV/ms, and narrow spikes that peak below -20 mV. The -20 mV rule at any rise and spikethresh with the slow events kept (rise above 5 mV/ms) are shown as readings.
    • Open details as the claims file's header lists them; main readings 1a (peak searched over the whole trace after onset), 2a, 3 (N - 1), 4a, 5a, 7 (20 shuffles); the others shown.
    • LFP-based culling follows PSP-based culling and is run on the trials it keeps, its mean LFP, threshold and scale factors recomputed over them (Figure 6 legend, "we repeat the above procedure after first performing the PSP-based culling procedure"; Figure 7B legend, "PSP-based culling followed by the LFP-based culling and estimation procedures"). Deciding both on the full ensemble is shown as a reading. Heights are measured once on the full ensemble and not re-measured after culling (a choice made here; the Methods say q is "recalculated"); heights taken again against the mean of the kept trials are shown as a reading.
    • The outcome also rests on the variance over N - 1 (detail 3), the claims file's main choice.
  • ReproducedResults, Spontaneous events veto evoked responses, paragraph 1; Figure 8C and 8D

    The size of spontaneous activity before the stimulus is anticorrelated with the evoked PSP, in the LFP (mean correlation -0.14, 29 neurons) and in the whole-cell record (-0.22, 33 neurons), the whole-cell correlation being negative in 30 of 33 neurons.

    there was a systematic negative correlation (correlation coefficient ��0.14 �0.02; n �29 neurons). Furthermore, we found that the magnitude of spontaneous subthreshold membrane potential events preceding the stimulus was anticorrelated with the strength of tone-evoked responses in 91% (30/33) of the neurons in our sample (correlation coefficient ��0.22 �0.03; n �33 neurons; Fig. 8, c and d).

    Results, Spontaneous events veto evoked responses, paragraph 1; Figure 8C and 8D

    Under readings (a) size as maximum minus minimum and (b) one correlation over all trials with heights normalized by the tone mean, the mean correlation is -0.155 +- 0.028 for the LFP (29) and -0.193 +- 0.031 for the whole cell (33), negative in 29 of 33 neurons. Readings (b, b) also hold (-0.149, -0.210, 28). Under (a, a) the LFP mean -0.168 is outside its tolerance. All combinations with either spike rule give negative means (-0.143 to -0.168 LFP, -0.179 to -0.238 whole cell). With the window ending 15 ms before onset, so that it does not overlap the PSP's baseline, the correlations remain (-0.134 to -0.158 LFP, -0.172 to -0.220 whole cell), and readings (a, a) and (b, b) still give all three values (-0.158, -0.220, 29; -0.135, -0.202, 28). Leaving out trials with a spike in the window changes the means by at most 0.014. 257 trials whose window overlaps the prepulse are left out. Under the window ending 15 ms before onset, (a, b) gives a whole-cell mean of -0.172, outside. With the -20 mV rule at any rise (b, b) gives -0.155, -0.205 and 29, and with spikethresh keeping the slow events -0.146, -0.201 and 28, all within.

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • Spike trials are found with the data set's per-neuron spikethresh above the trace's baseline, an assumption (the paper gives no rule). The claims file's rule, a fast rise (above 10 mV/ms, a standard choice) to above -20 mV after the 12 mV junction potential correction, is shown as a reading; it misses slow depolarizations to about -10 mV (regenerative events or large PSPs), broad spikes that cross -20 mV with a rise below 10 mV/ms, and narrow spikes that peak below -20 mV. The -20 mV rule at any rise and spikethresh with the slow events kept (rise above 5 mV/ms) are shown as readings.
    • The 100 ms before each tone is cut from the sweeps; trials whose window overlaps the prepulse are left out.
    • The combination of readings that matches is (a) for the size and (b) for the correlation; (b, b) also matches.
  • ReproducedMethods, LFP-based estimation procedure, paragraph 2rests on an assumption

    For the Figure 5 neuron, the shuffle control attributes 0.02 mV to overfitting of the linear estimator, and the corrected value after both cullings and estimation is 0.57 mV.

    the shuffling estimation control reduced the variability index for the neuron shown in Fig. 5 from �qLFP-culled,PSP-culled�tones �0.76 mV to �qLFP-culled,PSP-culled,shuffled,estimation�tones �0.74 mV, so about 0.02 mV was attributed to overfitting.

    Methods, LFP-based estimation procedure, paragraph 2

    In 020701md01b after both cullings <q>tones = 0.757 mV, shuffled estimation 0.743, estimation 0.524 uncorrected and 0.537 corrected; all within 10% of 0.76, 0.74, 0.55 and 0.57. The overfitting share is 0.014 mV against 0.02. The same with the claims file's spike rule, and 0.759, 0.746, 0.525, 0.538 with both cullings decided on the full ensemble. The peak-to-trough height over the trace (2b; corrected 0.892) and no negative scale factor rule in PSP culling (4b; 0.643) put the corrected value outside, which would give not reproduced. Per-tone and per-neuron corrections give the same tone average, so detail 7 has one reading. The two further spike rules and 1b give the main values; the variance over N 0.743, 0.730, 0.514, 0.528 and heights re-measured after culling 0.756, 0.742, 0.522, 0.536, all within.

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • Spike trials are found with the data set's per-neuron spikethresh above the trace's baseline, an assumption (the paper gives no rule). The claims file's rule, a fast rise (above 10 mV/ms, a standard choice) to above -20 mV after the 12 mV junction potential correction, is shown as a reading; it misses slow depolarizations to about -10 mV (regenerative events or large PSPs), broad spikes that cross -20 mV with a rise below 10 mV/ms, and narrow spikes that peak below -20 mV. The -20 mV rule at any rise and spikethresh with the slow events kept (rise above 5 mV/ms) are shown as readings.
    • Open details as the claims file's header lists them; main readings 1a (peak searched over the whole trace after onset), 2a, 3 (N - 1), 4a, 5a, 7 (20 shuffles); the others shown.
    • LFP-based culling follows PSP-based culling and is run on the trials it keeps, its mean LFP, threshold and scale factors recomputed over them (Figure 6 legend, "we repeat the above procedure after first performing the PSP-based culling procedure"; Figure 7B legend, "PSP-based culling followed by the LFP-based culling and estimation procedures"). Deciding both on the full ensemble is shown as a reading. Heights are measured once on the full ensemble and not re-measured after culling (a choice made here; the Methods say q is "recalculated"); heights taken again against the mean of the kept trials are shown as a reading.
  • ReproducedResults, Comparison to the contribution from stochastic synaptic release, paragraph 4

    In 14 of 33 neurons at least one tone elicits responses whose variability index is within a factor of 2 of the 0.38 mV expected from stochastic release alone.

    For 42% (14/33) of the neurons, at least one tone elicited responses whose variability index q was within a factor of 2 of the 0.38 mV expected from stochastic release alone.

    Results, Comparison to the contribution from stochastic synaptic release, paragraph 4

    14 of 33 neurons have an included tone with q within a factor of 2 of 0.38 mV, under both readings of the factor of 2; 15 with 0.39 mV (also within 14 +- 1.4). The claims file's spike rule and the peak in the culling window give the same; the variance over N 17 and 20. The two further spike rules give 14 and 15.

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • Spike trials are found with the data set's per-neuron spikethresh above the trace's baseline, an assumption (the paper gives no rule). The claims file's rule, a fast rise (above 10 mV/ms, a standard choice) to above -20 mV after the 12 mV junction potential correction, is shown as a reading; it misses slow depolarizations to about -10 mV (regenerative events or large PSPs), broad spikes that cross -20 mV with a rise below 10 mV/ms, and narrow spikes that peak below -20 mV. The -20 mV rule at any rise and spikethresh with the slow events kept (rise above 5 mV/ms) are shown as readings.
  • ReproducedMethods, LFP and whole cell patch-clamp recording, paragraph 3rests on an assumption

    Each tone was presented 18 times on average (10 to 63) per neuron; rest potential ranged from -58 to -92 mV (median -70) after junction potential correction, and input resistance from 19 to 141 MOhm (median 64).

    We presented 18 �2 repetitions (mean �SE, n �33 neurons; range �10 to 63 repetitions) of each tone to every neuron that was used in our variability analysis.

    Methods, LFP and whole cell patch-clamp recording, paragraph 3

    Repetitions 18.12 +- 1.93 (SE), 10 to 63. Rest potential from the 15-ms baselines corrected by -12 mV, median -70.0, range -58.4 to -92.0 mV; from the sweeps outside the prepulse, -68.6 and -55.3 to -87.9. Input resistance from the prepulse is 58.2 to 438.3 MOhm (median 137.9) as measured, outside the range; minus the recorded access resistance it is 19.1 to 140.6 MOhm (median 65.1), as in the paper (19 to 141, median 64).

    How the test differed from the paper’s method

    • Input resistance is the steady-state prepulse response (last 20% of the step, against the 5 ms before it) divided by the step current, minus the recorded access resistance. The Methods say there was no on-line series resistance compensation, so the step response includes the access resistance; the paper does not say it was subtracted. This is an assumption; without it the claim is not reproduced.
    • The prepulse height is read as a current in pA, as the docs table says for 110900md05, not the mV the documentation states.
    • Rest potential corrected by -12 mV (the paper), with the documentation's 10 mV shown.
  • ReproducedMethods, LFP and whole cell patch-clamp recording, paragraphs 2 and 3; Results, Trial-to-trial response variability, paragraph 2rests on an assumption

    Response variability is not correlated with access resistance (4.2 against 4.9 mV on either side of the 60 MOhm median) and does not vary systematically with recording depth.

    We found no correlation between response variability and access resistance (qR-series�median �4.2 �0.7 mV, n �17 cells; qR-series�median �4.9 �0.7 mV, n �16 cells; median access resistance �60 M�). ... Variability showed no systematic dependency on recording depth.

    Methods, LFP and whole cell patch-clamp recording, paragraphs 2 and 3; Results, Trial-to-trial response variability, paragraph 2

    Median access resistance 60 MOhm (10 to 320). <q> 4.88 +- 0.71 mV for the 17 at or above the median and 4.27 +- 0.63 mV for the 16 below, both within the tolerances of 4.2 +- 0.7 and 4.9 +- 0.7, though here the higher access resistance group is the more variable, the reverse of the paper. Access resistance, Pearson r = 0.087 (p = 0.631), Spearman 0.155 (p = 0.390); depth over 25 neurons with a recorded depth, r = -0.273 (p = 0.187), rho = -0.338 (p = 0.098). With the claims file's spike rule the groups are 5.70 and 4.99 mV, the first outside, and no correlation is significant (p = 0.125 to 0.661). The -20 mV rule at any rise gives 4.50 and 4.36 mV with no significant correlation; spikethresh keeping the slow events gives 5.22 mV for the upper group, outside.

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • Spike trials are found with the data set's per-neuron spikethresh above the trace's baseline, an assumption (the paper gives no rule). The claims file's rule, a fast rise (above 10 mV/ms, a standard choice) to above -20 mV after the 12 mV junction potential correction, is shown as a reading; it misses slow depolarizations to about -10 mV (regenerative events or large PSPs), broad spikes that cross -20 mV with a rise below 10 mV/ms, and narrow spikes that peak below -20 mV. The -20 mV rule at any rise and spikethresh with the slow events kept (rise above 5 mV/ms) are shown as readings.
    • Depth is missing for 8 of 33 neurons.

Partially reproduced7

  • Partially reproducedResults, Stereotypy of tone-evoked subthreshold responses, paragraph 1; Figure 2A and 2B

    Tones evoke brief, short-latency PSPs in all neurons. In the example neuron the mean responses to the 32 tones peak 27 to 42 ms after onset with heights of 1.2 to 18 mV, and are similar in shape once normalized and aligned.

    In all neurons, tones evoked brief short-latency PSPs ... For this neuron, the latency from stimulus onset to the peak of the mean response ranged from 27 to 42 ms, and the peak height ranged from 1.2 to 18 mV across the stimulus ensemble.

    Results, Stereotypy of tone-evoked subthreshold responses, paragraph 1; Figure 2A and 2B

    In 022801md01b (20 trials per tone) the 32 mean responses peak 26.5 to 41.5 ms after onset with heights 1.19 to 18.12 mV, within the widened ranges, under both peak windows. Normalized and aligned on their peaks they correlate with their average with median r 0.991 (minimum 0.933), above 0.9. The mean response to each neuron's largest tone peaks within 50 ms of onset in 19 of 33 neurons with the peak searched after onset (20.75 to 115.5 ms, median 45.25) or in the culling window (20.75 to 84.75 ms), so the all-neurons part fails under the claims file's 50 ms bound. The responses begin 14.25 to 38.5 ms after onset in 32 neurons (the mean over all tones first above 1 mV; 021401md02 never exceeds it).

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • The alignment window is 100 ms placed around each peak as the culling window sits around the median latency, with edge values where it runs off the trace (a plotting choice).
  • Partially reproducedResults, Partitioning variability based on the LFP, paragraph 6; Figure 7Arests on an assumption

    LFP-based culling and estimation together reduce the variability index by 40% on average, to 2.9 mV across 29 neurons, reduce it in 26 of 29 neurons, by as much as 71%, and have little effect when total variability is already small.

    The combined effect of the LFP-based culling and estimation procedures was to reduce the variability by an average of 40% across the population (qLFP-culled&estimation �2.9 mV � 0.3 mV; n �29 neurons; Fig. 7, a and c). In fact, these procedures reduced the variability in almost all (26/29) neurons

    Results, Partitioning variability based on the LFP, paragraph 6; Figure 7A

    <q LFP-culled&estimation> = 2.69 +- 0.33 mV over 29 neurons; reduced in 26 of 29; the average reduction is 43.9% by reading (a) and 40.1% by reading (b), so the 40% holds; the largest reduction is 80.0% (030301md03b), outside 71 +- 7.1. In that neuron LFP-based culling keeps 12 of 18 included tones; four of the six it drops have q of 5.7 to 35.8 mV, so <q>tones is 5.40 mV over all included tones and 1.73 mV over the kept ones before estimation (1.08 after). In the four neurons below 1.5 mV total, reductions of 8.6 to 49.1% (shown, not scored). With the tone criteria after culling (5b) 2.59 mV, 26 reduced, largest 73.9%, which would give reproduced; with the claims file's spike rule 3.22 mV, which would give not reproduced. The -20 mV rule at any rise (2.68 mV, 27, 80.0%), spikethresh keeping the slow events (2.85 mV, 29, 80.0%) and heights re-measured after culling (2.62 mV, 26, 80.5%) give partially reproduced.

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • Spike trials are found with the data set's per-neuron spikethresh above the trace's baseline, an assumption (the paper gives no rule). The claims file's rule, a fast rise (above 10 mV/ms, a standard choice) to above -20 mV after the 12 mV junction potential correction, is shown as a reading; it misses slow depolarizations to about -10 mV (regenerative events or large PSPs), broad spikes that cross -20 mV with a rise below 10 mV/ms, and narrow spikes that peak below -20 mV. The -20 mV rule at any rise and spikethresh with the slow events kept (rise above 5 mV/ms) are shown as readings.
    • Open details as the claims file's header lists them; main readings 1a (peak searched over the whole trace after onset), 2a, 3 (N - 1), 4a, 5a, 7 (20 shuffles); the others shown.
    • LFP-based culling follows PSP-based culling and is run on the trials it keeps, its mean LFP, threshold and scale factors recomputed over them (Figure 6 legend, "we repeat the above procedure after first performing the PSP-based culling procedure"; Figure 7B legend, "PSP-based culling followed by the LFP-based culling and estimation procedures"). Deciding both on the full ensemble is shown as a reading. Heights are measured once on the full ensemble and not re-measured after culling (a choice made here; the Methods say q is "recalculated"); heights taken again against the mean of the kept trials are shown as a reading.
  • Partially reproducedResults, Partitioning variability based on PSP shape, paragraphs 2 and 3rests on an assumption

    PSP-based culling alone reduces the mean variability index to 2.2 mV across 33 neurons, and leaves it at most about 3 mV in 32 of 33 neurons.

    On average, PSP-based culling reduced the variability index to a level comparable to that of the combined LFP-based procedures (qPSP-culled �2.2 �0.3 mV; n �33 neurons) ... in almost all (32/33) neurons, the variability index after PSP-based culling was at most about 3 mV

    Results, Partitioning variability based on PSP shape, paragraphs 2 and 3

    <q PSP-culled> = 2.19 +- 0.31 mV over 33 neurons, but 30 of 33 are at or below 3.3 mV (3.38, 3.60 and 10.98 mV above), outside 32 +- 1. The peak in the culling window (2.09 mV, 31) and the criteria after culling (1.92 mV, 32 of 32) would give reproduced; no negative scale factor rule in PSP culling (2.69 mV, 24) and the claims file's spike rule (2.72 mV, 25) would give not reproduced. The variance over N gives 2.02 mV and 32. The -20 mV rule at any rise (2.42 mV, 29), spikethresh keeping the slow events (2.54, 28) and heights re-measured after culling (2.18, 30) give partially reproduced.

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • Spike trials are found with the data set's per-neuron spikethresh above the trace's baseline, an assumption (the paper gives no rule). The claims file's rule, a fast rise (above 10 mV/ms, a standard choice) to above -20 mV after the 12 mV junction potential correction, is shown as a reading; it misses slow depolarizations to about -10 mV (regenerative events or large PSPs), broad spikes that cross -20 mV with a rise below 10 mV/ms, and narrow spikes that peak below -20 mV. The -20 mV rule at any rise and spikethresh with the slow events kept (rise above 5 mV/ms) are shown as readings.
    • Open details as the claims file's header lists them; main readings 1a (peak searched over the whole trace after onset), 2a, 3 (N - 1), 4a, 5a, 7 (20 shuffles); the others shown.
    • LFP-based culling follows PSP-based culling and is run on the trials it keeps, its mean LFP, threshold and scale factors recomputed over them (Figure 6 legend, "we repeat the above procedure after first performing the PSP-based culling procedure"; Figure 7B legend, "PSP-based culling followed by the LFP-based culling and estimation procedures"). Deciding both on the full ensemble is shown as a reading. Heights are measured once on the full ensemble and not re-measured after culling (a choice made here; the Methods say q is "recalculated"); heights taken again against the mean of the kept trials are shown as a reading.
  • Partially reproducedFigure 4C legendrests on an assumptionquestion to the reader

    PSP-based culling reduces the tone-averaged variability for all but one neuron, but increases the variability for at least one tone in half (18/33) of the neurons; in the Figure 4 neuron, it raises q for a 9.2-kHz tone from 0.47 to 0.57 mV.

    for this tone, PSP-based culling increased the variability index (q9.2 Hz �0.47 mV, q9.2 Hz-PSP-culled �0.57 mV). Although PSP-based culling reduced the tone-averaged variability for all but one neuron, variability increased after PSP-based culling for at least one tone in half (18/33) of the population.

    Figure 4C legend

    PSP-based culling lowers <q>tones in 32 of 33 neurons (021401md02 rises, 5.19 to 10.98 mV, one tone left). q rises for at least one tone in 16 of 33 neurons, outside 18 +- 1.8. Readings 1b, 2b, 3 and the intersected cullings give 16, as does spikethresh keeping the slow events; the claims file's spike rule and 5b 15; 4b 20. Heights re-measured after culling give 18 and the -20 mV rule at any rise 17, both within with 32 falling, which would give reproduced. The Figure 4 panels are labelled 030301md03b, which the claims file reads as not named (asked of a reader). The search the criterion sets finds 11 tones in 8 neurons fitting the Figure 4B legend, none with 0.47 and 0.57 mV at 9,188 Hz; the closest 9,188-Hz pair is 0.470 to 0.502 mV, in 030301md03b. At 31,118 Hz in that neuron PSP-based culling removes two traces (the aberrant one and one with a negative scale factor) and LFP-based culling four including both; without the negative scale factor rule (4b) PSP-based culling removes only the aberrant trace, as the Figure 4B legend says.

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • Spike trials are found with the data set's per-neuron spikethresh above the trace's baseline, an assumption (the paper gives no rule). The claims file's rule, a fast rise (above 10 mV/ms, a standard choice) to above -20 mV after the 12 mV junction potential correction, is shown as a reading; it misses slow depolarizations to about -10 mV (regenerative events or large PSPs), broad spikes that cross -20 mV with a rise below 10 mV/ms, and narrow spikes that peak below -20 mV. The -20 mV rule at any rise and spikethresh with the slow events kept (rise above 5 mV/ms) are shown as readings.
    • Open details as the claims file's header lists them; main readings 1a (peak searched over the whole trace after onset), 2a, 3 (N - 1), 4a, 5a, 7 (20 shuffles); the others shown.
    • LFP-based culling follows PSP-based culling and is run on the trials it keeps, its mean LFP, threshold and scale factors recomputed over them (Figure 6 legend, "we repeat the above procedure after first performing the PSP-based culling procedure"; Figure 7B legend, "PSP-based culling followed by the LFP-based culling and estimation procedures"). Deciding both on the full ensemble is shown as a reading. Heights are measured once on the full ensemble and not re-measured after culling (a choice made here; the Methods say q is "recalculated"); heights taken again against the mean of the kept trials are shown as a reading.
    • The outcome rests on the spike rule and on measuring heights once before culling (the readings that give 17 and 18 neurons).
  • Partially reproducedResults, Variability of large mean responses, paragraph 2rests on an assumption

    In coefficient of variation, both the total and the private variability of the large (at least 15 mV) responses are lower than for the full data set (total 0.37 against 0.73; private 0.27 against 0.41, significantly lower), unlike the variability index.

    Expressed in terms of the coefficient of variation, the total variability of the high mean responses (�CV�neurons �0.37 � 0.03, n �19 neurons) was lower than the total variability of the full data set (�CV�neurons �0.73 �0.04, n �33 neurons). However, the private contribution to the variability of the large mean responses (�CVPSP-culled,LFP-culled&estimation�neurons � 0.27 �0.01, n �16 neurons) was also significantly lower than the private contribution to the full data set

    Results, Variability of large mean responses, paragraph 2

    Total CV 0.371 +- 0.027 (19 neurons, at least 15 mV) and 0.734 +- 0.042 (33, all tones), matching 0.37 and 0.73. Private CV 0.265 +- 0.021 (14), within 0.27 +- 0.01, and 0.383 +- 0.026 (28), 0.007 below 0.41 +- 0.02. The large responses' private CV is significantly lower by Student's t-test (p = 0.0055) and the paired test (p < 0.0001). The criterion's reproduced outcome needs all four values, so the claim holds in part; the criterion defines no outcome for a value outside tolerance with the test holding, and partially reproduced is used. The peak-to-trough height over the trace (2b) gives 0.268 and 0.402, all four holding; the claims file's spike rule gives a large-response total CV of 0.491 and no significant difference (p = 0.475). Spikethresh keeping the slow events gives a total CV of 0.468 and no significant difference (p = 0.136); the -20 mV rule at any rise gives private CVs of 0.286 and 0.389 (p = 0.014), and heights re-measured after culling 0.267 and 0.385, partially reproduced.

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • Spike trials are found with the data set's per-neuron spikethresh above the trace's baseline, an assumption (the paper gives no rule). The claims file's rule, a fast rise (above 10 mV/ms, a standard choice) to above -20 mV after the 12 mV junction potential correction, is shown as a reading; it misses slow depolarizations to about -10 mV (regenerative events or large PSPs), broad spikes that cross -20 mV with a rise below 10 mV/ms, and narrow spikes that peak below -20 mV. The -20 mV rule at any rise and spikethresh with the slow events kept (rise above 5 mV/ms) are shown as readings.
    • LFP-based culling follows PSP-based culling and is run on the trials it keeps, its mean LFP, threshold and scale factors recomputed over them (Figure 6 legend, "we repeat the above procedure after first performing the PSP-based culling procedure"; Figure 7B legend, "PSP-based culling followed by the LFP-based culling and estimation procedures"). Deciding both on the full ensemble is shown as a reading. Heights are measured once on the full ensemble and not re-measured after culling (a choice made here; the Methods say q is "recalculated"); heights taken again against the mean of the kept trials are shown as a reading.
    • Private CV is the square root of the corrected residual variance over the mean, per tone, averaged as q is.
  • Partially reproducedDiscussion, paragraph 1; Figure 7C

    The private component of the variability is small, about 1 to 3 mV, and relatively homogeneous across neurons, while the shared component is often much larger and more heterogeneous, ranging from about 0 to 10 mV.

    We found that the private component was surprisingly small, about 1–3 mV, and relatively homogeneous across the population. The shared component was often much larger, and showed more heterogeneity across the population, ranging from about 0 to 10 mV.

    Discussion, paragraph 1; Figure 7C

    Shared (total minus private) ranges from 0.25 to 9.26 mV (total over all included tones) or 0.10 to 5.31 mV (total over the tones kept for the private estimate), within -1 to 11, and its SD (2.60 or 1.38 mV) exceeds the private SD (0.88). Only 20 of 28 private values (71%) lie in 0.9 to 3.3 mV, short of 80%: seven are below 0.9 (0.31 to 0.83) and one above (4.52). Every reading gives 57% to 75%.

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • Spike trials are found with the data set's per-neuron spikethresh above the trace's baseline, an assumption (the paper gives no rule). The claims file's rule, a fast rise (above 10 mV/ms, a standard choice) to above -20 mV after the 12 mV junction potential correction, is shown as a reading; it misses slow depolarizations to about -10 mV (regenerative events or large PSPs), broad spikes that cross -20 mV with a rise below 10 mV/ms, and narrow spikes that peak below -20 mV. The -20 mV rule at any rise and spikethresh with the slow events kept (rise above 5 mV/ms) are shown as readings.
    • LFP-based culling follows PSP-based culling and is run on the trials it keeps, its mean LFP, threshold and scale factors recomputed over them (Figure 6 legend, "we repeat the above procedure after first performing the PSP-based culling procedure"; Figure 7B legend, "PSP-based culling followed by the LFP-based culling and estimation procedures"). Deciding both on the full ensemble is shown as a reading. Heights are measured once on the full ensemble and not re-measured after culling (a choice made here; the Methods say q is "recalculated"); heights taken again against the mean of the kept trials are shown as a reading.
  • Partially reproducedResults, Comparison to the contribution from stochastic synaptic release, paragraph 2verifier disagreed at firstrests on an assumption

    Miniature EPSPs recorded in vivo after TTX have a mean size of 0.39 mV with a CV of 0.36 across 6 neurons, consistent with in vitro estimates.

    Our estimate of the distribution of mEPSPs in vivo (Qmini �0.39 �0.08 mV; n �6 neurons) is consistent with some previous estimates (0.35 mV) ... The width of the distribution of mEPSPs sizes (CV �0.36 �0.03; n �6 neurons)

    Results, Comparison to the contribution from stochastic synaptic release, paragraph 2

    The six TTX recordings are digitized in steps of 0.049 mV (0.012 for 030401md02h after its gain of 4), with 14% to 68% of the differences between samples 0. Template matching at thresholds of 3 to 5 finds up to 29 events per s and, on the inverted traces, 55% to 80% as many (mean over recordings), so most of its events are noise; its mean scale is 0.28 to 0.35 mV. The derivative detector with, per recording, the lowest threshold at which the inverted trace gives at most 10% as many events (k = 3 to 6) gives a mean size of 0.602 +- 0.164 mV over five recordings, outside 0.39 +- 0.08, and a mean CV of 0.354, within 0.36 +- 0.03. 050401md02 has one stable sweep (7.7 s) and no threshold meets the bound. The event-triggered averages rise within about 2 ms and peak at 0.15 to 0.99 mV; 6.7% to 12.0% of events fall in the 50 ms after a tone, against 10% by chance. The result moves with the choices: fixed thresholds k = 3 and 4 give 0.370 mV (CV 0.421) and 0.487 mV (CV 0.350), each with one value holding; bounds of 5% and 20% give 0.702 and 0.549 mV (CV 0.314 and 0.376); the documentation's file order gives 0.726 mV (CV 0.296). No reading with the false positives bounded gives both values. Question for the paper's authors: how were mEPSPs detected and sized (detector, threshold, filtering, baseline), and which sweeps of each TTX recording were used? Under the main reading event rates range from 0.11 Hz (030401md02h, 15 events, 1.04 mV) to 19 Hz (050801md01c, 0.20 mV, sizes down to 0.01 mV), so the mean rests on the threshold each recording gets. Without 030401md02h the four others give 0.492 mV and CV 0.385, both outside.

    How the test differed from the paper’s method

    • Sweeps are first_trial to last_trial of each TTX recording's cell_specific_parameters.m, read as ep_<trial - 1>, without their first 0.3 s; the documentation's name order (dir) and all sweeps are shown. 030401md02h is divided by its documented gain of 4.
    • The detector is assumed: a threshold on the derivative of the trace smoothed over 1 ms, in noise SDs (SD after clipping at 3 SD), set per recording as the lowest of k = 3 to 8 at which the sign-inverted trace gives at most 10% as many events; size is the peak in the 10 ms after onset minus the mean of the 2 ms before. Template matching (Clements and Bekkers 1997) is shown; on the inverted traces it finds most of the events it finds on the traces.

Not reproduced2

  • Not reproducedMethods, LFP-based culling and PSP-based culling procedures, paragraphs 3 and 4question to the reader

    Random culling of as many traces as LFP-based culling removes leaves the variability index unchanged (4.5 against 4.6 mV), while matching the PSP-based or both cullings lowers it (3.67 and 2.98 mV) because tones with fewer than 10 non-aberrant trials drop out; keeping only those tones without culling gives 2.9 mV. PSP-based culling removes 29% of trials and LFP-based culling 38%.

    we randomly culled an equal number of traces as the LFP-based culling procedure from each ensemble of responses corresponding to a given neuron and a given tone, and found no effect (after random culling: �q�neurons �4.5 �0.1 mV, n �20 simulations ... (after random culling: �q�neurons �3.67 �0.02 mV, n �20 simulations), despite the fact that the PSP-based culling procedure resulted in substantially fewer trials being culled (29%) than the LFP-based culling procedure (38%)

    Methods, LFP-based culling and PSP-based culling procedures, paragraphs 3 and 4

    Random culling matched to both cullings gives 2.997 mV (2.98 +- 0.02) and the second control 2.96 +- 0.32 mV over 28 neurons (2.9 +- 0.3), but random culling matched to LFP-based culling gives 3.82 mV over the 29 LFP neurons (3.80 over 33), not 4.5 +- 0.1. Removing as many trials per tone as LFP-based culling drops the same tones, 247 of 643, and <q> over the tones left is 3.72 against 4.80 mV. Matched to PSP-based culling 3.737 mV over 33 (3.86 over 29), against 3.67 +- 0.02, a miss of more than three SEs. PSP-based culling removes 24.1% of trials (pooled; 19.2% mean of neurons), outside 29 +- 2.9, and LFP-based culling 37.8%. Without the tone criteria applied again after random culling, the LFP-matched control gives 4.83 mV (SD over simulations 0.97) and the PSP-matched one is unstable (2.30 mV, SD 8.5). Whether the paper applied them again is asked of a reader (questions/crcns-ac-2.yaml). The claims file's spike rule gives 4.36, 3.74 and 3.98 mV; the outcome is the same. The -20 mV rule at any rise gives 3.68, 3.92 and 3.10 mV, spikethresh keeping the slow events 4.11, 5.09 and 3.47 mV. The controls use the heights of the full ensemble under every reading.

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • Spike trials are found with the data set's per-neuron spikethresh above the trace's baseline, an assumption (the paper gives no rule). The claims file's rule, a fast rise (above 10 mV/ms, a standard choice) to above -20 mV after the 12 mV junction potential correction, is shown as a reading; it misses slow depolarizations to about -10 mV (regenerative events or large PSPs), broad spikes that cross -20 mV with a rise below 10 mV/ms, and narrow spikes that peak below -20 mV. The -20 mV rule at any rise and spikethresh with the slow events kept (rise above 5 mV/ms) are shown as readings.
    • LFP-based culling follows PSP-based culling and is run on the trials it keeps, its mean LFP, threshold and scale factors recomputed over them (Figure 6 legend, "we repeat the above procedure after first performing the PSP-based culling procedure"; Figure 7B legend, "PSP-based culling followed by the LFP-based culling and estimation procedures"). Deciding both on the full ensemble is shown as a reading. Heights are measured once on the full ensemble and not re-measured after culling (a choice made here; the Methods say q is "recalculated"); heights taken again against the mean of the kept trials are shown as a reading.
    • The both-matched control removes as many trials as the two cullings remove in sequence.
  • Not reproducedMethods, LFP-based culling and PSP-based culling procedures, paragraph 4

    For tones of at least 15 mV, random culling and keeping only tones that survive both cullings give 3.2 mV, so the private estimate of 1.5 mV is not an undersampling effect.

    both the random culling control (�q�neurons �3.20 �0.05 mV, n �20 simulations) and the second control (�q�neurons �3.2 �0.5 mV, n �16 neurons) resulted in equivalent increases

    Methods, LFP-based culling and PSP-based culling procedures, paragraph 4

    For tones of at least 15 mV, random culling matched to both cullings gives 2.42 mV (SE over 20 simulations 0.02) and the second control 2.40 +- 0.39 mV over 14 neurons, against 3.20 +- 0.05 and 3.2 +- 0.5. The two agree with each other and lie above the 1.43 mV best estimate, as the paper argues, but both values are outside tolerance (4.91 and 4.96 mV with the claims file's spike rule). The -20 mV rule at any rise gives 2.49 and 2.51 mV, spikethresh keeping the slow events 3.96 and 3.88 mV, all outside.

    How the test differed from the paper’s method

    • Trials are each neuron's stable epoch, cycles first_cycle to last_cycle of the data set's cell_specific_parameters.m; the paper requires 10 consecutive stable repetitions, and these epochs give its repetition counts (mean 18.1, 10 to 63).
    • Spike trials are found with the data set's per-neuron spikethresh above the trace's baseline, an assumption (the paper gives no rule). The claims file's rule, a fast rise (above 10 mV/ms, a standard choice) to above -20 mV after the 12 mV junction potential correction, is shown as a reading; it misses slow depolarizations to about -10 mV (regenerative events or large PSPs), broad spikes that cross -20 mV with a rise below 10 mV/ms, and narrow spikes that peak below -20 mV. The -20 mV rule at any rise and spikethresh with the slow events kept (rise above 5 mV/ms) are shown as readings.
    • LFP-based culling follows PSP-based culling and is run on the trials it keeps, its mean LFP, threshold and scale factors recomputed over them (Figure 6 legend, "we repeat the above procedure after first performing the PSP-based culling procedure"; Figure 7B legend, "PSP-based culling followed by the LFP-based culling and estimation procedures"). Deciding both on the full ensemble is shown as a reading. Heights are measured once on the full ensemble and not re-measured after culling (a choice made here; the Methods say q is "recalculated"); heights taken again against the mean of the kept trials are shown as a reading.

Findings about the data

What attempts found about the dataset itself, for its owner and the archive.

  • Metadata errorAnesthetized_cell_info.txt gives a junction potential of about 10 mV and says the true potentials are about 10 mV more depolarized than the recorded values. The paper corrects by 12 mV in the other direction (more hyperpolarized), and only that correction gives the paper's rest potentials (median -70.0 mV, -58.4 to -92.0) from the recorded baselines; the documented direction gives -33 to -66 mV.
  • Selection not documentedEach anesthetized neuron's cell_specific_parameters.m gives first_cycle, last_cycle, spikethresh and sweeps_to_include_in_bumps_anl, which the documentation does not describe for these neurons. The cycles give the paper's stable epochs (repetitions 10 to 63, mean 18.1) and spikethresh its spike removal (with them the paper's population values follow), so they carry the paper's trial selection.
  • Metadata errorThe documentation calls the prepulse height a voltage ("height (negative mV)"); the values (100 and 200) are currents in pA, as the cell table's note for 110900md05 (-200 pA to -100 pA) and the input resistances they give show.
  • Data missingSutter_diag_depth is missing (empty or NaN) for 8 of the 33 anesthetized neurons, so the paper's depth range and its depth comparison can be checked on 25.
  • OtherThe TTX recording 030401md02h, after the documented division by 4, sits near +2.6 mV (1st to 99th percentile -5.0 to 6.1 mV), which is not a membrane potential; the other TTX recordings sit at -54 to -71 mV. Its offset or gain may be wrong.
  • Methods not availableThe paper does not say how spike trials were found. The claims file's rule (a fast rise to above -20 mV after the junction potential correction, with 10 mV/ms for the rise) and the data set's per-neuron spikethresh give different results (mean total variability 5.35 against 4.58 mV); without the rise criterion the -20 mV rule gives 4.43 mV. Spikethresh also removes slow depolarizations to about -10 mV, which may be regenerative events or large PSPs; keeping them gives 5.11 mV. Spikethresh was assumed. Question for the authors, were spikes removed with spikethresh, as a height above the 15-ms baseline, and were the slow depolarizations removed as spikes?
  • Methods not availableThe paper does not say whether input resistance had the access resistance subtracted; only with it subtracted do the prepulse responses give the paper's 19 to 141 MOhm (median 64). Question for the authors: was input resistance the step response minus the access resistance, and over which part of the step?
  • Methods not availableThe paper gives no method for detecting and sizing mEPSPs. With fixed thresholds of 3 to 5 noise SDs the mean size is 0.28 to 0.43 mV for template matching (CV 0.37 to 0.60) and 0.37 to 0.63 mV for the derivative detector (CV 0.33 to 0.53); with the derivative detector and a false-positive bound of 5% to 20% it is 0.55 to 0.77 mV (CV 0.27 to 0.40), over the two sweep mappings of the stable epochs. Question for the authors, which detector, threshold, filter and baseline were used, and which sweeps?
  • Methods not availableSeveral details of the variability procedures are not given and move counts near their tolerances: the window searched for the mean PSP's peak, the span of the culling threshold's peak-to-trough height, whether the negative scale factor rule applies to PSP-based culling, whether the tone criteria use the mean before or after culling, whether heights are re-measured after culling, and whether the random culling controls apply the 10-trial criterion again. Question for the authors: how was each set?
  • Description missingEach TTX recording's cell_specific_parameters.m gives first_trial and last_trial, but the documentation does not say which ep_<n>.ibw file is trial k. Its Matlab example lists the files with dir('ep_*.ibw'), which orders them by name (ep_0, ep_1, ep_10, ...), and that order picks different sweeps from the numeric one in four of the six recordings (050301md01, 050501md01, 050501md04, 050801md01c). 050401md02 has one stable sweep of 7.7 s.

Questions

Where the author or a verifier believed the claims file misread the paper, they asked, citing the paper, and a reader who had not seen the data answered from the paper alone, amending the claims file where it was wrong. The author then ran the claim as answered. Attempts before protocol pilot-11 also asked here for method details the claims file did not give.

  • openasked about culling-controls

    Does the paper's random culling control apply the 10-trial tone criterion again after the random removal? The criterion says "the tone criteria are then applied". Read that way, a control that removes as many trials per tone as LFP-based culling drops exactly the tones that LFP-based culling drops, and here it gives 3.82 mV, not the paper's 4.5 mV with "no effect". Without the criterion applied again (every tone included before culling kept), the LFP-matched control gives 4.83 mV (SD over 20 simulations 0.97). The paper's own results point both ways. Random culling of 38% of trials leaves q unchanged, while the PSP-matched control (29%) lowers it, which fits a control that does not drop tones. But the next paragraph explains the reductions by "the removal of ensembles of responses corresponding to tones that elicited fewer than 10 nonaberrant PSPs", which fits one that does. The same question applies to large-responses-controls, which follows culling-controls.

    Passage. Methods, LFP-based culling and PSP-based culling procedures, paragraphs 3 and 4, "As a control for the possibility that the variability index was reduced partly because of undersampling after the LFP-based culling procedure, we randomly culled an equal number of traces as the LFP-based culling procedure from each ensemble of responses corresponding to a given neuron and a given tone, and found no effect (after random culling ... 4.5 +- 0.1 mV, n = 20 simulations ...). Repeating this control for the PSP-based culling procedure did reduce the variability index (after random culling ... 3.67 +- 0.02 mV ...)" and "These reductions did not result from an undersampling bias, but were instead due to the removal of ensembles of responses corresponding to tones that elicited fewer than 10 nonaberrant PSPs from at least one neuron." Earlier in the same section, "In computing q values after each culling procedure, we avoided undersampling effects by including only those tones with at least 10 trials that remained after culling."

    Why it matters. The LFP-matched value is the control's main expected value and decides the outcome. It is 3.82 mV with the criterion applied again and 4.83 mV without it, against 4.5 +- 0.1 (the criterion counts a miss of less than three SEs as partial). The PSP-matched and both-matched values move as well, so the reading the claims file takes could change culling-controls and large-responses-controls from not reproduced.

    No reader has answered it yet.
  • The criterion says the Figure 4 neuron is not named and identifies it by a search. The panels of Figure 4 carry the label 030301md03b (under 4A, and in 4C), as Figures 2 and 3A carry 022801md01b. Should the Figure 4 example values be taken from 030301md03b? There the 9,188-Hz tone gives q 0.470 mV before PSP-based culling and 0.502 mV after it (0.47 and 0.57 in the paper). Its 31,118-Hz tone, the tone of Figure 4A and 4B, fits the Figure 4B legend (PSP-based culling removes one trace, LFP-based culling that trace and three others) only when the negative scale factor rule is left out of PSP-based culling (open detail 4, reading b); with it, PSP-based culling removes two traces.

    Passage. Figure 4 panel labels "030301md03b" (A, under the 31 KHz stimulus bar; C, beside the 9.2 kHz bar), and the Figure 4 legend, "In this example, PSP-based culling removes only the most aberrant trace, whereas LFP culling removes that trace and 3 others" and "for this tone, PSP-based culling increased the variability index (q9.2 Hz = 0.47 mV, q9.2 Hz-PSP-culled = 0.57 mV)".

    Why it matters. The example values are reported and do not decide the outcome, so the outcome does not change. The label settles which neuron the example is, and the 31-kHz tone of that neuron bears on open detail 4, which moves psp-culling-reduction, anesthesia-comparison and overfitting-correction-example.

    No reader has answered it yet.

Notebooks

Every registered entry that uses this dataset. An entry can hold several notebooks, and can use more than one dataset.

Reproduction Shared and private variability in the auditory cortex (DeWeese and Zador 2004)

Tests the findings of DeWeese and Zador (2004) on CRCNS data set ac-2, which holds the paper's 33 in vivo whole-cell recordings from rat auditory cortex during pure tones, 29 of them with a simultaneous LFP, and six recordings after TTX. It measures the variability index of tone-evoked PSPs, partitions it into private and shared parts with the paper's LFP- and PSP-based culling and estimation procedures, and checks the controls, the large-response and CV analyses, the spontaneous veto, the anesthesia groups and the recording properties. All files

Verification

  • Round 1
    2026-11-04
    Agreed with 22 of 23 outcomes. Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook). Reviewed: Round 1. The executed notebook with the conclusion cells and the summary removed, the helper module and the claims file, with the paper. The verifier checked the archive listing against the notebook's MD5 sums and tested the helper functions on synthetic inputs. 22 of its 23 outcomes agree with the record; for mepsp-size it gives partially reproduced where the record blocked the claim. Its general points: the main reading mixes readings across claims (psp-culling passes under 1b or 3, fig4c under the -20 mV rule, fig8 under (a,b)), and the -20 mV variant was not the claims file's rule.
    • mepsp-size: the verifier’s outcome was “Partially reproduced”. Labelled blocked although the archive holds all six TTX recordings and the documented gain, and the claims file's detector readings were run, so it is testable. The derivative detector at k = 3-4 gives 0.34-0.41 mV, within 0.39 +- 0.08; the CV is out under all six readings and the template detector gives 0.17 mV. For 050801md01c the derivative detector barely responds to its threshold, which suggests the robust noise SD has collapsed and the detector picks up noise; the template detector finds 15-28 Hz of 0.1-0.2-mV events; amplitude on the smoothed trace inflates noise events; the mapping of sweep files is not checked, and 050401md02 uses a single 7.7-s sweep.
  • Round 2
    2026-11-04
    Agreed with 23 of 23 outcomes. Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook). Reviewed: Round 2. The full executed notebook with the conclusions and the summary, the helper module, the attempt record and the claims file, with the paper. The verifier agreed with all 23 outcomes. It found statements the numbers do not support (cells 14, 29, 44, the lfp-culling notes and the mEPSP finding) and readings the record did not name. Remaining points, the mEPSP mean rests on per-recording thresholds; heights are not re-measured after culling and this was not run as a reading; the claims file's rule depends on the 10 mV/ms rise chosen here and the slow depolarizations that only spikethresh removes could be large PSPs; the culling-controls question is unanswered; the fig2, fig3a and fig4 panels had not been compared with the paper by eye.

Method differences

How the tests differed from the paper’s method or data, and the claims each applies to.

Provenance

Attempt record
crcns-ac-2-deweese-2004-01
Paper
10.1152/jn.00197.2004
Date
2026-11-04
Made by
An agent, run by CatalystNeuro
Dataset
CRCNS:ac-2
Version
unversioned (doi:10.6080/K0G44N6R)
Run on
GitHub Actions
Harness
Claude Code
Model
claude-opus-5-5
Protocol
pilot-13

Transcripts of the agent runs

Each run’s prompt, messages and tool calls, with long output cut and the text of the paper left out, and its final report.