{ "dataset": { "archive": "crcns", "id": "fcx-1", "label": "CRCNS:fcx-1" }, "page": "datasets/crcns/fcx-1/", "generated": "2026-11-07T21:11:16+00:00", "caution": "A failure to reproduce is not a finding that the paper is wrong. It may be a failure of the attempt.", "outcome_counts": { "reproduced": 15, "partially_reproduced": 14, "not_reproduced": 12, "blocked": 0 }, "outcome_counts_with_outside_data": { "reproduced": 0, "partially_reproduced": 0, "not_reproduced": 0, "blocked": 0 }, "outcome_counts_elsewhere": { "reproduced": 0, "partially_reproduced": 0, "not_reproduced": 0, "blocked": 0 }, "outcome_counts_links": { "reproduced": 0, "partially_reproduced": 0, "not_reproduced": 0, "blocked": 0 }, "attempts": [ { "id": "crcns-fcx-1-watson-2016-01", "dataset": { "archive": "crcns", "id": "fcx-1", "version": "unversioned", "doi": "10.6080/K02N506Q" }, "key": "crcns:fcx-1", "label": "CRCNS:fcx-1", "dataset_version": "unversioned", "paper": "10.1016/j.neuron.2016.03.036", "paper_title": "Network Homeostasis and State Dynamics of Neocortical Sleep", "date": "2026-11-07", "kind": "agent", "accountable": "CatalystNeuro", "agent": { "harness": "Claude Code", "model": "claude-opus-5-5", "protocol": "pilot-13" }, "runner": { "platform": "GitHub Actions", "url": "https://github.com/catalystneuro/encore/actions/runs/42353046463" }, "tier": "agent_verified", "deprecated": "", "deprecated_on": null, "entry": "crcns-fcx-1-watson-2016", "entry_title": "Network homeostasis and state dynamics of neocortical sleep (Watson et al. 2016)", "summary": "The paper's central finding, that sleep evens out cortical firing as fast-firing pyramidal cells slow down and slow-firing cells speed up, held up in this data across sleep and within nonREM packets (first-to-last packet slope 0.872), as did the overall decline in population rate. The reported drop in rate variability over sleep and the firing changes within single UP states did not reproduce. Most of the lasting effects of individual sub-states (REM, microarousals, waking between packets) did not reproduce, likely because the paper does not say how sequences of states were formed and no rule gave its cell counts. Lower firing in waking after sleep held only weakly, as most cycles lack scored waking right after sleep.", "description": "Tests the findings of Watson, Levenstein, Greene, Gelinas and Buzsaki (2016) on CRCNS data set fcx-1, which holds the spike trains, scored brain states, detected UP/DOWN states and spindles, and LFP of 27 frontal cortex recordings in 11 rats. It compares firing rates across states, their changes over sleep and within nonREM episodes and packets, the LFP and UP/DOWN dynamics, and the rate changes across sleep sub-state triplets.", "notebooks": [ { "path": "reproduce_watson_2016.ipynb", "title": "Reproducing Watson et al. (2016) from CRCNS fcx-1", "executed": "notebooks/crcns-fcx-1-watson-2016/reproduce_watson_2016.html" } ], "outcomes": [ { "claim": "table1-dataset-counts", "statement": "The data comprise 11 rats, 27 sessions and 54 WAKE-SLEEP episodes, with 995 putative pyramidal cells (mean 36.9 per session, range 10 to 104) and 126 putative interneurons (4.67 per session, range 0 to 13).", "source": "Results, Assessment of firing patterns in different brain states, and Firing pattern and LFP changes across the course of sleep; Table S1", "quote": "This resulted in 995 putative pyramidal (pE) cells (mean of 36.9 cells/session ... 10-104) and 126 putative interneurons (pI) (4.67 cells/session ... 0-13)", "statistics": [], "kind": "finding", "inferred": "", "outcome": "partially_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "995 pE and 126 pI cells (mean 36.85 pE per session, range 10 to 104; pI mean 4.67, range 0 to 13), 27 sessions and 11 rats match exactly. The WAKE-SLEEP cycles in _WSRestrictedIntervals.mat number 51, not 54; each session's Wake and Sleep intervals are exactly its WakeSleep pairs (51, 51 and 51 in all), with no unpaired WAKE or SLEEP, so no cycle was lost in parsing. The criterion names no partial outcome; partial is given because five of six counts match, and whether a partial rule applies is asked of a reader.", "deviations": [], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "ma-statistics", "statement": "There are 790 microarousals, with median duration 15.9 s, 0.32 per minute, taking 9.1% of SLEEP time; the fraction of time in MA decreases over nonREM episodes but not over SLEEP.", "source": "Results, Identification of brain states; Table S1", "quote": "There were a total of 790 MAs in our dataset (median duration = 15.9\u00b13.3sec; frequency = 0.32\u00b10.12 per minute; 9.1\u00b13.5% of time in SLEEP). Over nonREM episodes the fraction of time occupied by MAs decreased (correlation versus normalized nonREM episode time", "statistics": [ { "about": "MA time fraction against normalized nonREM episode time", "test": "correlation", "statistic": "R = -0.85", "p": 1e-10, "p_relation": "<" } ], "kind": "finding", "inferred": "", "outcome": "partially_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "434 MA intervals lie inside SLEEP (2130 in the files). Median duration 11.0 s over all MAs (11.5 s over session medians), per-session median incidence 0.184 per minute, per-session median 3.70% of SLEEP, against 790, 15.9 s, 0.32 and 9.1%; none within tolerance. The MA fraction falls over normalized episodes (20 bins, bin means R = -0.956, p = 4.5e-11; 10 bins R = -0.970, p = 3.3e-06), and over SLEEP the bin means are not significant (p = 0.15, 0.098; pooled period-bins p = 0.0057, 0.00061). The criterion names no partial outcome; partial is given because the trends hold and the four values do not. The 345 WakeInterruptions shorter than 40 s are not MAs by the paper's definition, since none lies between two packets (123 between two REM intervals, 80 between REM and a packet). Variant, MA reading (b), counting them as MAs, 779 events, 0.318 per minute (within tolerance), median 13.0 s, 7.85% of SLEEP, episodes R = -0.960 (p = 1.9e-11), SLEEP p = 0.93. With the Dino ACC sessions left out, episodes R = -0.949 (p = 1.7e-10), SLEEP p = 0.089.", "deviations": [ "The episode correlation uses the MA fraction in 10 or 20 normalized bins, both as bin means over episodes and as pooled episode-bins, as the criterion allows; episodes include MA and packet seconds." ], "assumed": false, "disagreed": true, "data": "dataset", "where": null, "proposed": null }, { "claim": "packet-duration", "statement": "nonREM packet durations span two orders of magnitude with a median of 111 s.", "source": "Results, Identification of brain states; Table S1", "quote": "We found that the duration of packets was distributed over two orders of magnitude, with a median of 111 seconds", "statistics": [], "kind": "finding", "inferred": "", "outcome": "reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Median of the 701 packets in SLEEP 111.0 s. Longest over shortest 52.5 (14 to 735 s), less than the two orders of magnitude of the statement, which the criterion only reports.", "deviations": [], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig2a-pe-rate-distribution", "statement": "Per-cell mean firing rates of pyramidal cells are strongly positively skewed and span three orders of magnitude, from 0.05 to 5 Hz, with medians of 0.76 Hz in WAKE, 0.69 Hz in nonREM and 0.88 Hz in REM; interneuron medians are 5.59, 4.69 and 4.25 Hz.", "source": "Results, Assessment of firing patterns in different brain states; Figure 2A; Figure S2C", "quote": "the distribution of per-cell mean firing rates for putative pyramidal neurons was strongly positively skewed ... spanning three orders of magnitude range from 0.05 Hz to 5 Hz ( Figure 2A ). Median rates (\u00b1SD) of putative pyramidal cells in each state were: WAKE: 0.76\u00b11.53 Hz, nonREM: 0.69\u00b10.86 Hz, REM: 0.88\u00b11.33 Hz; and for putative interneurons were: WAKE: 5.59\u00b17.25 Hz, nonREM: 4.69\u00b15.62 Hz, REM 4.25\u00b19.43 Hz", "statistics": [], "kind": "finding", "inferred": "", "outcome": "not_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Reading (a) medians pE 0.755, 0.706, 0.936 Hz and pI 6.001, 4.565, 5.724 Hz; reading (b) pE 0.784, 0.691, 0.922 and pI 5.632, 4.637, 5.724. Five of six within 10% under both; pI REM is 5.724 against 4.25. pE skew 2.18 to 2.83. The 1st to 99th percentile range covers 0.05 to 5 Hz within 10% in WAKE (0.004 to 7.14 Hz, reading b) and REM (0.008 to 6.03) but not nonREM (0.014 to 4.17; 4.05 under reading a); 5th to 95th percentiles stop below 4.5 Hz in every state. The criterion requires the shape for both the reproduced and the partial outcome, so the outcome is not reproduced. Whether the range applies to each state, when the paper itself says the nonREM tail is shorter, is asked of a reader. The pI REM median is equal under both readings because the middle cell is the same (Dino_061814_mPFC cell 46, 5.706 and 5.743 Hz in its two cycles); reading (a) has no duplicated rows.", "deviations": [ "The range test is applied in each state, as the criterion's 'in each state' reads." ], "assumed": false, "disagreed": true, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig2b-wake-nrem-slope", "statement": "Per-cell nonREM and WAKE rates are highly correlated (R = 0.82), but the slope of their relation is below 1 (95% interval 0.66 to 0.71), so state transitions affect cells differently across the rate spectrum, fast cells decreasing and slow cells increasing in nonREM.", "source": "Results, Assessment of firing patterns in different brain states; Figure 2B and legend", "quote": "Whereas nonREM and WAKE firing rates remained highly correlated on a per-cell basis (R = 0.82, p < 1 \u221210 ), the slope of this relationship significantly deviated from 1 ( Figure 2B ), demonstrating that state transitions exert a differential effect across the firing rate spectrum (95% confidence interval of slope ... 0.66 - 0.71)", "statistics": [ { "about": "correlation of nonREM and WAKE rates", "test": "correlation", "statistic": "R = 0.82", "p": 1e-10, "p_relation": "<" }, { "about": "slope of nonREM rate on WAKE rate", "ci": { "level": 0.95, "low": 0.66, "high": 0.71 } } ], "kind": "finding", "inferred": "", "outcome": "reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Reading (a) slope of log10 nonREM on log10 WAKE 0.684 (0.661 to 0.708), R 0.803; reading (b) 0.687 (0.658 to 0.717), R 0.824. Linear rates give slope 0.45. 82% of cells above the median WAKE rate fire less in nonREM, 55% of those below fire more.", "deviations": [], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig2a-fraction-above-2hz", "statement": "The right tail of the rate distribution is longer in WAKE and REM than in nonREM; the fraction of pE cells above 2 Hz is 19.2% in WAKE and in REM and 8.8% in nonREM.", "source": "Results, Assessment of firing patterns in different brain states; Figure 2A", "quote": "The right tail extended toward higher rates during WAKE and REM compared to nonREM ( Figure 2A ; fraction of >2 Hz neurons", "statistics": [ { "test": "Chi-Square Test", "p": 1e-10, "p_relation": "<" } ], "kind": "finding", "inferred": "", "outcome": "reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Reading (a) 18.8% WAKE, 20.6% REM, 9.3% nonREM; chi-square nonREM against WAKE p = 4.1e-16, against REM p = 4.4e-21. Reading (b) fractions 18.4, 19.5, 9.1% with p = 3.2e-09 and 8.3e-11.", "deviations": [], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "burst-index-by-state", "statement": "The burst index (fraction of spikes with ISIs below 15 ms) is highest in nonREM (median 11.8%), lowest in WAKE (4.0%), with REM (5.5%) slightly above WAKE.", "source": "Results, Assessment of firing patterns in different brain states", "quote": "Burst index was highest during nonREM (median in nonREM = 11.8%) and lowest during WAKE (4.0 %), with REM (5.5 %) slightly higher than WAKE (p < 0.05; Kruskal-Wallis test)", "statistics": [ { "test": "Kruskal-Wallis test", "p": 0.05, "p_relation": "<" } ], "kind": "finding", "inferred": "", "outcome": "partially_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Medians (nonREM, WAKE, REM) reading (a) 6.26, 2.43, 3.56%; (b) 4.93, 1.58, 2.33%; (c) 11.47, 4.51, 6.39%. Kruskal-Wallis p from 3.4e-46 to 3.6e-54; REM above WAKE by Dunn (p 1.3e-06 to 1e-05) and rank-sum. The order and the tests hold under every reading; no reading puts all three medians within tolerance.", "deviations": [ "Reading (c), the fraction of spikes with an ISI <= 15 ms before or after them, is added from the Results' wording, 'the fraction of spikes with ISIs below 15 ms' (Results, Assessment of firing patterns in different brain states), beside the claims file's readings of a proportion of ISIs." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig2c-autocorrelogram", "statement": "Average pE autocorrelograms peak at about 8 ms in nonREM and show smaller peaks near 125 ms (theta) in REM and WAKE, and decay faster in nonREM (median fit exponent -1.35) than in WAKE (-0.83) and REM (-0.80).", "source": "Results, Assessment of firing patterns in different brain states; Figure 2C", "quote": "a large peak occurred at approximately 8ms inter-spike interval during nonREM, whereas smaller peaks were present at ~125 ms (i.e., theta frequency) during REM and WAKE ( Figure 2C ). ... Time decay of the autocorrelograms of pyramidal neurons was significantly faster during nonREM compared to WAKE and REM ( Figure 2C ; decay fit exponent median nonREM = - 1.35, WAKE = - 0.83, REM = - 0.80; p < 0.05; Kruskal-Wallis test)", "statistics": [ { "test": "Kruskal-Wallis test", "p": 0.05, "p_relation": "<" } ], "kind": "finding", "inferred": "", "outcome": "partially_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Average nonREM autocorrelogram peaks at 8.5 ms; WAKE and REM have local maxima at 125 and 126 ms (nonREM also at 125 ms). Median decay, reading (a) WAKE -0.108, nonREM -0.538, REM -0.169; (b) -0.118, -0.336, -0.112; (c) -0.582, -1.623, -0.574. nonREM is the most negative and Kruskal-Wallis p <= 1.7e-95 under all three; no medians within tolerance. Cells with fewer than 100 counts dropped (158 of 995 in WAKE, 131 in nonREM, 284 in REM). The 125 ms bump is small, the average autocorrelogram at 115 to 135 ms being 1.027 (WAKE), 1.026 (REM) and 0.981 (nonREM) times its mean at 80 to 100 and 150 to 170 ms; the local-maximum test also fires at 125 ms in nonREM.", "deviations": [ "Per-cell peaks are found on a 5 ms boxcar-smoothed autocorrelogram in 1 to 50 ms, and cells with fewer than 100 counts are dropped; both are standard choices the paper does not state." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "synchrony-skew", "statement": "Population synchrony (fraction of pE cells firing in 50 ms windows) has a skewed distribution; more than 15% of pE cells rarely fire synchronously with others (in 4% of windows) while fewer than 4.2% fire synchronously in at least half of the windows.", "source": "Results, Assessment of firing patterns in different brain states; Figure S2F", "quote": "The fraction of principal cells firing in 50 ms overlapping time windows ('population synchrony') showed a skewed distribution. More than 15% of principal neurons rarely fired synchronously with others (4% of windows), while less than 4.2% discharged synchronously in at least half of the windows", "statistics": [], "kind": "finding", "inferred": "", "outcome": "reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Reading (a) 62.9% of pE cells fire with another pE cell in at most 4% of windows and 0.00% in at least half; reading (b) 54.2% and 0.00%. Per-cell distributions skewed (2.31, 2.26); population synchrony per window skew 1.28. The criterion's bounds hold with wide margins.", "deviations": [], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "updown-durations", "statement": "DOWN state durations are skewed with a median of 0.19 s and UP state durations with a median of 0.47 s.", "source": "Results, Assessment of firing patterns in different brain states; Figure S4A; Table S1", "quote": "The durations of DOWN states of 'slow oscillations' showed a skewed distribution (median = 0.19s) and so did the UP states (median = 0.47s; Figure S4A )", "statistics": [], "kind": "finding", "inferred": "", "outcome": "reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Deposited detections, 43230 UP and 86839 DOWN states (the paper's counts exactly), medians 0.473 and 0.188 s, skew 2.56 and 2.15. All start inside scored packets of the cycles.", "deviations": [ "Uses the deposited UP/DOWN detections (_UPDOWNINtervals.mat), reading (a) of updown-durations; a new detection was not run, since the LFP reduction kept has no power above 100 Hz." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig1e-up-state-dynamics", "statement": "Within UP states, firing rates of pyramidal cells and interneurons decrease over the course of the UP state, with decreasing E/(E+I) ratio and decreasing pE population synchrony; UP states begin with a surge of gamma power followed by increased sigma power.", "source": "Results, Assessment of firing patterns in different brain states; Figure 1E", "quote": "UP states were characterized by an initial surge of gamma power, followed by increased sigma power ( Figure 1E ), corresponding to UP state-related sleep spindles. Firing rates of both pyramidal cells and interneurons decreased over the time course of the UP state, accompanied by a decreasing gain of excitation (excitatory-inhibitory, E-I, ratio) and decreasing population synchrony of pyramidal neurons", "statistics": [], "kind": "finding", "inferred": "", "outcome": "not_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Per normalized bin over 43230 UP states, pE rate 1.823, 1.359 ... 1.310, 1.619 Hz (last not below the first two; R = -0.293); pI 15.88 to 11.63 Hz (R = -0.983); synchrony 0.079 to 0.071 (last below first two, R = -0.447); E/(E+I) 0.373 to 0.396 (R = +0.470). Sigma power peaks in bin 0 and gamma in bin 3, the reverse of the statement. Two of four measures decrease and the spectral order fails.", "deviations": [ "The criterion's 50 ms bins for synchrony and E/(E+I) are taken as the 10 normalized UP-state bins (47 ms for a median UP state).", "Gamma and sigma power come from an 8 ms band-power series (4th order Butterworth, mean of the squared signal) on goodeegchannel, log10 and z-scored per session; the paper's spectrogram method is not given.", "Uses the deposited UP/DOWN detections (_UPDOWNINtervals.mat), reading (a) of updown-durations; a new detection was not run, since the LFP reduction kept has no power above 100 Hz.", "The 8 ms series is zero-phase band-passed, which at 9 to 25 Hz spreads power over tens of ms, so the sigma peak in the first bin may carry power from the DOWN-UP edge; the outcome fails on the pE rate and E/(E+I) regardless." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig2de-fast-before-slow", "statement": "In the early UP state pE cells fire in sequences stable across the first and second halves of SLEEP, and a cell's onset latency is negatively correlated with its firing rate.", "source": "Results, Assessment of firing patterns in different brain states; Figure 2D and 2E", "quote": "pyramidal neurons fired in reliable spatiotemporal sequences during the early part of the UP state (<200 msec) and these stereotypical sequences remained stable across the entirety of sleep ( Figure 2D ). Additionally, the spike onset times of individual neurons in the sequences were negatively correlated with their firing rates", "statistics": [], "kind": "finding", "inferred": "", "outcome": "reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "First and second half latencies correlate across cells, R = 0.733 (halves of each SLEEP, n 932) and 0.660 (halves of the session, n 758). Latency against log10 WAKE rate R = -0.127, p = 7.6e-05; against log10 nonREM rate R = -0.238, p = 6e-14. Per session the WAKE-rate correlation is negative in 20 of 27 sessions.", "deviations": [ "A cell's mean latency in a half is computed when it has at least 5 UP states with a spike within 200 ms (a standard choice)." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig2f-rate-group-correlations", "statement": "In nonREM, pairwise spike-count correlations depend on firing rate group; pairs within the fastest group and within the slowest group are positively correlated, fast-firing pairs most highly, and fast and slow cells are negatively correlated with each other.", "source": "Results, Assessment of firing patterns in different brain states; Figure 2F", "quote": "the fastest and slowest firing neurons tended to be positively correlated within groups, but were actually negatively correlated across groups ( Figure 2F )", "statistics": [], "kind": "finding", "inferred": "", "outcome": "not_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Mean 100 ms count correlations in nonREM, WAKE sextiles, 6-6 0.0328 (p = 2.9e-70), 1-1 0.0062 (p = 2.1e-20), 1-6 0.0064 (p = 2e-12, positive). Correlations rise with rate group across the whole matrix; no negative group pair. nonREM sextiles, the claims file's reading (b) for this figure, give 1-6 0.0095 (p = 8.5e-66).", "deviations": [ "Sextiles follow M2. Reading (a) ranks each pE cell's rate over all scored WAKE of its session (before round 1's revision, the mean of its per-cycle WAKE rates, which put 65 of 995 cells in another sextile); reading (b) ranks each cell's rate in each cycle's WAKE and uses the cycle that holds the analysed period." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig3ab-population-rate-declines", "statement": "The mean population firing rate of pE cells declines over SLEEP.", "source": "Results, Firing pattern and LFP changes across the course of sleep; Figure 3A and 3B", "quote": "we found that the arithmetic mean of the population firing rates of pE cells declined over the course of sleep using a test of correlation of spike rate versus time ( Figure 3A-B ; R = - 0.11, p < 10 \u22123 )", "statistics": [ { "test": "correlation of spike rate versus time", "statistic": "R = -0.11", "p": 0.001, "p_relation": "<", "n": "995 cells, 54 sessions, 11 rats" } ], "kind": "finding", "inferred": "", "outcome": "reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Normalized mean pE rate against normalized SLEEP time, clock (i), R = -0.273 (a, p = 1.7e-13), -0.154 (b, p = 2.5e-57), -0.732 (c, p = 1.5e-09); clock (ii) R = -0.283, -0.163, -0.796. By rat (reading b) the decline is significant in 8 of 11 rats; Rizzo, Splinter and Templeton show R = -0.017, -0.049, -0.014.", "deviations": [ "Trends over normalized time follow M3 and M4 of the claims file; every reading (a, b, c) is run and, for SLEEP and episodes, two normalization clocks, (i) the period's clock time and (ii) the cumulative time of the used state, since the paper says only that SLEEP is time-normalized with nonREM time used (Results, Firing pattern and LFP changes across the course of sleep; Fig. 3B legend). Values are divided by the session mean before pooling (M3)." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "first-last-packet-mean-rate", "statement": "The mean pE rate is lower in the last than in the first nonREM packet of SLEEP (1.04 against 0.88 Hz).", "source": "Results, Firing pattern and LFP changes across the course of sleep, sentence citing Vyazovskiy et al., 2009", "quote": "Comparison of the population discharge rates in the first and last nonREM packets of SLEEP episodes showed an overall mean rate decrease (1.04\u00b11.20 Hz versus 0.88\u00b10.91 Hz; p < 0.02; Wilcoxon test)", "statistics": [ { "test": "Wilcoxon test", "p": 0.02, "p_relation": "<" } ], "kind": "finding", "inferred": "", "outcome": "reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Reading (a) first 1.027 +- 1.086, last 0.905 +- 0.936 Hz, signed-rank p = 1e-18; reading (b) 1.048 and 0.883 Hz, signed-rank p = 1.9e-20, rank-sum p = 0.0012. All 51 SLEEP periods have at least 2 packets.", "deviations": [], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "post-sleep-wake-lower", "statement": "pE rates in the 5 min of WAKE after SLEEP are lower than in the 5 min of WAKE before.", "source": "Results, Firing pattern and LFP changes across the course of sleep", "quote": "This effect was 'carried over' to subsequent WAKE as shown by the significantly decreased discharge rate of pE neurons in a 5-min post-sleep WAKE compared to a 5-min pre-sleep WAKE epoch (p < 10 \u22126 ; Wilcoxon test)", "statistics": [ { "test": "Wilcoxon test", "p": 1e-06, "p_relation": "<" } ], "kind": "finding", "inferred": "", "outcome": "partially_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Only 23 cycles have a scored WAKE after their SLEEP; in 17 it starts 166 to 10344 s later. pE rates are lower after, signed-rank p = 0.018 (reading a, 703 cell-cycles) and 0.048 (b, 494 cells); rank-sum p = 0.95 and 0.98. M4 gives partial (sign right, p < 0.05, not below 1e-6). Restricted to the 6 cycles whose next WAKE starts within 1 s, signed-rank p = 0.00026. Variant, the paper's wording read as the first 5 min after SLEEP ends (47 cycles), signed-rank p = 3.9e-10 (a) and 2.8e-06 (b); on average 0.18 of those 5 min is scored as a cycle's WAKE (none in 38 cycles). Whether that is the paper's window is asked of a reader; the outcome is the claims file's reading.", "deviations": [ "The data hold scored WAKE only as the WAKE of a WAKE-SLEEP cycle, so the WAKE after a SLEEP is the next cycle's WAKE, often hours later; the variant restricted to an immediate WAKE is shown." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig3d-first-last-packet-slope", "statement": "Comparing the same cells in the first and last nonREM packets of SLEEP, the slope of last on first rate is below 1 (95% interval 0.827 to 0.880); fast cells decrease and slow cells increase their rates over sleep.", "source": "Results, Firing pattern and LFP changes across the course of sleep; Figure 3D", "quote": "we assessed spike rates of the same neurons in the first versus the last nonREM packets of SLEEP and found the slope of this correlation significantly departed from unity (slope 95% confidence interval 0.827-0.880)", "statistics": [ { "ci": { "level": 0.95, "low": 0.827, "high": 0.88 } } ], "kind": "finding", "inferred": "", "outcome": "reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Reading (b) slope 0.872 (0.840 to 0.904), within 0.8535 +- 0.0265; reading (a) 0.788 (0.760 to 0.817). Zero-rate cells dropped, 23 (b) and 70 (a). 72% of cells above the median first-packet rate decrease; 49% of those below increase.", "deviations": [], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig3b-sextile-changes", "statement": "Over SLEEP (nonREM only), the mean rate of the highest-rate sextiles decreases while the lowest sextile shows a marginal increase.", "source": "Results, Firing pattern and LFP changes across the course of sleep; Figure 3B, bottom", "quote": "The mean spike discharge rate of highest-firing rate sextile groups significantly decreased over sleep whereas the activity of the lowest-firing rate sextile showed a marginal increase ( Figure 3B ; Pearson correlations versus normalized time)", "statistics": [], "kind": "finding", "inferred": "", "outcome": "reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Top sextile R = -0.333, -0.190, -0.716 (readings a, b, c; p <= 5e-09), bottom sextile R = +0.092, +0.046, +0.311 (p = 0.022, 8.5e-06, 0.028); clock (ii) agrees (top R = -0.344, -0.203, -0.799; bottom +0.090, +0.044, +0.360). M2 reading (b) agrees (top R = -0.291, -0.200, -0.756; bottom +0.167, +0.090, +0.550, p <= 3.6e-05).", "deviations": [ "Trends over normalized time follow M3 and M4 of the claims file; every reading (a, b, c) is run and, for SLEEP and episodes, two normalization clocks, (i) the period's clock time and (ii) the cumulative time of the used state, since the paper says only that SLEEP is time-normalized with nonREM time used (Results, Firing pattern and LFP changes across the course of sleep; Fig. 3B legend). Values are divided by the session mean before pooling (M3).", "Sextiles follow M2. Reading (a) ranks each pE cell's rate over all scored WAKE of its session (before round 1's revision, the mean of its per-cycle WAKE rates, which put 65 of 995 cells in another sextile); reading (b) ranks each cell's rate in each cycle's WAKE and uses the cycle that holds the analysed period." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "cv-decreases-over-sleep", "statement": "The coefficient of variation of firing rates decreases over SLEEP.", "source": "Results, Firing pattern and LFP changes across the course of sleep", "quote": "the coefficient of firing rate variation also showed a significantly decreasing slope over the course of SLEEP (R = - 0.05; p < 0.01)", "statistics": [ { "test": "Pearson correlation versus normalized time", "statistic": "R = -0.05", "p": 0.01, "p_relation": "<" } ], "kind": "finding", "inferred": "", "outcome": "not_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Reading (b), the criterion's, R = -0.003, p = 0.73 (clock ii R = -0.012, p = 0.2). Variants, one point per packet R = -0.101, p = 0.0073; 50 bins clock (ii) R = -0.483, p = 0.00039; 50 bins clock (i) R = -0.025, p = 0.86.", "deviations": [ "Trends over normalized time follow M3 and M4 of the claims file; every reading (a, b, c) is run and, for SLEEP and episodes, two normalization clocks, (i) the period's clock time and (ii) the cumulative time of the used state, since the paper says only that SLEEP is time-normalized with nonREM time used (Results, Firing pattern and LFP changes across the course of sleep; Fig. 3B legend). Values are divided by the session mean before pooling (M3)." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig3c-spindles-over-sleep", "statement": "Over SLEEP spindle incidence (1.94/min) decreases while spindle duration (0.52 s), frequency (13.93 Hz) and amplitude (203.3 uV) do not change.", "source": "Results, Firing pattern and LFP changes across the course of sleep; Figure 3C, bottom", "quote": "Incidence of spindles (1.94\u00b10.39/min) decreased (R = - 0. 093; p < 10 \u22125 ), while spindle duration (0.52\u00b10.32 sec), frequency (13.93\u00b10.93 Hz) and amplitude (203.3\u00b187.71 \u00b5V) did not change over SLEEP", "statistics": [ { "about": "spindle incidence against normalized SLEEP time", "test": "correlation", "statistic": "R = -0.093", "p": 1e-05, "p_relation": "<" } ], "kind": "finding", "inferred": "", "outcome": "partially_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Deposited spindles (5201 of 5203 start in nonREM of SLEEP); per-session mean incidence 2.852 +- 0.490 per minute, duration 0.616 s, frequency 13.923 Hz, amplitude 208.4 uV. Frequency and amplitude match; incidence matches Table S1 (0.049 Hz, 2.94 per minute) not the Results' 1.94; duration is outside 0.52 +- 0.052. Incidence over SLEEP, reading (b) R = -0.021, p = 0.03 (partial by M4); (a) p = 0.44; (c) p = 0.66; clock (ii) p >= 0.19. Absence of change holds for duration (p >= 0.068, clock i) but frequency changes under (b) (p = 0.011) and amplitude under (a) (p = 0.047). Over all 5201 spindles the duration is 0.614 +- 0.320 s, median 0.524 s, which matches the paper's 0.52 +- 0.32. With the two Dino ACC sessions left out, the incidence decrease is not significant (reading b p = 0.15).", "deviations": [ "Uses the deposited spindle detections (_Spindles.mat), reading (a) of the criterion; a new detection was not run. Amplitude converted from V to uV.", "Trends over normalized time follow M3 and M4 of the claims file; every reading (a, b, c) is run and, for SLEEP and episodes, two normalization clocks, (i) the period's clock time and (ii) the cumulative time of the used state, since the paper says only that SLEEP is time-normalized with nonREM time used (Results, Firing pattern and LFP changes across the course of sleep; Fig. 3B legend). Values are divided by the session mean before pooling (M3).", "The outcome depends on the normalization clock, which the paper does not state (Results; Fig. 3B legend); clock (i), the period's clock time, and clock (ii), the cumulative nonREM time, give different outcomes here. As for every trend, the outcome takes the result under one of them, as M4 does for readings; the question is put to the paper's authors.", "The incidence decrease also depends on the two Dino days that were recorded with two probes and are deposited as two sessions each (ACC and mPFC); without the ACC copies, reading (b) clock (i) gives p = 0.15. Both copies are kept, since the paper's 27 sessions, which the data set matches, include them." ], "assumed": true, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig3c-updown-over-sleep", "statement": "Over SLEEP the incidence of UP and DOWN states decreases, DOWN state duration increases, UP state duration does not change, and the fraction of time in neither UP nor DOWN increases.", "source": "Results, Firing pattern and LFP changes across the course of sleep; Figure 3C", "quote": "Incidence of both UP states (R = - 0.10; p <10 \u22126 ) and DOWN states (R = - 0.08; p < 10 \u22124 ) decreased significantly, while the duration of DOWN states increased (R slope = 0.09; p < 10 \u22124 ; Figure 3B ) and duration of UP states did not change. In addition, the fraction of time that could be identified as neither UP nor DOWN state ('other') increased significantly over the course of sleep (R = 0.08; p < 10 \u22122 )", "statistics": [ { "about": "UP incidence", "statistic": "R = -0.10", "p": 1e-06, "p_relation": "<" }, { "about": "DOWN incidence", "statistic": "R = -0.08", "p": 0.0001, "p_relation": "<" }, { "about": "DOWN duration", "statistic": "R = 0.09", "p": 0.0001, "p_relation": "<" }, { "about": "fraction neither UP nor DOWN", "statistic": "R = 0.08", "p": 0.01, "p_relation": "<" } ], "kind": "finding", "inferred": "", "outcome": "not_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Reading (b), UP incidence R = -0.013 (p = 0.19), DOWN incidence R = -0.031 (p = 0.0015), DOWN duration R = 0.117 (p = 2.4e-32), UP duration R = 0.020 (p = 0.067), other fraction R = -0.007 (p = 0.44). At most 2 of 5 parts hold under any reading and clock. With the Dino ACC sessions left out, reading (b) gives the same pattern (DOWN incidence p = 0.0023, DOWN duration p = 1.5e-30, UP incidence p = 0.23).", "deviations": [ "Trends over normalized time follow M3 and M4 of the claims file; every reading (a, b, c) is run and, for SLEEP and episodes, two normalization clocks, (i) the period's clock time and (ii) the cumulative time of the used state, since the paper says only that SLEEP is time-normalized with nonREM time used (Results, Firing pattern and LFP changes across the course of sleep; Fig. 3B legend). Values are divided by the session mean before pooling (M3).", "Uses the deposited UP/DOWN detections (_UPDOWNINtervals.mat), reading (a) of updown-durations; a new detection was not run, since the LFP reduction kept has no power above 100 Hz." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig3f-within-up-changes", "statement": "Within UP states, pE rates decrease over SLEEP, and from first to last packet the fastest sextile and interneurons decrease while the slowest sextile increases.", "source": "Results, Firing pattern and LFP changes across the course of sleep; Figure 3F; Figure S3C", "quote": "Comparison of firing rates within UP states showed a general decrease across the whole population ( Figure S3C ) and subtraction of UP state firing activity in early sleep from that in late sleep revealed that the firing rates of the fastest sextile group and putative interneurons decreased later in sleep, whereas firing of the slowest sextile increased", "statistics": [ { "about": "within-UP pE rate against normalized SLEEP time (Figure S3C)", "statistic": "R = -0.19849", "p": 3.3168e-09 } ], "kind": "finding", "inferred": "", "outcome": "reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Within-UP pE rate against SLEEP time, reading (b) R = -0.087, p = 9.4e-16 (meets p < 1e-8); (a) R = -0.159, p = 5.5e-05; (c) R = -0.568, p = 1.7e-05. Last minus first packet, M2 reading (a), fastest sextile -0.637 Hz (p = 1.2e-07, n 159), pI -0.572 Hz (p = 0.045, n 118), slowest +0.071 Hz (p = 0.27, n 151), which would be partial. M2 reading (b), one entry per cell and cycle, fastest -0.425 Hz (p = 3.7e-07, n 276), slowest +0.123 Hz (p = 0.047, n 262); every part holds under one reading of the claims file. On one unit throughout, M2 reading (b) gives all three parts with one entry per cell and cycle (M1 reading a, fastest -0.425 Hz, p = 3.7e-07; pI -0.672 Hz, p = 0.047, n 182; slowest +0.123 Hz, p = 0.047) and with one entry per cell (M1 reading b, fastest -0.573 Hz, p = 3.3e-07, n 181; pI -0.572 Hz, p = 0.045; slowest +0.141 Hz, p = 0.023, n 189). Under M2 reading (a) the slowest sextile is not significant with either unit (p = 0.48 and 0.27). The pI and slowest-sextile tests lie between 0.023 and 0.047.", "deviations": [ "Trends over normalized time follow M3 and M4 of the claims file; every reading (a, b, c) is run and, for SLEEP and episodes, two normalization clocks, (i) the period's clock time and (ii) the cumulative time of the used state, since the paper says only that SLEEP is time-normalized with nonREM time used (Results, Firing pattern and LFP changes across the course of sleep; Fig. 3B legend). Values are divided by the session mean before pooling (M3).", "Uses the deposited UP/DOWN detections (_UPDOWNINtervals.mat), reading (a) of updown-durations; a new detection was not run, since the LFP reduction kept has no power above 100 Hz.", "Sextiles follow M2. Reading (a) ranks each pE cell's rate over all scored WAKE of its session (before round 1's revision, the mean of its per-cycle WAKE rates, which put 65 of 995 cells in another sextile); reading (b) ranks each cell's rate in each cycle's WAKE and uses the cycle that holds the analysed period." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "figs3a-first-last-rem-ma-slopes", "statement": "Rates change differentially across the spectrum also from the first to last REM and first to last MA of SLEEP and from WAKE before to WAKE after sleep, with slopes below 1 (REM 0.76 to 0.83, MA 0.58 to 0.68, WAKE 0.80 to 0.86).", "source": "Results, Firing pattern and LFP changes across the course of sleep, last paragraph, and Sleep sub-states induce differential lasting firing rate changes; Figure S3A", "quote": "first versus last REM episode, also note significant deviation of slope from 1 (confidence interval ... 0.76-0.83). Right first versus last MA episode per sleep session, confidence interval of slope does not include 1 (0.58- 0.68)", "statistics": [ { "about": "first against last REM", "ci": { "level": 0.95, "low": 0.76, "high": 0.83 } }, { "about": "first against last MA", "ci": { "level": 0.95, "low": 0.58, "high": 0.68 } }, { "about": "WAKE before against WAKE after SLEEP", "ci": { "level": 0.95, "low": 0.8, "high": 0.86 } }, { "about": "correlation of WAKE before and WAKE after SLEEP rates (Figure S3A legend)", "test": "Pearson", "p": 1e-10, "p_relation": "<" } ], "kind": "finding", "inferred": "", "outcome": "partially_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Reading (b) REM 0.807 (0.769 to 0.845), MA 0.656 (0.601 to 0.710), WAKE 0.687 (0.627 to 0.746); reading (a) 0.742, 0.530, 0.713. REM and MA within tolerance under (b), WAKE outside under both; all intervals below 1. MA reading (b), short WakeInterruptions as MAs, MA slope 0.618 (0.563 to 0.674, reading b). With the post-sleep window read as the first 5 min after SLEEP, the WAKE slope is 0.793 (0.759 to 0.828, reading b) and 0.787 (reading a), still outside tolerance.", "deviations": [ "WAKE windows as in post-sleep-wake-lower, the next cycle's WAKE, which follows SLEEP immediately in only 6 of 23 cycles." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig3e-wake-before-after-sextiles", "statement": "From 5 min of WAKE before to 5 min of WAKE after SLEEP, the top three sextiles decrease their rates and the lowest sextile increases.", "source": "Results, Sleep sub-states induce differential lasting firing rate changes; Figure 3E", "quote": "Comparison of firing rates during 5 minutes of WAKE immediately before SLEEP versus 5 minutes of WAKE immediately after SLEEP showed a significant decrease firing in the top three sextile groups and an increase in the lowest sextile group (all ps < 0.05; stars in Figure 3E )", "statistics": [ { "test": "one-tailed Wilcoxon", "p": 0.05, "p_relation": "<" } ], "kind": "finding", "inferred": "", "outcome": "partially_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "One-tailed signed-rank per sextile. M2 reading (a), M1 reading (a), sextile 6 p = 0.0018, 5 p = 0.032, 4 p = 0.48, lowest (increase) p = 0.15, 2 of 4 parts; M1 reading (b) 0.00032, 0.073, 0.61, 0.1, 1 of 4. M2 reading (b), per-cycle sextiles, sextile 6 p = 3.2e-06, 5 p = 0.00013, 4 p = 0.23, lowest p = 2e-05, 3 of 4, the criterion's partial outcome. That sextile is ranked on the WAKE that holds the 5 min before SLEEP, so part of the change can be regression to the mean (the session-wide WAKE of reading a holds the window too). Under M2 reading (b) the second-lowest sextile, which the paper does not report as changing, also increases (p = 0.012, against 0.83 under M2 reading a), which fits that concern. Immediate-WAKE variant, 6 cycles, sextile 6 p = 0.00037, 5 p = 0.028, 4 p = 0.11, lowest p = 0.7. With the post-sleep window read as the first 5 min after SLEEP (asked of a reader), 6 p = 9.7e-13, 5 p = 1.7e-06, 4 p = 0.084, lowest p = 0.00096.", "deviations": [ "WAKE windows as in post-sleep-wake-lower.", "Sextiles follow M2. Reading (a) ranks each pE cell's rate over all scored WAKE of its session (before round 1's revision, the mean of its per-cycle WAKE rates, which put 65 of 995 cells in another sextile); reading (b) ranks each cell's rate in each cycle's WAKE and uses the cycle that holds the analysed period." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig4-lfp-within-nrem", "statement": "Within nonREM episodes and packets delta and sigma power increase, and gamma power decreases over episodes but not over packets; packet mean delta power is positively correlated with packet length.", "source": "Results, Firing rate and LFP changes within nonREM episodes and packets; Figure 4A-B i", "quote": "Over nonREM episodes and packets, delta and sigma band power steadily increased ( Trachsel et al., 1988 ), whereas gamma power decreased over episodes but not packets ( Figure 4 A-B i ). ... Mean delta power within whole packets (normalized by nonREM delta power) was positively correlated with packet length (R = 0.31; p < 0.05)", "statistics": [ { "about": "packet delta power against packet length", "statistic": "R = 0.31", "p": 0.05, "p_relation": "<" } ], "kind": "finding", "inferred": "", "outcome": "partially_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Delta and sigma increase within episodes and packets under every reading (packets, reading b, delta R = 0.329, sigma R = 0.306; episodes R = 0.276, 0.209). Gamma within episodes R = -0.220 (a), -0.129 (b), -0.155 (c, p = 0.28) with clock (i), and -0.245, -0.131, -0.121 (p = 0.4) with clock (ii); within packets R = -0.052, p = 6e-08 (b), +0.187, p = 0.19 (c). Packet delta against length R = 0.387, p = 1.8e-26. No single reading gives all six trends; the delta and sigma increases hold under all. Variant, seconds whose 10 s window reaches outside the packet left out, gamma within episodes R = -0.300, p = 0.034 (c) and within packets R = -0.059, p = 3.1e-09 (b), p = 0.14 (c), so under reading (c) all six trends hold with this variant; the claims file does not name it and the outcome is judged on its spectrogram. With the Dino ACC sessions left out, the trends are unchanged (gamma within packets, reading b, R = -0.064, p = 2.2e-10).", "deviations": [ "Band power from a spectrogram of goodeegchannel (10 s Hann windows every 1 s), as the claims file allows.", "Band power is log10-transformed before the per-session z-scoring the criterion names (a standard choice for power, not stated in the paper).", "Trends over normalized time follow M3 and M4 of the claims file; every reading (a, b, c) is run and, for SLEEP and episodes, two normalization clocks, (i) the period's clock time and (ii) the cumulative time of the used state, since the paper says only that SLEEP is time-normalized with nonREM time used (Results, Firing pattern and LFP changes across the course of sleep; Fig. 3B legend). Values are divided by the session mean before pooling (M3)." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig4-updown-within-packets", "statement": "Within packets UP and DOWN incidence increase, UP duration decreases and DOWN duration increases, so the UP/DOWN ratio falls.", "source": "Results, Firing rate and LFP changes within nonREM episodes and packets; Figure 4A-B ii", "quote": "The incidence of both UP states (R = 0.05, p < 10 \u22123 ) and DOWN states (R = 0.08; p < 10 \u221210 ) increased significantly within packets, whereas the duration of UP states decreased (R = - 0.02; p < 0.01) and DOWN states increased (R = 0.07; p < 10 \u221210 ), changing the UP-DOWN balance (UP/DOWN ratio negative correlation versus time p < 0.05)", "statistics": [ { "about": "UP incidence", "statistic": "R = 0.05", "p": 0.001, "p_relation": "<" }, { "about": "DOWN incidence", "statistic": "R = 0.08", "p": 1e-10, "p_relation": "<" }, { "about": "UP duration", "statistic": "R = -0.02", "p": 0.01, "p_relation": "<" }, { "about": "DOWN duration", "statistic": "R = 0.07", "p": 1e-10, "p_relation": "<" }, { "about": "UP/DOWN ratio", "p": 0.05, "p_relation": "<" } ], "kind": "finding", "inferred": "", "outcome": "reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Reading (b), UP incidence R = 0.042 (p = 1.3e-05), DOWN incidence R = 0.168 (p = 3.5e-68), UP duration R = -0.101 (p = 9.2e-21), DOWN duration R = 0.190 (p = 3.6e-83), UP/DOWN ratio R = -0.156 (durations) and -0.035 (time), p = 8.1e-48 and 0.00051. Reading (c) gives at most 1 of 5. Fraction in neither state R = -0.176 (b), the sign the paper prints. With the Dino ACC sessions left out, the four incidence and duration trends keep their signs (reading b, p <= 1.5e-05).", "deviations": [ "Trends over normalized time follow M3 and M4 of the claims file; every reading (a, b, c) is run and, for SLEEP and episodes, two normalization clocks, (i) the period's clock time and (ii) the cumulative time of the used state, since the paper says only that SLEEP is time-normalized with nonREM time used (Results, Firing pattern and LFP changes across the course of sleep; Fig. 3B legend). Values are divided by the session mean before pooling (M3).", "Uses the deposited UP/DOWN detections (_UPDOWNINtervals.mat), reading (a) of updown-durations; a new detection was not run, since the LFP reduction kept has no power above 100 Hz." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig4-spindles-within-nrem", "statement": "Spindle incidence and duration increase within nonREM episodes and packets (duration 0.61 to 0.67 s in episodes, 0.58 to 0.67 s in packets), while spindle frequency and amplitude do not change.", "source": "Results, Firing rate and LFP changes within nonREM episodes and packets; Figure 4A-B iii", "quote": "The incidence of spindles significantly increased both within nonREM episodes ( Figure 4A-B iii ; R = 0.08; p < 10 \u221210 ) and packets (R = 0.7; p < 10 \u221210 ). Spindle duration also increased (nonREM episode increase from 0.61 \u00b1 0.33 s to 0.67 \u00b1 0.30 s, R = 0.08; p < 10 \u22123 ; nonREM packet increase from 0.58 \u00b1 0.25 s to 0.67 \u00b1 0.27 s, R = 0.07, p < 10 \u22124 ), whereas spindle frequency and amplitude remained unchanged", "statistics": [ { "about": "spindle incidence within episodes", "statistic": "R = 0.08", "p": 1e-10, "p_relation": "<" }, { "about": "spindle incidence within packets", "statistic": "R = 0.7", "p": 1e-10, "p_relation": "<" }, { "about": "spindle duration within episodes", "statistic": "R = 0.08", "p": 0.001, "p_relation": "<" }, { "about": "spindle duration within packets", "statistic": "R = 0.07", "p": 0.0001, "p_relation": "<" } ], "kind": "finding", "inferred": "", "outcome": "reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Reading (b), incidence within episodes R = 0.083 (p = 1.1e-17), packets R = 0.142 (p = 2.5e-49); duration R = 0.075 (p = 5.4e-06) and 0.091 (p = 3.9e-08). First and last tenth durations 0.597 and 0.650 s (episodes), 0.603 and 0.651 s (packets), all within tolerance. The criterion's reproduced condition (increases and durations) holds. Amplitude does not change (p >= 0.16); frequency does not change within episodes (p = 0.11) but falls within packets (R = -0.043, p = 0.0096), so that part of the statement fails; the criterion's final rule does not judge it. With the Dino ACC sessions left out, incidence and duration still increase (reading b, p <= 2e-06).", "deviations": [ "Deposited spindle detections.", "Trends over normalized time follow M3 and M4 of the claims file; every reading (a, b, c) is run and, for SLEEP and episodes, two normalization clocks, (i) the period's clock time and (ii) the cumulative time of the used state, since the paper says only that SLEEP is time-normalized with nonREM time used (Results, Firing pattern and LFP changes across the course of sleep; Fig. 3B legend). Values are divided by the session mean before pooling (M3)." ], "assumed": false, "disagreed": true, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig4e-late-packet-sigma", "statement": "UP states late in nonREM packets have more sigma power than UP states early in packets.", "source": "Results, Firing rate and LFP changes within nonREM episodes and packets; Figure 4E; Figure S4B and S4D", "quote": "Spectral comparison of UP states in first third versus last third of packets showed increased sigma power ( Figure 4E ; p < 10 \u221210 ; Wilcoxon rank-sum test and per-pixel significance; Figure S4B )", "statistics": [ { "about": "UP state sigma power, first against last third of packets", "test": "Wilcoxon rank-sum test", "p": 1e-10, "p_relation": "<" }, { "about": "DOWN-UP transition-triggered sigma power, late against early packet (Results, Figure S4B)", "test": "t-test", "p": 1e-05, "p_relation": "<" } ], "kind": "finding", "inferred": "", "outcome": "reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "UP states of at least 1 s, late third mean sigma z 0.153 against early 0.009 (n 1708 and 1866), rank-sum p = 3.7e-34; UP states over 500 ms, 0.151 against 0.031, p = 2.5e-76.", "deviations": [ "Sigma power per UP state is the mean of an 8 ms band-power series (log10, z-scored per session), not an UP-onset-triggered spectrogram.", "Uses the deposited UP/DOWN detections (_UPDOWNINtervals.mat), reading (a) of updown-durations; a new detection was not run, since the LFP reduction kept has no power above 100 Hz." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig4-sextiles-within-nrem", "statement": "Within nonREM episodes and packets the fastest sextiles decrease and the slowest increase their rates, so the CV of population rates decreases, and the 15 ms burst index increases.", "source": "Results, Firing rate and LFP changes within nonREM episodes and packets; Figure 4A-B iv, v and vi", "quote": "Firing rates of the fastest-firing sextile groups decreased, whereas those of the slowest-firing sextile groups increased within both nonREM episodes and packets ( Figure 4A, B iv ; R and p values in figure), resulting in a decreasing coefficient of variation of binned population firing rates ( Figure 4 A-B v ; R episode = \u22120.13, p < 10 \u221210 ; R packet = - 0.12, p < 10 \u221210 ). An additional metric, the 15ms spike burst index of pyramidal neurons, also showed a positive slope within both nonREM episodes and packets ( Figure 4 A\u2013B vi ; R episode = 0.09, p < 10 \u221210 ; R packet = 0.08, p < 10 \u221210 )", "statistics": [ { "about": "CV within episodes", "statistic": "R = -0.13", "p": 1e-10, "p_relation": "<" }, { "about": "CV within packets", "statistic": "R = -0.12", "p": 1e-10, "p_relation": "<" }, { "about": "burst index within episodes", "statistic": "R = 0.09", "p": 1e-10, "p_relation": "<" }, { "about": "burst index within packets", "statistic": "R = 0.08", "p": 1e-10, "p_relation": "<" } ], "kind": "finding", "inferred": "", "outcome": "reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Reading (b), top sextile R = -0.139 (episodes), -0.137 (packets); bottom R = +0.029 (p = 0.0048), +0.044 (p = 2.3e-05); CV R = -0.103 (p = 1e-26), -0.139 (p = 5.2e-47); burst ratio within packets R = +0.089 (p = 2.9e-20). Within episodes the burst ratio meets p < 1e-10 with clock (ii) (R = 0.063, p = 8.5e-11) and misses it with clock (i) (R = 0.061, p = 2.6e-10), which would make that correlation partial by M4. The outcome takes the better of the two clocks, here clock (ii), as for every trend.", "deviations": [ "Trends over normalized time follow M3 and M4 of the claims file; every reading (a, b, c) is run and, for SLEEP and episodes, two normalization clocks, (i) the period's clock time and (ii) the cumulative time of the used state, since the paper says only that SLEEP is time-normalized with nonREM time used (Results, Firing pattern and LFP changes across the course of sleep; Fig. 3B legend). Values are divided by the session mean before pooling (M3).", "Sextiles follow M2. Reading (a) ranks each pE cell's rate over all scored WAKE of its session (before round 1's revision, the mean of its per-cycle WAKE rates, which put 65 of 995 cells in another sextile); reading (b) ranks each cell's rate in each cycle's WAKE and uses the cycle that holds the analysed period.", "The outcome depends on the normalization clock, which the paper does not state (Results; Fig. 3B legend); clock (i), the period's clock time, and clock (ii), the cumulative nonREM time, give different outcomes here. As for every trend, the outcome takes the result under one of them, as M4 does for readings; the question is put to the paper's authors." ], "assumed": true, "disagreed": true, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig4cd-packet-rate-lfp-correlations", "statement": "Across packets, the top sextile's rate correlates negatively with delta power, DOWN duration and DOWN incidence and positively with UP duration and gamma power, and the bottom sextile's rate correlates positively with sigma power in the last 20 s of the packet; other combinations are not significant.", "source": "Results, Firing rate and LFP changes within nonREM episodes and packets; Figure 4C and 4D", "quote": "Per-packet firing rate of the top sextile group was negatively correlated with per-packet LFP metrics, including whole-packet delta power (Z-scored per session; Figure 4C ; R= - 0.28; p < 10 \u221210 ), DOWN state duration (R = - 0.16, p < 10 \u22124 ), and DOWN state incidence rate (Z-scored per session; R = - 0.29, p<10 \u221210 ). A positive correlation was observed between firing rate of the top sextile group and UP state duration (R = 0.14, p < 10 \u22123 ) and gamma power ( Figure 4D ; R = 0.44; p < 10 \u221210 ). Finally, a positive correlation was found between normalized firing rates per packet in the bottom sextile group and sigma power in the last 20 sec of the packet (R = 0.25; p < 10 \u221210 ). Other tested relationship combinations were not significant", "statistics": [ { "about": "top sextile rate against delta power", "statistic": "R = -0.28", "p": 1e-10, "p_relation": "<" }, { "about": "top sextile rate against DOWN duration", "statistic": "R = -0.16", "p": 0.0001, "p_relation": "<" }, { "about": "top sextile rate against DOWN incidence", "statistic": "R = -0.29", "p": 1e-10, "p_relation": "<" }, { "about": "top sextile rate against UP duration", "statistic": "R = 0.14", "p": 0.001, "p_relation": "<" }, { "about": "top sextile rate against gamma power", "statistic": "R = 0.44", "p": 1e-10, "p_relation": "<" }, { "about": "bottom sextile rate against sigma power in the last 20 s", "statistic": "R = 0.25", "p": 1e-10, "p_relation": "<" } ], "kind": "finding", "inferred": "", "outcome": "partially_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "M2 reading (a), top sextile against delta R = -0.200 (p = 1.6e-06), DOWN duration R = -0.387 (p = 1.7e-21), DOWN incidence R = -0.300 (p = 2.9e-13), UP duration R = 0.248 (p = 2.8e-09), gamma R = 0.358 (p = 1.4e-18); bottom sextile against last-20 s sigma R = 0.215 (p = 7.7e-08). Four meet the paper's bounds and two have its sign at p < 0.05; 4 of 6, the criterion's partial outcome. (With the sextiles of the first revision, the mean of per-cycle WAKE rates, 3 of 6 held.) M2 reading (b) also gives 4 of 6, with no sigma correlation (R = 0.062, p = 0.12). Other combinations, bottom sextile against UP duration R = 0.110, p = 0.01, top sextile against late sigma R = -0.093, p = 0.026; the rest p >= 0.23. Sigma from 10 s windows inside the last 20 s, R = 0.199, p = 6.9e-07.", "deviations": [ "Band power is log10-transformed before the per-session z-scoring the criterion names (a standard choice for power, not stated in the paper).", "Uses the deposited UP/DOWN detections (_UPDOWNINtervals.mat), reading (a) of updown-durations; a new detection was not run, since the LFP reduction kept has no power above 100 Hz.", "Sextiles follow M2. Reading (a) ranks each pE cell's rate over all scored WAKE of its session (before round 1's revision, the mean of its per-cycle WAKE rates, which put 65 of 995 cells in another sextile); reading (b) ranks each cell's rate in each cycle's WAKE and uses the cycle that holds the analysed period." ], "assumed": false, "disagreed": true, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig4e-within-up-sextiles", "statement": "Within UP states, from early to later parts of packets, the fastest sextile's rate drops, interneuron rate increases, and the slowest sextile decreases slightly.", "source": "Results, Firing rate and LFP changes within nonREM episodes and packets; Figure 4E, bottom", "quote": "Comparison of within-UP state activity in the first versus second third of packets showed a drop in firing rate for the fastest sextile group (P < 10 \u22128 ; Wilcoxon rank-sum test), an increase in firing rate of putative interneurons (P < 10 \u22123 ), and a small but significant decrease for the slowest group (P < 0.003; Figure 4B, E )", "statistics": [ { "about": "fastest sextile", "test": "Wilcoxon rank-sum test", "p": 1e-08, "p_relation": "<" }, { "about": "putative interneurons", "test": "Wilcoxon rank-sum test", "p": 0.001, "p_relation": "<" }, { "about": "slowest sextile", "test": "Wilcoxon rank-sum test", "p": 0.003, "p_relation": "<" } ], "kind": "finding", "inferred": "", "outcome": "not_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Per UP state, first against second third, fastest 3.43 to 3.39 Hz (p = 5.8e-05), pI 13.11 to 13.65 Hz (p = 0.00078), slowest 0.29 to 0.31 Hz (p = 0.64, rising); against the last third, fastest p = 0.0069, pI p = 3.1e-05, slowest rising p = 5.3e-07. Per cell (unit ii, one entry per cell, its within-UP rate in each third over all its packets; 166, 126 and 165 cells) no group meets its bound (p >= 0.11). At most 1 of 3 holds under any reading, M2 reading (b) included.", "deviations": [ "Uses the deposited UP/DOWN detections (_UPDOWNINtervals.mat), reading (a) of updown-durations; a new detection was not run, since the LFP reduction kept has no power above 100 Hz.", "Sextiles follow M2. Reading (a) ranks each pE cell's rate over all scored WAKE of its session (before round 1's revision, the mean of its per-cycle WAKE rates, which put 65 of 995 cells in another sextile); reading (b) ranks each cell's rate in each cycle's WAKE and uses the cycle that holds the analysed period.", "Unit (ii) has one entry per cell, its spikes and UP time summed over all its packets in each third (the first revision had one entry per cell and packet)." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "figs5-within-wake-rem", "statement": "Within WAKE the two highest sextiles increase and the two lowest decrease their rates, opposite to nonREM; within REM all sextiles except the fourth increase.", "source": "Results, Firing rate and LFP changes within nonREM episodes and packets, last paragraph; Figure S5A and S5B", "quote": "We found that entry into WAKE corresponded with an increase in firing rate of the highest sextile two groups and a drop in firing rate of the lowest two sextile groups ( Figure S5A ), opposing the effects seen in SLEEP and nonREM. During REM, all sextile groups except the fourth showed an increase in firing rate", "statistics": [], "kind": "finding", "inferred": "", "outcome": "not_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "50 bins, sextiles numbered 1 lowest, M2 reading (a). WAKE, sextile 6 R = 0.107 (p = 0.46), 5 R = 0.471 (p = 0.00055), 2 R = 0.603 (p = 3.5e-06, increasing), 1 R = -0.286 (p = 0.044). REM, sextiles 1 to 6 R = -0.245, 0.443, 0.358, 0.394, -0.064, 0.308 (p = 0.086, 0.0013, 0.011, 0.0047, 0.66, 0.029); of the five other than sextile 3, three increase. M2 reading (b) holds less (WAKE sextile 6 p = 0.58; REM sextiles 1, 4, 5, 6 p >= 0.065).", "deviations": [ "Sextiles follow M2. Reading (a) ranks each pE cell's rate over all scored WAKE of its session (before round 1's revision, the mean of its per-cycle WAKE rates, which put 65 of 995 cells in another sextile); reading (b) ranks each cell's rate in each cycle's WAKE and uses the cycle that holds the analysed period." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig5b-ma-nrem-ma", "statement": "An intervening nonREM packet reduces the rates of the top two sextiles from one MA to the next, with an inverse relation between rate and change across cells.", "source": "Results, Sleep sub-states induce differential lasting firing rate changes; Figure 5B, blue; Figure S5C", "quote": "Firing rates in the top two firing rate sextiles were significantly reduced by the intervening nonREM packet (both p < 0.01; Figure 5B , blue)", "statistics": [ { "about": "top two sextiles, MA n+1 against MA n", "p": 0.01, "p_relation": "<" }, { "about": "log rate change against WAKE rate (Figure S5C)", "statistic": "R = -0.14923", "p": 3.9886e-06 } ], "kind": "finding", "inferred": "", "outcome": "not_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "169 MA-packet-MA triplets. Top two sextiles mean change -0.011 and -0.010 Hz (cells averaged), signed-rank and t-test p >= 0.24 under M1 a and b. S5C correlation R = 0.016 (p = 0.65, log10 WAKE rate, 843 cells), R = 0.006 (linear). M2 reading (b), top two p >= 0.12. MA reading (b), short WakeInterruptions as MAs, 234 triplets, top two p >= 0.28, S5C R = -0.054 (p = 0.11).", "deviations": [ "Triplets use adjacent scored intervals inside a SLEEP with gaps of at most 2 s (a standard choice for intervals scored at 1 s); the paper does not say how adjacency is decided. Assumed, as the reading closest to the Supplemental Experimental Procedures' description of consecutive sub-states.", "Sextiles follow M2. Reading (a) ranks each pE cell's rate over all scored WAKE of its session (before round 1's revision, the mean of its per-cycle WAKE rates, which put 65 of 995 cells in another sextile); reading (b) ranks each cell's rate in each cycle's WAKE and uses the cycle that holds the analysed period." ], "assumed": true, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "figs5d-rem-nrem-rem", "statement": "For REM, packet, REM triplets the rate change is inversely correlated with rate.", "source": "Results, Sleep sub-states induce differential lasting firing rate changes; Figure S5D", "quote": "Analysis of rate change versus firing rate for these triplets also showed an inverse correlation between rate and change ( Figure S5C ), as did a similar analysis for REM n - packet - REM n+1 triplets ( Figure S5D )", "statistics": [ { "statistic": "R = -0.39537", "p": 1.9003e-06 } ], "kind": "finding", "inferred": "", "outcome": "partially_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Adopted rule, MA or WakeInterruption allowed between REM n and the packet, 80 triplets in 17 sessions, 660 pE cells (paper about 136). S5D correlation R = -0.097 (p = 0.015, linear WAKE rate, reading b) and R = -0.077 (p = 0.054, log10, reading a); the paper's sign at p < 0.05, short of p < 1e-5, partial by M4. All states adjacent, 25 triplets in 3 sessions, 57 cells, R = 0.019 (p = 0.89) and -0.055 (p = 0.69). Interruptions allowed on both sides, 84 triplets, R = -0.071 (p = 0.071) and -0.103 (p = 0.0089). No rule gives the paper's number of cells.", "deviations": [ "REM n, packet, REM n+1 triplets allow an MA or WakeInterruption between REM n and the packet, the packet and REM n+1 being adjacent (gap at most 2 s). Assumed because the Supplemental Experimental Procedures tolerate an arousal after REM in nonREM-REM-nonREM triplets (claims file, M6), and REM in these files is followed by a WakeInterruption more often than by a packet, so the same tolerance is applied after REM n. The all-adjacent rule (the first revision's) and a rule allowing interruptions on both sides are shown; the question is put to the paper's authors." ], "assumed": true, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig5c-sigma-ma-change", "statement": "Higher sigma power in the last 20 s of a packet goes with a larger increase of the lowest sextile from the MA before to the MA after it.", "source": "Results, Sleep sub-states induce differential lasting firing rate changes; Figure 5C, lower right", "quote": "we found that sigma power in the last 20 seconds of nonREM packets was correlated with a greater increase in the lowest firing-rate sextile group in the MA n versus MA n+1 analysis ( Figure 5C ; R = 0.181, p < 0.01)", "statistics": [ { "statistic": "R = 0.181", "p": 0.01, "p_relation": "<" } ], "kind": "finding", "inferred": "", "outcome": "not_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "150 MA-packet-MA triplets with lowest-sextile cells; R = -0.042, p = 0.61. Sigma from 10 s windows inside the last 20 s R = -0.018 (p = 0.83); MA reading (b) R = -0.013 (p = 0.86, 209 triplets); M2 reading (b) R = -0.053 (p = 0.52).", "deviations": [ "Triplets use adjacent scored intervals inside a SLEEP with gaps of at most 2 s (a standard choice for intervals scored at 1 s); the paper does not say how adjacency is decided. Assumed, as the reading closest to the Supplemental Experimental Procedures' description of consecutive sub-states.", "Band power is log10-transformed before the per-session z-scoring the criterion names (a standard choice for power, not stated in the paper).", "Sextiles follow M2. Reading (a) ranks each pE cell's rate over all scored WAKE of its session (before round 1's revision, the mean of its per-cycle WAKE rates, which put 65 of 995 cells in another sextile); reading (b) ranks each cell's rate in each cycle's WAKE and uses the cycle that holds the analysed period." ], "assumed": true, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig5b-wake-between-packets", "statement": "WAKE between nonREM packets increases the rate of the fastest sextile, more so with more movement; its decrease of slow cells is not significant (24 triplets).", "source": "Results, Sleep sub-states induce differential lasting firing rate changes; Figure 5B, black; Figure 5C, upper left; Figure S5E", "quote": "WAKE episodes increased the activity of the fastest-firing sextile group of pyramidal cells from nonREM packet n to packet n+1 ( Figure 5B , black). Furthermore, WAKE periods with more movement had a proportionally larger effect ( Figure 5C ; R = 0.43; p < 0.05). WAKE also appeared to exert a rate-decreasing effect on slow firing neurons, although this effect was not significant", "statistics": [ { "about": "movement during WAKE against fastest sextile change", "statistic": "R = 0.43", "p": 0.05, "p_relation": "<", "n": "24 nonREM-WAKE-nonREM triplets" }, { "about": "log rate change against WAKE rate (Figure S5E)", "statistic": "R = 0.043591", "p": 0.2963 } ], "kind": "finding", "inferred": "", "outcome": "partially_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "Reading (a) 8 triplets, reading (b) 52 (paper 24). Fastest sextile change +0.092 Hz (p = 0.21) and +0.063 Hz (p = 0.15), cells averaged. Movement correlation, reading (a), raw motion R = 0.436 (p = 0.33, n 7), motion z-scored per session R = 0.869 (p = 0.011, n 7); reading (b) R = 0.124 and 0.248 (p = 0.43, 0.11). M2 reading (b), z-scored R = 0.924 (p = 0.0029, n 7). Slow cells, no significant decrease (lowest sextile p >= 0.12), as in the paper; S5E correlation absent (p >= 0.75), as in the paper. Partial rests on the z-scored motion under reading (a), with 7 triplets. The stored motion's per-session means run from 0.55 to 4076, so the raw values pooled across sessions mostly compare sessions.", "deviations": [ "Triplets use adjacent scored intervals inside a SLEEP with gaps of at most 2 s (a standard choice for intervals scored at 1 s); the paper does not say how adjacency is decided. Assumed, as the reading closest to the Supplemental Experimental Procedures' description of consecutive sub-states.", "The motion signal is z-scored per session before averaging over the WAKE period. Assumed because the data description says the signal is z-scored while the stored values are not (non-negative, per-session means 0.55 to 4076); the raw signal gives no correlation.", "Sextiles follow M2. Reading (a) ranks each pE cell's rate over all scored WAKE of its session (before round 1's revision, the mean of its per-cycle WAKE rates, which put 65 of 995 cells in another sextile); reading (b) ranks each cell's rate in each cycle's WAKE and uses the cycle that holds the analysed period." ], "assumed": true, "disagreed": true, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig5b-ma-between-packets", "statement": "MAs between packets do not change fast-firing cells but increase the rate of slow-firing cells, with an inverse relation between rate and change across cells.", "source": "Results, Sleep sub-states induce differential lasting firing rate changes; Figure 5B, yellow; Figure S5G", "quote": "In contrast to WAKE, MA epochs between packets did not affect fast-firing neurons but significantly increased the rate of slow-firing neurons from packet n to packet n+1 ( Figure 5B , yellow). Again, the differential effect across sextiles was confirmed by a correlation analysis", "statistics": [ { "about": "log rate change against WAKE rate (Figure S5G)", "statistic": "R = -0.20668", "p": 7.7388e-11 } ], "kind": "finding", "inferred": "", "outcome": "partially_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "264 packet-MA-packet triplets. Lowest sextile, one entry per cell and triplet (M1 a), mean +0.0075 Hz, median +0.0017 Hz, 51% positive, signed-rank p = 0.015, t-test p = 0.016; cells averaged +0.007 Hz, p = 0.36; lowest two together p = 0.92. M2 reading (b), lowest sextile +0.012 Hz cells averaged (p = 0.022), per cell and triplet p = 0.00088. Top sextile decreases (-0.140 Hz, p = 5.4e-08), where the paper finds no change. S5G R = -0.174, p = 4.9e-08 (log10), misses p < 1e-10. MA reading (b) gives the same 264 triplets.", "deviations": [ "Triplets use adjacent scored intervals inside a SLEEP with gaps of at most 2 s (a standard choice for intervals scored at 1 s); the paper does not say how adjacency is decided. Assumed, as the reading closest to the Supplemental Experimental Procedures' description of consecutive sub-states.", "Sextiles follow M2. Reading (a) ranks each pE cell's rate over all scored WAKE of its session (before round 1's revision, the mean of its per-cycle WAKE rates, which put 65 of 995 cells in another sextile); reading (b) ranks each cell's rate in each cycle's WAKE and uses the cycle that holds the analysed period." ], "assumed": true, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig5b-rem-between-packets", "statement": "REM between packets decreases the rates of all sextiles, by about the same proportion, with longer REM giving larger decreases; REM theta power correlates with the decrease of the slowest sextile only.", "source": "Results, Sleep sub-states induce differential lasting firing rate changes; Figure 5B, red; Figure 5C, upper right; Figure S5F", "quote": "Finally, REM episodes decreased the spike rates from packet n to packet n+1 in all sextile groups ( Figure 5B , red). Longer REM episodes exerted a proportionally larger effect on rate changes from packet n to packet n+1 for all sextile groups ( Figure 5C ; R = - 0.22; p < 10 \u22122 ; t-test) and REM theta power was significantly correlated with the packet n to packet n+1 rate decrease of the slowest sextile group only (R = - 0.15; p < 0.05). Correlational analysis did not reveal differential effects of REM on neurons across the rate spectrum", "statistics": [ { "about": "REM duration against rate change", "test": "t-test", "statistic": "R = -0.22", "p": 0.01, "p_relation": "<" }, { "about": "REM theta power against slowest sextile change", "statistic": "R = -0.15", "p": 0.05, "p_relation": "<" }, { "about": "log rate change against WAKE rate (Figure S5F)", "statistic": "R = -0.038346", "p": 0.2372 } ], "kind": "finding", "inferred": "", "outcome": "not_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "110 packet-REM-packet triplets. Sextile changes (1 lowest, cells averaged) -0.016, -0.041, -0.012, +0.006, +0.036, +0.164 Hz (signed-rank p 0.11, 0.00056, 0.72, 0.5, 0.23, 0.065). Duration correlation R = -0.019 (p = 0.85). Theta against slowest change R = 0.143 (ratio, p = 0.18), 0.068 (power, p = 0.53). S5F R = 0.068 (p = 0.06, log10), 0.122 (p = 0.00069, linear), where the paper finds none. M2 reading (b), one sextile decreases (the second lowest, p = 0.01).", "deviations": [ "Triplets use adjacent scored intervals inside a SLEEP with gaps of at most 2 s (a standard choice for intervals scored at 1 s); the paper does not say how adjacency is decided. Assumed, as the reading closest to the Supplemental Experimental Procedures' description of consecutive sub-states.", "The second packet is the next packet after the REM episode with any MA or WakeInterruption between them (M6, Supplemental Experimental Procedures).", "Sextiles follow M2. Reading (a) ranks each pE cell's rate over all scored WAKE of its session (before round 1's revision, the mean of its per-cycle WAKE rates, which put 65 of 995 cells in another sextile); reading (b) ranks each cell's rate in each cycle's WAKE and uses the cycle that holds the analysed period." ], "assumed": true, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "fig5c-delta-change-slow-cells", "statement": "Regardless of the intervening state, the increase of the slowest sextile from packet n to n+1 correlates positively with the change in delta power between the packets; the same is not found for the fastest sextile.", "source": "Results, Sleep sub-states induce differential lasting firing rate changes; Figure 5C, lower left", "quote": "The increase of spike rate from packet n to packet n+1 in the slowest sextile group was positively correlated with the delta power change from packet n to packet n+1 ( Figure 5C )", "statistics": [], "kind": "finding", "inferred": "", "outcome": "not_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "567 consecutive packet pairs; slowest sextile change against delta change R = 0.003 (p = 0.95); fastest R = -0.285 (p = 2.8e-11, 524 pairs). Delta from windows inside the packets R = -0.008 (p = 0.86); M2 reading (b) R = -0.011 (p = 0.78).", "deviations": [ "Band power is log10-transformed before the per-session z-scoring the criterion names (a standard choice for power, not stated in the paper).", "Sextiles follow M2. Reading (a) ranks each pE cell's rate over all scored WAKE of its session (before round 1's revision, the mean of its per-cycle WAKE rates, which put 65 of 995 cells in another sextile); reading (b) ranks each cell's rate in each cycle's WAKE and uses the cycle that holds the analysed period." ], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "figs2d-interneurons-unchanged", "statement": "Unlike pyramidal cells, interneuron rates in the first and last packets of SLEEP lie on a line of slope 1 (95% interval 0.996 to 1.005).", "source": "Figure S2D legend", "quote": "Comparison of log firing rates of individual putative interneurons during the first and last packets of SLEEP. Unlike for pE cells (Figure 2B) the regression line is not different from unity (slope 95% confidence interval ... 0.996-1.005)", "statistics": [ { "ci": { "level": 0.95, "low": 0.996, "high": 1.005 } } ], "kind": "finding", "inferred": "", "outcome": "not_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "pI slope of log10 last on first packet rate, reading (a) 0.836 (0.766 to 0.906, n 192), reading (b) 0.888 (0.802 to 0.973, n 123); neither interval includes 1.", "deviations": [], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null }, { "claim": "sleep-homogenizes-rates", "statement": "The net effect of sleep is to homogenize the firing rate distribution, fast cells decreasing and slow cells increasing, so the population rate dispersion falls.", "source": "Discussion, paragraph 1; Figure 6", "quote": "During nonREM, activity of high firing rate neurons decreased, whereas activity of slow firing neurons increased. REM sleep reduced firing rates relatively uniformly of all firing rate groups, whereas microarousals, interspersed between nonREM epochs, increased the discharge rates of slow firing neurons. Accordingly, the net result of sleep is that the firing rate variation of the population is decreased", "statistics": [], "kind": "finding", "inferred": "", "outcome": "partially_reproduced", "blocked_by": null, "notebook": "reproduce_watson_2016.ipynb", "notes": "fig3d-first-last-packet-slope reproduced (0.872, 0.840 to 0.904); fig3e-wake-before-after-sextiles partially reproduced (3 of 4 parts under M2 reading b, 2 under reading a); cv-decreases-over-sleep not reproduced (R = -0.003, p = 0.73). Of the sub-state claims reported beside it, fig5b-ma-between-packets is partially reproduced and fig5b-rem-between-packets is not.", "deviations": [], "assumed": false, "disagreed": false, "data": "dataset", "where": null, "proposed": null } ], "findings": [ { "kind": "metadata_error", "statement": "The files hold 51 WAKE-SLEEP cycles (WakeSleepTimePairFormat over the 27 sessions), where the paper reports 54; the Wake and Sleep intervals are the same 51 pairs, with none unpaired. All other counts (995 pE, 126 pI, 27 sessions, 11 rats, 43230 UP and 86839 DOWN states, 5203 spindles) match the paper exactly.", "found": null, "status": null }, { "kind": "metadata_error", "statement": "The paper reports 790 microarousals; 434 MA intervals lie inside SLEEP (2130 MA intervals in the files, most outside SLEEP). 345 WakeInterruption intervals inside SLEEP are shorter than the 40 s minimum the data description gives for them; none lies between two packets (123 between two REM intervals, 80 between REM and a packet), so they are not MAs by the paper's definition either, though counted with the MAs they give 779 events. One nonREM packet (14 s) is shorter than the 20 s minimum of Table S1.", "found": null, "status": null }, { "kind": "metadata_error", "statement": "In Rizzo_022715 and Splinter_020515 a WAKE-SLEEP cycle's WAKE starts before the previous cycle's SLEEP ends (WAKE 9919-14394 s against SLEEP 5655-10010 s; WAKE 5335-7210 s against SLEEP 3543-5455 s), so the cycles overlap.", "found": null, "status": null }, { "kind": "other", "statement": "Dino_061914 and Dino_062014 each appear as two sessions, ACC and mPFC probes recorded at the same time (the data set's WatsonSleepHomestasis2016Table.xlsx lists the same days), each with its own state files, and the two scorings of Dino_062014 differ (mean SLEEP per cycle 59.5 min in ACC and 27.1 min in mPFC; 41 and 17 packets). State- and LFP-level analyses over the 27 sessions count these days twice.", "found": null, "status": null }, { "kind": "metadata_error", "statement": "The data description says the motion signal in _Motion.mat is z-scored; it is non-negative in every session, with per-session means from 0.55 to 4076, and BWRat19_032513 holds NaN values.", "found": null, "status": null }, { "kind": "metadata_error", "statement": "RecordingFileIntervals in _BasicMetaData.mat ends at infinity in the three sessions of rat JennBuzsaki22, and the UP/DOWN file is named _UPDOWNINtervals.mat, where the description says _UPDOWNInts.mat and _UPDOWNIntervals.mat.", "found": null, "status": null }, { "kind": "paper_inconsistency", "statement": "The Results give spindle incidence as 1.94 +- 0.39 per minute; Table S1 gives 0.049 +- 0.008 Hz (2.94 per minute). The deposited spindles give 2.85 per minute, matching Table S1.", "found": null, "status": null }, { "kind": "metadata_insufficient", "statement": "Scored WAKE exists only as the WAKE of a WAKE-SLEEP cycle, so the WAKE that follows a SLEEP is in most cycles absent or hours later (23 of 51 cycles have a later scored WAKE, 6 within 1 s). The 5 min post-sleep WAKE windows of the paper cannot be located for most cycles.", "found": null, "status": null }, { "kind": "other", "statement": "Each session is one tar.gz of 0.9 to 6 GB, and the order of members differs between files (the LFP .eeg comes before the spike and state files in some), so reading any one small file needs the whole archive streamed. Reading all 27 takes 7 to 9 minutes from the bucket.", "found": null, "status": null }, { "kind": "methods_unavailable", "statement": "Questions for the paper's authors. (1) The sampling unit of the correlations against normalized time (per packet, 10 s bins, or bins of normalized time averaged over periods) and whether SLEEP is normalized by its clock time or by its concatenated nonREM time; the readings give different outcomes for the CV, spindle and UP/DOWN trends over SLEEP. (2) Whether the burst index is the proportion of ISIs <= 15 ms (Supplemental Experimental Procedures) or the fraction of spikes with such an ISI (Results); the latter gives the reported nonREM median. (3) The model and fit range of the autocorrelogram decay exponent. (4) How triplets are formed (adjacency of scored intervals, which WAKE counts between packets, giving the 24 nonREM-WAKE-nonREM triplets) and whether REM-packet-REM triplets allow an arousal after the first REM; the attempt assumes they do, which gives 80 triplets and 660 cells against the about 136 cells of Figure S5D. (5) Whether the motion signal was z-scored before the movement correlation. (6) How the WAKE sextiles are computed for cells recorded over several cycles (session WAKE or each cycle's WAKE); fig3e-wake-before-after-sextiles and fig3f-within-up-changes differ between the two.", "found": null, "status": null } ], "reconciliation": null, "verifications": [ { "date": "2026-11-07", "kind": "agent", "by": "Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook)", "reviewed": "Round 1. The first executed notebook with the conclusion cells and the summary removed, the helper module and the claims file, with the paper. Its report had no attempt record and gave its own outcome for each claim; 35 of 41 agree with the record as it then stood.", "agreed": 35, "of": 41, "disagreements": [ { "claim": "ma-statistics", "outcome": "partially_reproduced", "reason": "Cell 11, from the files' MA intervals, n 434 (790), median 11.0 s (15.9), 0.184/min (0.32) and 3.7% (9.1) are all outside tolerance. The episode trend holds (20 bins R -0.956, p 4.5e-11), and the SLEEP trend shows no change on bin means (p 0.10-0.15)." }, { "claim": "fig2a-pe-rate-distribution", "outcome": "not_reproduced", "reason": "Cell 16, 5 of 6 medians are within tolerance (pI REM 5.72 vs 4.25 is not). The shape test fails because the nonREM pE range stops short of 4.5 Hz under both percentile readings (1-99th; 4.05 and 4.17 Hz). See B11 on the criterion." }, { "claim": "fig4-spindles-within-nrem", "outcome": "reproduced", "reason": "Cell 65 (b), all four increases are below their bounds and all four durations are within tolerance. The packet frequency does change (p 0.0096), against the absence claim, but the criterion's final rule leaves this part out." }, { "claim": "fig4-sextiles-within-nrem", "outcome": "reproduced", "reason": "Cell 69 (b), all sextile, CV and burst parts hold under clock (ii). Under clock (i) the episode burst p is 2.6e-10, just above the 1e-10 bound." }, { "claim": "fig4cd-packet-rate-lfp-correlations", "outcome": "not_reproduced", "reason": "Cell 71, 3 of 6 meet their bounds; the criterion needs 4. All six have the right sign at p < 0.05." }, { "claim": "fig5b-wake-between-packets", "outcome": "not_reproduced", "reason": "Cell 85, 8 or 52 triplets (paper 24). The fastest sextile does not increase. Motion vs change; R 0.48, p 0.28 with raw motion. Raw motion is pooled across sessions whose means run from 0.55 to 4076; pooled that way the correlation is not meaningful. The z-scored variant matches the data description's own definition and would make the claim partially reproduced, but it rests on only n = 7 (B4)." } ] }, { "date": "2026-11-07", "kind": "agent", "by": "Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook)", "reviewed": "Round 2. The full executed notebook after the first revision, the helper module, the claims file and the attempt record, with the paper. It agreed with all 41 outcomes. Also corrected in reply to its general points, the read time in the README, the header cell and the findings (7 to 9 minutes; cell 2 printed 9.1 and 7.2 min in this round's two runs).", "agreed": 41, "of": 41, "disagreements": [] } ] } ], "entries": [ { "name": "crcns-fcx-1-watson-2016", "type": "reproduction", "title": "Network homeostasis and state dynamics of neocortical sleep (Watson et al. 2016)", "description": "Tests the findings of Watson, Levenstein, Greene, Gelinas and Buzsaki (2016) on CRCNS data set fcx-1, which holds the spike trains, scored brain states, detected UP/DOWN states and spindles, and LFP of 27 frontal cortex recordings in 11 rats. It compares firing rates across states, their changes over sleep and within nonREM episodes and packets, the LFP and UP/DOWN dynamics, and the rate changes across sleep sub-state triplets.", "datasets": [ "crcns:fcx-1" ], "papers": [ "10.1016/j.neuron.2016.03.036" ], "derived_from": [], "notebooks": [ { "path": "reproduce_watson_2016.ipynb", "title": "Reproducing Watson et al. (2016) from CRCNS fcx-1", "executed": "notebooks/crcns-fcx-1-watson-2016/reproduce_watson_2016.html" } ] } ] }