Claims 39
From the paper IBL behavioral data · claims file · checked 2026-10-22: ready
- cohort-counts
Of 206 mice that started training, 140 reached proficiency in the basic task and 98 in the full task.
Out of 206 mice that started training, 140 achieved proficiency in the basic task, and 98 in the full task
Results, opening section, paragraph 3How it will be judged
Counts of distinct mice under the paper's selection (project mice, criterion reached by 2020-03-23, code query_subjects), proficiency as P1 and P2 under each reading. A dataset that holds fewer mice is reported with the counts it holds; the claim is judged on the counts. P1 per the header (answers of 2026-10-22): trials completed are all trials; a mouse is proficient at the first session meeting the 1a or the 1b criteria, at any date (the pipeline's SessionTrainingStatus); 1b replacing 1a from 2019-09-12 (code text_trained1a_to_1b_sessions.py) is a variant.
- fig2g-days-to-proficiency
Basic task training took 18.4 +- 13.0 days (s.d., n = 140 mice), from 3 days for the fastest mouse to 59 days for the slowest.
The average training took 18.4 ± 13.0 days (s.d., n = 140, Figure 2g )
Results, Training succeeded but with different rates across mice and laboratories, paragraph 1; Figure 2gHow it will be judged
Per mouse, the number of training days (days with a session in the per-day summary the code reads) up to P1 (code figure2g_time_to_trained.py). The code's query branch counts days up to and including the P1 day, its CSV branch days before it; both readings are run. Mean and s.d. over the 140 mice (or those the dataset holds), and the minimum and maximum.
- fig2g-days-differ-across-labs
The number of days to basic task proficiency differed across laboratories; within-lab interquartile ranges ranged from 8 days (Lab 2) to 22 days (Lab 6).
The number of days needed to achieve basic task proficiency was different across laboratories ( Figure 2g , p<0.001, Kruskal-Wallis nonparametric test followed by a post-hoc Dunn's multiple comparisons test)
Results, Training succeeded but with different rates across mice and laboratories, paragraph 1; Figure 2gHow it will be judged
Days per mouse as in fig2g-days-to-proficiency, under both of its readings. Kruskal-Wallis test across the seven labs (the code switches to a one-way ANOVA when the days pass scipy's normality test; shown as a variant). Reproduced when p < 0.001. Interquartile ranges per lab are pandas' default (linear) quantiles, as in the code.
- fig2s1-trials-to-proficiency
Measured in trials, basic task training took 10.8 +- 8.6 thousand trials (s.d., n = 140), from one thousand for the fastest mouse to 43 thousand for the slowest; reaching 80% correct on easy trials took on average about 7.4 thousand trials.
For mice that learned the task, the average training took 10.8 ± 8.6 thousands of trials (s.d., n = 140) ... The fastest learner met training criteria in one thousand trials, the slowest 43 thousand trials. To reach 80% on easy trials took on average ~7.4 thousand trials.
Results, Training succeeded but with different rates across mice and laboratories, paragraph 2; Figure 2, figure supplement 1How it will be judged
Per mouse, the summed trial counts of the training days counted in fig2g-days-to-proficiency (code figure2g_time_to_trained.py), under both of its readings. The 7.4 thousand trials is printed by supp_figure2_performance_trials.py, and its method is followed. The 140 proficient mice, sessions (training days) with more than 100 trials, each mouse's easy-trial performance median-filtered over 3 sessions, training day counted from 0 at the mouse's first kept session, cumulative trials per mouse summed over its kept sessions; per training day, the mean over mice of performance and of cumulative trials; the value is the smallest mean cumulative trial count among training days whose mean performance exceeds 80%. Variant, shown only: the mean over mice of each mouse's cumulative trials at its first session above 80% correct on easy trials.
- fig2h-proportion-learned-40-days
About 80% of mice that started training would learn the basic task within the 40 days usually allotted (Kaplan-Meier estimate over all 206 mice).
there was a 80% probability that mice would learn the task within the 40 days that were usually allotted, and when mice were trained for a longer duration, the success rate rose even further ( Figure 2h )
Results, Training succeeded but with different rates across mice and laboratories, paragraph 4; Figure 2hHow it will be judged
Kaplan-Meier estimate (code figure2d_training_probability.py, lifelines) over all 206 mice that started training (Figure 2 legend, "Data in h is for all mice that started training (n = 206)"): time = number of sessions with status in_training or untrainable; event = the mouse's last status is neither, so mice still in training at the cutoff date are censored. The code leaves out mice still in training after the cutoff date; that is a variant. Without status labels in the dataset, time = training days to P1 for proficient mice and their total training days otherwise, with P1 as in the header (answer of 2026-10-22): "all contrasts introduced" in each of the three sessions (Appendix 1, table 1d) is the main reading, and the pipeline's check of the last session only (0% and 6.25% present) a variant. Reproduced when the cumulative proportion at day 40 is within the tolerance and higher at the last day than at day 40.
- fig2bc-threshold-bias-iqr
Contrast threshold and bias varied little during training; the threshold interquartile range slightly decreased from 12-29% in the first 10 training days to 11-23% thereafter, and the bias interquartile range went from -5.2 to 7.1% to -5.6 to 6.5%.
Estimates of contrast threshold did not vary much during training ( Figure 2b ), with an interquartile range that slightly decreased from 12–29% during the first 10 training days, and 11–23% thereafter. Likewise, bias did not vary much, with an interquartile range of −5.2% to 7.1% during the first 10 days to −5.6% to 6.5% thereafter ( Figure 2c ).
Results, Training succeeded but with different rates across mice and laboratories, paragraph 1; Figure 2b,cHow it will be judged
Per-session psychometric threshold and bias of the 140 proficient mice, as plotted in Figure 2b,c (code figure2af_learning_curves_all_parameters.py): 50:50 sessions (the basic-task sessions with all their trials, header; answer of 2026-10-22) with more than 100 trials, threshold and bias only from sessions with six or more signed contrasts (legend: from the first session with 12% contrast), each mouse's series median-filtered over 3 sessions, training day counted from 0 at the mouse's first session (the code), so the first 10 training days are code days 0 to 9. The code does not print the interquartile ranges, so the pooling is open; readings (1) all session values pooled over mice within days 1-10 and after day 10, (2) as (1) without the median filter, each with "thereafter" as (a) days 11 to 40 (the plotted range) and (b) all later days. Reproduced when all eight bounds are within tolerance under one reading and the threshold range after day 10 is narrower than in days 1-10.
- fig2-abs-bias-threshold-during-training
During training mice were not systematically biased to one side; absolute bias averaged 9.8 +- 12.1 (s.d.), and contrast threshold, which stabilized after about 10 sessions, averaged 17.8 +- 11.7%.
Mice across laboratories were not systematically biased towards a particular side and the absolute bias was similar across laboratories (9.8 ± 12.1 [s.d.] on average). Likewise, measures of contrast threshold stabilized after the first ~10 sessions, which was on average 17.8 ± 11.7% on average across institutions.
Results, Training succeeded but with different rates across mice and laboratories, paragraph 2; Figure 2b,cHow it will be judged
Same per-session values and readings as fig2bc-threshold-bias-iqr. No code computes these numbers; the averaging is open, with readings (1) mean and s.d. over all session values pooled over mice (absolute bias over all days; threshold after day 10), and (2) mean of the seven lab means ("on average across institutions") with the s.d. over pooled values. Reproduced when the two means are within tolerance under one reading.
- fig2s2-variability-decreases
Variability in performance across mice decreased with training but did not disappear; between training days 15 and 40, mean easy-trial performance rose from 80.7% to 91.1% while its s.d. across mice fell from 19.1% to 10.1%.
For example, between training days 15 and 40, when the average performance on easy trials of the mice increased from 80.7% to 91.1%, the variation across mice (s.d.) of performance on easy trials decreased from 19.1% to 10.1%.
Results, Training succeeded but with different rates across mice and laboratories, paragraph 3; Figure 2, figure supplement 2How it will be judged
Easy-trial performance per mouse and training day for the 140 proficient mice. No code prints these four numbers. Readings (1) supp_figure2_performance_trials.py, which computes the mean and s.d. over mice per training day (behav_summary, behav_summary_std) from sessions with more than 100 trials, median-filtered over 3 sessions, training day counted from 0 at the first kept session, at training days 15 and 40 of that count (and at 14 and 39, if the text counts from 1); (2) the Figure 2a,d data (code figure2af: as (1) and only 50:50 sessions) at the same days; (3) the unfiltered per-day performance binned as in suppfig_variability_over_time.py (training_day strictly within 2 days of the bin centre) at days 15 and 39, the nearest bin to 40. Reproduced when the mean rises and the s.d. falls, with all four values within tolerance under one reading.
- early-sessions-predict-slow-learners
A Random Forests decoder applied to the change in easy-trial performance over the first five sessions predicted whether a mouse would be in the bottom quartile of learning speed with 53% accuracy (chance 25%), and misclassified a top-quartile mouse as bottom-quartile only 7% of the time.
A Random Forests decoder applied to change in performance (% correct with easy, high-contrast stimuli) in the first five sessions was able to predict whether a mouse would end up in the bottom quartile of learning speed (the slowest learners) with accuracy of 53% (where chance is 25%). Conversely, the chance of misclassifying a fast-learning, top quartile mouse as a slow-learning, bottom quartile mouse, was only 7%.
Results, Training succeeded but with different rates across mice and laboratories, paragraph 3How it will be judged
The repository at the cited commit holds no code for this analysis and the Methods do not describe it. Learning-speed quartiles from days to P1 (fig2g-days-to-proficiency; trials to P1, as in Figure 2, figure supplement 1c, is a further reading). Open details and the readings the text allows: features (a) the four session-to-session differences in easy-trial performance over sessions 1-5, (b) the five performances; labels (A) the four quartiles, (B) bottom quartile against the rest (chance 25% then the base rate); scikit-learn Random Forest with default settings and leave-one-out cross-validation, as for the lab classifiers in the Methods; accuracy read as (i) recall of the bottom-quartile class, (ii) overall four-class accuracy under (A), and (iii) precision of the bottom-quartile class. If the readings disagree on whether accuracy is near 53% and well above 25%, the claim is methods_underspecified.
- fig3c-easy-errors-basic
At basic task proficiency, errors on easy (50% and 100%) contrasts were low, 9.5 +- 3.6%, with no significant difference across laboratories.
The lapse rates (i.e., the errors made in response to easy contrasts of 50% and 100%) were low (9.5 ± 3.6%, Figure 3c ) with no significant difference across laboratories.
Results, Performance in the basic task was indistinguishable across laboratories, paragraph 1; Figure 3cHow it will be judged
Per mouse, 100 minus the easy-trial performance over the pooled S1 trials (code figure3cde, perf_easy); mice with no valid trial duration are dropped, as in the code. Mean and s.d. over mice. The lab comparison is claim fig3cde-no-lab-difference-medians.
- fig3de-threshold-bias-basic
At basic task proficiency the contrast threshold was similar across laboratories at 14.3 +- 3.8 (s.d., n = 7 laboratories) and the response bias was small at 0.3 +- 8.4.
The slope of the curves, which measures contrast sensitivity, was also similar across laboratories, at 14.3 ± 3.8 (s.d., n = 7 laboratories, Figure 3d ). Finally, the horizontal displacement of the curve, which measures response bias, was small at 0.3 ± 8.4 (s.d., n = 7, Figure 3e ).
Results, Performance in the basic task was indistinguishable across laboratories, paragraph 1; Figure 3d,eHow it will be judged
Per mouse, psychometric threshold and bias fitted to the pooled S1 trials (code figure3cde). The code prints the mean and s.d. over mice (thres_mean, thres_std) while the text says n = 7 laboratories; readings (1) mean and s.d. over mice, (2) mean and s.d. over the seven lab means. The bias tolerance is 8.4/sqrt(7), the paper's s.d. over n = 7, because 10% of a value near zero is below any fitting precision.
- fig3cde-no-lab-difference-medians
At basic task proficiency, easy-trial performance, contrast threshold and bias did not differ in median across laboratories.
None of these measures showed a significant difference across laboratories, either in median (performance: p=0.63, threshold: p=0.81, bias: p=0.81, FDR corrected Kruskal-Wallis test)
Results, Performance in the basic task was indistinguishable across laboratories, paragraph 1; Figure 3c-eHow it will be judged
Per-mouse values from S1 as in fig3c-easy-errors-basic and fig3de-threshold-bias-basic. Kruskal-Wallis test across the seven labs for each variable, then Benjamini-Hochberg FDR over the six variables the code tests together (easy-trial performance, trial duration, trials per session, threshold, bias, trials per minute; code figure3cde). Reproduced when all three corrected p >= 0.05, not reproduced when any is below; the effect sizes are reported. The code's switch to ANOVA for normally distributed variables is shown as a variant.
- fig3cde-no-lab-difference-variances
At basic task proficiency, easy-trial performance, contrast threshold and bias did not differ in variance across laboratories.
or in variance (performance: p=0.09, threshold: p=0.57, bias: p=0.57, FDR corrected Levene's test)
Results, Performance in the basic task was indistinguishable across laboratories, paragraph 1; Figure 3c-eHow it will be judged
As fig3cde-no-lab-difference-medians, with scipy's Levene test (default, median-centred) and Benjamini-Hochberg FDR over the same six variables (code figure3cde). Reproduced when all three corrected p >= 0.05.
- fig3s2a-trial-duration
At basic task proficiency the median trial duration (stimulus onset to feedback) was 468 +- 221 ms, with some differences across laboratories.
The median time from stimulus onset to feedback (trial duration, a coarse measure of reaction time) was 468 ± 221 ms, showing some differences across laboratories ( Figure 3—figure supplement 2a , p=0.004, Kruskal-Wallis nonparametric test)
Results, Performance in the basic task was indistinguishable across laboratories, paragraph 4; Figure 3, figure supplement 2aHow it will be judged
Per mouse, the median of response time minus stimulus onset time over the pooled S1 trials, no-response trials left out (code figure3cde, reaction_time); the median and s.d. over mice (code rt_median, rt_std). Kruskal-Wallis across labs; the code FDR-corrects this p with the other five variables, the text does not say; readings uncorrected and corrected. Reproduced when the values are within tolerance and p < 0.05 under one reading.
- fig3s2b-trials-per-session
At basic task proficiency mice performed 719 +- 223 trials per session; the difference across laboratories was significant but only in one laboratory relative to the rest.
Mice on average performed 719 ± 223 trials per session ( Figure 3—figure supplement 2b ). This difference was significant (p<10 −6 , one-way ANOVA) but only in one laboratory relative to the rest.
Results, Performance in the basic task was indistinguishable across laboratories, paragraph 4; Figure 3, figure supplement 2bHow it will be judged
Per mouse, the mean number of trials over its S1 sessions (code figure3cde, n_trials); mean and s.d. over mice. One-way ANOVA across labs (code; the FDR correction it applies is a variant), then Tukey post hoc tests as in the code. Reproduced when the values are within tolerance, the ANOVA gives p < 1e-6, and the significant Tukey pairs all involve one lab. Partially reproduced when p < 0.05 but not below 1e-6, or the pairs involve more than one lab.
- fig3f-classifier-chance-basic
A Naive Bayes classifier could not identify a mouse's laboratory from its basic-task performance, threshold and bias; its accuracy was 0.12 +- 0.051 against a chance level of 0.11, and the 95% range of its distribution included chance.
Its mean accuracy was 0.12 ± 0.051, and a 95th percentile of its distribution included the chance level of 0.11 (mean of the null distribution).
Results, Performance in the basic task was indistinguishable across laboratories, paragraph 5; Figure 3fHow it will be judged
Lab classifier (shared definition) with scikit-learn GaussianNB on per-mouse easy-trial performance, threshold and bias from S1 (code figure3f_classifier_lab_membership_basic.py). The paper's test, from the code (figure3f_plot_classifier_basic.py), is that the classifier is not above chance when the 2.5th percentile of its 2000 accuracies is at or below the mean shuffled accuracy. Reproduced when that holds, not reproduced when the 2.5th percentile is above chance. The accuracy tolerance is the paper's stated s.d. of the subsample distribution. Subsample size per the Methods' rule (header, Lab classifier; answer of 2026-10-22): the lowest lab count over all datasets classified when it is below 8, the same for every classifier claim; 8 as in the code is a variant.
- fig3f-positive-control
With the laboratory's time zone added as a feature, the classifier performed above chance and confused labs in the same time zone.
If the classifier was given this variable during training and testing it performed above chance ( Figure 3f , Positive control ) and the confusion matrix showed clusters of labs located in the same time zone
Results, Performance in the basic task was indistinguishable across laboratories, paragraph 5; Figure 3f; Figure 3, figure supplement 3bHow it will be judged
As fig3f-classifier-chance-basic with the lab's time zone as a fourth feature (the code's offsets, Europe 0, New York -5, Los Angeles -7, from the lab's location). Reproduced when the 2.5th percentile of the control accuracies is above the mean shuffled accuracy (the code also computes a 0.001th percentile, sig_control, shown as a variant) and the mean confusion matrix puts more off-diagonal mass on labs in the same time zone than on labs in others.
- fig3s3-other-classifiers
Random Forest and Logistic Regression classifiers also failed to predict laboratory membership from basic-task behavior.
Similar results were obtained with two other classifier algorithms ( Figure 3—figure supplement 3d–i ).
Results, Performance in the basic task was indistinguishable across laboratories, paragraph 5; Figure 3, figure supplement 3d-iHow it will be judged
As fig3f-classifier-chance-basic with scikit-learn RandomForestClassifier (100 trees, random_state 424242) and LogisticRegression (solver liblinear), as in the code. Reproduced when neither is above chance by the same rule.
- lab-link
The institutions named in the dandiset's files are the paper's seven laboratories, so that the code's lab numbers (Lab 1 to 7) and time zones can be given to the dataset's mice.
How it will be judged
Each session's institution in the files is matched by name to the code's laboratories. Reproduced when the seven institutions match the seven laboratories one to one. Whether the session start times bear out the time zones (start hours shifted by the labs' offsets if they are in UTC, as stated) is reported. Added by the author of attempt 000045-ibl-2021-01 (protocol, Author, step 4) for fig3f-positive-control, whose time-zone feature comes from the paper's code by laboratory.
- fig3s1a-choice-consistency-basic
In the basic task, mouse choices were no more consistent within laboratories than across laboratories.
This averaged consistency was not significantly higher (p=0.73) than a null distribution generated by randomly shuffling lab assignments between mice and computing the average within-lab choice variability 10,000 times ( violin plot ). Therefore, choices were no more consistent within labs than across labs.
Figure 3, figure supplement 1a legendHow it will be judged
Per mouse, the fraction of rightward choices at each signed contrast; per lab, the variance across mice at each contrast, its mean over contrasts, and the inverse (consistency); the mean over labs compared with 10,000 shuffles of lab labels across mice; p = fraction of shuffles above the data (legend; code suppfig_choicevariability_withinacrosslabs.py, where 50% contrast trials are dropped). Sessions per the legend, S1 ("same data as in Figure 3"); the code instead uses the P1 day and the two days after (days_from_criterion [0, 2]), shown as a variant. Reproduced when p >= 0.05.
- fig3s1bc-choice-consistency-full
In the full task, neither choice fractions nor the bias shift between blocks were more consistent within laboratories than expected by chance.
Within-lab choice consistency on the full task was not higher than expected by chance, p=0.25. ... Within-lab consistency in bias shift was not higher than expected by chance (p=0.31).
Figure 3, figure supplement 1b,c legend; Results, Performance in the full task was indistinguishable across laboratories, paragraph 4How it will be judged
As fig3s1a-choice-consistency-basic on S2 trials (legend "same data as in Figure 4"; code), with consistency computed per signed contrast and block before averaging (b), and for the bias shift (c) per mouse and contrast the fraction rightward in 20:80 blocks minus 80:20 blocks. Reproduced when both p >= 0.05.
- fig3s4-looser-criterion
With a looser proficiency criterion (80% correct on easy trials over three sessions, n = 150 mice), behavior was again consistent across laboratories; only the number of trials per session differed slightly, and a classifier failed to identify a mouse's laboratory.
there was a small but significant difference in the number of trials per session across laboratories. All other measured parameters were similar.
Results, Performance in the basic task was indistinguishable across laboratories, paragraph 6; Figure 3, figure supplement 4How it will be judged
Sessions per the code (query_session_around_performance, suppfig_3-4a-f.py): for each mouse, the first three consecutive basic sessions each with more than 80% correct on easy trials; sessions with five or more signed contrasts kept; mice with three such sessions. Lab tests and FDR as in fig3cde-no-lab-difference-medians on the same six variables; Naive Bayes classifier on performance, threshold and bias as in fig3f-classifier-chance-basic (suppfig_classifier_lab_membership_perf.py). Reproduced when trials per session has corrected p < 0.05, the other five have p >= 0.05, and the classifier is not above chance.
- fig3s5-first-full-sessions
In the first three sessions of the full task, bias in each block, contrast threshold in each block and performance on 100% contrast trials did not vary significantly across laboratories, and a classifier could not predict lab membership.
Bias for each block prior did not vary significantly over labs (Kruskal-Wallis test, 20:80 blocks; p=0.96, 50:50 block; p=0.96, 80:20 block; p=0.89). ( b ) The contrast thresholds also did not vary systematically over labs (Kruskal-Wallis test, 20:80 block; p=0.078, 50:50 block; p=0.12, 80:20 block; p=0.17). ( c ) Performance on 100% contrast trials neither (Kruskal-Wallis test, p=0.15).
Figure 3, figure supplement 5 legend; Results, Performance in the basic task was indistinguishable across laboratories, paragraph 6How it will be judged
Per mouse and block, psychometric fits to the pooled trials of the first three full-task sessions, and performance on 100% contrast trials (legend; the code's correct_easy, which also counts any 50% contrast trials, is a variant). Sessions per the legend, the first three biased sessions from P1b, the first biased session on or after P1 (header; answer of 2026-10-22), with the code's P1b (first biased session with status trained_1b) a variant; the code (suppfig_variability_over_labs_first_biased.py) takes the biased sessions on training days 1 to 3 after P1b, and its classifier script days -1 to 3; both shown as variants. Kruskal-Wallis across labs for each of the seven variables, as the legend names it (the code corrects the seven p values with statsmodels' default Holm-Sidak method; variant). Naive Bayes classifier on the seven variables as in fig3f-classifier-chance-basic. Reproduced when all seven p >= 0.05 and the classifier is not above chance.
- fig3s5a-prior-used-from-first-sessions
Mice incorporated the block priors into their choices from the first sessions of the full task.
Mice incorporated block priors into their choices already from the first sessions in which they were exposed to this full task
Results, Performance in the full task was indistinguishable across laboratories, paragraph 2; Figure 3, figure supplement 5aHow it will be judged
Per-mouse bias in 80:20 and 20:80 blocks from the sessions of fig3s5-first-full-sessions (both readings). The paper reports no test for this; the test is the one the paper uses for the block effect on bias in Figure 4i (fig4i-bias-shifts-with-block), a Wilcoxon signed-rank test of bias in 80:20 against 20:80 blocks. Reproduced when bias is higher in 80:20 than in 20:80 blocks (median of paired differences) with p < 0.05. The median difference is also reported against the full-task proficiency threshold bias[80:20] - bias[20:80] > 5 (Appendix 1, table 1d), a criterion for counting mice and not a test, as a variant.
- full-task-training-time
Mice that reached full task proficiency took 31.5 +- 16.1 days, or 20,494 +- 10,980 trials (s.d., n = 98), from the start of training; the fastest took 8 days (4691 trials), the slowest 81 days (61,316 trials).
For mice that reached full task proficiency, the average training from start to finish took 31.5 ± 16.1 days, or 20,494 ± 10,980 trials (s.d., n = 98). ... The fastest learner achieved proficiency in 8 days (4691 trials), the slowest 81 days (61,316 trials).
Results, Performance in the full task was indistinguishable across laboratories, paragraph 2How it will be judged
Per mouse reaching P2, the last training-day number and the summed trials over training days up to and including the P2 day (code supp_queries.py, n_training_days = MAX(training_day), total_trials = SUM(n_trials_date)); mean, s.d., minimum and maximum over mice.
- fig4cd-biasshift-largest-at-zero
Block structure had the greatest effect on choices when sensory evidence was absent; the shift in rightward choices between 20:80 and 80:20 blocks was highest at 0% contrast in every laboratory, averaging 28.5%.
The average shift in rightward choices was invariably highest at 0% contrast, where rightward choices in the two blocks differed by an average of 28.5%.
Results, Performance in the full task was indistinguishable across laboratories, paragraph 2; Figure 4c-eHow it will be judged
Per mouse, psychometric fits to the pooled S2 trials of the 20:80 and 80:20 blocks; the bias shift at each signed contrast is the fitted rightward choice percentage in 20:80 minus 80:20 (code figure4de_psychfuncs_biased.py; the Figure 4c legend reads it from the curves). Mean over mice at 0%. Reproduced when the mean is within tolerance and, in each of the seven labs, the lab mean is highest at 0% contrast.
- fig4de-biasshift-no-lab-difference
The bias shift at 0% contrast did not differ across laboratories more than within them.
This value did not significantly differ across laboratories any more than it differed within laboratories (one-way ANOVA F(6) = 1.345, p=0.2455, Figure 4d,e ).
Results, Performance in the full task was indistinguishable across laboratories, paragraph 2; Figure 4d,eHow it will be judged
Per-mouse bias shift at 0% from fig4cd-biasshift-largest-at-zero; one-way ANOVA on lab (code, statsmodels ols and type 2 anova_lm). Reproduced when p >= 0.05.
- fig4f-threshold-unchanged-by-block
Block type did not significantly affect the contrast threshold.
Changes in block type did not significantly affect the contrast threshold (Wilcoxon Signed-Rank test, p=0.85, n = 98 mice, Figure 4f ).
Results, Performance in the full task was indistinguishable across laboratories, paragraph 3; Figure 4fHow it will be judged
Per mouse, psychometric fits to the pooled S2 trials of 80:20 and of 20:80 blocks (code figure4fghi_variability_over_labs_full.py); scipy Wilcoxon signed-rank test of threshold in the two blocks. Reproduced when p >= 0.05.
- fig4gh-lapses-change-with-block
Block type changed lapse rates, which were higher on the left in 20:80 blocks and higher on the right in 80:20 blocks.
However, it did change the lapse rates, which were consistently higher on the left in the 20:80 blocks and on the right in the 80:20 blocks
Results, Performance in the full task was indistinguishable across laboratories, paragraph 3; Figure 4g,hHow it will be judged
Fits as in fig4f-threshold-unchanged-by-block; left lapse gamma and right lapse lambda per block; Wilcoxon signed-rank tests between blocks. Reproduced when the left lapse is higher in 20:80 and the right lapse higher in 80:20 (median of paired differences) with p < 1e-6 and p < 1e-7; partially reproduced when the directions hold with p < 0.05 only.
- fig4i-bias-shifts-with-block
Block type changed the overall bias consistently, with curves shifting left in 20:80 blocks and right in 80:20 blocks.
Finally, as expected there was a highly consistent change in overall bias, with curves shifting to the left in 20:80 trials and to the right in 80:20 trials
Results, Performance in the full task was indistinguishable across laboratories, paragraph 3; Figure 4iHow it will be judged
Fits as in fig4f-threshold-unchanged-by-block; Wilcoxon signed-rank test of bias mu between blocks. Reproduced when mu is lower in 20:80 than in 80:20 blocks (median of paired differences) with p < 1e-16; partially reproduced when the direction holds with p < 0.05 only.
- fig4fi-no-lab-difference
Threshold, lapses and bias in each block type showed no significant differences across laboratories.
An analysis of the psychometric curves showed highly consistent effects of block type, with no significant differences across laboratories ( Figure 4f–i ).
Results, Performance in the full task was indistinguishable across laboratories, paragraph 2; Figure 4f-iHow it will be judged
The paper gives no test; the code (figure4fghi) runs the lab test (shared definition) on each of the eight per-block values (threshold, left lapse, right lapse, bias, in 80:20 and 20:80) and corrects the eight p values with Benjamini-Hochberg FDR. Reproduced when all eight corrected p >= 0.05.
- fig4j-classifier-chance-full
A classifier trained on the full-task psychometric parameters could not predict the laboratory of individual mice above chance.
Just as in the basic task ( Figure 3 ), a classifier trained on these variables could not predict above chance the origin laboratory of individual mice above chance ( Figure 4j ).
Results, Performance in the full task was indistinguishable across laboratories, paragraph 3; Figure 4jHow it will be judged
Lab classifier with GaussianNB on the eight per-block values of fig4fi-no-lab-difference from S2 (code figure4i_classifier_lab_membership_full.py); same rule as fig3f-classifier-chance-basic. Reproduced when not above chance.
- fig5cf-glm-weights
In a logistic choice model fitted to each mouse, visual weights grew with contrast to above 3 at 100% contrast, while weights for past choices were much smaller and positive after both rewarded (basic 0.19, full 0.42) and unrewarded (basic 0.33, full 0.46) trials, indicating perseverance.
As expected, visual terms had large weights, which grew with contrast to reach values above 3 at 100% contrast. Weights for non-sensory factors were much lower ( Figure 5c,f , note different scale). Weights for past choices were positive for both rewarded (basic 0.19, full 0.42) and unrewarded previous trials (basic 0.33, full 0.46)
Results, A probabilistic model reveals common strategies across mice and laboratories, paragraph 2; Figure 5c,d,f,gHow it will be judged
Model per Methods (Probabilistic choice model, equations 2-4): logistic regression of choice (no-response trials left out) on signed indicators per nonzero contrast, the previous choice split by its outcome (r and u), a constant, and in the full task the block (+1 for 20:80, -1 for 80:20, 0 for 50:50); design matrix by patsy, fit by statsmodels Logit.fit_regularized with its defaults (code figure5_GLM_modelfit.py), per mouse and task. Sessions per the code: training days P1b-2 to P1b+3, split into basic and full by protocol; the Methods say three sessions per mouse, read as (b) the last three basic sessions before P1b and the first three full sessions. Contrasts per the Methods include 50%; the code drops 50% trials (variant). The paper does not say what is done for a mouse shown no 50% trials in the sessions used; readings (answer of 2026-10-22): (a) the Methods' model with W50 left out for that mouse, (b) the code's model, 50% trials dropped for every mouse ("only rarely present (should not be its own regressor)", figure5_GLM_modelfit.py, fit_glm). Means over mice of each weight. Reproduced when the 100% weight's mean exceeds 3 in both tasks, the visual weights increase with contrast, and the four history weights are within tolerance under one reading.
- fig5dg-history-weights-higher-full
Previous choices influenced behavior more strongly in the full task than in the basic task, after both rewarded and unrewarded trials.
Indeed, previous choices more strongly influenced behavior in the full task, both after rewarded (t(125) = 10.736, p<10 −6 ) and unrewarded trials (t(125) = 4.817, p<10 −6 ).
Results, A probabilistic model reveals common strategies across mice and laboratories, paragraph 2; Figure 5d,gHow it will be judged
Per-mouse weights from fig5cf-glm-weights, mice with both fits; scipy ttest_rel of full against basic, NaN omitted (code figure5_GLM_plot.py). Reproduced when both t are positive with p < 0.05 (the code's num_star significance level). Note that t(125) = 4.817 gives a two-sided p of about 4e-6, above the printed bound of 1e-6; the t value, not the bound, is the reference.
- fig5s2-glm-accuracy
The model predicted individual mouse choices well, with cross-validated accuracy of 81.03 +- 5.1% in the basic task and 82.1 +- 5.8% in the full task, similar across laboratories; mean condition numbers were 2.4 and 3.2.
Importantly, fitted weights and the model's predictive accuracy (unbiased: 81.03 + 5.1%, biased 82.1 + 5.8%) ( Figure 5—figure supplement 2 ) were similar across laboratories, suggesting an overall common strategy.
Results, A probabilistic model reveals common strategies across mice and laboratories, paragraph 2; Figure 5, figure supplement 2cHow it will be judged
Fits as in fig5cf-glm-weights with 5-fold cross-validation, folds shuffled (the code sets no seed; a standard seed is used and the spread over seeds shown). Accuracy readings: (1) the Figure 5, figure supplement 2c legend, the fraction of test trials where the predicted probability of the actual choice exceeds 50%; (2) the code, the mean predicted probability of the actual choice. Condition number of the design matrix (numpy cond), as in the code. "Similar across laboratories" is fig5s2-weights-similar-across-labs.
- fig5s2-weights-similar-across-labs
Fitted model weights and predictive accuracy were similar across laboratories.
The model fit the mouse choices well and captured the relative importance of sensory and non-sensory information across mice and laboratories
Results, A probabilistic model reveals common strategies across mice and laboratories, paragraph 2; Figure 5, figure supplement 2How it will be judged
The paper reports no test; the code (figure5_GLM_plot.py) runs the lab test (shared definition, uncorrected) on each weight (6.25, 12.5, 25, 100, rewarded, unrewarded, bias, and block in the full task), pseudo-R2 and accuracy, per task. Reproduced when no test gives p < 0.05; partially reproduced when one or two of the up to 20 tests do; the p values are reported.
- fig5s1b-win-stay-lose-switch
After correcting for slow fluctuations in decision bound, most mice used a win-stay lose-switch strategy, in both the basic and full tasks.
When taking these slow state-changes into account, the majority of animals use a win-stay lose-switch strategy.
Figure 5, figure supplement 1b legend; Results, A probabilistic model reveals common strategies across mice and laboratories, paragraph 3How it will be judged
Code suppfig_history_strategy.py: trials from training days P1b-2 to P1b+3, split by task; signed contrasts with fewer than 100 trials over all mice dropped; per mouse, task, previous choice and previous outcome a psychometric fit; the history shift = fitted rightward % at 0% contrast after a right choice minus after a left choice, separately after rewarded and unrewarded trials, from fits with more than 50 trials; the same with the next trial's choice and outcome; corrected shift = history shift minus next-trial shift (Lak et al., 2020a). Reproduced when, in each task, more than half the mice have a positive corrected post-reward shift and a negative corrected post-error shift.
- fig5s1c-updating-by-contrast
After rewarded trials, choice updating was largest when the previous stimulus had low contrast and much smaller after high-contrast rewarded trials.
After rewarded trials, choice updating is largest when the visual stimulus was highly uncertain (i.e. had low contrast) but strongly diminished after more certain, rewarded trials.
Figure 5, figure supplement 1c legendHow it will be judged
Corrected updating as in fig5s1b-win-stay-lose-switch, computed separately by the previous trial's absolute contrast (code suppfig_history_bycontrast.py; read only in outline here). The paper reports no test. Reproduced when, after rewarded trials, the mean corrected updating after 0% and 6.25% contrast exceeds that after 100% contrast in both tasks.
- session-end-criteria
The session-ending criterion was not mandatory; 5,316 of 10,903 sessions ended earlier and 5,587 later, the latter continuing for a median of 19 +- 16 extra trials (m.a.d.).
The criterion to end a session was not mandatory: some sessions were ended earlier (5,316/10,903 sessions) and others ended later (5,587/10,903 sessions). The latter sessions typically continued for an extra 19 ± 16 trials (median ±m.a.d., 5,587 sessions) longer.
Results, Training progression in the basic task, paragraph 2; Figure 1, figure supplement 2How it will be judged
Code supp_queries.py: all mice that started training; per training day, an end criterion is met when a session that day has an end status (from the pipeline's SessionEndCriteriaImplemented table) or the day's summed session durations reach 90 minutes; extra trials = trials after the end-status trial. Without an end status in the dataset, it is recomputed from the three criteria of the Figure 1, figure supplement 2 legend (fewer than 400 trials in 45 min; over 400 trials and 90 min; over 400 trials and median reaction time over the last 20 trials above 5 times the session median), taking the first trial at which one is met.
Pipeline
- Paper
- done (1 of 1)
- Claims
- done
- Claims check
- done (ready)
- Attempt
- todo
- Questions
- none
- Verified
- none