Datasets / DANDI:000289

Two-photon calcium imaging of EPG cells in Drosophila Melanogaster presented with cues of varying reliability

behavioral cell population imaging microscopy Fruit fly 761 GB 221 files
  • 1 blocked

Counted separately: 12 claims on data the paper deposited in other archives.

  • 7 reproduced
  • 4 partially reproduced
  • 1 not reproduced

Tested on dataset version 0.240801.1040

View on DANDI Paper JSON How to read outcomes

Claims

From the paper 10.1073/pnas.2210622120

Testing Kutschireiter et al. (2023) with dandiset 000289 13

Notebook reproduce_kutschireiter_2023.ipynb

Reproduced7

  • ReproducedResults, Circular Kalman Filtering, HD Observations, paragraph 1; Fig. 1 D and Edata in Zenodo doi:10.5281/zenodo.7615975 (BayesianRingAttractor v1.0, commit a9db6f9), circularFiltering.py vM_Projection_Run

    In the circular Kalman filter, an HD observation consistent with the current estimate increases certainty, while one that strongly conflicts with it (opposite direction) decreases certainty.

    HD observations increase the observer’s certainty if they are confirmatory ... if HD observations strongly conflict with the current estimate (e.g., if they point in the opposite direction), they actually decrease certainty

    Results, Circular Kalman Filtering, HD Observations, paragraph 1; Fig. 1 D and E

    With the code's vM_Projection (kappa_phi 1, kappa_v 1, dt 0.01), for kappa_t 1, 2 and 5 and per-step reliabilities 0.1416 (gamma_z 1/s) and 0.4529 (10/s), a confirmatory observation raises and an opposite one lowers the certainty in all 6 cases, both for the observation update alone (for example kappa_t 1, 10/s, 1.453 and 0.547) and for the full step against the step without observation (1.447 and 0.546 against 0.997). The first comparison holds for any weight alpha between 0 and 2 kappa_t, so it is a weak test; the second is the informative one.

  • ReproducedResults, Neural Network Implementation, paragraph 6; Fig. 2Hdata in Zenodo doi:10.5281/zenodo.7615975 (BayesianRingAttractor v1.0, commit a9db6f9), Figure2h.ipynb with circularFiltering.py

    In a simulated Bayesian ring attractor of 80 neurons, the bump position tracks HD about as accurately as the circular Kalman filter, the bump amplitude tracks the filter's certainty, and when the visual cue is removed the amplitude decays toward its baseline.

    This network exhibits a spatially localized bump whose position tracks HD, with an accuracy similar to that of the circular Kalman filter itself ( Fig. 2 H ). Meanwhile, the amplitude of the bump accurately tracks the fluctuating HD posterior certainty in the circular Kalman filter ... When the visual cue is removed, the bump amplitude decays toward baseline

    Results, Neural Network Implementation, paragraph 6; Fig. 2H

    The simulation cells of Figure2h.ipynb were run from the deposit. With rate 15 (notebook) and 10/s (Methods), the network's position differs from the circKF's by a mean of 0.016 and 0.021 rad, against a circKF error to the true HD of 0.483 and 0.491 rad. Bump amplitude and circKF certainty correlate at r 0.994 and 0.993 over the cue periods. In darkness (5 to 10 s) the amplitude falls from 7.21 to 1.22 (rate 15) and from 5.89 to 1.21 (rate 10), closer to the baseline of 1 than at its start.

    How the test differed from the paper’s method

    • The notebook calls vM_Projection_Run with the keyword kappa_y, which the function names kappa_v; the keyword was renamed (claims file header). Nothing else in the cells was changed except the rate for the second reading.
  • ReproducedFig. 2I legend; Results, Neural Network Implementation, paragraph 7data in Zenodo doi:10.5281/zenodo.7615975 (BayesianRingAttractor v1.0, commit a9db6f9), data_processed/figure2i.npz with Figure2i.ipynb

    The Bayesian ring attractor tracks HD with the same accuracy as the circular Kalman filter when HD observations are reliable and with slightly lower accuracy when they are unreliable; in the unreliable regime its certainty stays near its baseline and overestimates the filter's certainty.

    The Bayesian ring attractor network tracks the true HD with the same accuracy ... if HD observations are reliable and, therefore, more informative but with slightly lower accuracy once they become less reliable, and therefore less informative. This drop co-occurs with an overestimate in the belief’s certainty

    Fig. 2I legend; Results, Neural Network Implementation, paragraph 7

    From figure2i.npz, the accuracies of the circKF and the Bayesian ring attractor differ by at most 0.0010 at rates of 1/s and above and 0.0003 at 3/s and above, so all four combinations hold. At 0.01, 0.03 and 0.1/s the attractor is less accurate (by 0.073, 0.084 and 0.057) and its mean certainty is higher (1.00 to 1.05 against 0.18 to 0.56).

  • ReproducedResults, Bayesian vs. Conventional Ring Attractors, paragraphs 2 and 3; Fig. 3Bdata in Zenodo doi:10.5281/zenodo.7615975 (BayesianRingAttractor v1.0, commit a9db6f9), data_processed/figure2i.npz with Figure3b,c,e,d_FigureS3.ipynb

    A conventional ring attractor (fast decay to a fixed bump amplitude, with that amplitude optimized) is less accurate than the Bayesian ring attractor at every HD observation reliability, and reaches its accuracy only when observations are highly unreliable.

    Even after this optimization of the conventional ring attractor, it does not rival the accuracy of the Bayesian ring attractor. The Bayesian attractor performs consistently better, regardless of the amount of information available to the network

    Results, Bayesian vs. Conventional Ring Attractors, paragraphs 2 and 3; Fig. 3B

    The optimal kappa* of the fixed-certainty filter is 1.7 (weighted accuracy 0.626 against 0.687 for the Bayesian attractor). The Bayesian attractor is more accurate at all 23 rates (smallest difference 0.0115 at 0.01/s, largest 0.131 at 100/s) and strictly so from 0.1 to 10/s (smallest 0.034). The reported part, not judged, holds only in part. The difference is smallest at 0.01/s, but at 0.03/s (0.036) it is larger than at 2 and 3/s (0.034 and 0.035).

  • ReproducedFig. 3C legend; Results, Bayesian vs. Conventional Ring Attractors, paragraph 3; SI Appendix, Fig. S3 legenddata in Zenodo doi:10.5281/zenodo.7615975 (BayesianRingAttractor v1.0, commit a9db6f9), data_processed/figure2i.npz with Figure3b,c,e,d_FigureS3.ipynb

    The weight given to a new HD observation by the Bayesian ring attractor depends on both the observation's reliability and the current certainty and is close to that of the circular Kalman filter, deviating only for very uninformative observations; a conventional attractor weights observations by their reliability alone.

    The update weight of the Bayesian attractor is close to optimal, visually indistinguishable from the circKF

    Fig. 3C legend; Results, Bayesian vs. Conventional Ring Attractors, paragraph 3; SI Appendix, Fig. S3 legend

    At rates of 1/s and above, the weights computed with the circKF's and the attractor's mean certainty differ by at most 0.0010 with the Methods' weight (dt 0.01) and 0.0062 with the notebook's weight (dt 0.1), inside both bounds (0.02, 0.05). The largest difference, 0.021 and 0.130, falls at the lowest rate, 0.01/s.

  • ReproducedResults, The Impact of Neural Noise on Inference Accuracy, paragraph 1; Fig. 3Fdata in Zenodo doi:10.5281/zenodo.7615975 (BayesianRingAttractor v1.0, commit a9db6f9), data_processed/figure3f/sigma=*.npz with Figure3f_FigureS4.ipynb

    In a network of 64 neurons with additive neural noise, inference accuracy drops below 70% of its noise-free value once the noise is large enough.

    Indeed, simulating a network of 64 noisy neurons shows that, once the noise becomes sufficiently large, inference accuracy drops to below 70% of its noise-free value

    Results, The Impact of Neural Noise on Inference Accuracy, paragraph 1; Fig. 3F

    The noise-free optimum is kappa* 0.5, beta 0.2. Kept at those values, the network's relative performance is 63.1% at noise SD 10 and 38.1% at SD 20, under the notebook's normalization (particle filter) and the legend's (same network at SD 0, 63.07% and 38.06%), below 70% under all four combinations. It first falls below 70% at SD 9 (67.4%).

  • ReproducedResults, The Impact of Neural Noise on Inference Accuracy, paragraph 3; Fig. 3F; SI Appendix, Fig. S4data in Zenodo doi:10.5281/zenodo.7615975 (BayesianRingAttractor v1.0, commit a9db6f9), data_processed/figure3f/sigma=*.npz with Figure3f_FigureS4.ipynb

    Retuning the bump decay speed and baseline amplitude for each noise level reduces the drop in inference accuracy by about 35% compared with the network tuned for noise-free performance, and raises the signal-to-noise ratio.

    Indeed, doing so reduced the drop in inference accuracy by about 35% when compared to the network tuned to optimize noise-free performance ( Fig. 3 D and F ) and also boosted the network’s signal-to-noise ratio

    Results, The Impact of Neural Noise on Inference Accuracy, paragraph 3; Fig. 3F; SI Appendix, Fig. S4

    The reduction of the drop by retuning is 34.64% at SD 10 with the particle filter normalization (the first reading, given as measured) and 34.56% with the legend's, and 31.67% for the mean over SD 1 to 10 with the particle filter normalization, inside 31.5 to 38.5%. The other readings are outside it, 31.19% (mean, legend normalization) and 29.80 and 29.76% (SD 20). The retuned signal-to-noise ratio is above the untuned one at every noise SD above 0 (ratio 1.05 to 3.00). Fig. S4, reported only, the best kappa* grows from 0.3 to 4.8 between SD 1 and 10, and the best beta from 0.1 to 2.0 but not steadily, and both fall again at SD 15 and 20.

    How the test differed from the paper’s method

    • The paper does not say at which noise level the 35% is taken; the claims file names six readings, three of them match, and all are shown.

Partially reproduced4

  • Partially reproducedResults, Bayesian vs. Conventional Ring Attractors, paragraph 4; Fig. 3Ddata in Zenodo doi:10.5281/zenodo.7615975 (BayesianRingAttractor v1.0, commit a9db6f9), data_processed/figure3d.npz with Figure3b,c,e,d_FigureS3.ipynb

    Across ring attractors with different bump decay speeds and baseline amplitudes, the best network has a slow decay (low beta) and performs about as well as the Bayesian ring attractor; performance drops as decay speed increases, which a larger baseline amplitude partly offsets.

    The network with the best performance overall had a slow bump decay speed (low β ), as expected ( Fig. 3 D and E ). While it featured similar performance to the Bayesian ring attractor, it had slightly different β and κ * parameters ... As the bump decay speed β increased further, performance dropped.

    Results, Bayesian vs. Conventional Ring Attractors, paragraph 4; Fig. 3D

    Normalized by the particle filter reference, the best network is kappa* 0.7, beta 0.3 (1.0012). Conditions 1, 2, 3 and 5 hold. The best beta is at most 1, the best network differs from (1, 0.5), the Bayesian attractor is 0.57% below the best, and above the best beta the maximizing kappa* is 0.8 to 1.6, never below 0.7. Condition 4 fails as written. The maximum over kappa* falls from 1.0012 (beta 0.3) to 0.9322 (beta 8), Spearman rho -0.98, but rises twice, by 0.0008 (beta 0.7 to 0.8) and 0.0014 (0.9 to 1). The two rises are smaller than the falls around them. With 10,000 runs per rate in each grid cell, the standard error of a cell's normalized performance is 0.0018 and that of a difference of two cells 0.0026, so the rises are 0.32 and 0.52 standard errors, within sampling error. The cells are independent samples, since performance_scan_figure3d.py runs each (kappa*, beta) as its own process and sets no seed (its lines are printed). The criterion gives no tolerance, so condition 4 fails as written.

  • Partially reproducedResults, The Impact of Neural Noise on Inference Accuracy, paragraphs 2 and 3; SI Appendix, section 5question to the readerdata in Zenodo doi:10.5281/zenodo.7615975 (BayesianRingAttractor v1.0, commit a9db6f9), circularFiltering.py RNN_filter_Run

    Neural noise makes the bump position diffuse without bias and makes the bump amplitude drift upward and diffuse, which leads to overconfidence; for additive noise, a four-fold larger network halves the noise's effective SD.

    First, it causes an unbiased random diffusion of μ t and, thus, an increasingly imprecise memory of the HD estimate. Second, it causes a positive drift and random diffusion of κ t

    Results, The Impact of Neural Noise on Inference Accuracy, paragraphs 2 and 3; SI Appendix, section 5

    2,000 runs of T 5 per setting, noise SD 1 and 2, N 64 and 256. (1) holds. The mean change of position is within 2 standard errors of 0 in all four settings (z -0.74 to 0.82) and its variance grows at each second (for N 64, SD 1, 0.034 at 1 s to 0.232 at 5 s). (3) holds. The SD of the one-step amplitude increments at N 256 is 0.500 and 0.501 times that at N 64. (2) fails. The mean amplitude falls throughout the run and ends at 0.83, 0.65, 0.82 and 0.42, all below the noise-free 1.00, so there is no upward drift and no overconfidence in this network. In RNN_filter_Run the noise also raises the rectified sum that drives the global inhibition (from 0.5 without noise to 0.57 to 0.77 at 1 s), and the zero mode stays near 0. With the inhibition held at its noise-free constant the mean amplitude rises to 1.08, 1.38, 1.01 and 1.08 after 5 s, within 1.9 standard errors of the mean that SI Eq. S124 implies (1.08, 1.35, 1.02, 1.08). That control removes both the noise's effect on the inhibition and the restoring force toward kappa* 1, so it confirms the SI's noise term but does not isolate the cause. With the inhibition set to the value a noise-free bump of the decoded amplitude gives, which keeps the restoring force, the mean amplitude ends at 1.002, 1.045, 1.003 and 1.005, above the noise-free 1.000 to 1.001 in all four settings (z 0.35, 6.2, 1.3, 1.1) and within 1.2 standard errors of the reduced two-dimensional SDE of the first mode. So the noise's direct effect gives a small upward drift, and in the released network the fall comes from the noise's own contribution to the rectified sum that drives the inhibition. The criterion tests the full network; whether the paper claims the drift for the full network is asked of a reader (questions/000289.yaml).

    How the test differed from the paper’s method

    • The runs use a batched copy of RNN_filter_Run (bra.rnn_batch), the same update term by term; on the same noise draws it matches the released function to 4e-16.
    • The released RNN_filter_Run has no z parameter and raises NameError; z=None was added to its signature, as the claims file says.
  • Partially reproducedFig. 4 B and C legends; SI Appendix, section 4Averifier disagreed at firstrests on an assumptionquestion to the readerdata in neuPrint hemibrain:v1.2.1 (Scheffer et al. 2020, eLife 9, e57443), synapse counts between EPG, Delta7 and PEN1 neurons

    The connectivity among EPG, Delta7 and PEN1 neurons in the hemibrain connectome resembles the connectivity of a network that implements the approximate circular Kalman filter, with shifted PEN1 to EPG projections and broad EPG to Delta7 to EPG inhibition.

    The connectivity profile of a recurrent neural network (RNN) ( Fig. 2 D – F ) that implements an approximate circKF is strikingly similar to the connectivity of neurons in the Drosophila HD system.

    Fig. 4 B and C legends; SI Appendix, section 4A

    Sorted by the PB glomerulus in the name (reading (a)), left PEN1 neurons send their EPG output a mean of +0.56 glomeruli from their EPG input (10 of 10 positive, Wilcoxon p 0.002) and right PEN1 neurons -0.48 (9 of 10 negative, p 0.004), among same-side EPG neurons; with partners on both sides, +0.37 and -0.35. The two edge neurons (L9, R9) have offsets near 0. Every Delta7 receives EPG input from 13 to 16 of the 16 EPG glomeruli and sends output to EPG neurons in 9 to 15, more than half. Counting only glomeruli with at least 3 (or 5) synapses, inputs still reach more than half for all 42 Delta7, but outputs do for 7 (or 0) of 42, so the breadth of the Delta7 to EPG side depends on the threshold. Reading (b), by ellipsoid body or gall position, could not be run, and the criterion requires both, so the claim holds in part (changed from reproduced after verification round 1). Whether reading (b) belongs to the claim is asked of a reader (questions/000289.yaml).

    How the test differed from the paper’s method

    • The criterion asks for two sorts. Reading (b), by the ellipsoid body or gall position in the neuron's name, cannot be formed: hemibrain instance names give only the PB glomerulus (EPG(PB08)_L1, PEN_a(PB06a)_R2). The outcome rests on reading (a) alone.
    • Assumed, as the Fig. 4B legend counts "the total number of synaptic connections between each cell pair" with no threshold, that a connection is any synapse; the counts at 3 and 5 synapses are shown.
    • Assumed a measure of the offset, since the paper gives none. For each PEN1 neuron, it is the synapse-weighted circular mean position (period 8 glomeruli) of its EPG targets minus that of its EPG inputs, among same-side EPG neurons, with both sides shown.
    • The connectome is the public hemibrain v1.2 export, not v1.2.1 as in the legend. neuPrint, which serves v1.2.1, now requires a personal API token for all data.
  • Partially reproducedResults, Tuning a Biological Ring Attractor, paragraph 3; Fig. 4 G and Hdata in Zenodo doi:10.5281/zenodo.7615975 (BayesianRingAttractor v1.0, commit a9db6f9), data_processed/figure4g.npz with Figure4g,h.ipynb and network_filter.py

    A Drosophila-like multipopulation network, constrained by the connectome, tracks HD with an accuracy indistinguishable from the idealized Bayesian ring attractor across observation reliabilities.

    We found that this network achieves a HD encoding accuracy which is indistinguishable from that of our idealized Bayesian ring attractor network ( Fig. 4 G and H ).

    Results, Tuning a Biological Ring Attractor, paragraph 3; Fig. 4 G and H

    From figure4g.npz, the Drosophila-like network's accuracy is within 0.02 of the Bayesian ring attractor's at all 23 rates (largest 0.019 at 0.1/s), so reading (b) holds, but it exceeds 0.01 at 0.1, 0.2 and 0.3/s (0.019, 0.014, 0.010), so reading (a) fails; the criterion needs both. At 0.4/s and above the difference is at most 0.004. The three differences above 0.01 are 1.95, 1.51 and 1.16 times the standard error of a difference of two independent scans (the two networks share their runs, so the actual error is smaller), and the same Bayesian attractor differs by 0.022 between figure4g.npz and figure2i.npz at 0.1/s, so the 0.01 bound is near the sampling error there. Mean certainties, reported only, are lower in the Drosophila-like network at every rate (0.88 against 1.00 at 0.01/s, 13.8 against 14.5 at 100/s).

Not reproduced1

  • Not reproducedSI Appendix, Fig. S2 legend; Results, Neural Network Implementation, paragraph 6verifier disagreed at firstquestion to the readerdata in Zenodo doi:10.5281/zenodo.7615975 (BayesianRingAttractor v1.0, commit a9db6f9), data_processed/figureS2.npz and vonMises_q_stats.npz with FigureS2.ipynb

    The Bayesian ring attractor's inference accuracy depends little on its number of neurons.

    Network inference performance is mostly independent of the number of neurons N in the Bayesian ring attractor network.

    SI Appendix, Fig. S2 legend; Results, Neural Network Implementation, paragraph 6

    Of the 160 values in figureS2.npz (7 network sizes, 23 rates, one missing), one is more than 0.02 from the quadratic circKF reference, N 16 at 0.1/s (0.041 below). Every other value, N 3 and 4 included, is within 0.019. Both readings include N 16, so neither holds, and the criterion requires both (changed from partially_reproduced after verification round 1). The deviation is not sampling error. With 10,000 runs per rate, the standard error of a difference of two independent scans at 0.1/s is 0.0096, and N 16 is 4.3 of them below the reference, with no other value beyond 3, and 0.029 below the mean of the other six sizes at that rate. The 0.02 bound is near the sampling error at low rates (about 1.2 of the values with N of 8 and more expected beyond it by chance alone), but this value is far beyond it. One value (N 128 at 100/s) is missing (NaN) in the stored array. Whether "mostly independent" allows single exceptions is asked of a reader (questions/000289.yaml).

Blocked1

  • BlockedDiscussion, paragraph 7· Methods underspecified

    The paper predicts that the bump amplitude in the HD system is higher in the presence of reliable external HD cues, such as salient visual landmarks. The paper does not test this.

    Second, we predict that bump amplitude in the HD system should vary dynamically, with higher amplitudes in the presence of reliable external HD cues, such as salient visual landmarks.

    Discussion, paragraph 7

    Tested on the 15 flies of the dandiset's cue-contrast experiment, high (more reliable) against low contrast, with bump amplitude the length of the population vector at the bump's spatial frequency across the PB midline ROIs (frequency 2, the spectral peak in every fly). The readings disagree, and the criterion blocks the claim when they do. With dF/F, amplitude is higher at high contrast in 10 of 15 flies (reading (a) holds) but the one-sided Wilcoxon test gives p 0.138 (reading (b) fails). With raw fluorescence minus baseline, 12 of 15 flies (a) and p 0.021 (b). Median high to low ratio 1.12 (dF/F) and 1.11 (raw). Variants, high against zero contrast give 12 of 15 flies (p 0.047) and 13 of 15 (p 0.006), and low contrast is not above zero contrast (7 of 15 with either signal). Without sub-2, which has one high-contrast block, dF/F gives 9 of 14 (p 0.21) and raw 11 of 14 (p 0.034). Across 18 variants of the raw signal (smoothing 0, 2 or 4 pixels, baseline at the 5th, 10th or 20th percentile, F - F0 or (F - F0) / F0), p is below 0.05 in all nine F - F0 variants (0.021 to 0.042) and in none of the nine divided by F0 (0.11 to 0.89), so the split between the readings follows the division by the baseline. Question for the paper's authors (finding below). The Discussion gives no experimental method for the prediction (which activity signal, which cue conditions are compared, which test). With these data the answer depends on the signal and the test, and no reading has more support in the paper than another.

    How the test differed from the paper’s method

    • The dandiset holds data of a separate study, not of this paper; the paper does not test its prediction.
    • Raw fluorescence is read from 10% of the frames (25 volumes every 250), averaged over planes and smoothed in the plane (Gaussian SD 4 pixels) before averaging in each ROI mask; the baseline is each ROI's 10th percentile. The projection and the smoothing were chosen as those that best reproduce the stored dF/F (median correlation over ROIs 0.93, against 0.66 without smoothing). Smoothing 0 and 2 pixels, baselines at the 5th and 20th percentile and the baseline-divided signal are shown as variants.

Findings about the data

What attempts found about the dataset itself, for its owner and the archive.

  • Metadata errorThe dandiset's related resources name the bioRxiv preprint "Bayesian inference in ring attractor networks" (10.1101/2021.12.17.473253 version 4) as the paper it supplements, but its description says the data belong to "Uncertainty and learning in a biological ring attractor" (Basnak et al., in preparation). The linked paper, published as Kutschireiter et al. 2023 (PNAS), is a theory paper that analyses none of these data, so the dandiset links no paper that uses them.
  • Metadata errorimaging_rate is 9.18 Hz in every file, but sub-4 and sub-10 (cue-contrast experiment) are imaged at 7.87 Hz (0.127 s frame interval, 9446 frames in 1200 s) with 14 planes, against 9.18 Hz and 12 planes in the other 13 sessions.
  • OtherIn sub-2 the trials table has five blocks (high, zero, low, zero, low) where every other fly of the cue-contrast experiment has six with two of each contrast. The last block lasts 405 s against about 200 s in the protocol, so it may hold a low and a high block labelled as one.
  • Description missingThe processing behind the stored dF/F is not described (which planes, the extent of each ROI, the baseline). The ROI masks are 2D, 3 to 11 pixels each (median 7), and the dF/F agrees with the raw imaging averaged in those masks only after spatial smoothing (median correlation over ROIs 0.66 without, 0.93 with a 4-pixel Gaussian, sub-1). The RoiResponseSeries unit is "na".
  • Methods not availableQuestion for the paper's authors. The Discussion predicts higher bump amplitude with reliable external HD cues but gives no experimental method. Which activity signal (dF/F or raw fluorescence), which comparison of cue conditions and which test would test it? On this dandiset's cue-contrast experiment, high contrast exceeds low contrast in most flies with either signal, but a one-sided Wilcoxon test over 15 flies gives p 0.021 with raw fluorescence and p 0.138 with dF/F.
  • Methods not availableQuestion for the paper's authors. Fig. 4B sorts neurons "by their spatial position as a proxy for their preferred HD". Which position was used? The hemibrain names give only the PB glomerulus for EPG and PEN1 neurons. This attempt sorted by it and counted any synapse as a connection.
  • Published code disagreesThe released code does not run as published. RNN_filter_Run (circularFiltering.py) uses z but has no z parameter, and Figure2h.ipynb and the scan scripts pass kappa_y to functions whose parameter is kappa_v. One NaN remains in figureS2.npz (N 128 at 100/s).
  • Published code disagreesIn the released single-population network (RNN_filter_Run) at the Bayesian ring attractor's parameters, additive neural noise lowers the mean bump amplitude (from 1.00 to 0.42 to 0.83 after 5 s, N 64 and 256, noise SD 1 and 2). The Results and SI Appendix section 5 say the noise causes a positive drift of the amplitude. With the inhibition set to the value a noise-free bump of the decoded amplitude gives, which keeps the restoring force and removes only the noise's own contribution to the rectified sum that drives the inhibition, the mean amplitude ends at 1.002 to 1.045, above the noise-free value, so the fall comes from that contribution.

Questions

Where the author or a verifier believed the claims file misread the paper, they asked, citing the paper, and a reader who had not seen the data answered from the paper alone, amending the claims file where it was wrong. The author then ran the claim as answered. Attempts before protocol pilot-11 also asked here for method details the claims file did not give.

  • openasked 2026-10-30about noise-effects-on-mu-and-kappa

    Condition (2) of the criterion compares the mean end amplitude of the full released network (RNN_filter_Run, with its rectified global inhibition) with a noise-free run. Does the paper claim that upward drift for the full network, or only for the noise's direct effect on the first Fourier mode, with the remaining dynamics unchanged, as SI section 5 derives it? If the latter, should condition (2) be tested with the global inhibition held at its noise-free level, or against the drift of SI Eqs. S122 and S123?

    Passage. "With neural noise, the population dynamics equation Eq. [S49] becomes dr_t(phi) = ... + I_ext(phi) + eta_t(phi) [S107] ... This shows that we can interpret neural noise as yet another stochastic input to the network." (SI Appendix, section 5A); "<dkappa> = sigma_nn^2 / (kappa N) dt" (Eq. S123); "it causes a positive drift and random diffusion of kappa_t" (Results, The Impact of Neural Noise on Inference Accuracy).

    Why it matters. In the full network the noise also raises the rectified sum that drives the global inhibition (from 0.5 to 0.57 to 0.77 after 1 s), and the mean amplitude falls to 0.42 to 0.83 after 5 s, so (2) fails and the claim is partially reproduced. With the inhibition held at its noise-free level the mean amplitude rises to 1.01 to 1.38, matching the mean that SI Eq. S124 implies (1.02 to 1.35), so (2) would hold and the claim would be reproduced. With the inhibition set by the decoded amplitude, which keeps the restoring force and removes only the noise's own contribution to the inhibition, the mean amplitude ends at 1.002 to 1.045, above the noise-free 1.000 to 1.001 in all four settings, so (2) would also hold.

    No reader has answered it yet.
  • openasked 2026-10-30about fig4b-connectome-motifs

    The criterion requires two sorts of the neurons, (a) by the protocerebral bridge glomerulus in the neuron's name and (b) by the ellipsoid body or gall position in the name. Hemibrain instance names give only the PB glomerulus for EPG and PEN1 neurons (EPG(PB08)_L1, PEN_a(PB06a)_R2), so reading (b) cannot be formed from the names. The legend names one sort, "spatial position". Is reading (b) part of the claim, or should the claim be judged on a sort that the connectome's names allow?

    Passage. "Neurons were sorted by their spatial position as a proxy for their preferred HD. The total number of synaptic connections between each cell pair was taken to indicate the functional connection strength between these cells." (Fig. 4B legend)

    Why it matters. Under reading (a), counting any synapse, both parts of the criterion hold (left PEN1 +0.56, right PEN1 -0.48 glomeruli; every Delta7 reaches more than half the EPG glomeruli for input and output). Because reading (b) cannot be formed and the criterion requires both, the outcome is partially reproduced; with reading (a) alone it would be reproduced.

    No reader has answered it yet.
  • openasked 2026-10-30about figs2-performance-independent-of-n

    The paper says accuracy is "mostly independent" of N. The criterion requires every value with N of 8 and more (or 16 and more) to be within 0.02 of the reference at every rate. Does "mostly" allow single exceptions, so that a criterion over all values is tighter than the paper's claim?

    Passage. "Network inference performance is mostly independent of the number of neurons N in the Bayesian ring attractor network." (SI Appendix, Fig. S2 legend)

    Why it matters. Of the 160 stored values, one (N 16 at 0.1/s, 0.041 below the reference, 4.3 standard errors of a difference) is beyond 0.02, and every other value, N 3 and 4 included, is within 0.019. The outcome is not reproduced on that one value; a criterion that allows isolated exceptions would give reproduced or partially reproduced.

    No reader has answered it yet.

Notebooks

Every registered entry that uses this dataset. An entry can hold several notebooks, and can use more than one dataset.

Reproduction Bayesian inference in ring attractor networks (Kutschireiter et al. 2023)

Tests the findings of Kutschireiter, Basnak, Wilson and Drugowitsch (2023), a theory paper. Its simulation findings are tested on the paper's released code and processed outputs (Zenodo), its connectome finding on the hemibrain connectome, and its prediction that bump amplitude is higher with a reliable cue on dandiset 000289, which holds two-photon imaging of EPG neurons in flies shown a visual cue at three contrasts, from a separate study by the same lab. All files

Verification

  • Round 1
    2026-10-30
    Agreed with 11 of 13 outcomes. Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook). Reviewed: Round 1. The first executed notebook with the conclusion cells and the summary removed, the helper module and the claims file, with the paper. The verifier recomputed the prediction's p-values from the printed table, tested bra.fourier on a synthetic bump and checked the asset count of the dandiset version. Its report had no attempt record and gave its own outcome for each claim; 11 of 13 agree with the record, and fig4b-connectome-motifs and figs2-performance-independent-of-n do not.
    • fig4b-connectome-motifs: the verifier’s outcome was “Partially reproduced”. Under reading (a) the offsets have opposite signs (left +0.56, right -0.48 on the same side) and all 42 Delta7 reach more than 8 EPG glomeruli for input and output at 1 synapse or more. Reading (b), which the criterion requires, cannot be formed because no instance name holds an EB or gall position. The Delta7 breadth depends on the synapse threshold (output in 42, 7 and 0 of 42 at 1, 3 and 5 synapses); the connectome is v1.2, not v1.2.1 (B4).
    • figs2-performance-independent-of-n: the verifier’s outcome was “Not reproduced”. N 16 at 0.1/s deviates by 0.041, so both readings (N of 8 and more, N of 16 and more, within 0.02) fail; it is the only value above 0.02, and N 3 and 4 stay within 0.0186. The bound is close to the run-to-run variability and the notebook gives no Monte Carlo SE (B2). One NaN value for N 128 is skipped by nanmax and its rate is not printed (B8).
  • Round 2
    2026-10-30
    Agreed with 13 of 13 outcomes. Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook). Reviewed: Round 2. The full executed notebook of the first revision with its conclusions and summary, the helper module, the README, the attempt record and the claims file, with the paper. The verifier checked each outcome, each assumption and each sentence of the conclusions against the printed outputs, and whether its round-1 points were resolved. It agreed with all 13 outcomes.

Method differences

How the tests differed from the paper’s method or data, and the claims each applies to.

  • The notebook calls vM_Projection_Run with the keyword kappa_y, which the function names kappa_v; the keyword was renamed (claims file header). Nothing else in the cells was changed except the rate for the second reading.fig2h-bump-tracks-circkf
  • The paper does not say at which noise level the 35% is taken; the claims file names six readings, three of them match, and all are shown.fig3f-retuning-reduces-drop
  • The runs use a batched copy of RNN_filter_Run (bra.rnn_batch), the same update term by term; on the same noise draws it matches the released function to 4e-16.noise-effects-on-mu-and-kappa
  • The released RNN_filter_Run has no z parameter and raises NameError; z=None was added to its signature, as the claims file says.noise-effects-on-mu-and-kappa
  • The criterion asks for two sorts. Reading (b), by the ellipsoid body or gall position in the neuron's name, cannot be formed: hemibrain instance names give only the PB glomerulus (EPG(PB08)_L1, PEN_a(PB06a)_R2). The outcome rests on reading (a) alone.fig4b-connectome-motifs
  • Assumed, as the Fig. 4B legend counts "the total number of synaptic connections between each cell pair" with no threshold, that a connection is any synapse; the counts at 3 and 5 synapses are shown.fig4b-connectome-motifs
  • Assumed a measure of the offset, since the paper gives none. For each PEN1 neuron, it is the synapse-weighted circular mean position (period 8 glomeruli) of its EPG targets minus that of its EPG inputs, among same-side EPG neurons, with both sides shown.fig4b-connectome-motifs
  • The connectome is the public hemibrain v1.2 export, not v1.2.1 as in the legend. neuPrint, which serves v1.2.1, now requires a personal API token for all data.fig4b-connectome-motifs
  • The dandiset holds data of a separate study, not of this paper; the paper does not test its prediction.prediction-bump-amplitude-cue-reliability
  • Raw fluorescence is read from 10% of the frames (25 volumes every 250), averaged over planes and smoothed in the plane (Gaussian SD 4 pixels) before averaging in each ROI mask; the baseline is each ROI's 10th percentile. The projection and the smoothing were chosen as those that best reproduce the stored dF/F (median correlation over ROIs 0.93, against 0.66 without smoothing). Smoothing 0 and 2 pixels, baselines at the 5th and 20th percentile and the baseline-divided signal are shown as variants.prediction-bump-amplitude-cue-reliability

Provenance

Attempt record
000289-kutschireiter-2023-01
Paper
10.1073/pnas.2210622120
Date
2026-10-29
Made by
An agent, run by CatalystNeuro
Dataset
DANDI:000289
Version
0.240801.1040 (doi:10.48324/dandi.000289/0.240801.1040)
Run on
GitHub Actions
Harness
Claude Code
Model
claude-opus-5-5
Protocol
pilot-12

Transcripts of the agent runs

Each run’s prompt, messages and tool calls, with long output cut and the text of the paper left out, and its final report.