{
 "dataset": {
  "archive": "dandi",
  "id": "000139",
  "label": "DANDI:000139"
 },
 "page": "datasets/dandi/000139/",
 "generated": "2026-11-05T07:41:23+00:00",
 "caution": "A failure to reproduce is not a finding that the paper is wrong. It may be a failure of the attempt.",
 "outcome_counts": {
  "reproduced": 14,
  "partially_reproduced": 2,
  "not_reproduced": 2,
  "blocked": 5
 },
 "outcome_counts_with_outside_data": {
  "reproduced": 0,
  "partially_reproduced": 0,
  "not_reproduced": 0,
  "blocked": 0
 },
 "outcome_counts_elsewhere": {
  "reproduced": 0,
  "partially_reproduced": 0,
  "not_reproduced": 0,
  "blocked": 0
 },
 "outcome_counts_links": {
  "reproduced": 0,
  "partially_reproduced": 0,
  "not_reproduced": 0,
  "blocked": 0
 },
 "attempts": [
  {
   "id": "000139-churchland-2010-01",
   "dataset": {
    "archive": "dandi",
    "id": "000139",
    "version": "0.220113.0408",
    "doi": "10.48324/dandi.000139/0.220113.0408"
   },
   "key": "dandi:000139",
   "label": "DANDI:000139",
   "dataset_version": "0.220113.0408",
   "paper": "10.1016/j.neuron.2010.09.015",
   "paper_title": "Cortical preparatory activity: representation of movement or first cog in a dynamical machine?",
   "date": "2026-10-28",
   "kind": "agent",
   "accountable": "CatalystNeuro",
   "agent": {
    "harness": "Claude Code",
    "model": "claude-opus-5-5",
    "protocol": "pilot-12"
   },
   "runner": {
    "platform": "GitHub Actions",
    "url": "https://github.com/catalystneuro/encore/actions/runs/40756774586"
   },
   "tier": "agent_verified",
   "deprecated": "",
   "deprecated_on": null,
   "entry": "000139-churchland-2010",
   "entry_title": "Preparatory activity and a PD in peri-movement space (Churchland et al. 2010)",
   "description": "Tests the findings of Churchland, Cunningham, Kaufman, Ryu and Shenoy (2010) on dandiset 000139 (MC_Maze_Medium), one session of monkey J on the maze task recorded with two Utah arrays and cut to 250 labelled trials over 27 conditions for the Neural Latents Benchmark. It tests the weak correlation between preparatory and peri-movement tuning, the performance of a preferred direction in peri-movement, kinematic and task spaces, the controls of Figure 7, and the simulation of Figure 8. The speed-task and single-electrode datasets and the EMG are not in the dandiset.",
   "notebooks": [
    {
     "path": "reproduce_churchland_2010.ipynb",
     "title": "Reproducing Churchland et al. (2010) from dandiset 000139",
     "executed": "notebooks/000139-churchland-2010/reproduce_churchland_2010.html"
    }
   ],
   "outcomes": [
    {
     "claim": "j-array-recordings",
     "statement": "The J-array dataset holds 146 single- and multi-unit isolations recorded with two 96-electrode arrays during the maze task, over 108 conditions, with a mean of 2166 trials per neuron.",
     "source": "Results, Behavior and recordings, paragraph 1; Methods, Neural recordings and datasets",
     "quote": "a pair of implanted 96-electrode arrays (J-array dataset, 146 single- and multi-unit isolations)",
     "statistics": [],
     "kind": "finding",
     "inferred": "",
     "outcome": "partially_reproduced",
     "blocked_by": null,
     "notebook": "reproduce_churchland_2010.ipynb",
     "notes": "The dandiset holds 152 units (within 146 +/- 14.6), 75 and 77 on the two arrays by unit id, recorded over 27 conditions (9 mazes, each in the no-barrier, barrier and distractor versions) and 250 labelled trials per unit, against the paper's 108 conditions and 2166 trials per neuron. The test file adds 100 unlabelled segments for the 114 held-in units only. The trial count is reported only, as the criterion says. The criterion's partial outcome applies because the units match and the conditions are a 27-condition maze set; that these 9 mazes are among the paper's 108 is consistent with the data (same monkey, task and three-version design) but cannot be shown, as nothing in the files links the session (2009-09-29) to the paper's J-array dataset. The paper's J-array dataset interleaved all 108 conditions; the file says its trials come from the full session of 2009-09-29, and its trial timeline is rebuilt (each trial starts 0.014 to 0.109 s after the previous one ends), so the files cannot show whether that session held other mazes. If it ran only these 27 conditions it is a different recording, and the unit count match says nothing about the paper's dataset. Whether the criterion's partial clause should apply to a session that may not be the J-array recording is asked of a reader (questions/000139.yaml).",
     "deviations": [
      "Assumed that the 27 conditions are among the paper's 108, as the criterion's partial clause requires. The monkey, task and three-version design match the paper's, but nothing in the files links this session to the paper's J-array dataset, so the subset relation cannot be shown.",
      "The 100 test trials of the Neural Latents Benchmark carry no condition labels or go cue times, so only the 250 training and validation trials are used in every analysis."
     ],
     "assumed": true,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig4d-mean-correlation-j-array",
     "statement": "For the J-array dataset, preparatory and peri-movement tuning are positively but only weakly correlated on average across neurons (mean r = 0.13, p = 0.03; mean Spearman r = 0.11).",
     "source": "Results, Correlations between preparatory and peri-movement tuning, paragraph 1; Figure 4D",
     "quote": "Across the four datasets the mean correlation was positive (p=0.01, p=0.01, p=0.12, p=0.03) but only very modestly so (mean r = 0.21, 0.19, 0.06 and 0.13). Non-parametric correlations were similarly modest (mean Spearman's r = 0.20, 0.20, 0.06 and 0.11)",
     "statistics": [
      {
       "about": "mean correlation against zero, J-array",
       "p": 0.03
      }
     ],
     "kind": "finding",
     "inferred": "",
     "outcome": "partially_reproduced",
     "blocked_by": null,
     "notebook": "reproduce_churchland_2010.ipynb",
     "notes": "Under the adopted reading 17 neurons pass both thresholds; mean Pearson r = 0.172 and mean Spearman r = 0.139, both outside tolerance. The mean is above zero by the t-test (p = 0.042) and the Fisher-z t-test (p = 0.044), not by the Wilcoxon test (p = 0.071). Over the 8 trial-selection readings mean r is 0.141 to 0.207 (all inside 0.054 to 0.231) and p < 0.05 by some test in 5; in the other 3 (onset a with condition-mean proto and separate profiles, onset a with medoid and concatenated profiles, onset b with medoid and separate profiles) the smallest p is 0.052, 0.056 and 0.080, and the outcome would be not reproduced. No reading puts both means in tolerance with p < 0.05 (onset b, medoid, separate has both in tolerance, 0.141 and 0.119, with p = 0.080). The SD tuning measure leaves no neuron passing both thresholds in any reading. The result moves with the thresholds (at 1.5 times, 3 to 30 neurons and mean r 0.114 to 0.210 for preparatory thresholds of 10 to 3 spikes/s; table in the notebook).",
     "deviations": [
      "Assumed the tuning strength is the range across conditions. The Methods give the thresholds in spikes/s of tuning (\"at least 10 spikes/s of preparatory tuning\", Methods, Neural recordings and datasets) without the measure; the SD reading leaves 5 to 8 of 152 units above 5 spikes/s and none passing both thresholds, which cannot be the paper's array analysis, so the range is the reading the paper supports.",
      "Assumed the proto-trial is the condition-mean velocity profile and that the horizontal and vertical profiles are concatenated. The Supplemental Experimental Procedures (Task design and behavioral control) keep \"the subset of trials (typically ~90% of all trials)\" that correlate with the proto-trial at r > 0.9; concatenation keeps 93 to 96% of trials, comparing the profiles separately keeps 70 to 82%. Mean against medoid is not settled by the paper; a template is usually a mean. All 8 readings are shown.",
      "Movement onset is the dataset's move_onset_time (reading a); onset from hand speed (10% of peak, a standard choice, a median 12 ms later) is run as reading b.",
      "6 to 10 trials per condition, against about 20 in the paper's J-array dataset."
     ],
     "assumed": true,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig4abc-mean-correlation-other-datasets",
     "statement": "For monkeys A, B and J (single electrodes) the mean correlation between preparatory and peri-movement tuning is 0.21, 0.19 and 0.06 (p = 0.01, 0.01, 0.12), with mean Spearman r of 0.20, 0.20 and 0.06.",
     "source": "Results, Correlations between preparatory and peri-movement tuning, paragraph 1; Figure 4A to 4C",
     "quote": "Across the four datasets the mean correlation was positive (p=0.01, p=0.01, p=0.12, p=0.03) but only very modestly so (mean r = 0.21, 0.19, 0.06 and 0.13)",
     "statistics": [
      {
       "about": "monkey A",
       "p": 0.01
      },
      {
       "about": "monkey B",
       "p": 0.01
      },
      {
       "about": "monkey J, single electrodes",
       "p": 0.12
      }
     ],
     "kind": "finding",
     "inferred": "",
     "outcome": "blocked",
     "blocked_by": "data_missing",
     "notebook": null,
     "notes": "The dandiset holds one maze-task array session of monkey J. The speed-task datasets of monkeys A and B and the single-electrode monkey J dataset are not in it (assets and metadata checked) and the paper names no other archive for them.",
     "deviations": [],
     "assumed": false,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig4-broad-unimodal",
     "statement": "Individual neurons span a broad range of preparatory to peri-movement correlations centred near zero, with little sign of separate populations of concordant (r near 1) and opposing (r near -1) tuning, except perhaps weakly for monkey B (Figure 4B).",
     "source": "Results, Correlations between preparatory and peri-movement tuning, paragraph 2; Figure 4A to 4D",
     "quote": "different neurons show markedly different relationships between preparatory and peri-movement activity, with a broad range of correlations centered near zero",
     "statistics": [],
     "kind": "finding",
     "inferred": "",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_churchland_2010.ipynb",
     "notes": "In every selection reading the per-neuron correlations run from negative (-0.36 to -0.50) to positive (0.54 to 0.68) values, the mean is 0.141 to 0.207, no neuron has |r| > 0.8 while 13 to 18 have |r| < 0.5, and Hartigan's dip test does not reject unimodality (p = 0.195 to 0.993). Main reading n = 17, SD 0.321, dip p = 0.788. With 15 to 20 neurons the test has little power.",
     "deviations": [
      "Rests on the tuning measure and trial selection adopted in fig4d-mean-correlation-j-array."
     ],
     "assumed": true,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig4e-correlation-time-course",
     "statement": "The correlation between preparatory tuning and peri-movement activity at a single time point begins to decline about 200 ms before movement onset and is near zero by movement onset.",
     "source": "Results, Correlations between preparatory and peri-movement tuning, paragraph 2; Figure 4E",
     "quote": "The correlation with preparatory activity began to decline \u223c200 ms prior to movement onset, and reached values near zero by movement onset",
     "statistics": [],
     "kind": "finding",
     "inferred": "",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_churchland_2010.ipynb",
     "notes": "With the neurons of Figure 4D and the spliced trace, mean r is 0.703 at 400 ms before onset, 0.492 at 200 ms before and 0.118 at onset (not above zero, t-test p = 0.127), and the decline begins at -263 ms. These two readings meet the criterion in 7 of 8 selection readings. The splice decides this result. Its go-locked part (go cue -200 to +100 ms) is the preparatory window, mapped with the median RT to -554 to -254 ms, so the value at 400 ms before onset is taken inside the window the preparatory rate is averaged over, and the decline at -263 ms is the end of that window shifted by the median RT. In the main selection no other combination meets the criterion. The movement-locked trace alone puts the decline at -307 ms, just outside -300 to -100, and with all preparatory neurons the onset value (0.107, n = 76) is above zero (p = 0.0036) and above 0.1. Without the splice the criterion is met by the movement-locked trace with all preparatory neurons in 6 of 8 selection readings (decline at -283 to -299 ms, onset value 0.054 to 0.098) and with the neurons of Figure 4D in 3 of 8 (decline at -298 to -300 ms). The outcome holds under the criterion through those readings, with the decline at the early edge of the window. 22 of 32 combinations meet the criterion.",
     "deviations": [
      "Assumed the neurons of Figure 4A to 4D (both thresholds) and the spliced trace. The legend of Figure 4 places E with A to D, and the Supplemental Experimental Procedures (Computing the mean firing rate) say the spliced trace \"was also useful for the analysis in Figure 4E\". The splice maps the go-locked trace (-200 to +100 ms) to movement time with the median RT (354 ms) and continues with the movement-locked trace; where the paper splices is not given, and the median RT is an assumption. The decline under the spliced trace (-263 ms) falls near the splice point (-254 ms) and is set by it; the movement-locked trace alone gives -307 ms."
     ],
     "assumed": true,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "suppfig1-population-psth-weak-movement-tuning",
     "statement": "In population averages sorted by each neuron's preparatory preference, the preparatory differences carry weakly into the peri-movement epoch and show little relationship with preparatory tuning by movement onset.",
     "source": "Results, Correlations between preparatory and peri-movement tuning, paragraph 2; Supplemental Figure 1",
     "quote": "By movement onset even population averages show little relationship with preparatory tuning",
     "statistics": [],
     "kind": "finding",
     "inferred": "",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_churchland_2010.ipynb",
     "notes": "Under the main reading (17 neurons) the strongest-minus-weakest difference of normalized rate is 0.108 in the preparatory window and 0.101 at movement onset (ratio 0.942); it is smaller (criterion reading a) in all 8 selection readings but less than half (reading b) only under the separate-profile readings (0.184 to 0.357). The statement's \"little relationship\" by onset is supported under those readings and with all preparatory neurons (ratio 0.047 to 0.213), not under the concatenated-profile readings. Ranking the conditions and measuring on the same noisy rates favours the claim. With conditions ranked on half of each condition's trials and measured on the other half (main selection), the preparatory difference of the 17 neurons falls to 0.040 and the onset difference (0.045) is no longer smaller (ratio 1.115); with all 76 preparatory neurons it is 0.055 against 0.030 (ratio 0.545). The outcome follows the legend's method, which the criterion names; on these data the result for the claim's neurons does not survive cross-validation.",
     "deviations": [
      "The two intermediate conditions are those at one third and two thirds of the rank order of preparatory rate, a standard choice (the legend says only \"two intermediate conditions\").",
      "Rests on the tuning measure and trial selection adopted in fig4d-mean-correlation-j-array."
     ],
     "assumed": true,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig6d-perimovement-best-j-array",
     "statement": "For the J-array dataset, the highest cross-validated PD performance over all tested spaces and dimensionalities is reached when the PD is expressed in the peri-movement space.",
     "source": "Results, PD performance in a variety of spaces, paragraph 2; Figure 6D",
     "quote": "For every dataset, the highest level of performance was achieved ... when PDs were expressed in the peri-movement space",
     "statistics": [],
     "kind": "finding",
     "inferred": "",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_churchland_2010.ipynb",
     "notes": "76 neurons fitted. Best peri-movement performance 0.117 (k = 6) with S fixed and 0.226 (k = 13) with S recomputed in each fold, against the best kinematic space 0.016 to 0.024 and the best task space 0.030 (initial velocity), under all 8 combinations of S, kinematic window and replication. Over the selection readings the peri-movement space is highest under all combinations in 7 of 8 and under some in the eighth (onset a, medoid, separate). All levels are low; the split-half reliability of preparatory tuning is a median r of 0.374. The paper's upper limit (T-matrix including the fitted neuron) is 0.178 with S fixed and 0.300 recomputed; the variant without the fitted neuron (0.068, 0.184) is below the peri-movement best and is no upper limit. The array-1 neurons reach 0.291 and the array-2 neurons -0.006. Chance (conditions shuffled, 10 draws) was computed for the peri-movement space, the kinematic space with window a and replication a, and the task spaces, and each is above its chance level at every k; chance is -0.120 to -2.732 for the peri-movement space with S fixed, -0.102 to -0.308 recomputed, and -0.172 to -1.128 for the task spaces. It was not computed for kinematic window b, replication b or the upper limits. With 6 to 10 trials per condition the 5 spikes/s threshold passes a median of 59 of 152 units on shuffled labels against 75 observed, so noise matches most of the passing count, and 43 of the 76 PD neurons have tuning distinguishable from noise (permutation p < 0.05). On those 43 the peri-movement best is 0.336 (S fixed) and 0.415 (recomputed), against 0.116 to 0.117 kinematic and 0.143 for the best task space.",
     "deviations": [
      "Rests on the tuning measure and trial selection adopted in fig4d-mean-correlation-j-array.",
      "Reach end, used for endpoints and the trajectory-based task spaces, is where the condition-mean speed falls below 10% of its peak, a standard choice; acceleration is the derivative of the smoothed velocity; the T-matrices are sampled every 10 ms, a standard choice.",
      "The EMG space cannot be built; the paper recorded no EMG for J-array."
     ],
     "assumed": true,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig6-perimovement-best-each-dimensionality",
     "statement": "Even taken one dimensionality at a time, PD performance is almost always higher in the peri-movement space than in other spaces of the same dimensionality.",
     "source": "Results, PD performance in a variety of spaces, paragraph 2; Figure 6A to 6D",
     "quote": "Even if we consider each dimensionality individually, performance was almost always higher for the peri-movement space",
     "statistics": [],
     "kind": "finding",
     "inferred": "",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_churchland_2010.ipynb",
     "notes": "The peri-movement space wins 20 or 21 of 22 comparisons (14 kinematic dimensionalities and 8 task spaces), 0.909 to 0.955, under every reading of S, kinematic window and replication, and both sets of task-space variants.",
     "deviations": [
      "Rests on the tuning measure and trial selection adopted in fig4d-mean-correlation-j-array."
     ],
     "assumed": true,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig6-dimensionality-peak",
     "statement": "Performance of the PCA-based spaces typically improves with dimensionality, peaks usually at a middle range, and declines at dimensionalities above 10, a sign of over-fitting.",
     "source": "Results, PD performance in a variety of spaces, paragraph 1; Figure 6A to 6D",
     "quote": "Performance typically improved with dimensionality. This increase is nontrivial, as generalization performance is being assessed ... Performance typically declined at dimensionalities >10, an indication of over-fitting",
     "statistics": [],
     "kind": "finding",
     "inferred": "",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_churchland_2010.ipynb",
     "notes": "With S fixed the peri-movement curve goes from -0.006 at k = 1 to 0.117 at k = 6 and -0.980 at k = 14; the kinematic curves peak at k = 2 (0.016 to 0.022) and fall to about -2 at k = 14. With S recomputed in each fold the peri-movement curve peaks at k = 13 (0.226), outside 2 to 10, and is flat above k = 10 (0.221 to 0.226), so the criterion fails under that reading. With S fixed the best k is 2 to 6 in all 8 selection readings; with S recomputed it is 11 to 14. The chance curves (conditions shuffled) show the two readings penalize extra dimensions differently; the peri-movement chance curve falls by 2.292 from k = 6 to k = 14 with S fixed and by 0.094 with S recomputed, so under the recomputed reading little decline is possible even for noise. The verifier would give partially reproduced; the outcome is kept on the adopted reading and the disagreement recorded.",
     "deviations": [
      "Assumed S is computed once from all conditions and only the PD regression leaves the condition out. The Methods (Using a PD, PD performance) define S as \"a c x k matrix containing the location of every condition\" and say \"One condition was left out and we found the PD that best captured tuning for the remaining conditions\", which refits only the PD. The recomputed reading is shown."
     ],
     "assumed": true,
     "disagreed": true,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig6-maze-task-spaces",
     "statement": "Segment directions and initial velocity are among the task spaces that perform reasonably well, segment directions doing so for the maze task, while for the maze task the endpoints + maximum speed space does not perform well.",
     "source": "Results, PD performance in a variety of spaces, paragraphs 1 and 2; Figure 6C and 6D; Figure 7D legend",
     "quote": "A number of task spaces performed reasonably well, including ... ' segment dirs. ' ... and ' initial vel. ' ... The former is undefined for the maze task, but the latter did not perform well",
     "statistics": [],
     "kind": "finding",
     "inferred": "",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_churchland_2010.ipynb",
     "notes": "The claims file names both segment divisions as readings, and the criterion holds under one. Under the main reading segment directions with segments equal in time (0.008) and initial velocity (0.030) are above zero, and endpoints + maximum speed (-0.045) is below the 3-D peri-movement space (0.052 S fixed, 0.065 recomputed; reading a) and below both (reading b). With segments equal in path length segment directions are -0.027, below zero. All three are far above their chance levels (conditions shuffled; -0.429, -0.172 and -0.222). The result is fragile across the trial selection; segment directions with time segments are above zero in 2 of 8 selection readings (-0.156 to 0.008) and initial velocity in 4 (-0.028 to 0.040). On the 43 PD neurons whose tuning passes a permutation test it is clearer (segment directions 0.143, initial velocity 0.102, endpoints + maximum speed 0.020). Measured values are segment directions (time), initial velocity and endpoints + maximum speed. The segment division is asked of the paper's authors (findings).",
     "deviations": [
      "Rests on segments equal in time, the reading under which the criterion holds; the paper (\"the individual horizontal and vertical displacements after dividing the reach trajectory into three segments\", Methods, Task spaces) does not say how the segments are divided.",
      "Rests on the tuning measure and trial selection adopted in fig4d-mean-correlation-j-array."
     ],
     "assumed": true,
     "disagreed": true,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig6abc-perimovement-best-other-datasets",
     "statement": "For monkeys A, B and J (single electrodes) too, the best PD performance is in the peri-movement space, though among 3-D spaces the targets + instructed speed and endpoints + maximum speed spaces perform very well for the speed-task datasets.",
     "source": "Results, PD performance in a variety of spaces, paragraph 2; Figure 6A to 6C",
     "quote": "An exception was that, among 3-dimensional spaces, the 'targs+instr. spd.' and 'endpoints + max. spd.' performed very well for the datasets collected using the speed task",
     "statistics": [],
     "kind": "finding",
     "inferred": "",
     "outcome": "blocked",
     "blocked_by": "data_missing",
     "notebook": null,
     "notes": "Needs the speed-task and single-electrode datasets and EMG, which are not in the dandiset.",
     "deviations": [],
     "assumed": false,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig6e-summary-rank",
     "statement": "Across monkeys A, B and J, performance averaged over neurons is highest for the peri-movement space, with the gap growing at higher dimensionalities; task spaces are outcompeted by the kinematic and EMG spaces, which are outperformed by the peri-movement space.",
     "source": "Results, PD performance in a variety of spaces, paragraph 3; Figure 6E",
     "quote": "Performance (averaged across neurons) was highest when the PD employed the space based on peri-movement neural activity. This performance differential increased at higher dimensionalities",
     "statistics": [
      {
       "about": "spaces against the best space, per dimensionality limit",
       "p": 0.001,
       "p_relation": "<"
      }
     ],
     "kind": "finding",
     "inferred": "",
     "outcome": "blocked",
     "blocked_by": "data_missing",
     "notebook": null,
     "notes": "Pools monkeys A, B and J (single electrodes) and EMG spaces, which are not in the dandiset; the summary excludes J-array.",
     "deviations": [],
     "assumed": false,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig6-preparatory-dimensionality",
     "statement": "Preparatory activity needs 7 to 10 dimensions, depending on the dataset, to capture 90% of its variance.",
     "source": "Supplemental Experimental Procedures, Estimating the upper limit of PD performance; Results, PD performance in a variety of spaces, paragraph 1",
     "quote": "depending on the dataset, 7-10 dimensions were required to capture 90% of the data variance",
     "statistics": [],
     "kind": "finding",
     "inferred": "",
     "outcome": "not_reproduced",
     "blocked_by": null,
     "notebook": "reproduce_churchland_2010.ipynb",
     "notes": "90% of the variance of the c x n preparatory matrix takes 12 PCs for the main reading (PD neurons, not normalized), and 12 to 19 over all readings of neurons, normalization and trial selection, outside 6.3 to 11 and below the bound of 26. With condition labels shuffled the same count is 17, so with 6 to 10 trials per condition it measures noise more than the structure of preparatory activity; the paper had about 20 per condition and 108 conditions. A count that removes noise independent between halves of the trials (cross-validated PCA, odd against even trials) gives 7 PCs for both the PD neurons and all units, inside 6.3 to 11, while the plain count on each half gives 15 to 16. The criterion's count is the plain one, so the outcome stands; the failure is likely due to the few trials per condition.",
     "deviations": [],
     "assumed": false,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig7a-shuffled-space-poor",
     "statement": "A peri-movement space built after randomly inverting each condition's peri-movement response on half the conditions performs poorly.",
     "source": "Results, Controls, paragraph 1; Figure 7A",
     "quote": "The PCA-based 'shuffled-space' performed poorly",
     "statistics": [],
     "kind": "finding",
     "inferred": "",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_churchland_2010.ipynb",
     "notes": "Averaged over 20 shuffles the shuffled space's best performance is -0.069 with S fixed and -0.010 recomputed (single draws -0.099 to 0.042), below the unshuffled peri-movement best (0.117, 0.226), reading a, and below the best kinematic (0.022, 0.024) and task (0.030) spaces, reading b.",
     "deviations": [
      "Rests on the tuning measure and trial selection adopted in fig4d-mean-correlation-j-array."
     ],
     "assumed": true,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig7b-window-start-slide",
     "statement": "Sliding the start of the peri-movement window from 150 ms before movement onset to movement onset barely lowers PD performance, to 97% (monkey J) and 94% (monkeys A, B) of its original value.",
     "source": "Results, Controls, paragraph 1; Figure 7B",
     "quote": "Even when the window began at movement onset ... performance was still at 97% (monkey J), and 94% (monkeys A, B) of its original value",
     "statistics": [],
     "kind": "finding",
     "inferred": "",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_churchland_2010.ipynb",
     "notes": "With the window end fixed at 800 ms, performance at the best k falls from 0.117 to 0.107 as the start moves from -150 ms to onset (fraction 0.911) with S fixed, and from 0.226 to 0.218 (0.962) with S recomputed, both within 0.97 +/- 0.097. Shifting the whole window gives 0.853 and 0.937; 0.853 is outside 0.97 +/- 0.097 but inside reading b's 0.846 to 1.067.",
     "deviations": [
      "S fixed across folds, as adopted in fig6-dimensionality-peak; the recomputed reading also reproduces."
     ],
     "assumed": true,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "ap-location-no-effect-j-array",
     "statement": "For the J-array dataset the correlation between preparatory and peri-movement tuning does not depend on anterior-posterior location (r = 0.20, p = 0.27).",
     "source": "Results, Comparison of anterior versus posterior sites, paragraph 1",
     "quote": "Regressing tuning correlation (as in Figure 4 ) against AP location yielded little or no effect",
     "statistics": [
      {
       "about": "tuning correlation against AP location, J-array",
       "test": "regression",
       "statistic": "r = 0.20",
       "p": 0.27
      }
     ],
     "kind": "finding",
     "inferred": "",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_churchland_2010.ipynb",
     "notes": "Regressing each neuron's tuning correlation on its array (PMd 1, M1 0) gives r = 0.165, p = 0.526 (17 neurons, 9 on array 1) under the main reading and p = 0.53 to 0.91 over the selection readings, so the paper's test finds no effect at p < 0.05. Mean r is 0.221 on array 1 and 0.117 on array 2.",
     "deviations": [
      "The electrode table has no positions, so AP location is read as the array (the claims file's reading b) and the median split of the paper cannot be made within an array.",
      "Assumed unit id prefix 1 is the PMd array. The units' electrode references all point at PMd rows; those of prefix-1 units (rows 0 to 89) are consistent with PMd, those of prefix-2 units (rows 14 to 94) with M1 channels missing the offset of 96 rows, and the electrode table lists PMd first. The regression's p does not depend on which array is coded 1."
     ],
     "assumed": true,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "ap-location-no-effect-other-datasets",
     "statement": "For monkeys A, B and J (single electrodes) the tuning correlation does not depend on AP location (r = -0.05, p = 0.82; r = -0.03, p = 0.85; r = -0.07, p = 0.52).",
     "source": "Results, Comparison of anterior versus posterior sites, paragraph 1",
     "quote": "r =-0.05, p =0.82; r =-0.03, p =0.85; r =-0.07, p =0.52; and r =0.20, p =0.27 for the four datasets",
     "statistics": [
      {
       "about": "monkey A",
       "test": "regression",
       "statistic": "r = -0.05",
       "p": 0.82
      },
      {
       "about": "monkey B",
       "test": "regression",
       "statistic": "r = -0.03",
       "p": 0.85
      },
      {
       "about": "monkey J",
       "test": "regression",
       "statistic": "r = -0.07",
       "p": 0.52
      }
     ],
     "kind": "finding",
     "inferred": "",
     "outcome": "blocked",
     "blocked_by": "data_missing",
     "notebook": null,
     "notes": "The single-electrode datasets are not in the dandiset.",
     "deviations": [],
     "assumed": false,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig7c-anterior-posterior-performance",
     "statement": "The peri-movement space captures preparatory tuning at both anterior and posterior sites, with normalized performance only slightly different from that of randomly divided data, and slightly better at anterior sites.",
     "source": "Results, Comparison of anterior versus posterior sites, paragraph 1; Figure 7C",
     "quote": "The ability of the peri-movement space to capture tuning also held for both anterior and posterior sites",
     "statistics": [],
     "kind": "finding",
     "inferred": "",
     "outcome": "not_reproduced",
     "blocked_by": null,
     "notebook": "reproduce_churchland_2010.ipynb",
     "notes": "With the paper's upper limit (T-matrix including the fitted neuron) neither array is within 10% of the random halves' normalized value under any reading of S or of the upper limit's dimensionality. At 6-D with S fixed array 1 gives 0.761, array 2 -0.402 and random halves 0.455 (SD 0.153 over the 40 halves); with S recomputed 0.766, 0.230 and 0.564; at the best k of the upper limit 0.472, -0.374 and 0.350 (S fixed) and 0.438, 0.144 and 0.336 (recomputed). Array 2 is below zero with S fixed. Array 1 is higher in every reading. Since the within-10% test fails for both halves, the outcome does not depend on which array is PMd. The variant without the fitted neuron, used in round 1, gives 6-D upper limits from -0.107 to 0.170, at or below zero in 2 of 8 (array 2 and random halves, S fixed), and ratios from -0.740 to 2.111. Unnormalized 6-D performance is 0.271 on array 1 and -0.060 on array 2 with S fixed, 0.270 and 0.049 recomputed, against 0.095 and 0.140 for random halves.",
     "deviations": [
      "Halves are the two arrays, not a median split of AP location (no positions in the dandiset); array 1 is taken as PMd, as in ap-location-no-effect-j-array.",
      "Random halves are 20 random splits of the units into two equal halves, a standard choice.",
      "The upper limit's T-matrix holds every neuron of the half, the fitted one included, as the Supplemental Experimental Procedures (Estimating the upper limit of PD performance) give it, \"T was c x n with each row containing the mean preparatory activity for every neuron\"."
     ],
     "assumed": true,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "prep-activity-more-anterior",
     "statement": "Preparatory activity is more prevalent at anterior sites.",
     "source": "Results, Comparison of anterior versus posterior sites, paragraph 1",
     "quote": "We similarly found preparatory activity to be more prevalent at anterior sites (data not shown)",
     "statistics": [],
     "kind": "finding",
     "inferred": "",
     "outcome": "blocked",
     "blocked_by": "metadata_insufficient",
     "notebook": "reproduce_churchland_2010.ipynb",
     "notes": "Blocked after the analysis. The direction of the comparison depends on which unit id prefix is PMd, and the files cannot say; the electrode table's location column puts PMd on ids 1 to 96 and M1 on 97 to 192, but every unit references a PMd row, the x, y, z columns are empty, and the units table has no location. With the range measure 35 of 75 units on array 1 (0.467) and 41 of 77 on array 2 (0.532) pass the 5 spikes/s preparatory threshold (Fisher p = 0.517), and array 1 has the lower fraction in all 8 selection readings; with the SD measure array 1 is higher (4 of 75 against 1 of 77, p = 0.206). If array 1 is PMd the range result goes against the claim; if it is M1 it favours it, without reaching p < 0.05 in any reading. With labels shuffled a median of 59 of 152 units pass the range threshold, so the fractions are largely noise.",
     "deviations": [
      "AP location read as the array; tuning measured as the range, as adopted in fig4d-mean-correlation-j-array."
     ],
     "assumed": false,
     "disagreed": true,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig7d-pd-consistency",
     "statement": "The PD is most stable across random condition subsets when measured in the peri-movement space, for every comparison and all four datasets.",
     "source": "Results, Stability of the PD, paragraph 1; Figure 7D",
     "quote": "Yet for every comparison, and for all four datasets, the PD was most stable when measured in the peri-movement space",
     "statistics": [],
     "kind": "finding",
     "inferred": "",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_churchland_2010.ipynb",
     "notes": "Mean PD consistency over 76 neurons is higher in the peri-movement space than in endpoints + maximum speed (0.698 against 0.597, Wilcoxon p < 0.0001), segment directions (0.613 against 0.605 with time segments, p = 0.897; 0.615 against 0.542 with path segments, p = 0.023) and the kinematic space at its best k = 2 (0.729 against 0.670, p = 0.0004). The segment-directions comparison with time segments is a tie in practice.",
     "deviations": [
      "100 pairs of draws per neuron, a standard choice; S from all conditions; the peri-movement space at the dimensionality of each comparison space.",
      "Rests on the tuning measure and trial selection adopted in fig4d-mean-correlation-j-array."
     ],
     "assumed": true,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "some-neurons-better-elsewhere",
     "statement": "Although the peri-movement space performs best on average, some individual neurons are better captured in one of the other spaces.",
     "source": "Discussion, paragraph 3",
     "quote": "while the peri-movement space performed better on average, there were individual neurons whose activity was better captured in one of the other spaces",
     "statistics": [],
     "kind": "finding",
     "inferred": "",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_churchland_2010.ipynb",
     "notes": "60 of 76 neurons (0.789) with S fixed and 44 of 76 (0.579) with S recomputed perform better in at least one other space than in the peri-movement space at its best k (6 and 13). One value is compared with the best of 12 alternatives on noisy rates, so a large fraction is expected from noise alone; the claim asks only for at least one neuron.",
     "deviations": [
      "Rests on the tuning measure and trial selection adopted in fig4d-mean-correlation-j-array."
     ],
     "assumed": true,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "high-dimensional-task-spaces-underperform",
     "statement": "The high-dimensional task spaces, including the kitchen-sink space, always perform below the peri-movement space, so high dimensionality alone does not explain its performance.",
     "source": "Results, Controls, paragraph 2; Methods, Task spaces",
     "quote": "the high-dimensional task spaces (including the gratuitously high-dimensional 'kitchen-sink' space) always underperformed the peri-movement space",
     "statistics": [],
     "kind": "finding",
     "inferred": "",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_churchland_2010.ipynb",
     "notes": "The kitchen-sink spaces (-0.244 and -0.221) and segment directions (0.008 and -0.027) are below the peri-movement space at the same dimensionality (11-D -0.184 and 6-D 0.117 with S fixed; 0.221 and 0.145 recomputed) and below its best (0.117, 0.226), under both readings.",
     "deviations": [
      "Rests on the tuning measure and trial selection adopted in fig4d-mean-correlation-j-array."
     ],
     "assumed": true,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig8-linear-dynamics-simulation",
     "statement": "In a simulated two-unit linear dynamical system with 25 random initial states, preparatory activity correlates weakly with peri-movement activity at the single-unit level (r = 0.08), yet a PD in the PCA-based peri-movement space captures preparatory tuning perfectly.",
     "source": "Results, Tuning from a dynamical systems perspective, paragraph 2; Figure 8",
     "quote": "The correlation of preparatory activity with peri-movement activity (averaged over the gray interval) was 0.08 ... The PD points towards the pattern recruited by the activity of unit one (panel A ), and perfectly captures preparatory 'tuning'",
     "statistics": [],
     "kind": "finding",
     "inferred": "",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_churchland_2010.ipynb",
     "notes": "With the gray interval read as one full cycle (360 steps), the central 95% of the correlation over 1000 draws is -0.397 to 0.387 (uniform initial states) and -0.400 to 0.404 (Gaussian), containing 0.08; over a quarter cycle it is 0.496 to 0.847 and 0.455 to 0.871, not containing it. The 2-D PCA peri-movement space fits unit 1's initial rates with performance 1.0 in every draw.",
     "deviations": [
      "The distribution of initial states and the gray interval are not given (Figure 8 legend); both readings of each are run and the outcome rests on the full-cycle interval."
     ],
     "assumed": true,
     "disagreed": false,
     "data": "dataset",
     "where": null,
     "proposed": null
    }
   ],
   "findings": [
    {
     "kind": "metadata_error",
     "statement": "Every unit's electrode reference points at a row of the PMd electrode group, although the unit ids (array x 1000 + channel x 10 + sort) put 77 of the 152 units on a second array. The references of those units (rows 14 to 94) match their channel numbers without the offset of 96 that would place them on the M1 rows, so the M1 units are labelled as PMd.",
     "found": null,
     "status": null
    },
    {
     "kind": "metadata_insufficient",
     "statement": "The electrode table's x, y, z and impedance columns are empty, so the anterior-posterior location of a unit cannot be read; the array is known only through the unit id convention, which the files do not describe, and nothing says whether id prefix 1 or 2 is the PMd array. This blocks prep-activity-more-anterior.",
     "found": null,
     "status": null
    },
    {
     "kind": "data_missing",
     "statement": "The dandiset holds one array session of monkey J on the maze task with 27 conditions and 250 labelled trials. The paper's J-array dataset had 108 conditions and about 2166 trials per neuron, and the speed-task datasets of monkeys A and B, the single-electrode monkey J dataset and all EMG are not deposited, so 4 of the paper's main findings cannot be tested and the others rest on 6 to 10 trials per condition.",
     "found": null,
     "status": null
    },
    {
     "kind": "identifier_missing",
     "statement": "Nothing in the files or the dandiset metadata says whether this session (2009-09-29) is the paper's J-array recording or a different one, or how its 27 conditions relate to the paper's 108.",
     "found": null,
     "status": null
    },
    {
     "kind": "selection_undocumented",
     "statement": "The files keep 250 successful trials split into train and val, and 100 test trials as 700 ms segments without condition labels, go cue or behaviour. How the trials and the 9 mazes were chosen from the session is not described.",
     "found": null,
     "status": null
    },
    {
     "kind": "methods_unavailable",
     "statement": "For the paper's authors. The supplement keeps trials whose velocity profiles correlate with a reference proto-trial at r > 0.9. Is the proto-trial the condition mean or one trial, and are the horizontal and vertical profiles correlated separately or together? Here separate comparison keeps 70 to 82% of trials and concatenation 93 to 96%, against the stated ~90%; the mean with concatenation was assumed, and the mean correlation of Figure 4D passes p < 0.05 under 5 of 8 readings.",
     "found": null,
     "status": null
    },
    {
     "kind": "methods_unavailable",
     "statement": "For the paper's authors. How is \"spikes/s of preparatory tuning\" measured (range or standard deviation of condition means)? The range was assumed; with the standard deviation almost no array unit passes 5 spikes/s.",
     "found": null,
     "status": null
    },
    {
     "kind": "methods_unavailable",
     "statement": "For the paper's authors. In the leave-one-condition-out performance, is the PCA-based space S recomputed without the left-out condition? S fixed was assumed (one S per space, only the PD refitted). With S recomputed, peri-movement performance here does not decline at high dimensionality (best k = 13).",
     "found": null,
     "status": null
    },
    {
     "kind": "methods_unavailable",
     "statement": "For the paper's authors. For the segment-directions and kitchen-sink spaces, is the reach divided into three segments of equal time or equal path length? Here segment directions perform above zero with time segments (0.008) and below with path segments (-0.027), which decides fig6-maze-task-spaces, reproduced under time segments.",
     "found": null,
     "status": null
    },
    {
     "kind": "methods_unavailable",
     "statement": "For the paper's authors. How was movement onset detected, where is the go-locked trace spliced to the movement-locked one in Figure 4E, and what is the gray interval of Figure 8? Here the dataset's onsets, a splice at the median RT and a full cycle were used, and the other readings shown.",
     "found": null,
     "status": null
    }
   ],
   "reconciliation": null,
   "verifications": [
    {
     "date": "2026-10-28",
     "kind": "agent",
     "by": "Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook), verifier-1",
     "reviewed": "The handover of the first run, the executed notebook with the conclusions and summary removed, the helper module and the claims file, with the paper under paper/ and the archive's API.",
     "agreed": 20,
     "of": 23,
     "disagreements": [
      {
       "claim": "fig6-dimensionality-peak",
       "outcome": "partially_reproduced",
       "reason": "It holds under S reading (a), best k = 6, k = 14 at -0.98. It fails under S (b), best k = 13, k = 14 equals the best, and best k is 11 to 14 in every selection reading. The result rests on an open reading. A random PCA space under S (b) gives about -0.1 flat across k, against -0.18 to -1.34 under S (a), so the missing decline above k = 10 under (b) partly reflects how out-of-sample projection shrinks the left-out condition's scores, and the two S readings are not on the same footing (problem 5)."
      },
      {
       "claim": "fig6-maze-task-spaces",
       "outcome": "reproduced",
       "reason": "Marginal. Main selection gives segment directions (time-split) 0.008 > 0, initial velocity 0.030 > 0, endpoints + max speed -0.045, below both references. The notebook's blocked, methods_underspecified is not supported, because the criterion supplies the readings (problem 4). It is fragile, segment directions are > 0 in only 2 of 8 selection readings, and the length-split variant fails."
      },
      {
       "claim": "prep-activity-more-anterior",
       "outcome": "blocked",
       "reason": "Blocked, metadata_insufficient. The direction depends on which id prefix is PMd. The electrode x, y, z are NaN and every unit points at a PMd row, so the file cannot say. As labelled, the range measure gives fewer passing units anterior; swapping the labels reverses that. No reading reaches Fisher p < 0.05. The archive API metadata hold no electrode locations either."
      }
     ]
    },
    {
     "date": "2026-10-28",
     "kind": "agent",
     "by": "Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook), verifier-2",
     "reviewed": "The full executed notebook of the author's round-1 revision, with its conclusions and summary, the README, the helper module, the claims file and the attempt record, with the paper under paper/.",
     "agreed": 23,
     "of": 23,
     "disagreements": []
    }
   ]
  }
 ],
 "entries": [
  {
   "name": "000139-churchland-2010",
   "type": "reproduction",
   "title": "Preparatory activity and a PD in peri-movement space (Churchland et al. 2010)",
   "description": "Tests the findings of Churchland, Cunningham, Kaufman, Ryu and Shenoy (2010) on dandiset 000139 (MC_Maze_Medium), one session of monkey J on the maze task recorded with two Utah arrays and cut to 250 labelled trials over 27 conditions for the Neural Latents Benchmark. It tests the weak correlation between preparatory and peri-movement tuning, the performance of a preferred direction in peri-movement, kinematic and task spaces, the controls of Figure 7, and the simulation of Figure 8. The speed-task and single-electrode datasets and the EMG are not in the dandiset.",
   "datasets": [
    "dandi:000139"
   ],
   "papers": [
    "10.1016/j.neuron.2010.09.015"
   ],
   "derived_from": [],
   "notebooks": [
    {
     "path": "reproduce_churchland_2010.ipynb",
     "title": "Reproducing Churchland et al. (2010) from dandiset 000139",
     "executed": "notebooks/000139-churchland-2010/reproduce_churchland_2010.html"
    }
   ]
  }
 ]
}