Sit & Goard (2023) on dandiset 000483: rotation direction and arena control (Figures S5, S6)

Paper. Sit KK, Goard MJ (2023), Nature Communications 14:1992, doi:10.1038/s41467-023-37704-5; claims file written from the version of record without its supplement, so Figures S5 and S6 were not read.

Dataset. DANDI:000483, version 0.230421.2321, the only published version (the paper cites none). See single_cue.ipynb for the data check of all files.

Claim	Here
figs5-rotation-direction	tested
figs6-arena-control	tested
fig5k-visual-cue-only	blocked: no recording shares the visual-only files' ROIs (structure shown below)

The claims file gives two readings of what the bootstrapped KS test compares: (a) differences in preferred heading between the two conditions for heading-selective cells, against the same differences with cells paired at random; (b) the correlation of each cell's tuning curves in the two conditions, against randomly paired cells. p is one minus the fraction of 1000 bootstrap iterations in which a two-sample KS test passes at alpha 0.05 (Methods, Statistical information); in each iteration both samples are drawn with replacement and the random pairing is drawn afresh (a standard choice). Cells: selective by the two-term test with the centred fit (see single_cue.ipynb) in the motor-driven recording.

import os

for _v in ["OMP_NUM_THREADS", "OPENBLAS_NUM_THREADS", "MKL_NUM_THREADS"]:
    os.environ[_v] = "1"   # one BLAS thread, so that results do not depend on the number of cores
import time
import warnings
from multiprocessing import Pool

import numpy as np
import pandas as pd
from scipy import stats

import rsc

warnings.filterwarnings("ignore")
pd.set_option("display.width", 200)
started = time.time()
cat = rsc.catalog(full=False)
sub = cat[cat.experiment.isin(["motor_both", "motor_cw", "motor_ccw", "animal_controlled"])].reset_index(drop=True)
recs = rsc.load_many(sub.url.tolist())
with Pool(4) as pool:
    for r in recs:
        r["sp"] = rsc.spikes(rsc.dff(r["F"]), pool)
        del r["F"]
print(f"dandiset {rsc.DANDISET_ID} version {rsc.DANDISET_VERSION}: {len(sub)} files loaded and deconvolved "
      f"in {time.time() - started:.0f} s")

dandiset 000483 version 0.230421.2321: 13 files loaded and deconvolved in 163 s

rows = []
for m, r in zip(sub.itertuples(), recs):
    hc = rsc.clean_heading(r["heading"])
    step = rsc.wrap180(np.diff(hc))
    segs = rsc.trial_segments(r["heading"])
    sign = [np.sign(np.median(np.diff(hc[s]))) for s in segs]
    rows.append(dict(experiment=m.experiment, subject=m.subject, file=m.identifier[:6] + " " + m.identifier.split("_")[-1],
                     frames=m.n_frames, rois=m.n_rois, frac_moving=round(float(np.mean(np.abs(step) > 0.5)), 3),
                     trials_pos=int(np.sum(np.array(sign) > 0)), trials_neg=int(np.sum(np.array(sign) < 0))))
info = pd.DataFrame(rows)
print(info.to_string(index=False))

       experiment subject                            file  frames  rois  frac_moving  trials_pos  trials_neg
animal_controlled SKKS124           220720 SKKS124-FR-001    7550    62        0.300           6           4
animal_controlled SKKS125           220721 SKKS125-FR-003    7550   333        0.056           0           0
animal_controlled SKKS125           220726 SKKS125-FM-001    7550   670        0.329           3           9
animal_controlled SKKS126           220727 SKKS126-FR-001    7550   352        0.183           7           1
animal_controlled SKKS127           220729 SKKS127-FR-001    7550   423        0.070           3           0
       motor_both SKKS125 220729 SKKS125-DualRotation-002   15050   399        0.939          41          42
       motor_both SKKS126 220729 SKKS126-DualRotation-002   15050   463        0.923          39          42
       motor_both SKKS127      220727 SKKS127-DualDir-005    7550   213        0.885          17          21
       motor_both SKKS127 220729 SKKS127-DualRotation-002   15050   423        0.891          41          36
        motor_ccw SKKS125          220721 SKKS125-CCW-002    7550   333        0.944           0          37
         motor_cw SKKS124           220720 SKKS124-CW-002    7550    62        0.944           0          40
         motor_cw SKKS125           220726 SKKS125-CW-002    7550   670        0.941          40           0
         motor_cw SKKS126           220727 SKKS126-CW-002    7550   352        0.917          39           0

The animal-controlled files (FR, FM) have few frames in which the chamber turns, so their tuning curves are built from all turning frames, not from full rotations. The files do not mark light and dark, so all frames are used.

def null_boot(real, rand_fn, rng, n_iter=1000, closer="less"):
    passes = 0
    for _ in range(n_iter):
        a = rng.choice(real, len(real))
        b = rng.choice(rand_fn(rng), len(real))
        ks = stats.ks_2samp(a, b)
        side = np.median(a) < np.median(b) if closer == "less" else np.median(a) > np.median(b)
        passes += (ks.pvalue < 0.05) and side
    return 1 - passes / n_iter

def compare(tc1, tc2, cells, rng, label):
    """Readings (a) and (b) for two sets of tuning curves of the same ROIs (ROIs x 60)."""
    idx = np.flatnonzero(cells)
    p1, p2 = rsc.peak_heading(tc1[idx]), rsc.peak_heading(tc2[idx])
    # dual cues: a peak and its opposite are the same cue; distance taken modulo 180
    fold = lambda d: np.abs(rsc.wrap180(2 * d)) / 2
    real_a = fold(p1 - p2)
    rand_a = lambda g: fold(p1 - p2[g.permutation(len(idx))])
    real_b = rsc._rowcorr(tc1[idx], tc2[idx])
    rand_b = lambda g: rsc._rowcorr(tc1[idx], tc2[idx][g.permutation(len(idx))])
    pa = null_boot(real_a, rand_a, rng, closer="less")
    pb = null_boot(real_b[np.isfinite(real_b)], lambda g: rand_b(g)[np.isfinite(rand_b(g))], rng, closer="greater")
    ra = rand_a(rng); rb = rand_b(rng)
    print(f"{label}: n={len(idx)}; (a) median |pref diff| (mod 180) real {np.median(real_a):.0f} deg, random {np.median(ra):.0f} deg, "
          f"bootstrapped KS p = {pa:.3g}; (b) median correlation real {np.nanmedian(real_b):.3f}, random {np.nanmedian(rb):.3f}, "
          f"bootstrapped KS p = {pb:.3g}")
    return dict(n=len(idx), a_real=np.median(real_a), a_rand=np.median(ra), p_a=pa,
                b_real=np.nanmedian(real_b), b_rand=np.nanmedian(rb), p_b=pb)

figs5-rotation-direction

The diversity of RSC responses (a second peak in light, as in visual axons, and a single peak in the dark, as in ADN axons) persists regardless of the direction of chamber rotation (bootstrapped KS test, p = 0.0094, 4 recordings over 2 mice).

The four files that rotate both ways; clockwise and counterclockwise trials within each file.

both = sub.index[sub.experiment == "motor_both"].tolist()
trials = [rsc.trial_tuning(recs[i]["sp"], recs[i]["heading"]) for i in both]
with Pool(4) as pool:
    sel = rsc.selection_many(trials, "double", 401, pool, centre=True)
rng = np.random.default_rng(402)
tc_pos, tc_neg, cells, mice = [], [], [], []
for i, s in zip(both, sel):
    r = recs[i]
    hc = rsc.clean_heading(r["heading"])
    segs = rsc.trial_segments(r["heading"])
    sign = np.array([np.sign(np.median(np.diff(hc[g]))) for g in segs])
    pos = np.concatenate([g for g, x in zip(segs, sign) if x > 0])
    neg = np.concatenate([g for g, x in zip(segs, sign) if x < 0])
    tc_pos.append(rsc.bin_tuning(r["sp"][pos], hc[pos])); tc_neg.append(rsc.bin_tuning(r["sp"][neg], hc[neg]))
    cells.append(s.is_hd.values); mice.append(np.repeat(sub.subject[i], len(s)))
    print(f"{info.file[i]}: {s.is_hd.sum()} of {len(s)} selective")
tc_pos, tc_neg, cells, mice = np.vstack(tc_pos), np.vstack(tc_neg), np.concatenate(cells), np.concatenate(mice)
res5 = {"selective": compare(tc_pos, tc_neg, cells, rng, "selective cells, 4 files"),
        "all active": compare(tc_pos, tc_neg, (tc_pos.sum(1) > 0) & (tc_neg.sum(1) > 0), rng, "all active cells, 4 files")}
for m in np.unique(mice):
    compare(tc_pos, tc_neg, cells & (mice == m), rng, f"selective cells, {m}")

220729 SKKS125-DualRotation-002: 74 of 399 selective
220729 SKKS126-DualRotation-002: 153 of 463 selective
220727 SKKS127-DualDir-005: 18 of 213 selective
220729 SKKS127-DualRotation-002: 162 of 423 selective

selective cells, 4 files: n=407; (a) median |pref diff| (mod 180) real 24 deg, random 30 deg, bootstrapped KS p = 0.16; (b) median correlation real 0.210, random 0.118, bootstrapped KS p = 0.002

all active cells, 4 files: n=1492; (a) median |pref diff| (mod 180) real 36 deg, random 42 deg, bootstrapped KS p = 0.512; (b) median correlation real 0.023, random 0.007, bootstrapped KS p = 0.543

selective cells, SKKS125: n=74; (a) median |pref diff| (mod 180) real 24 deg, random 36 deg, bootstrapped KS p = 0.348; (b) median correlation real 0.218, random 0.032, bootstrapped KS p = 0.044

selective cells, SKKS126: n=153; (a) median |pref diff| (mod 180) real 24 deg, random 36 deg, bootstrapped KS p = 0.45; (b) median correlation real 0.128, random 0.068, bootstrapped KS p = 0.119

selective cells, SKKS127: n=180; (a) median |pref diff| (mod 180) real 30 deg, random 30 deg, bootstrapped KS p = 0.898; (b) median correlation real 0.279, random 0.259, bootstrapped KS p = 0.699

Result: reproduced under reading (b). For the 407 cells selective in the four files that rotate both ways, the correlation of each cell's clockwise and counterclockwise tuning curves (median 0.210) exceeds that of randomly paired cells (0.118), bootstrapped KS p = 0.002. Reading (a), differences in preferred heading, gives p = 0.16 (median 24 against 30 degrees). The pass is fragile: the pooled random pairs are drawn across files, and by mouse, where random pairs come from the same mouse, (b) passes for SKKS125 only (p = 0.044; SKKS126 0.119, SKKS127 0.699). Over all active cells neither reading passes (p = 0.512 and 0.543). The files hold 3 mice where the paper reports 2, and their light condition is not marked, so the light and dark responses the claim names are not separated.

figs6-arena-control

The diversity of RSC responses persists whether or not the mouse is in control of the arena (bootstrapped KS test, p = 0.031, 7 recordings over 3 mice).

Pairs: each animal-controlled file (FR or FM) with the motor-driven file of the same mouse and day, which has the same ROIs. Cells selective in the motor-driven file. Tuning in the animal-controlled file from all frames in which the chamber turns (more than 0.5 degree per frame).

The result is also shown without any pair whose animal-controlled file visits fewer than half of the 60 heading bins (bin_tuning sets an unvisited bin to 0), and for each pair alone, where random pairs come from the same pair of files.

pairs6 = []
for i in sub.index[sub.experiment == "animal_controlled"]:
    j = sub.index[(sub.experiment != "animal_controlled") & (sub.date == sub.date[i]) & (sub.subject == sub.subject[i])
                  & (sub.n_rois == sub.n_rois[i])]
    if len(j):
        pairs6.append((i, j[0]))
print("pairs (animal-controlled, motor):", [(info.file[i], info.file[j]) for i, j in pairs6])
need = [j for _, j in pairs6]
with Pool(4) as pool:
    sel6 = rsc.selection_many([rsc.trial_tuning(recs[j]["sp"], recs[j]["heading"]) for j in need], "double", 403, pool, centre=True)
tc_a, tc_m, cells6, frames, pair_id = [], [], [], [], []
for (i, j), s in zip(pairs6, sel6):
    hi = rsc.clean_heading(recs[i]["heading"])
    moving = np.r_[False, np.abs(rsc.wrap180(np.diff(hi))) > 0.5]
    tc_a.append(rsc.bin_tuning(recs[i]["sp"][moving], hi[moving]))
    tc_m.append(rsc.trial_tuning(recs[j]["sp"], recs[j]["heading"]).mean(2))
    cells6.append(s.is_hd.values)
    pair_id.append(np.repeat(info.file[i], len(s)))
    occ = np.bincount(np.clip(((hi[moving] + 180) // 6).astype(int), 0, 59), minlength=60)
    frames.append((info.file[i], int(moving.sum()), int((occ > 0).sum())))
    print(f"{info.file[i]} / {info.file[j]}: {s.is_hd.sum()} of {len(s)} selective in the motor file; "
          f"{moving.sum()} turning frames, {int((occ > 0).sum())} of 60 heading bins visited")
tc_a, tc_m, cells6, pair_id = np.vstack(tc_a), np.vstack(tc_m), np.concatenate(cells6), np.concatenate(pair_id)
rng = np.random.default_rng(404)
res6 = {"selective": compare(tc_a, tc_m, cells6, rng, "selective cells, all pairs")}
# without the pairs whose animal-controlled file visits fewer than half of the 60 heading bins
# (bin_tuning sets an unvisited bin to 0, so their curves are mostly zeros)
sparse = [f for f, _, nb_ in frames if nb_ < 30]
res6["without sparse"] = compare(tc_a, tc_m, cells6 & ~np.isin(pair_id, sparse), rng, f"selective cells, without {sparse}")
for f in np.unique(pair_id):
    compare(tc_a, tc_m, cells6 & (pair_id == f), rng, f"selective cells, pair {f} only")
print(f"\nPaper: 7 recordings over 3 mice; here {len(pairs6)} pairs over {sub.subject[[i for i, _ in pairs6]].nunique()} mice")

pairs (animal-controlled, motor): [('220720 SKKS124-FR-001', '220720 SKKS124-CW-002'), ('220721 SKKS125-FR-003', '220721 SKKS125-CCW-002'), ('220726 SKKS125-FM-001', '220726 SKKS125-CW-002'), ('220727 SKKS126-FR-001', '220727 SKKS126-CW-002'), ('220729 SKKS127-FR-001', '220729 SKKS127-DualRotation-002')]

220720 SKKS124-FR-001 / 220720 SKKS124-CW-002: 3 of 62 selective in the motor file; 2265 turning frames, 60 of 60 heading bins visited
220721 SKKS125-FR-003 / 220721 SKKS125-CCW-002: 112 of 333 selective in the motor file; 420 turning frames, 6 of 60 heading bins visited
220726 SKKS125-FM-001 / 220726 SKKS125-CW-002: 46 of 670 selective in the motor file; 2487 turning frames, 60 of 60 heading bins visited
220727 SKKS126-FR-001 / 220727 SKKS126-CW-002: 57 of 352 selective in the motor file; 1382 turning frames, 60 of 60 heading bins visited

220729 SKKS127-FR-001 / 220729 SKKS127-DualRotation-002: 163 of 423 selective in the motor file; 530 turning frames, 52 of 60 heading bins visited

selective cells, all pairs: n=381; (a) median |pref diff| (mod 180) real 42 deg, random 48 deg, bootstrapped KS p = 0.889; (b) median correlation real 0.027, random -0.011, bootstrapped KS p = 0.026

selective cells, without ['220721 SKKS125-FR-003']: n=269; (a) median |pref diff| (mod 180) real 36 deg, random 36 deg, bootstrapped KS p = 0.973; (b) median correlation real 0.093, random 0.086, bootstrapped KS p = 0.9

selective cells, pair 220720 SKKS124-FR-001 only: n=3; (a) median |pref diff| (mod 180) real 42 deg, random 42 deg, bootstrapped KS p = 1; (b) median correlation real 0.098, random 0.192, bootstrapped KS p = 1

selective cells, pair 220721 SKKS125-FR-003 only: n=112; (a) median |pref diff| (mod 180) real 60 deg, random 66 deg, bootstrapped KS p = 0.984; (b) median correlation real -0.017, random -0.031, bootstrapped KS p = 0.88

selective cells, pair 220726 SKKS125-FM-001 only: n=46; (a) median |pref diff| (mod 180) real 39 deg, random 42 deg, bootstrapped KS p = 0.983; (b) median correlation real -0.011, random 0.007, bootstrapped KS p = 0.974

selective cells, pair 220727 SKKS126-FR-001 only: n=57; (a) median |pref diff| (mod 180) real 60 deg, random 42 deg, bootstrapped KS p = 0.994; (b) median correlation real 0.022, random 0.019, bootstrapped KS p = 0.899

selective cells, pair 220729 SKKS127-FR-001 only: n=163; (a) median |pref diff| (mod 180) real 36 deg, random 30 deg, bootstrapped KS p = 0.967; (b) median correlation real 0.182, random 0.156, bootstrapped KS p = 0.781

Paper: 7 recordings over 3 mice; here 5 pairs over 4 mice

Result: not reproduced, resting on leaving out the pair with 6 of 60 heading bins (an assumption). Over all 5 pairs, reading (b) passes: for 381 cells selective in the motor-driven file, animal-controlled and motor-driven tuning curves correlate more (median 0.027) than randomly paired cells (-0.011), bootstrapped KS p = 0.026; reading (a) gives p = 0.889. That result rests on the pair SKKS125 2022-07-21, whose animal-controlled file visits 6 of 60 heading bins, so its 112 cells have tuning curves that are 0 in the other 54 bins. Without it neither reading passes (n = 269; (a) p = 0.973, (b) p = 0.9, median correlation 0.093 against 0.086 at random), and no pair passes on its own (p 0.781 to 1). The pooled pass comes from random pairs drawn across pairs of files. The dataset holds 5 pairs over 4 mice, where the paper reports 7 recordings over 3.

fig5k-visual-cue-only (blocked)

When only the visual cues rotate (no proprioceptive or somatosensory input), the visual contributions to tuning are retained, but heading-cell responses are strongly degraded.

The claim needs neurons matched between a standard-rotation recording and a visual-cue-only one (Figure 5K legend). The dataset's two visual-cue-only ("Suspended") files are listed below with every file of the same mice, their numbers of ROIs, and their structure in blocks of 1250 frames: the heading step and the share of still frames per block, and the mean raw fluorescence. A standard rotation inside the same file would show as blocks that differ in some way other than the direction of rotation.

same_mice = cat[cat.subject.isin(cat.subject[cat.experiment == "visual_only"])]
print(same_mice[["experiment", "subject", "identifier", "n_frames", "n_rois"]].to_string(index=False))
vis = cat[cat.experiment == "visual_only"].reset_index(drop=True)
vrecs = rsc.load_many(vis.url.tolist())
for m, r in zip(vis.itertuples(), vrecs):
    hc = rsc.clean_heading(r["heading"])
    st = rsc.wrap180(np.diff(hc))
    blocks = range(0, len(st), 1250)
    print(f"\n{m.identifier}: {m.n_frames} frames, {m.n_rois} ROIs, {len(rsc.trial_segments(r['heading']))} trials")
    print("  median heading step per block (deg/frame):", [round(float(np.median(st[k:k + 1250])), 2) for k in blocks])
    print("  share of still frames per block:", [round(float(np.mean(np.abs(st[k:k + 1250]) < 0.5)), 2) for k in blocks])
    print("  mean raw fluorescence per block:", [round(float(r["F"][k:k + 1250].mean()), 0) for k in blocks])
print("\nROI counts shared between a visual-only file and another file of the same mouse:",
      sorted(set(vis.n_rois) & set(same_mice[same_mice.experiment != "visual_only"].n_rois)))

       experiment subject                                            identifier  n_frames  n_rois
animal_controlled SKKS125           220721_KKS_SKKS125_FR_CW_CCW_SKKS125-FR-003      7550     333
animal_controlled SKKS125           220726_KKS_SKKS125_FR_CW_CCW_SKKS125-FM-001      7550     670
animal_controlled SKKS127           220729_KKS_SKKS127_FR_CW_CCW_SKKS127-FR-001      7550     423
       motor_both SKKS125 220729_KKS_SKKS125_FR_CW_CCW_SKKS125-DualRotation-002     15050     399
       motor_both SKKS127      220727_KKS_SKKS127_FR_CW_CCW_SKKS127-DualDir-005      7550     213
       motor_both SKKS127 220729_KKS_SKKS127_FR_CW_CCW_SKKS127-DualRotation-002     15050     423
        motor_ccw SKKS125          220721_KKS_SKKS125_FR_CW_CCW_SKKS125-CCW-002      7550     333
         motor_cw SKKS125           220726_KKS_SKKS125_FR_CW_CCW_SKKS125-CW-002      7550     670
      visual_only SKKS125      220810_KKS_SKKS125_Suspended_SKKS125-DualDir-002     15050     446
      visual_only SKKS127      220811_KKS_SKKS127_Suspended_SKKS127-DualDir-002     15050     691

220810_KKS_SKKS125_Suspended_SKKS125-DualDir-002: 15050 frames, 446 ROIs, 75 trials
  median heading step per block (deg/frame): [2.02, -2.09, 1.99, -2.08, 2.0, -2.09, 1.89, -2.09, 2.02, -2.09, 0.88, -2.09, 0.0]
  share of still frames per block: [0.15, 0.05, 0.15, 0.05, 0.17, 0.05, 0.26, 0.05, 0.16, 0.05, 0.46, 0.05, 1.0]
  mean raw fluorescence per block: [2102.0, 2129.0, 2090.0, 2114.0, 2095.0, 2130.0, 2141.0, 2117.0, 2112.0, 2130.0, 2143.0, 2197.0, 2777.0]

220811_KKS_SKKS127_Suspended_SKKS127-DualDir-002: 15050 frames, 691 ROIs, 63 trials
  median heading step per block (deg/frame): [1.97, -2.07, 1.71, -2.07, 1.13, -2.08, 0.02, -2.07, 0.08, -2.08, 0.89, -2.07, 0.0]
  share of still frames per block: [0.2, 0.05, 0.37, 0.05, 0.45, 0.06, 0.85, 0.06, 0.62, 0.05, 0.46, 0.05, 1.0]
  mean raw fluorescence per block: [2390.0, 2335.0, 2232.0, 2265.0, 2230.0, 2230.0, 2266.0, 2279.0, 2345.0, 2243.0, 2304.0, 2296.0, 2767.0]

ROI counts shared between a visual-only file and another file of the same mouse: []

Result: blocked (metadata_insufficient). Each visual-cue-only file shares its number of ROIs with no other file of its mouse, so no standard recording holds the same ROIs, and no ROI matching is given. Inside each file the blocks alternate between the two directions of rotation at about 2 degrees per frame, and nothing in the heading or the fluorescence marks some blocks as standard rotation: the files do not say which frames, if any, are standard rotation. The last block of each is still.

Summary

Claim	Outcome
figs5-rotation-direction	reproduced under reading (b) (p = 0.002); (a) p = 0.16
figs6-arena-control	not reproduced, assumed (the 6-bin pair left out): p = 0.973 and 0.9; with it (b) p = 0.026
fig5k-visual-cue-only	blocked: no recording shares the visual-only files' ROIs

A failure to reproduce here may be a failure of this attempt, not of the paper: see the reasoning and the differences from the paper's method.

How the analysis differs from the paper's: Figures S5 and S6 were not read, so what the test compares follows the two readings of the claims file. Spikes are computed without neuropil correction, cells are selected with the centred fit, and light and dark are not separated, as the files do not mark them. For figs6, the pair whose animal-controlled file visits 6 of 60 heading bins is left out, an assumption; the result with it is shown.

What was found about the dataset: the recordings in both directions are over 3 mice (paper 2), and the animal-controlled pairs are 5 over 4 mice (paper 7 over 3). The files do not mark the light condition. The two visual-cue-only files share their ROIs with no other file, and nothing in them marks standard and visual-only rotation.