Decoding variants for Watters, Gabel, Tenenbaum & Jazayeri (2026) on DANDI 000620¶
Paper: doi:10.64898/2026.01.27.702062 (bioRxiv preprint version 1); claims file claims/000620/10.64898-2026.01.27.702062.yaml; paper code https://github.com/jazlab/multi_object_memory_2025 at commit 6e4b333b78cd4461e6373a42378535e8fd392cdb (figures/supp_fig_decoding/). Dataset DANDI 000620 version 0.261019.2000 (doi:10.48324/dandi.000620/0.261019.2000), pinned as in the main notebook, reproduce_watters_2026.ipynb, whose header gives the reasoning and the table of all claims.
The outcomes of ext3ab-decoding and ext3c-decoder-orthogonality are decided in decoding_watters_2026.ipynb, with the code's decoders. This notebook runs variants that are not the code's, asked for in verification, to show how much those results depend on two details: the scale of the inputs (the code feeds rates in Hz, as dataset.py leaves its per-unit normalisation commented out) and, for the Ring, the sigmoid output of models.py, whose range (0, 1) does not cover the Ring targets in [-0.7, 0.7]. It is kept apart so that each notebook runs within the time limit.
| claim | status |
|---|---|
| ext3ab-decoding | variant (z-scored inputs) |
| ext3c-decoder-orthogonality | variants (z-scored inputs; Ring without the output sigmoid) |
import warnings
warnings.filterwarnings("ignore")
import time
import numpy as np, pandas as pd
import wm
t_start = time.time()
assets = wm.list_assets()
print(f"Dandiset {wm.DANDISET} version {wm.VERSION}: {len(assets)} assets,", assets.kind.value_counts().to_dict())
print("Raw ecephys files (not used):", f"{assets[assets.kind == 'ecephys']['size'].sum() / 1e12:.2f} TB")
keys = sorted(set(zip(assets.subject, assets.session)))
S = [wm.load_session(assets, subj, sess) for subj, sess in keys]
print(f"Loaded {len(S)} sessions in {time.time() - t_start:.0f} s")
Dandiset 000620 version 0.261019.2000: 142 assets, {'behavior+task': 48, 'ecephys': 47, 'spikesorting': 47}
Raw ecephys files (not used): 5.47 TB
Loaded 48 sessions in 265 s
import numpy as np, pandas as pd, scipy.stats as st
import matplotlib
import matplotlib.pyplot as plt
pd.set_option("display.width", 200); pd.set_option("display.max_columns", 30)
MONKEY = {"Perle": "P", "Elgar": "E"}
for x in S:
x["trials"]["subject"] = x["subject"]; x["trials"]["session"] = x["session"]
x["trials"]["task"] = x["task"]; x["trials"]["trial"] = np.arange(len(x["trials"]))
# Physiology sessions of each dataset: sessions with spike sorting. Ring sessions are those
# with 1- and 2-object trials (Methods, Behavioral Task); Perle 2022-06-06 has 2-object
# trials only and is kept apart (see the data check).
tri = [x for x in S if x["task"] == "triangle" and x["units"] is not None]
ring = [x for x in S if x["task"] == "ring" and x["units"] is not None
and (x["trials"].n_obj == 1).any()]
other = [x for x in S if x not in tri and x not in ring]
T = pd.concat([x["trials"] for x in S], ignore_index=True)
T["physiology"] = T.session.isin([x["session"] for x in tri + ring])
print("Triangle sessions:", {m: sum(x["subject"] == m for x in tri) for m in MONKEY})
print("Ring sessions:", {m: sum(x["subject"] == m for x in ring) for m in MONKEY})
print("Other sessions:", [(x["subject"], x["session"], x["task"], x["units"] is not None,
sorted(x["trials"].n_obj.unique())) for x in other])
Triangle sessions: {'Perle': 10, 'Elgar': 14}
Ring sessions: {'Perle': 10, 'Elgar': 12}
Other sessions: [('Perle', '2022-06-06', 'ring', True, [np.int64(2)]), ('Perle', '2022-06-08', 'ring', False, [np.int64(1), np.int64(2)])]
Data check¶
inv = pd.DataFrame([dict(monkey=MONKEY[x["subject"]], session=x["session"], task=x["task"], trials=len(x["trials"]),
completed=int(x["trials"].completed.sum()),
n1=int((x["trials"].n_obj == 1).sum()), n2=int((x["trials"].n_obj == 2).sum()), n3=int((x["trials"].n_obj == 3).sum()),
units=np.nan if x["units"] is None else len(x["units"])) for x in S])
print(inv.to_string(index=False))
r = np.concatenate([np.nanmean(x["rates"][w], 1) for x in tri + ring for w in ["stimulus", "delay"]])
print("Unit mean rates, stimulus and delay phases (Hz): min", r.min().round(3), "median", np.median(r).round(2), "max", r.max().round(1))
xy = T[["x0", "y0"]].dropna().values
print("Object positions (arena widths from centre): eccentricity", np.unique(np.hypot(*xy.T).round(4)))
print("Ring 2-object positions in decoder units (2x, 2y): range", (2 * T[T.task == "ring"][["x0", "y0", "x1", "y1"]]).min().min().round(3),
"to", (2 * T[T.task == "ring"][["x0", "y0", "x1", "y1"]]).max().max().round(3))
print("The main notebook (reproduce_watters_2026.ipynb) checks the clock and the metadata in full.")
monkey session task trials completed n1 n2 n3 units
E 2022-08-19 triangle 2847 1253 234 928 1685 323.0
E 2022-08-20 triangle 2619 1361 251 991 1377 414.0
E 2022-08-21 triangle 2928 1517 289 992 1647 229.0
E 2022-08-22 triangle 1964 1157 187 722 1055 234.0
E 2022-08-23 triangle 1816 1299 149 631 1036 217.0
E 2022-08-24 triangle 1401 849 130 501 770 283.0
E 2022-08-25 triangle 2083 1353 205 768 1110 164.0
E 2022-08-26 triangle 2132 1514 240 845 1047 141.0
E 2022-08-31 triangle 1578 1238 154 595 829 206.0
E 2022-09-01 triangle 1947 1373 189 720 1038 225.0
E 2022-09-02 triangle 1512 1193 148 574 790 151.0
E 2022-09-03 triangle 2267 1735 240 931 1096 203.0
E 2022-09-04 triangle 1954 1336 200 751 1003 290.0
E 2022-09-05 triangle 1598 1238 172 621 805 181.0
E 2022-09-06 ring 2080 1335 389 1691 0 197.0
E 2022-09-07 ring 2307 1297 437 1870 0 78.0
E 2022-09-09 ring 2186 1656 481 1705 0 127.0
E 2022-09-10 ring 1828 1448 376 1452 0 165.0
E 2022-09-11 ring 2435 1722 488 1947 0 148.0
E 2022-09-12 ring 2008 1402 428 1580 0 134.0
E 2022-09-13 ring 1922 1361 387 1535 0 71.0
E 2022-09-14 ring 2019 1350 414 1605 0 98.0
E 2022-09-15 ring 1849 1339 384 1465 0 93.0
E 2022-09-18 ring 1722 1323 315 1407 0 77.0
E 2022-09-19 ring 1598 1214 322 1276 0 114.0
E 2022-09-20 ring 1680 1242 306 1374 0 63.0
P 2022-05-26 triangle 1756 1002 241 587 928 101.0
P 2022-05-27 triangle 1629 1239 200 602 827 111.0
P 2022-05-28 triangle 1840 1412 198 711 931 137.0
P 2022-05-29 triangle 2040 1326 275 768 997 129.0
P 2022-05-30 triangle 1555 992 223 545 787 185.0
P 2022-05-31 triangle 1481 957 172 555 754 200.0
P 2022-06-01 triangle 1621 1097 209 648 764 226.0
P 2022-06-03 triangle 2091 1335 270 791 1030 198.0
P 2022-06-04 triangle 1862 1123 234 687 941 243.0
P 2022-06-05 triangle 1338 896 193 507 638 108.0
P 2022-06-06 ring 2378 1559 0 2378 0 77.0
P 2022-06-07 ring 2660 1481 538 2122 0 14.0
P 2022-06-08 ring 1517 1001 281 1236 0 NaN
P 2022-06-09 ring 2417 1416 488 1929 0 193.0
P 2022-06-10 ring 1476 1206 295 1181 0 216.0
P 2022-06-11 ring 1474 1170 280 1194 0 207.0
P 2022-06-12 ring 1724 1275 330 1394 0 231.0
P 2022-06-13 ring 1365 968 257 1108 0 232.0
P 2022-06-14 ring 1762 965 347 1415 0 182.0
P 2022-06-15 ring 2291 1516 465 1826 0 136.0
P 2022-06-16 ring 1372 968 283 1089 0 217.0
P 2022-06-19 ring 3678 1645 748 2930 0 193.0
Unit mean rates, stimulus and delay phases (Hz): min 0.009 median 6.0 max 133.9
Object positions (arena widths from centre): eccentricity [0.35]
Ring 2-object positions in decoder units (2x, 2y): range -0.7 to 0.7
The main notebook (reproduce_watters_2026.ipynb) checks the clock and the metadata in full.
Sessions and units of the decoders¶
# Units entering the models (T7): delay-phase AUC > 0.65 with p < 0.05 for position or identity,
# rate > 1 Hz. AUC on 1-object trials as in the main notebook (T6), Methods p-value (1,000
# shuffles); the sessions it ranks highest are the ones the code's decoding scripts name.
rng_sel = np.random.default_rng(620)
ID_NUM = {"a": 0, "b": 1, "c": 2}
sel = []
for x in tri + ring:
tr = x["trials"]; R = x["rates"]["delay"]
m = (tr.n_obj == 1).values & tr.completed.values
if x["task"] == "triangle":
m &= tr.on_triangle.values; pos = tr.loc0.values
else:
pos = wm.ring_sector(tr.theta0.values)
idn = tr.id0.map(ID_NUM).fillna(-1).astype(int).values
rate = np.nanmean(R, 1)
for k in np.where(rate > wm.MIN_RATE_HZ)[0]:
r = R[k]; ok = m & np.isfinite(r)
pa, pp = wm.unit_auc(r[ok], pos[ok], rng_sel, "methods")
ia, ip = wm.unit_auc(r[ok], idn[ok], rng_sel, "methods")
if (pa > 0.65 and pp < 0.05) or (ia > 0.65 and ip < 0.05):
sel.append(dict(subject=x["subject"], session=x["session"], task=x["task"], unit_index=k))
sel = pd.DataFrame(sel)
n_mod = sel.groupby(["task", "subject", "session"]).size().rename("modeled_units").reset_index()
top = {t: n_mod[n_mod.task == t].sort_values("modeled_units", ascending=False).head(3) for t in ["triangle", "ring"]}
for t in top:
print(f"{t}, three sessions with the most modeled units:"); print(top[t].to_string(index=False))
print("Code's sessions: Triangle Perle 2022-06-01, 2022-06-04, 2022-05-31 (run_1_triangle.py); Ring Perle 2022-06-09, 2022-06-10, 2022-06-11 (run_2_ring.py)")
SESS = {x["session"]: x for x in S}
UNITS = {s: g.unit_index.values for s, g in sel.groupby("session")}
triangle, three sessions with the most modeled units:
task subject session modeled_units
triangle Perle 2022-06-01 171
triangle Perle 2022-05-31 117
triangle Perle 2022-06-04 116
ring, three sessions with the most modeled units:
task subject session modeled_units
ring Perle 2022-06-10 113
ring Perle 2022-06-11 113
ring Perle 2022-06-09 89
Code's sessions: Triangle Perle 2022-06-01, 2022-06-04, 2022-05-31 (run_1_triangle.py); Ring Perle 2022-06-09, 2022-06-10, 2022-06-11 (run_2_ring.py)
ext3ab-decoding¶
Single-trial delay-phase activity carries information about object positions and identities, decoded above a shuffled-label control.
Source: Results, The Triangle Dataset, paragraph 1; Extended Data Fig. 3a and 3b. Paper: "We also found that neural activity contained information about object positions and identities ( Extended Data Fig. 3a-b )"
Criterion (claims file): As the Extended Data Fig. 3 legend and figures/supp_fig_decoding/ fix it, on the three Triangle sessions with the most units entering the models (T7; the code's analysis notebook uses Perle 2022-05-31, 2022-06-01 and 2022-06-04): a linear decoder (MSE loss to a one-hot target, L1 weight 0.003, SGD at learning rate 0.01, 10,000 steps, config_1_triangle.py) from single-trial rates of the stimulus and delay phases, ten 80/20 train/test splits, against the same with shuffled labels. Panel a decodes the location or identity of 1-object trials; panel b, on 2-object trials, the absent location or identity. The paper sets no rule for "above control"; readings (a) the neural accuracy exceeds the shuffled accuracy in all ten splits; (b) the mean neural accuracy exceeds the shuffled mean by more than two standard deviations of the shuffled splits. Reproduced when, under both readings, delay-phase position and identity decoding of panel a are above control in all three sessions; panel b is reported, with the legend's 66% mark.
Variant here: inputs z-scored per unit.
# ext3ab-decoding, variant with z-scored inputs (not the code's): as decoding_watters_2026.ipynb,
# with each unit's rates z-scored by the mean and SD of the training trials of each split, the
# test trials with the same mean and SD. Own generator.
rng_dec_z = np.random.default_rng(13)
dec = []
for kind, opt, lr in [("1", "sgd", 0.01), ("complement", "adam", 0.001)]:
for s in top["triangle"].session:
for phase in ["stimulus", "delay"]:
for feature in ["position", "identity"]:
X, Y = wm.decoding_data(SESS[s], UNITS[s], phase, kind, feature)
n = len(X); nt = int(0.2 * n)
Xs, Ys, Xt, Yt = [], [], [], []
for f in range(20):
r = np.random.default_rng(f % 10)
y = Y[np.random.default_rng(100 + f).permutation(n)] if f >= 10 else Y
perm = r.permutation(n)
a, t = X[perm[nt:]], X[perm[:nt]]
mu, sd = a.mean(0), a.std(0) + 1e-8
Xs.append((a - mu) / sd); Ys.append(y[perm[nt:]]); Xt.append((t - mu) / sd); Yt.append(y[perm[:nt]])
W, b = wm.train_linear(np.array(Xs, np.float32), np.array(Ys, np.float32), rng_dec_z, opt=opt, lr=lr)
for f in range(20):
acc = float(np.mean(np.argmax(Xt[f] @ W[f] + b[f], 1) == np.argmax(Yt[f], 1)))
tr_acc = float(np.mean(np.argmax(Xs[f] @ W[f] + b[f], 1) == np.argmax(Ys[f], 1)))
dec.append(dict(panel="a" if kind == "1" else "b", session=s, phase=phase, feature=feature,
labels="shuffled" if f >= 10 else "neural", split=f % 10, accuracy=100 * acc,
train_accuracy=100 * tr_acc, n_trials=n, n_units=X.shape[1]))
dec = pd.DataFrame(dec)
print("Training accuracy (%), mean over splits:")
print(dec.groupby(["panel", "phase", "feature", "session", "labels"]).train_accuracy.mean().unstack("labels").round(1).to_string())
judge = []
for (p, s, ph, ft), g in dec.groupby(["panel", "session", "phase", "feature"]):
nn = g[g.labels == "neural"].sort_values("split").accuracy.values
sh = g[g.labels == "shuffled"].sort_values("split").accuracy.values
judge.append(dict(panel=p, session=s, phase=ph, feature=ft, neural_mean=nn.mean(), neural_sd=nn.std(ddof=1),
shuffled_mean=sh.mean(), shuffled_sd=sh.std(ddof=1),
a_all_splits=bool((nn > sh).all()), b_two_sd=bool(nn.mean() > sh.mean() + 2 * sh.std(ddof=1))))
judge = pd.DataFrame(judge)
print("\nTest accuracy (%), z-scored inputs:"); print(judge.round(1).to_string(index=False))
ja = judge[(judge.panel == "a") & (judge.phase == "delay")]
print("\nPanel a, delay phase, position and identity above control in all three sessions: reading (a)",
bool(ja.a_all_splits.all()), "; reading (b)", bool(ja.b_two_sd.all()))
for ft in ["position", "identity"]:
jf = ja[ja.feature == ft]
print(f" {ft}: reading (a) in {int(jf.a_all_splits.sum())} of 3 sessions, reading (b) in {int(jf.b_two_sd.sum())} of 3")
jb = judge[judge.panel == "b"]
print("Panel b, mean neural accuracy above 66%:")
print(jb[["session", "phase", "feature", "neural_mean"]].assign(over_66=jb.neural_mean > 66).round(1).to_string(index=False))
print("All accuracies lie in [0, 100]:", bool(dec[["accuracy", "train_accuracy"]].stack().between(0, 100).all()))
Training accuracy (%), mean over splits:
labels neural shuffled
panel phase feature session
a delay identity 2022-05-31 90.7 86.4
2022-06-01 95.9 89.1
2022-06-04 58.8 58.9
position 2022-05-31 100.0 88.7
2022-06-01 100.0 90.1
2022-06-04 96.2 60.1
stimulus identity 2022-05-31 90.1 87.7
2022-06-01 92.2 87.6
2022-06-04 62.3 57.6
position 2022-05-31 100.0 88.2
2022-06-01 100.0 90.1
2022-06-04 83.8 55.4
b delay identity 2022-05-31 63.9 58.6
2022-06-01 62.1 58.8
2022-06-04 42.8 45.9
position 2022-05-31 99.8 61.0
2022-06-01 99.7 59.4
2022-06-04 80.9 44.6
stimulus identity 2022-05-31 65.3 60.8
2022-06-01 66.1 58.2
2022-06-04 47.4 44.6
position 2022-05-31 100.0 60.7
2022-06-01 99.9 60.1
2022-06-04 82.3 44.1
Test accuracy (%), z-scored inputs:
panel session phase feature neural_mean neural_sd shuffled_mean shuffled_sd a_all_splits b_two_sd
a 2022-05-31 delay identity 48.1 5.2 31.4 10.8 True False
a 2022-05-31 delay position 96.2 3.8 34.8 12.1 True True
a 2022-05-31 stimulus identity 38.6 9.9 30.0 8.4 False False
a 2022-05-31 stimulus position 95.7 3.5 40.5 11.3 True True
a 2022-06-01 delay identity 50.8 10.2 35.0 4.9 False True
a 2022-06-01 delay position 99.2 1.8 35.0 10.8 True True
a 2022-06-01 stimulus identity 46.3 9.9 33.7 8.9 False False
a 2022-06-01 stimulus position 99.2 1.8 30.8 8.1 True True
a 2022-06-04 delay identity 32.9 8.9 30.8 7.4 False False
a 2022-06-04 delay position 89.2 6.6 35.0 9.3 True True
a 2022-06-04 stimulus identity 49.2 10.0 31.2 8.8 False True
a 2022-06-04 stimulus position 71.7 5.5 29.6 8.9 True True
b 2022-05-31 delay identity 36.5 5.6 34.8 5.4 False False
b 2022-05-31 delay position 98.6 1.6 34.3 3.5 True True
b 2022-05-31 stimulus identity 40.3 7.1 37.9 5.6 False False
b 2022-05-31 stimulus position 99.2 0.8 34.8 8.6 True True
b 2022-06-01 delay identity 35.5 6.3 32.7 5.1 False False
b 2022-06-01 delay position 99.5 0.7 35.3 6.0 True True
b 2022-06-01 stimulus identity 43.5 5.1 33.1 3.4 False True
b 2022-06-01 stimulus position 99.5 0.9 34.4 5.0 True True
b 2022-06-04 delay identity 32.6 4.4 32.3 3.1 False False
b 2022-06-04 delay position 79.2 4.7 31.9 4.6 True True
b 2022-06-04 stimulus identity 35.0 4.1 31.5 3.0 False False
b 2022-06-04 stimulus position 76.2 3.4 34.5 3.2 True True
Panel a, delay phase, position and identity above control in all three sessions: reading (a) False ; reading (b) False
position: reading (a) in 3 of 3 sessions, reading (b) in 3 of 3
identity: reading (a) in 1 of 3 sessions, reading (b) in 1 of 3
Panel b, mean neural accuracy above 66%:
session phase feature neural_mean over_66
2022-05-31 delay identity 36.5 False
2022-05-31 delay position 98.6 True
2022-05-31 stimulus identity 40.3 False
2022-05-31 stimulus position 99.2 True
2022-06-01 delay identity 35.5 False
2022-06-01 delay position 99.5 True
2022-06-01 stimulus identity 43.5 False
2022-06-01 stimulus position 99.5 True
2022-06-04 delay identity 32.6 False
2022-06-04 delay position 79.2 True
2022-06-04 stimulus identity 35.0 False
2022-06-04 stimulus position 76.2 True
All accuracies lie in [0, 100]: True
Result: the outcome of the code's decoder stands with z-scored inputs. Delay-phase position decoding is above control in all three sessions under both readings (89.2% to 99.2% against 34.8% to 35.0%). Delay-phase identity decoding is 48.1% (2022-05-31), 50.8% (2022-06-01) and 32.9% (2022-06-04) against 31.4%, 35.0% and 30.8% shuffled, above control in all ten splits only in 2022-05-31 and by two standard deviations only in 2022-06-01, so in 1 of 3 sessions under each reading, where with the code's rates in Hz it passes in 0 of 3. In panel b the absent location is above 66% everywhere (76.2% to 99.5%) and the absent identity nowhere (32.6% to 43.5%). Input scale does not explain the identity result: identity fails the criterion either way.
ext3c-decoder-orthogonality¶
The coding axes of a two-object decoder come out nearly orthogonal even for synthetic data with no slot structure, so decoder orthogonality does not diagnose slots.
Source: Methods, Decoding Analyses, paragraphs 1 and 4; Extended Data Fig. 3c. Paper: "However, we found that this orthogonality resulted even on synthetic data that was specifically not Slot-like. In other words, orthogonality of these decoding subspaces is not diagnostic of object slots."
Criterion (claims file): The 2-object decoder of Methods, Decoding Analyses (two linear decoders and a per-trial assignment sigma pressed to be binary; figures/supp_fig_decoding/modeling/models.py, LinearModel2Objects) fitted on 2-object trials of the three Triangle sessions of ext3ab-decoding and of the three Ring sessions with the most units entering the models (the Extended Data Fig. 3 legend picks three sessions from each dataset; the code's run_2_ring.py uses Perle 2022-06-09, 2022-06-10 and 2022-06-11), and on the Receptive Field and Orthogonal synthetic datasets built from them as Methods describe; angles between corresponding columns of W0 and W1 for each location, over ten bootstraps, stimulus and delay phases. The paper gives no threshold for "orthogonal". Reproduced when the angles for the neural data and for the Receptive Field data are both centred near 90 degrees, read as (a) a median between 75 and 105 degrees and (b) a median within 10 degrees of the Orthogonal dataset's median; both cut-offs are the agent's. Not reproduced when neither reading holds; methods_underspecified when they differ.
Variants here: inputs z-scored per unit; for the Ring, decoders with a linear output.
# ext3c-decoder-orthogonality, variants (not the code's): "z/sigmoid", each unit's rates
# (neural or synthetic) z-scored over the training trials; "raw/linear" (Ring only), the
# decoders without the sigmoid that models.py applies, since the Ring targets 2x - 1 and
# 2y - 1 (labelers.py) lie in [-0.7, 0.7] and a sigmoid cannot reach the negative ones.
# Otherwise as decoding_watters_2026.ipynb. Own generator per variant.
RNG_V = {"z/sigmoid": np.random.default_rng(14), "raw/linear": np.random.default_rng(15)}
ang = []
for task in ["triangle", "ring"]:
for s in top[task].session:
for phase in ["stimulus", "delay"]:
X0, Y = wm.decoding_data(SESS[s], UNITS[s], phase, "2obj", "position")
n = len(X0); nt = int(0.2 * n)
for mode in ["neural", "receptive_fields", "orthogonal"]:
Xs, Ys = [], []
for seed in range(10):
r = np.random.default_rng(seed)
X = X0 if mode == "neural" else wm.synthetic_2obj(X0, Y, task, mode, np.random.default_rng(1000 + seed))
perm = r.permutation(n)
Xs.append(X[perm[nt:]]); Ys.append(Y[perm[nt:]])
Xs, Ys = np.array(Xs, np.float32), np.array(Ys, np.float32)
for variant, rng_v in RNG_V.items():
if variant == "raw/linear" and task == "triangle":
continue
if variant == "z/sigmoid":
W0, W1 = wm.train_linear_2obj((Xs - Xs.mean(1, keepdims=True)) / (Xs.std(1, keepdims=True) + 1e-8), Ys, rng_v)
else:
W0, W1 = wm.train_linear_2obj(Xs, Ys, rng_v, output="linear")
a = wm.angles_deg(W0, W1)
for seed in range(10):
for i, v in enumerate(a[seed]):
ang.append(dict(variant=variant, task=task, session=s, phase=phase, mode=mode, seed=seed,
feature=i, angle=v))
ang = pd.DataFrame(ang)
def judge3c(med):
rows = []
for (t, ph), g in med.iterrows():
for mode in ["neural", "receptive_fields"]:
rows.append(dict(task=t, phase=ph, data=mode, median=g[mode], orthogonal=g["orthogonal"],
a_75_105=bool(75 <= g[mode] <= 105), b_within_10_of_orth=bool(abs(g[mode] - g["orthogonal"]) <= 10)))
return pd.DataFrame(rows)
for v in RNG_V:
g = ang[ang.variant == v]
r = judge3c(g.groupby(["task", "phase", "mode"]).angle.median().unstack("mode"))
print(f"\nVariant {v} (inputs/output): medians (degrees) over sessions, seeds and columns, and readings:")
print(r.round(1).to_string(index=False))
for (t, d), rr in r.groupby(["task", "data"]):
print(f" {t}, {d}: reading (a) {bool(rr.a_75_105.all())}, reading (b) {bool(rr.b_within_10_of_orth.all())}")
print("Median per session:")
print(g.groupby(["task", "phase", "session", "mode"]).angle.median().unstack("mode").round(1).to_string())
print("Range of angles:", g.groupby(["task", "mode"]).angle.agg(["min", "max"]).round(1).to_dict("index"))
Variant z/sigmoid (inputs/output): medians (degrees) over sessions, seeds and columns, and readings:
task phase data median orthogonal a_75_105 b_within_10_of_orth
ring delay neural 99.099998 90.0 True True
ring delay receptive_fields 102.000000 90.0 True False
ring stimulus neural 101.599998 90.0 True False
ring stimulus receptive_fields 102.199997 90.0 True False
triangle delay neural 96.099998 90.0 True True
triangle delay receptive_fields 90.000000 90.0 True True
triangle stimulus neural 98.500000 90.0 True True
triangle stimulus receptive_fields 90.000000 90.0 True True
ring, neural: reading (a) True, reading (b) False
ring, receptive_fields: reading (a) True, reading (b) False
triangle, neural: reading (a) True, reading (b) True
triangle, receptive_fields: reading (a) True, reading (b) True
Median per session:
mode neural orthogonal receptive_fields
task phase session
ring delay 2022-06-09 108.000000 90.0 95.400002
2022-06-10 99.099998 90.0 106.699997
2022-06-11 95.099998 90.0 102.000000
stimulus 2022-06-09 115.599998 90.0 97.800003
2022-06-10 101.400002 90.0 109.199997
2022-06-11 95.300003 90.0 100.300003
triangle delay 2022-05-31 92.800003 90.0 90.000000
2022-06-01 93.199997 90.0 90.000000
2022-06-04 110.800003 90.0 90.000000
stimulus 2022-05-31 95.599998 90.0 90.000000
2022-06-01 94.699997 90.0 90.000000
2022-06-04 112.900002 90.0 90.000000
Range of angles: {('ring', 'neural'): {'min': 82.5999984741211, 'max': 137.6999969482422}, ('ring', 'orthogonal'): {'min': 90.0, 'max': 111.0999984741211}, ('ring', 'receptive_fields'): {'min': 85.9000015258789, 'max': 140.10000610351562}, ('triangle', 'neural'): {'min': 65.5999984741211, 'max': 142.0}, ('triangle', 'orthogonal'): {'min': 76.0, 'max': 110.69999694824219}, ('triangle', 'receptive_fields'): {'min': 61.5, 'max': 120.5999984741211}}
Variant raw/linear (inputs/output): medians (degrees) over sessions, seeds and columns, and readings:
task phase data median orthogonal a_75_105 b_within_10_of_orth
ring delay neural 140.199997 104.800003 False False
ring delay receptive_fields 103.300003 104.800003 True True
ring stimulus neural 142.100006 105.800003 False False
ring stimulus receptive_fields 103.000000 105.800003 True True
ring, neural: reading (a) False, reading (b) False
ring, receptive_fields: reading (a) True, reading (b) True
Median per session:
mode neural orthogonal receptive_fields
task phase session
ring delay 2022-06-09 139.500000 102.300003 97.699997
2022-06-10 140.800003 111.900002 104.500000
2022-06-11 142.500000 104.300003 103.800003
stimulus 2022-06-09 143.000000 104.300003 102.699997
2022-06-10 139.300003 110.599998 104.900002
2022-06-11 145.100006 104.599998 101.800003
Range of angles: {('ring', 'neural'): {'min': 90.0999984741211, 'max': 159.39999389648438}, ('ring', 'orthogonal'): {'min': 82.30000305175781, 'max': 134.89999389648438}, ('ring', 'receptive_fields'): {'min': 77.30000305175781, 'max': 129.89999389648438}}
Result: the angles depend on the decoder's details. With z-scored inputs the Orthogonal data give medians of 90.0 degrees everywhere; the Triangle neural data move to 96.1 (delay) and 98.5 degrees (stimulus), and pass both readings, as do the Triangle Receptive Field data (90.0); the Ring neural data (99.1, 101.6) and Receptive Field data (102.0, 102.2) pass reading (a) and fail reading (b) except the Ring neural delay phase. Without the output sigmoid, the Ring neural decoders are far from orthogonal (medians 140.2 and 142.1 degrees) and fail both readings, while the Receptive Field data (103.3, 103.0) stay within 10 degrees of the Orthogonal data (104.8, 105.8). Under neither variant do both readings hold for the neural and Receptive Field data of both datasets. These variants are not the code's and do not change the outcome decided in decoding_watters_2026.ipynb, but the Ring pass there rests on the code's sigmoid, so that outcome is marked assumed.
print(f"Notebook ran in {(time.time() - t_start) / 60:.1f} min")
Notebook ran in 21.4 min
Summary¶
| claim | variant | result |
|---|---|---|
| ext3ab-decoding | z-scored inputs | identity above control in 1 of 3 sessions under each reading, against 0 of 3 with the code's inputs |
| ext3c-decoder-orthogonality | z-scored inputs | Triangle neural passes both readings; Ring neural and Receptive Field data fail reading (b) in 3 of 4 cases |
| ext3c-decoder-orthogonality | Ring, no output sigmoid | Ring neural medians 140 to 142 degrees, far from orthogonal |
The outcomes are decided in decoding_watters_2026.ipynb with the code's decoders: ext3ab-decoding and ext3c-decoder-orthogonality are partially reproduced. A failure to reproduce here may be a failure of this attempt, not of the paper: see the reasoning and the differences from the paper's method.
How the analysis differs from the paper's. As in decoding_watters_2026.ipynb, with the two changes this notebook tests: inputs z-scored per unit over the training trials, and for the Ring decoders a linear output in place of the sigmoid of models.py. Neither is the paper's or the code's.
What was found about the dataset. Nothing beyond the main notebook's summary.