Reproducing Dehaqani, Michelon et al. (2024) from dandiset 000931¶
Paper. Dehaqani AA, Michelon F, Patella P, Petrucco L, Piasini E, Iurilli G (2024). A mechanosensory feedback that uncouples external and self-generated sensory responses in the olfactory cortex. Cell Reports 43, 114013. doi:10.1016/j.celrep.2024.114013.
Text. The claims file (claims/000931/10.1016-j.celrep.2024.114013.yaml) was written from the
version of record (main text, figure legends, STAR Methods and the legends of Figures S1 to S6),
and checked by a second reader. This attempt read the same text.
Dandiset. DANDI:000931, version 0.261011.1559 (doi:10.48324/dandi.000931/0.261011.1559). The paper cites no version of the dandiset; its data statement says processed data "will be shared by the lead contact upon request". The protocol then pins the most recent published version. All three published versions and the draft hold the same single asset (same asset id and SHA-256, printed below); only the most recent published version and the draft name the paper in their metadata. So the choice of version does not change the data.
What the dandiset holds. One NWB file, sub-001/sub-001.nwb: the spike times of 1013 sorted
units from one mouse, recorded on 2021-10-27, with the keyword "Concentration Series". The paper
recorded 22 mice in six groups (STAR Methods, Mice: 4 + 2 + 4 + 3 + 4 + 5). The file has no breathing signal or inhalation times, no odor
trials, no artificial airflow pulses, no unit waveforms and no brain areas. Every claim needs at
least one of these (printed below, per claim), so no claim can be tested.
Claims.
| Claim | Outcome |
|---|---|
| All 37 claims of the claims file (inhalation clustering, phase and speed decoding, odor responses and their dependence on inhalation speed, GLM and linear models, airflow tuning in anesthetized, awake, neurectomized and bulbectomized mice, motor cortex, population decoding of concentration, the simulation, and the waveform control) | Blocked: data missing |
This notebook lists what the dandiset and its file hold, checks the unit table against its description and the paper's spike-sorting criteria, and shows, per claim, which data it needs and which are missing.
import time
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import mechano
pd.set_option("display.width", 200)
pd.set_option("display.max_columns", 40)
pd.set_option("display.max_rows", 200)
pd.set_option("display.max_colwidth", 90)
started = time.time()
archive = mechano.versions()
f = mechano.open_file()
units, trains = mechano.load_units(f)
print(f"dandiset {mechano.DANDISET_ID}, pinned version {mechano.DANDISET_VERSION}; "
f"loaded {len(units)} units and {units['count'].sum()} spikes in {time.time() - started:.0f} s")
dandiset 000931, pinned version 0.261011.1559; loaded 1013 units and 22086262 spikes in 4 s
Data check¶
Each check of the metadata against the values is stated under its output.
print("Every version of the dandiset and its assets (SHA-256 truncated to 16 characters):")
print(archive.to_string(index=False))
print("\nDistinct assets over all versions:", archive.asset_id.nunique(),
"; distinct SHA-256:", archive.sha256.nunique())
pinned = archive[archive.version == mechano.DANDISET_VERSION]
print("Pinned version:", mechano.DANDISET_VERSION, "; files:", len(pinned),
"; subjects:", int(pinned.subjects.iloc[0]), "; variables measured:", pinned.variables.iloc[0])
Every version of the dandiset and its assets (SHA-256 truncated to 16 characters):
version status path size asset_id sha256 related_papers variables subjects
draft Published sub-001/sub-001.nwb 177095408 46205384-bce2-42be-adad-db216a2a6972 1419b98184bf7486 doi:10.1016/j.celrep.2024.114013 Units 1
0.240305.0939 Valid sub-001/sub-001.nwb 177095408 46205384-bce2-42be-adad-db216a2a6972 1419b98184bf7486 Units 1
0.240305.0945 Valid sub-001/sub-001.nwb 177095408 46205384-bce2-42be-adad-db216a2a6972 1419b98184bf7486 Units 1
0.261011.1559 Valid sub-001/sub-001.nwb 177095408 46205384-bce2-42be-adad-db216a2a6972 1419b98184bf7486 doi:10.1016/j.celrep.2024.114013 Units 1
Distinct assets over all versions: 1 ; distinct SHA-256: 1
Pinned version: 0.261011.1559 ; files: 1 ; subjects: 1 ; variables measured: Units
The archive lists one file and one subject in every version. The three published versions (0.240305.0939, 0.240305.0945, 0.261011.1559) and the draft share the asset, so they hold the same data. The two 2024 versions do not name the paper among their related resources; the pinned one does.
print("Everything the file holds, outside /specifications:")
print(mechano.contents(f).to_string(index=False))
print("\nGeneral metadata:")
for key, value in mechano.general(f).items():
print(f" {key}: {value}")
for group in ["acquisition", "analysis", "processing", "stimulus/presentation", "stimulus/templates"]:
print(f"/{group} holds {len(f[group])} objects")
print("/intervals present:", "intervals" in f, "; electrodes table present:",
"general/extracellular_ephys" in f)
print("\nColumns of the units table (units.attrs['colnames']):", mechano._text(f["units"].attrs["colnames"]))
print("\nDatasets anywhere outside /units, /general, /specifications and the file's own metadata:",
len(mechano.data_outside_units(f)))
print(mechano.data_outside_units(f).to_string(index=False))
print("\nNamespaces cached under /specifications, and the data types each defines:")
print(mechano.specifications(f).to_string(index=False))
Everything the file holds, outside /specifications:
path kind neurodata_type shape dtype description
/acquisition group
/analysis group
/file_create_date dataset (1,) object
/general group
/general/experimenter dataset (1,) object
/general/institution dataset () object
/general/keywords dataset (1,) object
/general/lab dataset () object
/general/related_publications dataset (1,) object
/general/session_id dataset () object
/general/subject group Subject
/general/subject/age dataset () object
/general/subject/sex dataset () object
/general/subject/species dataset () object
/general/subject/subject_id dataset () object
/identifier dataset () object
/processing group
/session_description dataset () object
/session_start_time dataset () object
/stimulus group
/stimulus/presentation group
/stimulus/templates group
/timestamps_reference_time dataset () object
/units group Units Autogenerated by neuroconv.
/units/Amplitude dataset VectorData (1013,) float64 No description.
/units/ContamPct dataset VectorData (1013,) float64 No description.
/units/KSLabel dataset VectorData (1013,) object No description.
/units/amp dataset VectorData (1013,) float64 No description.
/units/ch dataset VectorData (1013,) float64 No description.
/units/depth dataset VectorData (1013,) float64 No description.
/units/fr dataset VectorData (1013,) float64 No description.
/units/id dataset ElementIdentifiers (1013,) int32
/units/n_spikes dataset VectorData (1013,) float64 No description.
/units/original_cluster_id dataset VectorData (1013,) float64 No description.
/units/quality dataset VectorData (1013,) object Quality of the unit as defined by phy (good, mua, noise).
/units/sh dataset VectorData (1013,) float64 No description.
/units/spike_times dataset VectorData (22086262,) float64 the spike times for each unit
/units/spike_times_index dataset VectorIndex (1013,) uint32 Index for VectorData 'spike_times'
/units/unit_name dataset VectorData (1013,) object Unique reference for each unit.
General metadata:
session_description: Auto-generated by neuroconv
session_start_time: 2021-10-27T15:30:00+02:00
identifier: f6c23c74-efd0-4bb2-a3b5-77691700a893
experimenter: ['Michelon, Filippo']
institution: Istituto Italiano di Tecnologia
keywords: ['Concentration Series']
lab: Systems Neurobiology Lab
related_publications: ['https://doi.org/10.1101/2023.04.04.535405']
session_id: 1
subject/age: P56D
subject/sex: M
subject/species: Mus musculus
subject/subject_id: 001
/acquisition holds 0 objects
/analysis holds 0 objects
/processing holds 0 objects
/stimulus/presentation holds 0 objects
/stimulus/templates holds 0 objects
/intervals present: False ; electrodes table present: False
Columns of the units table (units.attrs['colnames']): ['spike_times', 'unit_name', 'KSLabel', 'sh', 'Amplitude', 'quality', 'n_spikes', 'ContamPct', 'fr', 'amp', 'original_cluster_id', 'depth', 'ch']
Datasets anywhere outside /units, /general, /specifications and the file's own metadata: 0
Empty DataFrame
Columns: [path, shape, dtype]
Index: []
Namespaces cached under /specifications, and the data types each defines:
namespace version file types_defined types
core 2.6.0-alpha namespace 0
core 2.6.0-alpha nwb.base 9 NWBContainer, NWBDataInterface, TimeSeries, ProcessingModule, Images, NWBData, TimeSeriesReferenceVectorData, Image, ImageReferences
core 2.6.0-alpha nwb.behavior 8 SpatialSeries, BehavioralEpochs, BehavioralEvents, BehavioralTimeSeries, PupilTracking, EyeTracking, CompassDirection, Position
core 2.6.0-alpha nwb.device 1 Device
core 2.6.0-alpha nwb.ecephys 10 ElectricalSeries, SpikeEventSeries, FeatureExtraction, EventDetection, EventWaveform, FilteredEphys, LFP, ElectrodeGroup, ClusterWaveforms, Clustering
core 2.6.0-alpha nwb.epoch 1 TimeIntervals
core 2.6.0-alpha nwb.file 4 NWBFile, LabMetaData, Subject, ScratchData
core 2.6.0-alpha nwb.icephys 16 PatchClampSeries, CurrentClampSeries, IZeroClampSeries, CurrentClampStimulusSeries, VoltageClampSeries, VoltageClampStimulusSeries, IntracellularElectrode, SweepTable, IntracellularElectrodesTable, In
core 2.6.0-alpha nwb.image 7 ImageSeries, ImageMaskSeries, OpticalSeries, IndexSeries, GrayscaleImage, RGBImage, RGBAImage
core 2.6.0-alpha nwb.misc 5 AbstractFeatureSeries, AnnotationSeries, IntervalSeries, DecompositionSeries, Units
core 2.6.0-alpha nwb.ogen 2 OptogeneticSeries, OptogeneticStimulusSite
core 2.6.0-alpha nwb.ophys 11 OnePhotonSeries, TwoPhotonSeries, RoiResponseSeries, DfOverF, Fluorescence, ImageSegmentation, PlaneSegmentation, ImagingPlane, OpticalChannel, MotionCorrection, CorrectedImageStack
core 2.6.0-alpha nwb.retinotopy 1 ImagingRetinotopy
hdmf-common 1.8.0 base 3 Container, SimpleMultiContainer, Data
hdmf-common 1.8.0 namespace 0
hdmf-common 1.8.0 sparse 1 CSRMatrix
hdmf-common 1.8.0 table 6 DynamicTable, AlignedDynamicTable, VectorData, VectorIndex, ElementIdentifiers, DynamicTableRegion
hdmf-experimental 0.5.0 experimental 1 EnumData
hdmf-experimental 0.5.0 namespace 0
hdmf-experimental 0.5.0 resources 1 HERD
ndx-dandi-icephys 0.3.0 namespace 0
ndx-dandi-icephys 0.3.0 ndx-dandi-icephys.extensions 2 DandiIcephysMetadata, CreSubject
The file holds only general metadata and the Units table. acquisition, analysis,
processing, stimulus/presentation and stimulus/templates are empty; there is no intervals
group (no trials or epochs) and no electrodes table. The units table has the 13 columns printed,
none of them a waveform, a location or an event time. No dataset is stored anywhere else in the
file, so no inhalation times, odor trials or pulses are held in a column or table outside the
groups mechano.streams_present looks at. Besides the NWB core and hdmf namespaces,
/specifications caches one extension, ndx-dandi-icephys, which defines only two metadata types
(DandiIcephysMetadata, CreSubject) and is not used by any object in the file. The Methods describe four signals recorded
besides the spikes, all at 30 kHz: the nasal airflow from a flow sensor, the odor signal from a
photo-ionization detector, a TTL at each odor trial and a TTL at each detected inhalation onset
(STAR Methods, Experimental signals). None is in the file.
The keyword "Concentration Series" places this mouse in the group of four mice given alpha-pinene
and eucalyptol at three concentrations (STAR Methods, Mice), as far as a keyword can; the file does
not say which odors were given, and the subject age (P56D) is within the paper's 6 to 9 weeks. The
file's related_publications names the bioRxiv preprint (10.1101/2023.04.04.535405), not the
version of record. The session description is the converter's default ("Auto-generated by
neuroconv").
import pathlib
import yaml
claims_path = pathlib.Path("../../claims/000931/10.1016-j.celrep.2024.114013.yaml")
if claims_path.exists():
with open(claims_path) as fh:
claim_ids = [c["id"] for c in yaml.safe_load(fh)["claims"]]
print("Claims in the claims file:", len(claim_ids), "; every one listed in mechano.CLAIM_NEEDS:",
sorted(claim_ids) == sorted(mechano.CLAIM_NEEDS))
else:
print("Claims file not found at", claims_path, "; the ids below can be compared with it by hand.")
print("Claim ids in mechano.CLAIM_NEEDS:", len(mechano.CLAIM_NEEDS))
print(", ".join(mechano.CLAIM_NEEDS))
n_subjects = int(pinned.subjects.iloc[0])
present = mechano.streams_present(f, n_subjects)
print("Kinds of data the claims use, and whether the file holds them:")
for kind, (held, looked) in present.items():
print(f" {kind:15s} {'held' if held else 'NOT HELD':9s} looked for: {looked}")
print(f" {'':15s} {'':9s} needed: {mechano.STREAMS[kind]}")
rows = []
for claim, needs in mechano.CLAIM_NEEDS.items():
missing = [k for k in needs if not present[k][0]]
rows.append({"claim": claim, "needs": ", ".join(needs), "missing": ", ".join(missing),
"testable": not missing})
needs_table = pd.DataFrame(rows)
print("\nPer claim:")
print(needs_table.to_string(index=False))
print("\nClaims:", len(needs_table), "; testable on this file:", int(needs_table.testable.sum()))
print("Claims missing the breathing signal or inhalation times:",
int(needs_table.missing.str.contains("breathing").sum()))
print("Claims missing odor trial times:", int(needs_table.missing.str.contains("odor_trials").sum()))
print("Claims missing artificial airflow pulses:", int(needs_table.missing.str.contains("airflow_pulses").sum()))
print("Claims missing unit waveforms:", int(needs_table.missing.str.contains("waveforms").sum()))
Claims in the claims file:
37 ; every one listed in mechano.CLAIM_NEEDS: True
Claim ids in mechano.CLAIM_NEEDS: 37
inhalation-sorting, figs1a-phase-decoding, figs1b-speed-decoding-single-mice, fig1-odor-responsive-fraction, fig1hi-sniff-changes-responses, fig1h-direction-of-change, figs1c-early-70ms, figs1d-rs-fs-similar, fig1j-sniff-change-larger-than-concentration, fig1l-cdi-mixed-lower, fig2-glm-variance, fig2c-interaction-small, fig2d-mixed-sensitivities, figs2ab-rs-fs-heterogeneous, figs2c-waveform-no-prediction, figs2d-70ms-similar, fig3c-airflow-tuning-anesthetized, fig3b-latency-shortens, fig3f-airflow-tuning-awake, figs3-aen-neurectomy, figs4-bulbectomy-no-odor-no-phase, fig4-sniff-preferring-fraction, fig4e-sniff-response-smaller, fig4f-sniff-response-more-variable, fig4g-sniff-decoding-reduced, fig4d-pre-sniff-rise, figs4-motor-cortex-sniff, fig5b-pc1-speed-pc23-concentration, fig5c-encoding-angles, fig5d-sniff-invariant-decoding, figs5gh-other-windows-and-codes, fig5ef-correlated-subpopulations, fig5gh-mechanosensitive-as-good, fig5i-time-course-similar, figs5i-coefficients-similar, fig6-simulation-mechanosensory-rescues, figs6h-waveforms-homogeneous
Kinds of data the claims use, and whether the file holds them:
spikes held looked for: units/spike_times
needed: sorted spike times of single units
breathing NOT HELD looked for: any series in acquisition, processing or stimulus
needed: nasal airflow signal from the flow sensor, or inhalation onset, peak and offset times (STAR Methods, Experimental signals; Analysis of the breathing signal)
odor_trials NOT HELD looked for: any table under intervals, or a stimulus series
needed: odor onset times with the odor and concentration of each trial (Experimental signals, the odor-trial TTL; Odor delivery)
airflow_pulses NOT HELD looked for: any table under intervals, or a stimulus series
needed: times and flow rates of the artificial suction pulses in tracheostomized mice (Artificial mechanical stimulation)
waveforms NOT HELD looked for: units/waveform_mean or waveforms
needed: mean spike waveform of each unit, for trough-to-peak latency and asymmetry (Spike sorting and inclusion criteria)
regions NOT HELD looked for: electrodes table with location, or a location column in units
needed: brain area of each unit (PCx layers 2 and 3, motor cortex, or the olfactory areas of fig3_C), from the probe reconstruction (Histology)
other_groups NOT HELD looked for: subjects in the dandiset: 1
needed: recordings of the other experimental groups or mice the claim pools
Per claim:
claim needs missing testable
inhalation-sorting breathing, other_groups breathing, other_groups False
figs1a-phase-decoding spikes, breathing, regions, other_groups breathing, regions, other_groups False
figs1b-speed-decoding-single-mice spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
fig1-odor-responsive-fraction spikes, odor_trials, regions, other_groups odor_trials, regions, other_groups False
fig1hi-sniff-changes-responses spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
fig1h-direction-of-change spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
figs1c-early-70ms spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
figs1d-rs-fs-similar spikes, breathing, odor_trials, waveforms, regions, other_groups breathing, odor_trials, waveforms, regions, other_groups False
fig1j-sniff-change-larger-than-concentration spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
fig1l-cdi-mixed-lower spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
fig2-glm-variance spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
fig2c-interaction-small spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
fig2d-mixed-sensitivities spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
figs2ab-rs-fs-heterogeneous spikes, breathing, odor_trials, waveforms, regions, other_groups breathing, odor_trials, waveforms, regions, other_groups False
figs2c-waveform-no-prediction spikes, breathing, odor_trials, waveforms, regions, other_groups breathing, odor_trials, waveforms, regions, other_groups False
figs2d-70ms-similar spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
fig3c-airflow-tuning-anesthetized spikes, airflow_pulses, regions, other_groups airflow_pulses, regions, other_groups False
fig3b-latency-shortens spikes, airflow_pulses, regions, other_groups airflow_pulses, regions, other_groups False
fig3f-airflow-tuning-awake spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
figs3-aen-neurectomy spikes, breathing, regions, other_groups breathing, regions, other_groups False
figs4-bulbectomy-no-odor-no-phase spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
fig4-sniff-preferring-fraction spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
fig4e-sniff-response-smaller spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
fig4f-sniff-response-more-variable spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
fig4g-sniff-decoding-reduced spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
fig4d-pre-sniff-rise spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
figs4-motor-cortex-sniff spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
fig5b-pc1-speed-pc23-concentration spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
fig5c-encoding-angles spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
fig5d-sniff-invariant-decoding spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
figs5gh-other-windows-and-codes spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
fig5ef-correlated-subpopulations spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
fig5gh-mechanosensitive-as-good spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
fig5i-time-course-similar spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
figs5i-coefficients-similar spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
fig6-simulation-mechanosensory-rescues spikes, breathing, odor_trials, regions, other_groups breathing, odor_trials, regions, other_groups False
figs6h-waveforms-homogeneous waveforms, other_groups waveforms, other_groups False
Claims: 37 ; testable on this file: 0
Claims missing the breathing signal or inhalation times: 33
Claims missing odor trial times: 31
Claims missing artificial airflow pulses: 2
Claims missing unit waveforms: 4
Each claim's needs are read from its criterion in the claims file and the STAR Methods sections it
cites (the docstring of mechano.py says so). No claim can be tested: 33 need the breathing
signal or inhalation times, 31 need odor trial times, 2 need the artificial airflow pulses of the
tracheostomized mice, and 4 need unit waveforms. The claims on odorless inhalations in the ten
control mice (Figures 3F and 4) count as needing odor trial times: the Methods define odorless
inhalations as "those occurring within the -13 to -4 seconds window before the delivery of an
odor" (Analysis of sniff-by-sniff responses), and the ten control mice are the mice of the odor
experiments (STAR Methods, Mice: four concentration-series, two four-odor and four odor-identity
mice), so odor onsets are needed to find those inhalations. The Methods do not say whether the
neurectomized mice were given odors, so figs3-aen-neurectomy is not counted as needing them;
it is blocked by the missing breathing signal and mice either way. Beyond that, every claim pools mice of one or
more groups (the four concentration-series mice, the six of Figure 1, the ten control, four
bulbectomized, five neurectomized or three anesthetized mice), and most select PCx layer 2 and 3
units, for which the file has no brain area. The spikes of one mouse alone cannot stand in for
any of them, since the paper's every measure on spikes is aligned to an inhalation, an odor
onset or an airflow pulse. No outside source holds these data publicly: the paper says the
processed data are available on request, and its code deposit (Zenodo
10.5281/zenodo.10381293, read by the reader at commit 3bec502) loads preprocessed .mat files it
does not contain.
The check of claim ids against the claims file reads it from the registry
(../../claims/...); where that file is not beside the notebook, the cell prints the 37 ids in
mechano.CLAIM_NEEDS for a comparison by hand.
With no trial table or event times in the file, there is no second stream whose clock could be checked against the spikes (protocol, Author, step 2); that check could not be made.
duration = float(units.last_spike.max())
print("Units:", len(units), "; unit_name unique:", units.unit_name.nunique(),
"; original_cluster_id unique:", units.original_cluster_id.nunique())
print("\nKilosort label (KSLabel) against Phy curation (quality):")
print(pd.crosstab(units.quality, units.KSLabel, margins=True).to_string())
print("\nContamPct values:", units.ContamPct.value_counts().to_dict())
print("sh values:", units.sh.value_counts().to_dict())
print("\nSpike times (s): first", round(units.first_spike.min(), 4), "; last", round(duration, 4),
f"({duration / 3600:.2f} h); every train sorted:", bool(units.sorted.all()))
print("n_spikes equals the number of spike times for every unit:", bool((units.n_spikes == units["count"]).all()))
ratio = units["count"] / duration / units.fr
print("count / last spike time / fr: min", round(ratio.min(), 6), "max", round(ratio.max(), 6),
"; implied duration behind fr (s):", round(float((units["count"] / units.fr).median()), 2))
print("\nRanges:")
for col, unit in [("fr", "Hz"), ("depth", "um (assumed)"), ("ch", "channel"), ("Amplitude", "none given"),
("amp", "none given"), ("n_spikes", "spikes")]:
print(f" {col:10s} {units[col].min():10.3f} to {units[col].max():10.3f} [{unit}]")
good = units.quality == "good"
print("\nUnits curated good in Phy:", int(good.sum()),
f"; of these, at or above {mechano.MIN_RATE_HZ} Hz:", int((good & (units.fr >= mechano.MIN_RATE_HZ)).sum()),
"; lowest rate among them:", round(units.fr[good].min(), 3), "Hz")
print(f"All units at or above {mechano.MIN_RATE_HZ} Hz:", int((units.fr >= mechano.MIN_RATE_HZ).sum()))
pred = 20 + 20 * (units.ch // 2)
print("\ndepth against ch: correlation", round(float(np.corrcoef(units.ch, units.depth)[0, 1]), 5),
"; largest |depth - (20 + 20 * floor(ch / 2))|:", float((units.depth - pred).abs().max()),
"; equal for", f"{(units.depth == pred).mean():.1%}", "of units")
print("Depth of the units curated good in Phy:", float(units.depth[good].min()), "to", float(units.depth[good].max()))
print("\nPhy quality by depth (um):")
print(pd.crosstab(units.quality, pd.cut(units.depth, [0, 1000, 2000, 3000, 4000])).to_string())
print("\nPaper: 464 neurons in the four concentration-series mice (claims file, fig2c-interaction-small), "
"116 per mouse on average, from PCx layers 2 and 3.")
Units: 1013 ; unit_name unique: 1013 ; original_cluster_id unique: 1013
Kilosort label (KSLabel) against Phy curation (quality):
KSLabel good All
quality
good 359 359
mua 10 10
nan 644 644
All 1013 1013
ContamPct values: {100.0: 1013}
sh values: {0.0: 1013}
Spike times (s): first 0.0035 ; last 7676.0356 (2.13 h); every train sorted: True
n_spikes equals the number of spike times for every unit: True
count / last spike time / fr: min 1.000176 max 1.000176 ; implied duration behind fr (s): 7677.39
Ranges:
fr 0.014 to 37.187 [Hz]
depth 80.000 to 3840.000 [um (assumed)]
ch 6.000 to 382.000 [channel]
Amplitude 782.900 to 14438.300 [none given]
amp 35.148 to 167.703 [none given]
n_spikes 107.000 to 285500.000 [spikes]
Units curated good in Phy: 359 ; of these, at or above 0.5 Hz: 359 ; lowest rate among them: 0.515 Hz
All units at or above 0.5 Hz: 629
depth against ch: correlation 0.99998 ; largest |depth - (20 + 20 * floor(ch / 2))|: 20.0 ; equal for 73.2% of units
Depth of the units curated good in Phy: 80.0 to 3840.0
Phy quality by depth (um):
depth (0, 1000] (1000, 2000] (2000, 3000] (3000, 4000]
quality
good 97 127 53 82
mua 5 3 0 2
nan 95 172 298 79
Paper: 464 neurons in the four concentration-series mice (claims file, fig2c-interaction-small), 116 per mouse on average, from PCx layers 2 and 3.
Checks of the unit table against its description and the paper:
n_spikesequals the number of spike times of every unit, and every spike train is sorted.fris each unit's spike count divided by 7677.39 s, a little longer than the last spike (7676.04 s), so it is the mean rate over the whole recording, consistent with its name; the table does not describe it.KSLabelis "good" for all 1013 units andContamPctis 100 for all of them. In Kilosort a cluster is labelled good when its estimated contamination is low, so these two columns contradict each other and carry no information about the units. Both are undescribed.quality(Phy curation) is "good" for 359 units, "mua" for 10, and missing for 644. The paper analyses single units after manual curation in Phy; the 359 good units all fire at or above 0.5 Hz (lowest 0.515 Hz), so the paper's rate threshold removes none of them, while 629 units of any label pass it. Which units the paper kept is not stated in the file.shis 0 for every unit (one shank, as a Neuropixels 1.0 probe has).chruns from 6 to 382 anddepthfrom 80 to 3840, the length of the probe's recording sites in micrometres, though no unit is given.depthrises withch(correlation 0.99998) and lies within 20 of 20 + 20 floor(ch / 2), the height of channelchin the standard Neuropixels 1.0 channel map, where channel 0 is the site nearest the tip. So small depths are near the probe tip, if the probe is a Neuropixels 1.0 with its standard map; the file names no probe and has no electrodes table, so this is assumed. Good units are found along the whole probe, so they come from PCx and from the structures above it; the paper keeps only PCx layer 2 and 3 units (464 in four mice, 116 per mouse on average), and the file gives no area to select them by.Amplitudeandampare two amplitude measures with no description or unit.
good_idx = np.where(good)[0]
isi = np.array([mechano.isi_violation_rate(trains[i], duration) for i in good_idx])
presence = np.array([mechano.presence_ratio(trains[i], duration) for i in good_idx])
q = lambda x: f"median {np.median(x):.4f}, Q1 {np.percentile(x, 25):.4f}, Q3 {np.percentile(x, 75):.4f}, max {x.max():.4f}"
print(f"Phy-good units ({len(good_idx)}):")
print(" ISI violation rate (Hill et al. 2011, 1.5 ms):", q(isi))
print(" units with an ISI violation rate >= 0.5 (the paper's exclusion threshold):", int((isi >= 0.5).sum()))
print(" presence ratio (100 blocks):", q(presence))
print("Paper (STAR Methods, Spike sorting): ISI violation rate threshold < 0.5, median 0.01, Q1 0, Q3 0.04;"
" presence ratio median 1, Q1 1, Q3 1.")
Phy-good units (359): ISI violation rate (Hill et al. 2011, 1.5 ms): median 0.0066, Q1 0.0000, Q3 0.0291, max 0.3638 units with an ISI violation rate >= 0.5 (the paper's exclusion threshold): 0 presence ratio (100 blocks): median 1.0000, Q1 1.0000, Q3 1.0000, max 1.0000 Paper (STAR Methods, Spike sorting): ISI violation rate threshold < 0.5, median 0.01, Q1 0, Q3 0.04; presence ratio median 1, Q1 1, Q3 1.
The ISI violation rate of the good units (median 0.0066, Q1 0, Q3 0.029, none at or above 0.5) and their presence ratio (1 in every quartile) are consistent with the ranges the Methods report for the paper's units (median 0.01, Q1 0, Q3 0.04; presence ratio 1). The Methods do not give the formula or the refractory period, so this comparison uses a standard one (Hill et al. 2011, 1.5 ms) and is a consistency check only.
identical, pairs = mechano.duplicates(units, trains)
print("Units with a spike train identical to another unit's:", identical)
print(f"Pairs within {mechano.NEAR_CHANNELS} channels compared:", len(pairs))
print(f"Pairs where more than 10% of the smaller unit's spikes coincide within "
f"{mechano.COINCIDENCE_S * 1000:.1f} ms:", int((pairs.coincident > 0.1).sum()))
top = pairs.sort_values("coincident", ascending=False).head(5).copy()
top["quality_a"] = units.quality.loc[top.unit_a].to_numpy()
top["quality_b"] = units.quality.loc[top.unit_b].to_numpy()
print(top.round(3).to_string(index=False))
both_good = units.quality.loc[pairs.unit_a].to_numpy() == "good"
both_good &= units.quality.loc[pairs.unit_b].to_numpy() == "good"
print("Pairs of two Phy-good units:", int(both_good.sum()), "; largest coincident fraction among them:",
round(float(pairs.coincident[both_good].max()), 3))
Units with a spike train identical to another unit's: 0
Pairs within 4 channels compared: 13767
Pairs where more than 10% of the smaller unit's spikes coincide within 0.2 ms: 2
unit_a unit_b ch_a ch_b spikes_smaller coincident quality_a quality_b
672 695 249.0 251.0 136 0.559 nan nan
668 695 249.0 251.0 136 0.140 nan nan
759 764 275.0 275.0 126 0.071 nan good
499 1004 205.0 209.0 1951 0.063 nan good
669 695 247.0 251.0 136 0.059 nan nan
Pairs of two Phy-good units: 1749 ; largest coincident fraction among them: 0.026
No two units have identical spike trains. Of the pairs of units within four channels, two share more than 10% of the smaller unit's spikes within 0.2 ms; both pair unit 695 (136 spikes) with unit 672 or 668, and Phy left all three unlabelled. Among the 1749 pairs of two good units the largest shared fraction is 0.026. The file holds one recording, so no recording is deposited twice.
bands = [(0, 1000), (1000, 2000), (2000, 3000), (3000, 4000)]
fig, ax = plt.subplots(figsize=(7, 3.5))
# The peak finder on synthetic trains: a regular 3 Hz train, whose spectrum has its largest
# power at the 6 Hz harmonic, and a 3 Hz sinusoidally modulated Poisson train.
rng = np.random.default_rng(0)
grid = np.arange(0, 600, 1 / 3)
regular = [np.sort(np.clip(grid + rng.normal(0, 0.005, len(grid)), 0, 600)) for _ in range(20)]
t_ms = np.arange(0, 600, 0.001)
poisson = [t_ms[rng.random(len(t_ms)) < 0.02 * (1 + np.cos(2 * np.pi * 3 * t_ms))] for _ in range(20)]
for name, synth in [("regular 3 Hz", regular), ("Poisson 3 Hz", poisson)]:
fq, pw = mechano.rate_spectrum(synth, 600)
b = (fq >= 1) & (fq <= 15)
print(f"synthetic {name}: largest power at {fq[b][np.argmax(pw[b])]:.2f} Hz, "
f"peak after the subharmonic check {mechano.spectral_peak(fq, pw):.2f} Hz")
rows = []
for lo, hi in bands:
sel = np.where(good & (units.depth > lo) & (units.depth <= hi))[0]
freq, power = mechano.rate_spectrum([trains[i] for i in sel], duration)
base = np.median(power[(freq >= 15) & (freq <= 40)])
band = (freq >= 1) & (freq <= 15)
rows.append({"depth_um": f"{lo}-{hi}", "units": len(sel), "largest_power_hz_1_15": round(freq[band][np.argmax(power[band])], 2),
"peak_hz_1_15": round(mechano.spectral_peak(freq, power), 2),
"peak_over_15_40Hz_median": round(power[band].max() / base, 1)})
ax.semilogy(freq, power / base, label=f"{lo}-{hi} um ({len(sel)} units)")
ax.set_xlim(0, 20)
ax.set_xlabel("frequency (Hz)")
ax.set_ylabel("power / median power 15-40 Hz")
ax.legend(fontsize=8)
plt.show()
print(pd.DataFrame(rows).to_string(index=False))
shown = freq <= 20
print("Plotted range of power / baseline over 0-20 Hz is printed per band above as the peak; frequency axis 0-20 Hz,",
"x values beyond it not shown:", int((~shown).sum()), "frequencies.")
synthetic regular 3 Hz: largest power at 6.01 Hz, peak after the subharmonic check 2.98 Hz synthetic Poisson 3 Hz: largest power at 2.98 Hz, peak after the subharmonic check 2.98 Hz
depth_um units largest_power_hz_1_15 peak_hz_1_15 peak_over_15_40Hz_median 0-1000 97 2.78 2.78 9.7 1000-2000 127 1.03 1.03 8.6 2000-3000 53 2.93 1.51 2.9 3000-4000 82 1.03 1.03 10.2 Plotted range of power / baseline over 0-20 Hz is printed per band above as the peak; frequency axis 0-20 Hz, x values beyond it not shown: 615 frequencies.
The Methods say the probe was lowered to "a region with high firing rates and breathing-coupled
activity" (Stereotaxic probe insertion). Without a breathing signal this cannot be shown, but the
summed activity of the 97 good units with depth at most 1000 has its spectral peak between
1 and 15 Hz at 2.78 Hz, 9.7 times the 15 to 40 Hz baseline, a frequency within the range of
resting breathing in mice. By the channel map assumed above, these units are the ones nearest
the probe tip, where the PCx would be; the file does not confirm the probe or its map. The
largest power in a band can lie on a harmonic of the rhythm: on a regular synthetic 3 Hz train it
lies at 6.01 Hz. mechano.spectral_peak moves the peak to its subharmonic when the power there is
at least half the peak power, which gives 2.98 Hz on that train and leaves a 3 Hz Poisson train at
2.98 Hz. For the 0 to 1000 band the largest power and the checked peak agree (2.78 Hz). The bands
further from the tip peak near 1 Hz (1.03 Hz at 1000 to 2000 and 3000 to 4000) or have a weaker
peak (2000 to 3000: largest power at 2.93 Hz, checked peak 1.51 Hz, 2.9 times baseline).
This is consistent with these units being in a breathing-coupled region such as the PCx,
and does not show that they are.
Claims¶
No claim has a section of its own: none can be run on this file. The table printed under "Data
check" gives, for each of the 37 claims, the data its criterion needs and which of them are
missing; it is the output each blocked outcome in the attempt record points to (cell tag
blocked-claims).
print(f"notebook ran in {time.time() - started:.0f} s")
notebook ran in 15 s
Summary¶
| Claim | Outcome | Missing from the dandiset |
|---|---|---|
| inhalation-sorting | Blocked: data missing | breathing signal; the other awake mice |
| figs1a-phase-decoding | Blocked: data missing | breathing signal, brain areas, other mice |
| figs1b-speed-decoding-single-mice | Blocked: data missing | breathing signal, odor trials, brain areas, the other five mice of Figure 1 |
| fig1-odor-responsive-fraction | Blocked: data missing | odor trials, brain areas, the other five mice |
| fig1hi-sniff-changes-responses, fig1h-direction-of-change, figs1c-early-70ms, fig1j-sniff-change-larger-than-concentration, fig1l-cdi-mixed-lower | Blocked: data missing | breathing signal, odor trials, brain areas, other mice |
| figs1d-rs-fs-similar | Blocked: data missing | as above, and unit waveforms |
| fig2-glm-variance, fig2c-interaction-small, fig2d-mixed-sensitivities, figs2d-70ms-similar | Blocked: data missing | breathing signal, odor trials, brain areas, the other three concentration-series mice |
| figs2ab-rs-fs-heterogeneous, figs2c-waveform-no-prediction | Blocked: data missing | as above, and unit waveforms |
| fig3c-airflow-tuning-anesthetized, fig3b-latency-shortens | Blocked: data missing | airflow pulses, brain areas, the anesthetized mice |
| fig3f-airflow-tuning-awake | Blocked: data missing | breathing signal, odor trials, brain areas, the control mice |
| figs3-aen-neurectomy | Blocked: data missing | breathing signal, brain areas, the control and neurectomized mice |
| figs4-bulbectomy-no-odor-no-phase, fig4-sniff-preferring-fraction, fig4e-sniff-response-smaller, fig4f-sniff-response-more-variable, fig4g-sniff-decoding-reduced, fig4d-pre-sniff-rise, figs4-motor-cortex-sniff | Blocked: data missing | breathing signal, odor trials, brain areas, the control and bulbectomized mice |
| fig5b-pc1-speed-pc23-concentration | Blocked: data missing | breathing signal, odor trials, brain areas, the other concentration-series mice and the odor-identity mice |
| fig5c-encoding-angles, fig5d-sniff-invariant-decoding, figs5gh-other-windows-and-codes, fig5ef-correlated-subpopulations, fig5gh-mechanosensitive-as-good, fig5i-time-course-similar, figs5i-coefficients-similar, fig6-simulation-mechanosensory-rescues | Blocked: data missing | breathing signal, odor trials, brain areas, the other concentration-series mice |
| figs6h-waveforms-homogeneous | Blocked: data missing | unit waveforms, the other groups of mice |
How the analysis differs from the paper's. No analysis of the paper was run. The notebook checks the file against its description and against the spike-sorting criteria of the Methods.
What was found about the dataset.
- The dandiset holds one NWB file from one of the paper's 22 mice: spike times of 1013 units and their Kilosort and Phy labels. The breathing signal, the inhalation and odor-trial TTLs, the odor signal, the artificial airflow pulses, unit waveforms and brain areas are not in it, and neither are the other 21 mice. None of the paper's findings can be tested from it.
- The file does not say which experiment it belongs to except through the keyword "Concentration Series", which odors were given, or which units the paper analysed.
KSLabelis "good" andContamPctis 100 for every unit, which contradict each other;qualityis missing for 644 of 1013 units. The unit table's columns other thanquality,spike_timesandunit_namehave no description, anddepth,Amplitudeandampno unit.- The file's
related_publicationsnames the preprint, not the published paper; the two 2024 versions of the dandiset do not name the paper. - What could be checked is consistent with the paper: the good units' ISI violation rates and presence ratios fall in the ranges the Methods report, no unit is duplicated, and the summed activity of the good units nearest the probe tip (by the Neuropixels 1.0 channel map, which is assumed, as the file names no probe) has its spectral peak at 2.78 Hz, within the range of mouse breathing; the subharmonic check leaves that peak at 2.78 Hz.