The variation in number of matched ROIs across session orderings for somata was generally far less than that for dendrites due to their larger sizes and more regular shapes.
Technical Validation, paragraph 5; Figure 8a and 8b legends
Tested on DANDI:000037, six orderings each of the three tracked sessions of the 13 tracked mice (sessions 1 to 3 of the 11 analysed mice, sessions 4 to 6 of 411400 and 412933), with results for the 11 analysed mice and for all 13. Somatic mice vary by at most one matched ROI over orderings (range over mean 0 to 0.017, mean 0.004; SD over mean, mean 0.002; 4 of 6 mice do not vary). For the 11 analysed mice, the dendritic mice with the 10% threshold range over mean 0.036 to 0.259 (mean 0.136), SD over mean, mean 0.055; with 20%, 0.053 to 0.405 (mean 0.161) and 0.079. Every dendritic mouse varies more than every somatic one under both thresholds and both measures, and also when the threshold applies only to pairs with a direct pairwise match (range over mean 0.029 to 0.400; means 0.113 at 10% and 0.172 at 20%). Without a threshold the dendritic mean is still larger (0.040 against 0.004), though 433451 (0.015) then varies less than 408021 (0.017). With all 13 tracked mice the dendritic means stay larger under every reading (range over mean 0.104 at 10% and 0.134 at 20%, SD over mean 0.042 and 0.064, against 0.004 and 0.002; 0.034 without a threshold); 412933 matches 60 ROIs in every ordering at 10%, so with it not every dendritic mouse varies more than every somatic one. The somatic match counts (12 to 151) are within 4 of the tracked counts in the files; the dendritic counts under the IoU thresholds (10 to 143 for the analysed mice) are well below them (51 to 353). Per dendritic mouse 19 to 307 three-session matches (on average over orderings) hold a pair of ROIs without a direct pairwise match, against 1 to 4 per somatic mouse; applying the threshold only to linked pairs keeps 3 to 28 more matches per dendritic mouse at 10%, still below the tracked counts (for example 34.3 against 118 for 433414 and 44.2 against 112 for 433448), so the IoU threshold removes most matches. Without a threshold the dendritic counts are 1.3 to 2.6 times the tracked counts. The package's own NwayMatching.run gives the same count as the reconstruction in all 12 orderings of 418779 and 420011. The dendritic values printed in this run differ from those of the run the verifier read in round 2 (for example 433448 range over mean at 10%, 0.259 here and 0.309 there), with the same conclusion; repeating the matching of 420011 and 433414 twice on one machine while preparing (not in the notebook) gave identical counts, so the difference is between runs on different machines. Open for the paper's authors - what overlap measure and threshold the modified matching used for dendrites, and whether the modified code can be published; the Methods say only "overlapped by at least 10-20%", and a measure relative to the smaller ROI would keep more matches.
How the test differed from the paper’s method
- The public package (version 0.6.0, default parameters) in place of the paper's custom-modified version, which is not public, as the claims file directs.
- Assumed that the dendritic overlap is the intersection over union of the registered masks that the package reports, applied to every pairwise match within a three-session match. The paper does not define the overlap; IoU is the overlap measure the package itself computes. The result holds at both thresholds the criterion names.
- The package expects an average intensity projection for registration; the files hold only a max projection, which is used in its place.
- The six directed pairwise matchings of each mouse are computed once and combined per ordering, as NwayMatching.run combines them (itertools.combinations in the given order); the notebook checks this against NwayMatching.run on all six orderings of 418779 and 420011 (12 of 12 equal). The package's summary plots are not made; they fail with the locked pandas (a read-only array in numpy.fill_diagonal) and do not change the matches.
- A pair of ROIs within a three-session match that has no direct pairwise match counts as IoU 0, so the match fails any threshold; the variant that applies the threshold only to linked pairs is shown and gives the same outcome.