Detroit Automation Academy

Live Transit Detroit · Installment 01 · October 2026

Postmortem: Activating Empirical Timetable Substitution

Published October 6, 2026 · By Justin Spencer Smith

Phase 2b-ii of the Forge Line live transit layer replaces scheduled bus segment times with measured typical times, once enough observations exist. It was reported active on October 5, found completely inert on October 6, misdiagnosed twice, and finally root-caused and activated the same day. This is the record.

Summary

The timetable stores stops as page indices into ddot-routes-3d.json; timing cells are keyed by GTFS stop_id. The client looked up cells with page indices, so all 154 mature cells missed on every query. One translation function (ttStopId) restored the lookup. Post-fix verification against production data: 18,002 substitutions applied across 2,199 trips.

Timeline

2026-10-05UAT Pass 2 audit reports 2b-ii "verified and active" based on 73 mature cells in segments.json. No functional test performed.
2026-10-06 ~11:10UAT Pass 3 finds 2b-ii inert. Defect D1: fetchTimingCells() stored the segments.json document envelope ({schema, built_at, cells, segments}); applyEmpiricalBaseline() expected the flat cell map. The hasCells gate could never pass.
2026-10-06 ~11:10Same pass finds D2: 0 of 154 cells match schedule-adjacent stop pairs — all span multiple stops.
2026-10-06 ~11:20D1 fixed (unwrap d.segments). D2 addressed client-side with a covering-span lookup and schedule-proportioned apportioning. Deployed as stamp v20261006-1115 and reported as "2b-ii actually engaging."
2026-10-06 ~11:31Independent accuracy review reports a stop-ID space mismatch between the detector reference and the timetable (cell 60|741|764 vs timetable stop 741 on route 68) and prescribes a pipeline rebuild.
2026-10-06 ~11:35Direct GTFS verification (stop_times.txt, feed S1000182): GTFS stops 741 and 764 are both served by route 60. The cell was correctly keyed. The review compared timetable page indices to GTFS stop IDs as one space — the same category error as the D2 analysis.
2026-10-06 ~11:40Root cause identified in build-timetable.py: timetable stops arrays store page indices (stop_to_idx), not GTFS IDs. Timetable page 741 = GTFS stop 846 (Freud & St Jean, route 68). Client lookups used page indices against GTFS-keyed cells.
2026-10-06 ~11:45Fix deployed: ttStopId() translates page index → GTFS stop_id via the loaded stopData array, applied to exact and spanning-span lookups. Stamp v20261006-1145. Served bytes verified.
2026-10-06 ~11:50Node verification on production data (real timetable, real stopData, real cells, wkd_pm bucket): 18,002 substitutions, 2,199 trips with baselines.

Root cause

Two distinct numeric ID spaces overlap in range but identify different stops:

  • GTFS stop_id (e.g. "741" = Evergreen & 10 Mile): used by the telemetry detector, reference/route-stops.json, and timing-cell keys (route|from|to|bucket).
  • Page index (e.g. 741 = position 741 in the ddot-routes-3d.json stops array = GTFS "846", Freud & St Jean): used by ddot-timetable-S1000182.json trip stop lists.

applyEmpiricalBaseline() concatenated timetable page indices directly into cell keys. Every lookup missed. The failure was silent: the hasCells gate (fixed by D1) passed, the spanning-span scan (added by D2) ran, but stopPos was keyed by page index while cell keys carried GTFS IDs, so no span ever matched either.

Why it took three passes

  1. Pass 2 inferred "active" from cell count, never executing the lookup path. Count of data is not evidence of function.
  2. D1 was a real bug (envelope unwrap) but fixing it only moved the failure one layer down — lookups then ran and still missed.
  3. D2 and the accuracy review both assumed the numeric IDs were one space. The D2 fix (spanning-span apportioning) was correct logic applied to wrong keys; the review's "pipeline rebuild" prescription would not have fixed anything, since the pipeline was correct.
  4. Resolution came from reading build-timetable.py (stop_to_idx) and cross-checking one cell against GTFS stop_times.txt — from the data, not from re-reading the client code.

Resolution

  • live-transit.js: ttStopId(pageIdx) → String(stopData[pageIdx].id), applied in applyEmpiricalBaseline() exact and span lookups. empiricalSpeed() required no change (client map-matched segments already carry GTFS IDs). raptor.js required no change (already consumed depL2/arrL2 and marked provenance: 'empirical').
  • scripts/build_timing.py (transit-data repo): ghost-route traversals no longer emitted as segment.traversed — a latent contamination vector from the accuracy review. Existing cells verified clean (145/145).
  • No reference rebuild was required. The detector reference and pipeline were correct throughout.

Verification

  • node --check clean on both changed files.
  • Unit: apportioning logic produced [0, 150, 390, 615] on a synthetic 4-stop trip (exact cell + spanning cell).
  • Production data: 18,002 substitutions / 2,199 trips (40 patterns, wkd_pm). Pair-delta distribution: p50 0.0s, p90 2.0s, p99 157.3s.
  • Largest delta (1647.9s, route 12 pair 5664→5321, scheduled 24s) traced to cell 12|5664|5321|wkd_pm, n=3, mean 1671.9s / p50 1723.3s / p90 1798.4s — three concordant observations of a ~28-minute dwell. The substitution is honest; the schedule was the fiction.
  • Served bytes verified: HTML references live-transit.js?v=20261006-1145; JS contains the ttStopId translation.

Lessons

  1. A count of data is not evidence of function. Pass 2's "73 mature cells, therefore active" was the original error. Functional verification — execute the lookup, count substitutions — is the bar.
  2. Overlapping numeric ranges are not one ID space. Page indices and GTFS IDs are both small integers; nothing in the client code marked which was which. The ttStopId comment now records the distinction at the use site.
  3. Two independent wrong diagnoses can agree. The D2 analysis and the review's defect-2 analysis made the same category error from different directions. Agreement between analyses is not validation; the GTFS cross-check was.
  4. Fix the layer that's actually broken. The pipeline-rebuild prescription was a large, disruptive change to a correct system. Reading the builder showed the translation belonged in the client.
  5. Silent fallbacks need tripwires. 2b-ii failed safe (scheduled times) for its entire life, which is why the inertness went unnoticed. A substitution counter existed but was never surfaced. Logging substitution counts where they can be observed is the follow-up.

← Live Transit Detroit series index