Skip to content

#1 ETA approximation before we have our own data - #43

Open
vbo3376 wants to merge 3 commits into
wncc:mainfrom
vbo3376:eta-approx-2
Open

vbo3376 wants to merge 3 commits into
wncc:mainfrom
vbo3376:eta-approx-2

Conversation

@vbo3376

@vbo3376 vbo3376 commented Oct 29, 2026 •

Copy link
Copy Markdown

Closes #1

Summary

Implements steps 1–3 of #1: ETAs are no longer distance / 30 km/h before we have telemetry.

  1. Per-vehicle, per-class free-flow speeds. New free_flow_kmh tables for car, auto and bike (indiviual max speeds per vehicle class per road type).
    free_flow_speed_kmh() returns min(class speed, legal limit), ×0.9 at speed breakers.
    osm_import builds EdgeMeta.speed_kmh from it, and baseline_model() now limits speed to
    car_average_speed (the edge's own speed) and then to the legal limit.
  2. Starting congestion prior. CongestionPrior / DEFAULT_CONGESTION_PRIOR hold the table from the
    issue (main vs minor roads × hour band; Saturday 0.8× and Sunday 0.5× the weekday excess).
    FittedSpeedModel starts its traffic term from ln(slowdown) and shrinks toward the prior, not toward 0,
    so learned data replaces the table gradually. RoutingService initialises the model with the prior
    whenever time buckets are configured, so ETAs are time-aware before the first refresh_telemetry().
  3. Per-bucket ETA calibration. EtaCalibrator learns a correction factor per time bucket from
    (predicted, actual) durations, with log ratios clipped to ±ln 3 and k = 20. It tracks median absolute
    % error per bucket (median_ape(bucket, calibrated)).
    RoutingService.route() applies it, record_trip_eta() feeds it.

Acceptance criteria (#1)

  • Weekday 09:00 ETA > 03:00 ETA for the same route with no telemetry (tested)
  • Starting table scaled by per-bucket calibration factors; median absolute % ETA error tracked per bucket
  • Speed model shrinks toward the starting table; existing learning tests pass
  • Provider terms reviewed before reference-API data is stored: not applicable yet, because nothing
    stores reference-API data in this PR. Calibration here uses actual trip durations only.

Step 4 (trip-level residual model) is not part of this PR (adddressed in the follow up section ahead).

Things worth knowing

  • All numbers are placeholders. Free-flow speeds (anchored on TomTom Mumbai 2025) and the congestion
    table are guesses, as the issue says. The auto/bike ratios are assumptions.
  • Baseline no longer carries the speed-breaker rule. The ×0.9 factor now lives only in
    EdgeMeta.speed_kmh; keeping both applied it twice and tripped _check_expected_speeds. The GraphHopper
    export re-adds the rule when extensions=True.
  • GraphHopper vs Python engine differ. All three deploy/graphhopper/*.json now use car_average_speed,
    so GraphHopper uses its own car speeds for bikes and autos, while the Python engine uses the per-vehicle tables.
  • Rebuild data/roads.pkl. An old pickle loads fine but keeps the 30 km/h speeds.
  • Calibrator learns from the raw ETA. route() returns a calibrated eta_s plus
    uncalibrated_eta_s; trips must be recorded with the latter, otherwise the correction drifts back toward 1.
  • Two-stage shrinkage. The global residual is itself shrunk toward 0 (Σr / (N + k)), so one early
    outlier trip can't shift every empty bucket. The issue doesn't specify how global is computed.
  • Prior is on by default in fit_speed_model and RoutingService, but FittedSpeedModel(...)
    constructed directly defaults to no prior. diagnose() only checks peaks in buckets with observations, so the
    static table doesn't flag normal roads.
  • Restriction tests. RestrictionRoutingTests now builds its service with congestion_prior=None. NOW
    is Friday 17:30 IST, where the prior would otherwise halve speeds and hide what those tests check.
  • Custom groupings (group_of other than default_group) now get the prior too, for road-class group
    names. Unknown group names get no slowdown.
  • some end checks brought up minor bug fixes / syntax changes across steps which were amended into the step 2 & 3 commit

Known limitations / follow-ups

  • Nothing calls record_trip_eta() yet (no trip-completion hook), so the calibrator only learns in tests.
  • Road-based pickup ETAs in matching/eta.py get the prior but not the calibrator.
  • StraightLineEta and the backend straight-line fallback are still a flat 30 km/h.
  • UNIFORM_SPEED_KMH, expected_speed_kmh and VehicleProfile.uniform_speed_kmh are kept (still used by
    EdgeMeta's default and some fixtures); cleanup can follow.

Docs

Docs have not been edited in this PR. ROUTING.md, MAP_DATA.md, ROAD_LEARNING.md,
VEHICLES_AND_TURNS.md, OVERVIEW.md and docs/issues/001-eta-approximation.md still describe the uniform
30 km/h behaviour. I'm happy to update them in this PR or a follow-up, at the maintainers' discretion.

Testing

python -m pytest tests backend/tests: all pass. New tests: tests/test_free_flow.py,
tests/test_congestion_prior.py (prior, calibrator, route-level calibration, custom-group prior).

Follow-up (not in this PR)

A later PR could feed EtaCalibrator with reference ETAs from the TomTom Routing API before we have
enough real trips, as the issue suggests for a reference API on 1–5% of trips.

  • TomTom's pricing page lists a free tier of 20K Routing API requests/month. Rate limits and
    paid pricing still need checking.
  • Sampling would be small (1–5% of trips), so the free tier should cover early traffic.

Add FREE_FLOW_KMH (car), AUTO_FREE_FLOW and BIKE_FREE_FLOW,
a required VehicleProfile.free_flow_kmh field, and free_flow_speed_kmh(),
which returns min(class speed, legal limit) with the speed-breaker factor.

Values are assumptions based on TomTom Traffic Index, Mumbai, 2025
(~34 km/h all-in free flow, 54.1 km/h observed highway average);
the auto and bike ratios are assumptions built upon the generic data found.
Legal limits alone are not usable as speeds: unmapped cars default to 70 km/h on side streets.

No behaviour change yet: nothing calls the new function.

Part of issue wncc#1
- osm_import builds EdgeMeta.speed_kmh with free_flow_speed_kmh(), using
  each vehicle profile's own table.
- baseline_model() limits speed to car_average_speed (EDGE_AVERAGE_SPEED),
  i.e. the edge's own speed, then to the legal limit.
- The speed-breaker factor now lives only in EdgeMeta.speed_kmh. Keeping
  the baseline statement too applied it twice and tripped
  _check_expected_speeds. The GraphHopper export re-adds the rule when
  extensions=True, since GraphHopper has no per-edge speed from us.
- Regenerate deploy/graphhopper/*.json. All three now reference
  car_average_speed, so GraphHopper uses its own car speeds for bikes
  and autos too; the Python engine uses the per-vehicle tables.
- Tests: update the GraphHopper serialization expectation and two
  OsmImportTests assertions (living street 20 -> 15, primary with a
  speed breaker 27 -> 36); add a test that the baseline returns the edge
  speed for every class, vehicle, limit and breaker combination.

Existing learning tests are unchanged and pass. Rebuild data/roads.pkl
(python -m routing.osm_import ...) to pick up the new speeds; an old
pickle loads without error but keeps the 30 km/h speeds.

Part of issue wncc#1
…ket ETA calibration

- Add CongestionPrior and DEFAULT_CONGESTION_PRIOR in routing/speed_model.py
  with starting slowdown multipliers across (main vs. minor road class) x
  (weekday, Saturday 0.8x excess, Sunday 0.5x excess) x hour band.
- FittedSpeedModel starts its traffic term from ln(slowdown) instead of 0
  and shrinks observed group residuals toward the prior, so learned telemetry
  replaces the starting table gradually as samples arrive.
- RoutingService initializes speed_model with the prior whenever time buckets
  are configured, making routing and ETAs time-aware before the first
  refresh_telemetry() call.
- Scope SegmentDiagnosis traffic peak checks to observed buckets in
  self.traffic so static priors in unvisited hours do not flag normal roads.
- Add EtaCalibrator for per-time-bucket correction from (predicted, actual)
  duration pairs, clipping each log ratio to +-ln 3 and shrinking bucket
  means toward the global log residual (k = 20).
- Tests: add congestion prior and cold-start ETA tests in
  tests/test_congestion_prior.py (verifying weekday 09:00 ETA > 03:00 ETA
  before telemetry exists) and update two RestrictionRoutingTests speed
  assertions in tests/test_restrictions.py for the new baseline speeds.

Part of issue wncc#1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ETA approximation before we have our own data

1 participant