Workspace/Coding labs
Loading progress

Compare complete paired runs

Intermediate45 min

Implement compare_runs(baseline, candidate). Each list contains rows with unique string id, boolean success, and finite nonnegative numeric ms. Require identical nonempty ID sets. Return n, baseline_rate, candidate_rate, wins (false to true), regressions (true to false), and mean_latency_delta_ms (candidate minus baseline). Reject duplicates, missing cases, invalid booleans, or bad latency. A failed outcome still belongs in the denominator.

Your task

  1. Complete the starter function using the contract above.
  2. Use the examples and visible tests to check normal inputs, boundaries, and rejected inputs.
  3. Run tests to record your result, then compare with the explained reference solution.

Examples

EXAMPLE 1

Inputbaseline=[{"id":"x","success":False,"ms":10}]; candidate=[{"id":"x","success":True,"ms":15}]

Outputn=1, baseline_rate=0.0, candidate_rate=1.0, wins=1, regressions=0, mean_latency_delta_ms=5.0

The quality improvement and latency cost are both retained.
solution.pyPython 3.12