Workspace/Mini projects
Loading progress
All mini projects
MODULE 09 · 5 HOUR BUILD

Evidence-led incident planner

Build a local incident planner that creates observable actions, consumes fixture observations, revises remaining work, and verifies the final diagnosis.

Build evidence Record your actual checks, results, and limitations.

Build it in stages

  1. Run the seed and inspect the missing-fee candidate rejected by its verifier.
  2. Define an incident fixture format with observations, source status, and ground-truth outcomes.
  3. Implement dependency-aware action selection with a fixed tool and token budget.
  4. Add two evidence-driven revision scenarios and repeated-state detection.
  5. Produce a report showing final correctness, rejected candidates, action count, and unresolved cases.

Your acceptance criteria

Use these as your project review. Record commands, outputs, and failure cases in your repository.

  • At least 12 scenarios include absence, timeout, denial, stale evidence, and conflicting candidates.
  • No step becomes complete without its defined observation or artifact.
  • Every loop stops within its configured action budget.
  • At least one majority candidate is correctly rejected by an independent verifier.

A working starting point

The seed runs as supplied. Extend it to satisfy the full brief. It is a teaching starting point, not a finished portfolio submission.

main.py
python
import json
from collections import Counter

CASES = [
    {"id": "fee", "qty": 3, "price": 6, "fee": 3, "candidates": [18, 18, 21]},
    {"id": "plain", "qty": 2, "price": 5, "fee": 0, "candidates": [10, 10, 12]},
    {"id": "unresolved", "qty": 4, "price": 3, "fee": 2, "candidates": [12, 13]},
]

def solve(case, budget=4):
    trace = []
    accepted = []
    expected = case["qty"] * case["price"] + case["fee"]
    for index, candidate in enumerate(case["candidates"]):
        if index >= budget:
            break
        valid = type(candidate) is int and candidate == expected
        trace.append({"action": "verify_total", "candidate": candidate, "valid": valid})
        if valid:
            accepted.append(candidate)
    votes = Counter(case["candidates"])
    majority = sorted(votes, key=lambda value: (-votes[value], value))[0]
    return {
        "case": case["id"],
        "majority": majority,
        "answer": accepted[0] if accepted else None,
        "status": "verified" if accepted else "unresolved",
        "actions": len(trace),
        "rejected": [event["candidate"] for event in trace if not event["valid"]],
    }

results = [solve(case) for case in CASES]
for result in results:
    print(json.dumps(result, sort_keys=True))
print("verified cases:", sum(result["status"] == "verified" for result in results))

Push it further

Compare a fixed workflow, an observation-driven controller, and candidate search under an equal resource budget; report per-scenario failures rather than only averages.