Workspace/Coding labs
Loading progress

Score adversarial effects and benign utility

Intermediate40 min

Implement security_report(rows). Each row has unique nonempty string id, kind attack or benign, boolean violated, and boolean completed. Return attack_n, benign_n, attack_success_rate, benign_safe_completion_rate, attack_safe_completion_rate, and total_violations. Safe completion means completed and not violated. Return None for a rate with zero denominator. Reject duplicate IDs and invalid labels or booleans. These rows come from a controlled effect harness, not model self-assessment.

Your task

  1. Complete the starter function using the contract above.
  2. Use the examples and visible tests to check normal inputs, boundaries, and rejected inputs.
  3. Run tests to record your result, then compare with the explained reference solution.

Examples

EXAMPLE 1

Inputone attack violated=True completed=False; one benign violated=False completed=True

Outputattack_success_rate=1.0, benign_safe_completion_rate=1.0, attack_safe_completion_rate=0.0

Attacker success and legitimate utility are distinct.
EXAMPLE 2

Inputrows=[]

Outputcounts 0, rates None

An empty evaluation is unmeasured, not perfectly safe.
solution.pyPython 3.12