Workspace/Coding labs
Loading progress

Implement one masked attention query

Intermediate65 min

Implement masked_attention(query, keys, values, allowed), returning normalized weights and a context vector with stable exponentiation and strict shape checks.

Your task

  1. Accept a nonempty list query, nonempty equally sized lists keys and values, and a boolean list allowed with one entry per key.
  2. Require every key to match query length and every value to share a positive output dimension. Require finite int/float coordinates, excluding booleans.
  3. Compute q dot k / sqrt(query dimension); normalize allowed positions with max-subtracted softmax and assign exactly 0.0 weight to masked positions.
  4. Reject an all-false mask and invalid inputs with ValueError. Return a dictionary with weights and context lists; do not mutate inputs.

Examples

EXAMPLE 1

Inputq=[1], keys=[[1],[0]], values=[[10],[20]], allowed=[true,true]

Outputweights≈[0.7311,0.2689], context≈[12.6894]

The output is a mixture, not a selected value.
EXAMPLE 2

Inputq=[1], keys=[[1],[2]], values=[[3],[9]], allowed=[true,false]

Outputweights=[1,0], context=[3]

Masking overrides a larger score.
solution.pyPython 3.12