Workspace/Mini projects
Loading progress
All mini projects
MODULE 12 · 6 HOUR BUILD

Grounded configuration answer service

Build a local retrieval service over versioned configuration notes, with hybrid candidate fusion, explicit evidence bundles, and claim-level support checks.

Build evidence Record your actual checks, results, and limitations.

Build it in stages

  1. Run the seed to retrieve an environment-specific configuration fact with a source ID.
  2. Create a corpus containing policies, exceptions, tables, and multiple document revisions.
  3. Implement structural chunking and compare one semantic-boundary experiment.
  4. Add lexical retrieval, a verified dense-retrieval adapter or clearly labeled vector simulation, and rank fusion.
  5. Add query-aware reranking, claim support checks, and an abstention path for unsupported questions.

Your acceptance criteria

Use these as your project review. Record commands, outputs, and failure cases in your repository.

  • At least 20 labeled questions include exact identifiers, paraphrases, exceptions, and unanswerable cases.
  • Every factual answer cites a stable document revision and passage.
  • No cross-environment passage validates an environment-specific claim.
  • Report sparse, dense/simulated-dense, and hybrid retrieval results separately without invented performance numbers.

A working starting point

The seed runs as supplied. Extend it to satisfy the full brief. It is a teaching starting point, not a finished portfolio submission.

main.py
python
import json
import re

DOCUMENTS = [
    {"id": "staging-v4-p1", "entity": "staging", "field": "timeout", "value": 30,
     "text": "Staging revision 4 timeout is 30 seconds."},
    {"id": "production-v4-p1", "entity": "production", "field": "timeout", "value": 60,
     "text": "Production revision 4 timeout is 60 seconds."},
    {"id": "staging-v4-p2", "entity": "staging", "field": "retries", "value": 2,
     "text": "Staging revision 4 permits 2 retries."},
]

def words(text):
    return set(re.findall(r"\w+", text.lower()))

def retrieve(query):
    query_words = words(query)
    return sorted(DOCUMENTS, key=lambda doc: (-len(query_words & words(doc["text"])), doc["id"]))

def answer(entity, field):
    candidates = retrieve(entity + " " + field)
    for doc in candidates:
        if doc["entity"] == entity and doc["field"] == field:
            claim = {"entity": entity, "field": field, "value": doc["value"], "citation": doc["id"]}
            supported = all(claim[key] == doc[key] for key in ["entity", "field", "value"])
            if supported:
                return {"status": "supported", "claim": claim}
    return {"status": "insufficient evidence", "entity": entity, "field": field}

print(json.dumps(answer("staging", "timeout"), sort_keys=True))
print(json.dumps(answer("staging", "rationale"), sort_keys=True))

Push it further

Add parent-context expansion and optimize evidence coverage under a measured model-token budget; compare its gains with simply increasing candidate count.