Put Retrieval Before Generation

Tip: Do not ask the model to remember the repo

For codebase Q&A, retrieve the relevant files first, then generate. A 200k context window is not a search index.

hits = retriever.search(question, k=8)
context = format_hits(hits)
answer = generate(question, context)

Gotcha: Stuffing the whole monorepo burns money

Dumping every file into the prompt looks thorough and then truncates the actual answer. Cap tokens and rank by relevance.

prompt = repo.read_all()  # will truncate the useful tail

Source: https://docs.anthropic.com/en/docs/build-with-claude/rag