Long Context and Working with Documents
How to use 200k–1M tokens of context for good, instead of just dumping junk into it.
Long context isn't magic
Claude can "hold in mind" very long documents (up to a million tokens on advanced plans). But that doesn't mean you should pour everything into it. The more context, the higher the chance of "losing" something in the middle.
What works well
- Summarizing a long document (a legal contract, report, book).
- Comparing two versions of a contract.
- Q&A over a book / document.
- Carrying knowledge from one long reference into a new format (e.g., a FAQ).
A hack about lost-in-the-middle
Information in the middle of a long document is found by the model worse than at the beginning and the end. If an important point sits "in the middle", ask Claude to explicitly cite it and show the page/section.
When long context is NOT needed
- If the document fits in 30-40 pages — use any model.
- If you have many separate documents — RAG is better than "everything into one window".
Upload a long document (50+ pages) into Claude and ask for 5 typical tasks: summary, key numbers, risks, questions for a lawyer, changes in the new version.
Task grader
Copy and adapt to your context. Text in angle brackets should be replaced.
Read the attached document. Do 5 tasks: 1. A summary in 5 sentences. 2. All key numbers with the page noted. 3. The top 5 risks with a quote from the source text. 4. A list of points requiring legal clarification. 5. If there are tables — restructure them into markdown. If you didn't find something — say "didn't find it", don't make it up.
Prompt sandbox
- They cram the whole corporate archive into one window — the model gets confused.
- They don't ask for quotes — it's impossible to verify the answer.
- They rely on the answer without verifying the pages.
- Ask for quotes with the paragraph noted — that's easy to check.
- For very long tasks — a two-pass scheme: first a "map of the document", then targeted queries.
- If there are several documents and they don't fit — move to RAG.
Long documents, legal, reporting.
When there are too many documents — you need RAG.
Quiz — 2 questions
Discussion
No comments yet. Be the first!