One-trillion-token context will arrive before perfect memory
A forecast for 2028–2030: trillion-token context becomes practical, while selection, memory, provenance, latency, and attention remain harder engineering problems.
Our prediction is that context windows around one trillion tokens will emerge in specialized systems by 2028 and become commercially practical for broader high-value workloads by 2030. That will probably happen before most teams build trustworthy long-term memory.
This is not a prediction that every request will contain one trillion useful tokens, or that a model will pay equal attention to all of them. It is a prediction about addressable capacity. Hardware, inference methods, distributed memory, caching, and model architecture will keep pushing the maximum upward because large windows unlock valuable new workloads.
The harder question is what an AI system should do with that capacity.
Capacity is not attention
A database can store a billion rows without reading all of them for every query. Context should be viewed similarly.
A model accepting a trillion tokens does not guarantee that every detail influences the answer equally. Relevant information may be separated from the question by enormous amounts of noise. Multiple versions of a fact may conflict. Repeated text may distort emphasis. The position and structure of information can matter.
Long context expands the possible working set. It does not remove the need to curate that working set.
The economics will shape usage
Even when a model supports a huge window, repeatedly processing it has costs:
- Time to read, transfer, and tokenize input.
- Memory required for inference and attention state.
- Cost charged by providers.
- Energy consumed locally or in a data center.
- Additional latency before the first useful output.
Prompt caching and reusable prefixes can make repeated context cheaper. Hierarchical attention and retrieval can reduce work. But the economically sensible architecture will still avoid sending an entire company drive when the task concerns three current files.
Our 2028–2030 forecast
We expect the path to one trillion tokens to happen in stages rather than through one sudden model release.
| Period | What we expect |
|---|---|
| By 2028 | Specialized research, code, intelligence, and enterprise systems advertise trillion-token addressable workspaces using retrieval, caching, sparse attention, and hierarchical representations |
| 2028–2030 | Providers reduce the latency and cost of repeatedly working across those spaces, while applications build better provenance and context inspection |
| By 2030 | Trillion-token capacity becomes commercially practical for selected high-value workloads, although most individual model decisions still use a much smaller active context |
The distinction between an addressable workspace and an active attention set will matter. A system may make a trillion tokens available without processing every token with full attention on every step.
Large context will change software interfaces
The immediate value will appear in tasks with a naturally bounded but large workspace:
- A complete repository plus build and issue history.
- A long legal matter with contracts, evidence, and correspondence.
- A research collection with papers, notes, and datasets.
- A video or meeting archive with transcripts and timestamps.
- A design project containing decisions across many artifacts.
Applications will need context maps that show what is loaded, summarized, omitted, stale, or conflicting. Users will want to pin critical material, exclude sensitive sources, and understand why an answer used one version over another.
The context inspector may become as important as the chat box.
Memory remains a different problem
Context is what the model can access during a run. Memory is the policy for what persists across runs.
Perfect memory requires decisions about identity, consent, relevance, correction, expiration, and deletion. It must distinguish an enduring preference from a passing thought. It must update facts without silently erasing history. It must avoid leaking one project into another.
Those are product and governance problems, not token-limit problems.
Retrieval will evolve, not disappear
With very large windows, retrieval may move from “find five chunks” to “select regions, summaries, and active versions from a large addressable workspace.” Systems can keep broad context cached while dynamically focusing the model on a smaller decision set.
The boundary between retrieval and context management will blur. Provenance will become more important because users need to know which part of a vast workspace supported a claim.
What we would build for
We would not design an AI platform around a fixed context limit. We would build for:
- Model-independent context objects with source and permission metadata.
- Multiple representations of the same material: raw, indexed, summarized, and cached.
- Explicit retention and expiration rules.
- An inspectable record of what each model actually received.
- Routing that balances privacy, quality, latency, and cost.
Then larger windows become an upgrade to the runtime, not a reason to redesign the entire product.
Our prediction
One-trillion-token contexts will make impressive demonstrations possible between 2028 and 2030. The winning systems will still be those that use fewer tokens intelligently for each decision.
Abundant capacity does not reduce the value of context engineering. It increases it, because the cost of choosing the wrong information grows with the amount available.