Rendered at 16:25:26 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
langs 1 days ago [-]
I don't get it. Why benchmark the latency instead of recall/precision? Optimizing for millisecond-level latency is meaningless in the context of LLM calls. Accuracy is the tool's greatest value, yet there is no testing for it?
esafak 24 hours ago [-]
Has anyone else benchmarked all these tools for precision/recall? I too want to know if agent memory is something I should add. I only do session memory for now and that is quite useful.
glub 5 hours ago [-]
Yes. See LongMemEval, LoCoMo. Tons of research here.
But precision/recall is relatively "solved". What nobody has gotten close to solving is maintenance and provenance - what goes into memory, what qualifies as truth, how stale memory gets invalidated/superseded.
We're now in the phase of re-discovering 30+ years of pain of knowledgebases.
vshulcz 10 minutes ago [-]
[flagged]
straumat 9 hours ago [-]
Nice tool! If you want to publish your okf bundles for humans to read (like on a github pages or your intranet), i developped an open source solution to do that : https://github.com/oak-invest/kiso - It's like Hugo for OKF
kimseungyong 16 hours ago [-]
I like the idea.
To avoid losing context, I mainly conduct the planning session and the implementation session separately.
From the standpoint of building enterprise products, what worries me most is whether the agent we are implementing may not have understood a completely different context.
If okf_memory maintains domain knowledge very well, it is expected that implementation will be possible in unit functional units within a consistently smooth session.
However, there is a risk in applying this idea directly to practical work, so I’ll have to test it separately on a personal project.
calebkaiser 2 days ago [-]
Love seeing projects like this. The performance benchmarks are nice to see. Have you done any benchmarks against approaches like OpenAI's Symphony for things like token usage or task completion?
Personally I've always seen AI 'memory' as a pain point for people in their experience using LLMs than a benefit from the agent remembering the last unrelated thing you were working on. It wastes context similarly to 'skills'. The most efficient workflow imo is having a few well written (not by ai) md files across a clean codebase.
continuitykit 19 hours ago [-]
[flagged]
practicalsystem 2 days ago [-]
cool going to check it out, I've got an opensource project that might compliment it that I'm excited to try
I want this, but also for cross-project memory. Save me from building my own, which I have planned but figure something would eventually pop up in HN...
How well does the model adhere to using this in a harness like Codex where it may be directed to use the built in memory tooling? Maybe I'll need to try an experiment directing it to save to its native memory to use OKF instead
mbreese 2 days ago [-]
This is usually my main concern with tooling like this that isn’t a first party project. Anthropic can tune Opus, Fable, etc and their harness to use their memory format or preferred method of tool calling. I have had mixed results getting LLMs to consistently use third party tools.
I’m very much in favor of things like OKF wikis for memory or knowledge storage/retrieval. So I too would love to know how well this really integrates into one of the coding harnesses (Claude code or Codex mainly).
steammaho 1 days ago [-]
honestly didn't notice much degrading of standard compact. Have several sessions which already lasts for several month and they are perfectly fine
nullbio 2 days ago [-]
Now that Astra is moving to a new compaction model (aka, ditching compaction altogether), is there any need for this sort of thing still?
pdimitar 2 days ago [-]
Huh? Can you show source on the "it will not compact"? Very interesting.
esafak 2 days ago [-]
"With Astra, we’re introducing a new way for Codex to preserve and retrieve context when the context window fills. Historically, models have used compaction to summarize work during long sessions, such as when debugging complex issues or tackling large refactors. Each compaction can leave out details about why a fix failed or how a component behaves. In Codex, Astra can keep notes across context windows, preserving accumulated details without repeatedly compressing them into a single summary. Earlier context windows remain searchable, so Astra can find requirements or test results from previous messages and tool outputs—even if that information wasn’t captured in its notes. You can enable this experimental feature in your Codex config.toml, (opens in a new window) and it will become the default for Astra in the coming weeks."
This clearly is Codex-specific, not as much a feature of the model (though obviously they probably have trained it to be great at working with their own tools). Sounds somewhat similar to pi-observational-memory I'm using with Pi.
svyatov 2 days ago [-]
The performance is cool and all, but what about capture/retrieval quality? Are there any benchmarks for that?
muhammadwaqar1 1 days ago [-]
[dead]
okf_memory 2 days ago [-]
Hey HN,
We built OKF Agent Memory because we were frustrated with how AI coding agents (Claude Code, Cursor, Windsurf, local models) handle long-term project context.
Every time a context window closes or a session resets, the agent forgets architectural decisions, domain discoveries, and operational rules. The existing solutions fall into two extremes:
1. Ad-hoc flat files (CLAUDE.md, AGENTS.md, .cursorrules) that inevitably balloon into 20k-token monoliths, degrade agent focus, and cause "lost-in-the-middle" attention failure.
2. Vector databases / background daemons (Mem0, Letta, Zep) that introduce heavy runtimes (Python/Node), docker containers, proprietary storage silos, and recurring embedding API costs (adding 200–800ms per retrieval call).
Our approach: The "LLM Wiki" in pure Go.
OKF Agent Memory (v0.1.0) is a single, zero-dependency Go binary that turns your Git repository into a structured, self-validating knowledge corpus based on Google's Open Knowledge Format (OKF) v0.2 specification:
• In-Memory BM25 Search (<300µs): Fast lexical ranking across titles, YAML metadata, tags, and bodies directly in memory. No embedding APIs, zero network overhead, zero runtime cost.
• Progressive Disclosure: Slashes prompt overhead by up to 90%. Instead of loading thousands of lines of context, the agent searches the bundle index and pulls only the exact 300-token concept required for the current task.
• 100% Git-Native: Everything lives in `knowledge/` as human-readable Markdown. You audit your agent's memory via `git diff`, `git blame`, and code reviews.
• Built-In Stdio MCP Server: `okf mcp knowledge` exposes native Model Context Protocol tools (`okf_search`, `okf_show`, `okf_create`, `okf_validate`) directly to Claude Code and Cursor.
• Trust Tiers: Distinguishes authoritative human law (`verified: human:...`) from agent-generated drafts (`generated: agent:...`).
• Sub-4ms Cold Starts (<15MB RSS): Starts in milliseconds with no VM spin-up.
Try it in 30 seconds:
$ brew install okf-memory/tap/okf
$ cd your-project && okf bootstrap .
We'd love your feedback on the architecture, the Go implementation, and how your coding agents behave with progressive disclosure memory!
lukevp 2 days ago [-]
I love the idea! OKF 0.2 solves for the problem of the AIs generating massive amounts of documentation (far more than humans ever created) and giving it equal-weight over what a human actually approved and committed to. I've been using it to attribute my decisions with specific directions on how to ensure that it takes strong direction from my explicit decisions and clarifies implicit / AI-driven decisioning.
Adding on progressive disclosure to this is brilliant, and I love the idea of a fast, in-memory, single-binary tool. This is a great way to approach the solution to this problem.
One thing I will say though - I would never be able to use this in my enterprise. It would just be too much of an uphill battle to purchase something that is so niche in utility - this tool is not a ton different than just having the md files locally and having it use ripgrep to search over them, and telling CLAUDE to write the OKF files as well as an index when it makes changes, is it? is the index generated dynamically / is anything about the progressive disclosure different than just having the agent manage it while it documents?
If you're going for smaller teams that can buy tools without a ton of approval / procedural overhead, I think that might have some success. another possible solution would be to make the cross-repo search something that you can handle with OSS but you have to self-host, and then pay for support. if you got enough usage and penetration within an enterprise from the teams just using OSS and self-hosting, they might consider buying support after-the-fact.
But precision/recall is relatively "solved". What nobody has gotten close to solving is maintenance and provenance - what goes into memory, what qualifies as truth, how stale memory gets invalidated/superseded.
We're now in the phase of re-discovering 30+ years of pain of knowledgebases.
To avoid losing context, I mainly conduct the planning session and the implementation session separately.
From the standpoint of building enterprise products, what worries me most is whether the agent we are implementing may not have understood a completely different context.
If okf_memory maintains domain knowledge very well, it is expected that implementation will be possible in unit functional units within a consistently smooth session.
However, there is a risk in applying this idea directly to practical work, so I’ll have to test it separately on a personal project.
https://github.com/ucsandman/declick
Maybe?
https://github.com/fellowgeek/mcp-memory
https://news.ycombinator.com/item?id=49286073
I’m very much in favor of things like OKF wikis for memory or knowledge storage/retrieval. So I too would love to know how well this really integrates into one of the coding harnesses (Claude code or Codex mainly).
https://openai.com/index/gpt-6-astra/
We built OKF Agent Memory because we were frustrated with how AI coding agents (Claude Code, Cursor, Windsurf, local models) handle long-term project context.
Every time a context window closes or a session resets, the agent forgets architectural decisions, domain discoveries, and operational rules. The existing solutions fall into two extremes: 1. Ad-hoc flat files (CLAUDE.md, AGENTS.md, .cursorrules) that inevitably balloon into 20k-token monoliths, degrade agent focus, and cause "lost-in-the-middle" attention failure. 2. Vector databases / background daemons (Mem0, Letta, Zep) that introduce heavy runtimes (Python/Node), docker containers, proprietary storage silos, and recurring embedding API costs (adding 200–800ms per retrieval call).
Our approach: The "LLM Wiki" in pure Go.
OKF Agent Memory (v0.1.0) is a single, zero-dependency Go binary that turns your Git repository into a structured, self-validating knowledge corpus based on Google's Open Knowledge Format (OKF) v0.2 specification:
• In-Memory BM25 Search (<300µs): Fast lexical ranking across titles, YAML metadata, tags, and bodies directly in memory. No embedding APIs, zero network overhead, zero runtime cost. • Progressive Disclosure: Slashes prompt overhead by up to 90%. Instead of loading thousands of lines of context, the agent searches the bundle index and pulls only the exact 300-token concept required for the current task. • 100% Git-Native: Everything lives in `knowledge/` as human-readable Markdown. You audit your agent's memory via `git diff`, `git blame`, and code reviews. • Built-In Stdio MCP Server: `okf mcp knowledge` exposes native Model Context Protocol tools (`okf_search`, `okf_show`, `okf_create`, `okf_validate`) directly to Claude Code and Cursor. • Trust Tiers: Distinguishes authoritative human law (`verified: human:...`) from agent-generated drafts (`generated: agent:...`). • Sub-4ms Cold Starts (<15MB RSS): Starts in milliseconds with no VM spin-up.
Try it in 30 seconds: $ brew install okf-memory/tap/okf $ cd your-project && okf bootstrap .
GitHub: https://github.com/okf-memory/okf-agent-memory Docs & Landing Page: https://okf-memory.dev
We'd love your feedback on the architecture, the Go implementation, and how your coding agents behave with progressive disclosure memory!
Adding on progressive disclosure to this is brilliant, and I love the idea of a fast, in-memory, single-binary tool. This is a great way to approach the solution to this problem.
One thing I will say though - I would never be able to use this in my enterprise. It would just be too much of an uphill battle to purchase something that is so niche in utility - this tool is not a ton different than just having the md files locally and having it use ripgrep to search over them, and telling CLAUDE to write the OKF files as well as an index when it makes changes, is it? is the index generated dynamically / is anything about the progressive disclosure different than just having the agent manage it while it documents?
If you're going for smaller teams that can buy tools without a ton of approval / procedural overhead, I think that might have some success. another possible solution would be to make the cross-repo search something that you can handle with OSS but you have to self-host, and then pay for support. if you got enough usage and penetration within an enterprise from the teams just using OSS and self-hosting, they might consider buying support after-the-fact.