I don't really understand how async tool calls translate to token savings

It says that it removes tokens wasted while a model is waiting on synchronous tool calls. What tokens exactly is Pi using when waiting?

I think a running agent periodically checks if a process it started has finished.

The headline graph is kind of bizarre.

For some reason they're comparing their harness running on Astra xhigh to Codex with Astra max?

---

Also worth noting that OpenAI just added support for async tool calling to their harness, which isn't 1:1 with this approach, but is slowly ramping up in being able to provide something similar.

A big part of why Codex uses so many tokens is that it basically hot loops on polling tasks it starts for... absolutely no good reason: https://www.reddit.com/r/codex/comments/1wdlp7q/weve_discove...

I fixed it on my fork of Codex too, also back in Jan/Feb – I keep this patch rebased, for anyone who wants it: https://github.com/tekacs/codex/commit/9ffcf8db9078eae43d411...

It results in token savings similar in scale to those displayed here by Unreal.

---

My harness has used a slightly fancier version of the approach that Unreal is using since ~Feb, and... it definitely works excellently, but it's also assuredly smoother with Astra and other recent models that are more aware of async tool calling.

To reduce the codex polling I slapped this into my ~/.codex/config.toml:

  [features.multi_agent_v2]
  enabled = true
  wait_agent_enabled = true
  min_wait_timeout_ms = 10000 # 10s
  default_wait_timeout_ms = 300000 # 5m
  max_wait_timeout_ms = 3600000 # 1h

Seems to do the job and reduce usage; I just ran Astra for ~5 hours (using a goal) and it used the last 30% of my usage. And now they released GPT-6 Sol and Luna (which is basically 5.6 Sol and Luna, but a bit better and also 50% cheaper) ;_;

Honestly the fact that we're still modeling agent harnesses like chat and strapping them into shell sessions instead of building async actor systems with sandboxed OS functionality access actors is bananas to me. So much easy wins can be had just by building on right abstractions... Hopefully will get enough time to play with this idea soon on my own.

Do you have any examples of this approach?

I'm actually taking the time to hash out the details to do this as a side experiment project, but basically what I read this project as showing is that if you leverage asynchrony you can let agents be more efficient - and you get this "for free" if you model your "agents" as actors that interact with the system through message passing. Then all the system operations become messages to different actors, need to read a file => message the fs actor => get reply from FS actor as a message when it's done. You'd probably need some out of mailbox ways to share blob resources and sockets for realtime (audio), but for the most part simple message passing should handle most of what LLM agents do.

Actors can have identities and roles for RBAC, etc. you - cross agent communication is the same as sending any other message to a actors.

Not to mention that actors can be on your device, another device, etc. whatever the router can resolve - it's transparent to the agents.

Sounds like a trademark issue when Epic ships a wildly popular Unreal Engine

But I don't think you can trademark a generic word like Unreal...

"But I don't think you can trademark a generic word like Unreal..."

Sure you can, trademarks are contextual. If they were a landscaping business it wouldn't matter. But within the same industry absolutely.

I guess? but we are talking Game engine vs AI harness...

They've trademarked it for quite a few closely-related domains: https://tsdr.uspto.gov/#caseNumber=87709072&caseSearchType=U...

Computer software, namely, game engine software for video game development and operation; Computer software, namely, software development tools for the creation of computer-generated imagery and graphics for the production of video games; Computer software, namely, software development tools for the creation of computer-generated imagery and graphics for the production of content for virtual worlds and 3D platforms; Computer software, namely, software development tools for the creation of computer-generated imagery and graphics for the production of motion pictures, television shows, videos, 3D animations, 3D simulations, 3D visualizations, virtual reality motion pictures, virtual reality television shows; Computer software, namely, software development tools for the creation of computer-generated imagery and graphics for the production of virtual reality video games; Virtual reality game software; Virtual reality software for creating multimedia content; Augmented reality game software; Augmented reality software for use in mobile devices for integrating electronic data with real world environments for the purposes of entertainment

Apple? https://en.wikipedia.org/wiki/Apple_Corps_v_Apple_Computer

Unreal is trademarked. This is not in question. And personally I 100% thought this was an agent specifically for dealing with the Unreal Engine, and was interpreting all of that information in that context. Very weird name for a company/product.

Same, I thought this was going to be a plugin for creating content in Unreal engine.

I had to scroll to the bottom to realize that this had absolutely nothing to do with Unreal Engine or Epic. There's no reason to think that Epic wouldn't have an "Unreal Labs" creating harnesses to help them with software engineering.

This plainly seems like a trademark issue in progress considering it's in the same exact domain and considering how many others were confused the way I probably was.

Mildly disappointed that this has nothing to do with Unreal Engine, the popular game engine.

Yeah, I was hoping it would be something to make it easier for agents to interface with Unreal Engine games.

Can I use it with my codex sub?

This feels like it's begging for a lawsuit from Epic.

I think this space is very untapped. Models are interesting, but I am absolutely obsessed with some things I've been researching/working on for the past few years:

Fractal tool discovery: tool taxonomy where an agent can "drill deeper" to find what specific tool it's looking for. Helps if/when polluting context with a zillion (mostly unnecessary) tools.

Leveraging splay trees: this is my favorite data structure and I think relatively unused in the context of agents/harnesses. A lot of times, recently-used workflows/tool-chains will be used again, so having those at the top of the search hierarchy is an awesome optimization.

Virtual containerized notebooks: models working in sandboxed (WASI) Python notebooks is incredible. Even local models (if given enough time) will usually converge on a good solution. Being able to mount tools/resources/fs is again, imo quite untapped. Some problems here are running native things (thing numpy/pandas) in containers is a nightmare (or impossible).

Anyway, happy to see other folks seriously doing stuff in this space. If anyone wants to collaborate on anything don't hesitate to reach out :) I'm also actively looking for a job or some contract gigs.

Fun times ahead.

What makes it fractal?

It's kind of self-expanding/looping; fractal is just a cute name I like, but it's technically a directed cyclic graph (since you always have/want cycles).

I think recursive fits better, but I get it.

Helps if/when polluting context with a zillion (mostly unnecessary) tools.

Ugh, I can't help but respond to this one point. The fact that this is even an issue in the current year just tells us how screwed the software field is in a lot of ways. I don't mean that in existential terms, but of how divided we've become in terms of what's happened to human reasoning. On the one hand, you have people who apply deep thinking to develop the sort of approaches you described, and there's the exponentially growing segment of not-even-programmers who seem to never ask themselves whether any of their ideas have any sort of consequences.

Take MCPs for instance. Sure, I guess it can sometimes make sense to have a stateful API that is optimized for agents. Yet, more often than not, these MCPs frontload a ton of context where it's not needed, and solve problems where none existed. Merely sticking an API (MCP) in front of an API (CLI, REST, GraphQL) without a benefit that can be explained in a single sentence is lunacy and demonstrates a real lack of complex thinking.

that's like the worst name you could have picked

Does it support subscriptions ? If not it's a no go for me.

In my projects i use unreal and me and my team have tested out multiple things. We have settled on no MCP complications whatsoever other than simply exposing the python scripting from the editor + a export step that can write the blueprints and asset data into plaintext so that the bot can grep them. This has given by far the strongest results, and we now have no issues having the agents edit game code and do operations.

We found this massively outperforms any kind of agent like this and the official unreal MCP systems. Its similar to the Blender MCP which also just exposes scripting + very minimal api to claude code/others.

OP is a coding agent and harness tool named Unreal, not the game engine by Epic.

Not to be confused with Epic's Unreal Engine. Epic will probably be having words with this startup about the name.

OK, but isn't this just programmatic tool calling, which is beneficial in some cases but not others?

This is available in DSH via PTC mode.

Basically, instead of the completion API returning a sequence of tool calls, it returns a program that invokes the tools.

I think most are returning Typescript program

With ordinary tool calling, you may have to wait for a tool to return its result to the completion API before the model can continue and issue the next tool call.

With PTC, the generated program can invoke independent tools asynchronously, collect their results, and dispatch the relevant results back to the completion API in a single turn.

Claude and Codex added this as well, but there are some papers on this that show it's not good across all tasks, and models need to be trained for this specific technique, which is what AI labs are doing now.

its not really, model expresses tool calls as a list of bash tools

For everyone self-hosting models, optimising for cost is an anti-feature. It makes the results worse for no benefit (except a little speed).

What I'd love to see is a harness that deeply optimises for the best results obtainable out of non-frontier models. Many of these have 1M context windows, and most of it remains unused and under utilised in these harnesses, in my opinion.