upvote
In general, I've wondered whymost (all?) harnesses perform the API calls to the provider from the same machine that runs the tools. Why not have a client/server architecture? Then one session could talk to as many tool harnesses as it wanted to and you wouldn't have this kind of arbitrary limit of the session being tied to one specific workspace.
reply
My harness actually has 3 distinct components: the LLM gateway, the agent session manager, and the tool server, and they can be run on different machines. The most obvious use case is the LLM gateway being a central router for all provider queries with multiple machines running the agent sessions.

The agent session manager talks to the LLM gateway over HTTP using a standard internal protocol, then the gateway has adapters for each upstream provider and translates between their external API format and the standard internal one.

The tool server is a separate jar running as a separate server process with its own systemd unit and the agent session manager talks to the tool server over a beanstalkd queue. The primary motivation for having 2 processes was total separation for unprivileged agent tool call execution (ie. the tool server systemd unit runs as a user with no sudo whereas the agent session manager can execute as root, so the user can run scripts through the interface that do things like install packages and access secrets, but the agent can't do that.

The tool server could just as easily be talking to beanstalkd over a Cloudflare tunnel or ssh port forward, though, and a session could be routed to different machines, allowing execution of tool calls in different environments within the same context window/agent session.

reply
[flagged]
reply
A typical scenario is that you walk into a small company and find they're still using Windows 7 or an early version of Windows 10. Then you have to help them deal with all sorts of problems.
reply