Hacker News

I don't think there is really any convergence going on. The agentic ecosystem is continuing to multiply on a daily basis and everyone and their grandma has written a new agent framework--people are stepping over each other to get these new projects out the door.

That said, I think Google's ADK ecosystem and this new AX platform is promising--I would expect Google to maintain this and other tooling around this for years to come.

To the Googlers out there: is Google using this at any capacity for internal projects?

I'm working on something in the "cloud VMs for agents" space[1], so I have some battle scars and opinions!

IMO, you want the flexibility to create either: (a) permanent devbox VMs, and (b) per-task VMs

Agent sandbox platforms tend to be tuned for the latter, which sometimes involves VMM hackery for fast boot, snapshotting VM filesystem and RAM, etc.

Some workflows are a lot simpler if the multiple agents share a VM. These are workflows where agents must share state. A simple one we have: making related changes in our public OSS repo and our private repo, and then testing the change.

And other times you want to split up the tasks onto isolated VMs so they don't interfere with each other (ie run two dev servers without database or port collisions).

I tweeted a bit about this (https://x.com/dbmikus/status/2099264325231771878) and had a little debate with folks about ephemeral vs persistent VMs for agents

[1]: https://github.com/gofixpoint/amika

Basically everyone has a sandbox of some sort to run agents inside. Everyone has a registry of some sort for tools. Everyone has a way of running agents inside a sandbox and giving it some tools.

Now the stuff people are coming up with is: how do you do authorization in this model? do you need a full sandbox all the time or can it be a workflow? how do you specify an agent is it a prompt or does it have some kind of control flow structure? How do you coordinate among many running agents?

I would say thats where we are now is there’s loads of people all solving the same problems a bit like when CoreOS, Kube etc. were all competing.

From what I've seen, the vast majority of agent sandboxes with funding aren't for developers to use when coding, but for production applications that want to have LLMs do work. It's just a different model - APIs are better than a great terminal experience for a coding harness.

I've been working on https://lullabot.github.io/sandbar/latest/ which works with Proxmox for VMs (and lima for locals or regular linux hosts over ssh). There's a diagram in https://lullabot.github.io/sandbar/latest/why/#recommended-w... with what we're currently recommending. Though, after some feedback, I'm in the process of integrating a colleague's web-based review tool as it turns out many preferred fully reviewing locally instead of using draft PRs.

It's got some opinions in terms of default tools for our team and industry so it may not fit yours. Forgive some of the AI-isms in the docs, I want to get the UX and feature set to a solid place before doing a full review.

I have no idea, I just live in my terminal at this point, on linux. Don't use Visual Studio or Jetbrains products anymore at all. I use visual studio code to view the occasional diff and run sql queries. Other than that, zero desktop apps, just terminal & cli tooling. Its great! I have about 20 terminals open at any given time though. I have no idea how some of my colleagues stay productive as they are messing around with all these workflow tools, desktop apps etc.

My advice: stop your fomo. Just get useful work done.

"Is giving the agent a temporary scratchbox really that valuable?"

Yes, but, wrong layer here. Giving the agent a computer use (a la bash) is what folks are after. A temporary sandbox with lots of control knobs and security bits is how you do that in (as you noted) an enterprise.

For my workflow I need both, a permanent-ish VM and emphemeral sandboxes. Both have their place and pros and cons.

VMs are better for personal assistant work, GUI clicktesting, investigating bugs in your personal dogfooding dev instance and anything you haven‘t yet made repeatable and fast to set up.

Sandboxes are better when you need resource isolation or security and have a graph of tasks to work through. My agents often starve each other on one VM, so if they don‘t need any of the above it‘s just easier to isolate them.

Everyone is working in this area, including me [0], but either option really isn‘t that convenient to use yet. It‘s a bit of a „isn‘t Dropbox just FTP on a VM“ moment right now.

[0] https://github.com/madeinorbit/podium

Personally I think microVM are the future but nothing wrong with a solid proxmox setup.

Probably we'll converge on a virtualised IO / Storage layer running microVMs beneath for isolation and security. Keep the network and storage layer separate for compatibility running a variety of stuff and a second security boundary.

> What's wrong with that?

Nothing at all. You'll know when you've outgrown it.

> what the general workflow is now that people are converging to?

Graph-based workflows where agents pick up work as it becomes available, structured output, while you manage the work queue and outcomes. Maybe? IDK really, it's all moving quite fast.

The general workflow I have seen for non-technical people building software is something like:

- Do multiple tasks in the same context window / session, conflate different changes into the same prompt

- Repo mixed with old markdown files from previous tasks, excel and word docs and 300 playwright screenshots

- 5 tools all calling each other, test and deployment scripts are all markdown skills

Personally I prefer a ticketing system and isolated work trees

I wrote a little program to spin up isolated rootless workspaces: https://github.com/jgillich/tpd

Works pretty well for me but I haven't put any effort into promoting it

This sort of works.

The problem is that you'll end up wanting to run 2 or 3 (or 20, 100, 10,000) agents at once and that gets very hard with a single VM.

There's also an argument that you should be using a separate sandbox for each code operation a LLM performs (or at least each set of related operations). That's even harder to do with conventional VMs.

Compared to enterprise yours is missing egress control and secrets management, if you make the isolation watertight you cripple the agent's performance, and then the careful game of whack-a-mole begins when you stand up local package mirrors, authentication brokers etc. etc.

From their own example "Setting up a Python 3 environment" is best it can do.

The goal is "ticket in JIRA -> solution in production" without human intervention. Right now agentic frameworks are multiplying to bring us closer to that solution like JavaScript frameworks did ~10y ago. We still haven't uncovered the "React" of this space yet, the one that business decides is good enough to standardize on.

> I've been still just like, making VM's with proxmox, then putting my agent in the machine and letting it run free

I outgrew this when I wanted to bring different sets of skills and templates to different machines, wanted to be able to share a small number of credentials, different agents in different machines, different egress rules etc. I wrote https://github.com/pjlsergeant/byre which gives you a TUI and some machinery for doing this easily on top of Docker or Podman.

There's no convergence, but there sure as hell is a lot of pseudo-scientific voodoo and overengineering going on.

I'm stuck on Windows, so oh-my-pi has been really nice. The others I've tried such as kilo do alright but tool calling can mess up a bit.

Only complaint is that connecting the agent harness to my local model took more work getting configured right than I'd like, but that's been true of most harnesses I've tried as well. Most assume you're using a cloud model and local model configuration is a bit of an afterthought.

What are you using Jules for? I want to like it but it fails too often. If I could use Gemini 3.8 I would be happy but 3.6 rarely succeeds.

Give https://juggler.studio a shot if you want a nice desktop UX with all the extensibility of things like Pi

Deepseek Harness is great, at least with deepseek.

Does agy has auto mode?

goose has now native support for local models

Since local models are largest constrained by context window you want to have a tiny system prompt. I know Hax: https://github.com/OleksandrChekhovskyi/hax was designed with local models in mind, but I haven't used it.

pi.

Always Pi.

https://github.com/google/ax/blob/main/docs/concepts.md

>A Model is not a model. It is a named model configuration: ...

Remember kids, a model is not a model.

it feels like the kind of interface a devops engineer who hates AI would design

Fully agreed. Google just can't stop losing.

It is on Google's github https://github.com/google/ax and the title comes from there.

It's got a formal announcement on the blog. 4 months ago.

https://cloud.google.com/blog/products/ai-machine-learning/a...

I have no insider knowledge but https://x.com/rakyll is working on it and she is tweeting about it and I got the impression there is a quite a team behind it. It looks like an effort in GCP.

This /looks/ at least more official. Most unofficial Google projects have a disclaimer in the repo.

the repo description https://github.com/google/ax is "Google's open agentic orchestrator"

>The reality with releases like this is that I'm 90% sure most Google bigwigs have never heard of it

Google has around 200,000 employees. They probably haven't heard of most things Google releases.

I have 6 and ended up needing to use my gaming PC for a build server.

I think you could get by with 1 computer, but it’ll have to have a pretty decent machine.

Between agents running tests, CI, docker image builds, an average $400 mini PC won’t cut it.

Don’t forget also many older mini PCs don’t support KVM. Some newer ones don’t support AVX/ mongodb.

It’s not so easy to buy any old hardware sadly.

I think it will be, but I don't think you need a standalone machine! If you run things inside a VM, you can get safety and control over access and networks

A standalone machine is nice if you need more compute resources or if you want an always-on machine you can connect to from your laptop, phone, etc.

It doesn't look like Google's AX is quite the plug-and-play fit for running agents on a computer you own, since it requires setting up a K8S cluster, etc.

I think what's needed is something like a zero-setup combo of Tailscale and Firecracker

I'm trying to work towards that with my startup (https://github.com/gofixpoint/amika) but the bring-your-own-computer part doesn't work quite yet.

Everything is always better with more YAML, are you perhaps new to this industry?

Next we also need an instruction style guide and CoC. It's important to treat your agents with respect. I almost forgot, the YAML template meta-language to YAML the YAML. Then we will need a foundation employing 12 FTEs to maintain it all and of course to run the certification process. You are certified, right? Statistics show a 10x increased chance of an agent going rogue and hacking competitors if it has been mistreated or been run in an unvalidated sandbox. It goes without saying the sandbox certification process is separate and must be repeated yearly by a trusted third party auditing company.

k8s, but for agents

must look cool for people who want to solve every problem with k8s

it starts with interesting misnomers like "Task" which is not a work item but a sandbox.

"billions of tasks" is a "solution" to problem nobody has (maybe some RL labs? but they solve it other way and with orders of magnitude better optimizations).

freezes design too early – unless they'll actually focus on developing it and make tons of breaking changes it looks shit.

shared state in the same workspace, identity, authority, etc – stuff like that needs to be solved

The best part of all of this is the most if not all people have no idea what the hell is going on when every day a new paradigm/tooling/harness emerges. It's hard to keep up. On the plus side, it's a great equalizer.

Offload the work of an LLM agent to a box in the cloud, so it doesn’t run on your laptop. This has security benefits (no access to your laptop’s files) and you can scale it up (run a lot of agents at the same time).

Then put a “sandbox” around these agents, that word has many meanings. In this case they fence the network traffic, so likely some kind of allowlist for network requests so that the agent doesn’t exfil crap to random websites. They also limit the resource limits of the sandbox, so that is beneficial to the cost of running these agents.

Same. I don't get it.

Not sure if this is connected, but I've found Gemini pretty hopeless within its own notebooks. I've been doing some job applications, and added cv and docs into a notebook. I then create a new chat to say "here is a job description, help me write a cover letter" or some such.

After about 3 messages in any given chat, a follow-up to "rewrite that with a more friendly tone" will result in a letter for a completely different job from another chat within the notebook.

On your Sheets + Gemini integration point, I've genuinely tried to give the Gemini integration into Google Docs & Google Sheets a chance. It is so incompetent that it is fully useless to me. I have not gotten a single correct solution each time I tried to use it, even something I consider table stakes. I often write my work reports in Vim in Markdown format, but they need to go to the corporate Google space. No matter how hard I tried, no matter how many prompts I have, it was completely unable to manage the command to "convert the Markdown format markers into native Google Docs markers". And I want to note, this was 2 pages of extremely simple Markdown with no "advanced" patterns, like tables or quotes, I think all I used was heading-marks, bolding, italicizing, and code blocks. This is something I would expect even GPT 3.5 to succeed in, and even more so Luna, but somehow it destroyed the formatting throughout half the document. This leads me to believe that they apply the absolute cheapest model they have there, or they have the model a harness which can barely be considered working. I found it absurd when I found out that they suddenly made this Gemini integration an additional paid plan recently, there's absolutely no way I can consider that in good faith.

This is an Apache 2.0 open source project

Making Kubernetes a centre of everything. This one, they won’t sunset, because it helps selling GCP services.

I assume they expect agents will be writing most of those

It depends on where they decide to go with it, I guess. Kubernetes, Go, Tensorflow, Chromium, gRPC are some examples that obviously went incredibly well.

Well, they're not above forking their own project to patch security holes and never upstreaming the fixes.

https://grapheneos.social/@GrapheneOS/117282080803799576

> Google should not be gatekeeping security patches to the standard Android platform code from Android OEMs but that's what they've started doing.

Maintain it briefly then slowly let it die.

Didn't you see the example on the website? It can set-up a Python 3 environment. Duh!

I believe it's a stylized lo-fi rendition of an axolotl.

It's most likely an axolotl.

Axolotl

Not sure about billions, but companies doing evals or RL or training will create really big bursty agent workloads. I think they are the best fit for AX, as opposed to individual dev teams building software, etc.

billions? who is running BILLIONS of agents?

tens, hundreds, maybe a couple thousand at a time? absolutely.

you can run it yourself, it's open source, you can use any harness (req. custom image), you can use any token vendor (config)

Google already has gVisor running in Kubernetes as a product (GKE Sandbox), which provides the security guarantees necessary for secure sandboxes (regular k8s isn't great in this respect). They also have pod snapshots running at scale (which run on gVisor), so you can spin up process(es) and snapshot the memory and fs of a pod at a point in time, ship it to a blob in GCS, and then rehydrate those snapshots very quickly (or fork into new instances), which allows for the fast/cheap startup and suspend times and the instant scaling they advertise here. One of these snapshots can be created in one cluster and spun up in another.

Not sure if this is an extension of tech they already have had in their systems, but I've experimenting with it to build my own orchestrator and it's been a pretty neat set of tools and abstractions so far.

Future of platforms is operators in k8s to abstract the developer need to the underlying systems. On local it maps to kvm, on gke it maps to their stuff, on AWS to RDS. It's "interfaces" on a platform level so devs can just ask for a thing.

Overall I agree though, this is a bit of an abuse of that concept.

EDIT: I'm sure op is familiar with this workflow but I'm being overly verbose to clarify what I think they mean and my thoughts.

I can launch Astra to deploy changes to my homelab via creating Forgejo issues from my phone. That kind of system is pretty hard to set up without kubernetes.

Even if you confine yourself to a dev workstation, having 5 agents concurrently building testing deploying code makes your computer loud and/or hot.

Agree. Probably an unpopular opinion, but I strongly dislike YAML.

EDIT: if I HAD to use YAML, I'd prefer KYAML: https://dev.to/mechcloud_academy/goodbye-yaml-hell-meet-kyam...

The OP is not really a workflow manager, it’s a workspace manager that facilitates creating controlled environments where your skills can run. Everything you said is compatible with (and complementary of) the OP project.

With that said, I’ll somewhat disagree with you. I’ve been down the path you’re talking about and while it is incredibly flexible and powerful, it became too difficult to maintain, and too inconsistent between workflow runs, and a pretty hefty waste of tokens to use AI on things that could instead be handled by deterministic scripts. I ended up creating an orchestrator for myself that uses skills as the primary way to tell agents how to execute a step in a workflow, but also directly orchestrates running scripts and managing state in a deterministic way rather than leaving it all up to agents.

How does your criticism relate to the specifics of what OP posted? https://github.com/google/ax/blob/main/docs/concepts.md#work... This says it has skill registries.

You really, really need different skills depending on the model.

If you are using Qwen 27B you need very prescriptive skills.

If you are using Astra you usually want very minimal skills (because it will follow them but be unnecessarily constrained in some contexts)

If you are using Fable then it depends - it will take the skills as general guidelines but ignore them a lot more than Astra does. Sometimes this is good, sometimes not at all.

Restoring memory is useful if, when you resume an agent VM, you want the apps to be in the exact state when they were suspended.

But for most things, I find resuming with memory is more trouble than it's worth. If you always resume from memory, you lose the ability to control the state of a VM. It's much easier to define which services should run than to define which active RAM state should be purged

Similar to why "did you try turning it on and off again?" is good for system reliability.

what about msb? https://github.com/superradcompany/microsandbox

For microvms, take a look at https://smolmachines.com

For microVM, smolvm is quite impressive.

For further isolation, I like to use nono inside a smolvm instance.

K8s jobs don’t run in a secure runtime. K8s jobs don’t give you dynamic zero trust permissions scopes. Restoring a harness in 500ms is really fast, much faster than naively creating a new job downloading session and ‘codex resume’ etc.

> you are opting in to a pretty huge stack that is just Agent Substrate, that is their runners, their harness, their substrate

The website makes me think the contrary: It is described as “low opinion” and explicitly mentions that the running tasks don’t even have to be AI agents. Can you explain in what ways you’re more locked in than the website suggests?

Scion at the same time talks much more about concrete agents, giving me the opposite initial impression.

what are your points of comparison ?

No to confuse with Ax, the DSPy inspired agent framework

https://axllm.dev/

Ah yes, k8s8n.

Dropbox comment