upvote
Yes, and we haven't even really begun to nail down computer-use agents yet (can you believe they're still basically just OCR'ing screenshots?)

Once we have something that experiences a desktop interface more like a human does, an entire swathe of tooling that has heretofore been nigh-impossible to automate moves into the fold, and that'll be another explosion of folks finally getting to join the agentic workflow world on their industry specific apps...

reply
How do you think humans experience desktop interfaces? “Basically just OCR'ing screenshots” is exactly what humans do.
reply
It's not the same thing. For example, given a GUI with a titlebar, title, subtitle, text, and buttons, a human can instantly understand spatially the relationship between these items. But a naive OCR of such a GUI would be a flat stream of text that loses a ton of information.
reply
But that’s not how models handle images either. They spatially segment and reason about title bars, placement, etc.
reply
I was also under the impression modern AI agents have moved on from just OCR'ing screenshots to leveraging native vision model capabilities.
reply
They do. They all use ViTs and have for quite a while.
reply
Can you share what's your setup for all that orchestration? I feel way behind just asking Claude Code for code edits. Is there any site where people share different AI setups, besides youtube?
reply
Fwiw, don't buy into all the hype that you're falling behind. Yes, AI does cool things now, but I would say the impact is still unproven past indie hackers or early-stage startups. And a lot of the esoteric setups people have created with things like OpenClaw have become outdated as quickly as they were conceived.

The popular thing is now to setup loops (eg I setup hourly integrations for Claude/Codex to 1) scrape my Linear, claim achievable tasks, and push PRs or 2) do root cause analysis on customer issues that evaded automated filters, to name a few)

Though for me, my setup still feels mundane. I have AGENTS.md, CLAUDE.md etc and a few skill files. These are purposefully light - tons of examples online you can pull from online. Mine are fairly personal to my setup and products.

Importantly, I also allow Claude and Codex to bypass permissions. Yes, there is a risk they wipe my machine. The productivity upside has been worth it, for me (haven't been burned yet, ~9+ months into running models this way, I have backups, use cloud etc).

As far as maintaining quality, one of the most helpful guardrails over the past year, for me, has been requiring my agents to pipe their changes to local reviewers through OpenCode, Cursor, etc agents to have a council of models with different biases reviewing the changes, and autonomously working towards a completed objective. No matter how good Claude or Codex gets, for example, I will probably always want a different model checking its work. Like GLM, (now with 4.5) Grok, Composer.

Several OpenAI, Anthropic, xAI employees, and popular AI engineers post on X and share helpful tips & updates. Highly recommend for keeping a pulse on startups and AI. I haven't found something close, honestly, other than when I spend time in SF talking to people.

reply
I always thought bootstrap was pretty good. All the gradients and sparkles don't do much for me.
reply
without taking down prod even 1% of the time

Literally every major company that has embraced AI coding has suffered devastating downtime this year as a direct result of AI induced failures.

reply
Wasn't writing about major companies. That's obviously next, if we follow the trend lines.
reply
Most companies have already given up on AI. It was a bigger disappointment than big data.

The only companies sticking with AI at this point are the major players who have chasing their masssive overinvestments to the bitter end, crappy coding shops, and for some reason Starbucks and they'll all suffering badly in a year their customers begin demanding massive amounts of human coding to repair all the issues with the AI code.

reply
We're still in the early adoption phase. Betting against the internet wasn't a great idea, and betting against AI doesn't look like one either ;)
reply