You just get an LLM to do the bounding box stuff or use the ComfyUI node that provides a GUI for bounding box generation
replyI always hated Comfy's node based UI, but agents make it tolerable. Now I just have them set up a workflow and I go in and tweak it manually if the results aren't where I want them. I even have agents cherry doing multiple runs and cherry picking the best outputs, models have gotten good enough that it's a real time saver, assuming you have references they can and a rubric to check against.
reply> I always hated Comfy's node based UI
replyIt's one of the most uniquely hostile user experiences I've ever had the (dis)pleasure of working with
Awhile back I built a project which reads comfy's API and builds a typescript sdk from it. Much nicer to work with and still benefits from node caching and the comfy ecosystem. Perhaps I should clean it up and open source it, though I haven't looked around and there may already be other projects doing this out there.
replyIt's definitely not for me.
replyFrom where I'm sitting, it's just turning python functions into boxes and instead of write the function yourself, you drag from the output of one box to the input of another. For 2 or 3 boxes, this is cool, but I opened up a professional workflow and was taken into a view with 100s of boxes and wires all over the place. Uhh, ok?
For myself, I'd rather just create my own python environment, write some quick pytorch or mlx calls, wire up some cli to it and share that in a GitHub.
I recently found a project[0] that takes a ComfyUI workflow and turns it into a simple UI. I have no affiliation with it and haven't tested it myself, but it looks like it might be handy once you have a finished workflow you want to use.
replyHow do you get agents to set up a workflow? You just get them to modify the JSON directly and then import it, or do you have a tighter integration (e.g. in the UI)?
replyThe agents can interact with Comfy via API pretty well, which afaik ends up being directly with JSON.
replyYeah, given how much better Ideogram v4 outputs are when you use the proper structured JSON (background, elements, etc.) I think most users probably have stuck some kind of Qwen/Gemma-based LLM between their raw prompt and the CLIP encoder.
replyIdeogram 4.5 released 2 days ago with more features along these lines
replyAdobe has had something like this for a while with Generative Fill. Drag a box, specify contents.
reply