Yeah, I agree with that. We have started Kapso for predictive modelling on relational databases before RelArena came out, so Relbench was the only popular public benchmark. We surely try Kapso on it too.
replyTo test it further in live environment (additional to enterprise environment of our customers), we participated with Kapso in AI olympiad this year, and surpassed all human participants, and became top 3 AI systems: https://ioai-official.org/ai-models-track/ioai%c2%b2-2026-re.... In IOAI2, six expert-designed tasks, a single-GPU, and 2 hours budget were given to autonomous agents for deliver solutions without any human intervention.
Woah, now that is extremely impressive.
replyI think Kapso could be used as an _incredible_ resource to AB test internal architectural changes to complex models with many moving parts ...
> Woah, now that is extremely impressive.
I think Kapso could be used as an _incredible_ resource to AB test internal architectural changes to complex models with many moving parts ...
replyI think if a task is iterative and needs experimentation to find the optimal solution, then Kapso can be used there (hopefully helps)!