> I find a ton of variability in this interview and a lot of people will just take the entire ticket and one shot into Claude without prepping. This is lead to the biggest disparity in results of candidates so we stopped doing it.

Agentic engineering is the new microservices craze. Everyone's doing it, and some will do it exceptionally badly. This is always how these trends go. The anecdote you give here of some people just one shotting it in Claude Code without thinking about it and this producing extreme variability matches my experience. Basically, the longer you go down this road you realise some people will still actually think about things and some people will just cede cognitive control and stop thinking much at all. If you're the former, working with the latter is exceptionally draining over the long-term. I have in recent times become pretty bearish on agentic engineering for this reason.

> The first was a traditional interview problem where we give the candidate a codebase and a docker image to run against

Is that also on a company provided computer?

There are a lot more scams these days where a job-seeker compromises their computer by running provided code, and often docker isn't much against malice.

[dead]