Yeah I think this is the main issue at play. Bad engineers are now 10x bad engineers with the ability to hide behind the mask cause they can output something that looks right, but they don't understand it. You can "make" an agent do ANYTHING. There is no rigor, there is no taste, there is no risk. Anecdotally I have seen this play out IRL. I worked with a 3-month boot-camper that somehow stumbled into a software developer role at a small company. The amount of slop that he was creating was unprecedented. No formal training, no idea of abstraction, letting/making the agent reinvent the wheel for everything. The LLM was doing the whole "you're absolutely right" so he was also under the impression that he was a real engineer doing the "right thing". Concretely he convinced our manager that we need to re-write our web-app in Go because it was so much faster(0 users btw) and safer than python. He had an agent build out this test suite and there was a metric like Go vs Python DB operations, Go being 107x better. Seemed sus, looked into it and he(his agent) removed connection pooling for the python side but for the Go side, 100 concurrent connections, 10 idle and a connection lifespan of 1 hour. His reason for that was that the python code kept crashing every time his agent adding pooling so he had the agent removed it. This sheet got printed as is and distributed to our C-suite to much praise. I know this is a very specific example but I fear this is happening at some level everywhere. Where in the past a bad engineer to this degree would stick out like a sore thumb if they were surrounded by competent people that have rigorous standards. An LLM/AI doesn't have to worry about what happens if things don't work and when it's confirming that YOU are a genius every step of the way. It's going to be a disaster when the reckoning comes. I am not a AI luddite but software engineering is just as much as a culture as it is art and science. Without that backstop I just think that we will see a huge degradation in quality because there is no incentive for LLMs/AI to write "good software".

Sure theoretically. practically this isn't a real problem for me or anyone else i know

You don't give it tasks. You give it what you want. It decides the tasks, and then executes. Then you give it feedback. "black-box approach" here is just a fancy word for vibe coding.