How could we "force" the students to use an LLM that confronted their doubts with more questions? We could tell them to start each chat with a specific prompt (to use the socratic method, etc), but they could eventually jail-break it..
But nevertheless, I like your idea! This is something that a document highlighting methodologies for students on how to use LLMs effectively could/should contain..
As an undergrad, I hope schools move to educating students to use LLMs in a more responsible way. You can't put the genie back in the bottle, and resisting progress is futile, might as well use the tool we now have to help students learn even faster and better (e.g., making feedback instant and not answers, helping digest or split up material, checking answers).
I know opinions about AI at (not only at) my faculty are very mixed, but I think the answer is going to be in the rational mean, just like how technorealism reacted to the internet[0].
In our last program board sitting, some teachers said that they think programming as a job will be completely irrelevant in two years, while other pushed for more adoption. And meanwhile I know of some students that are basically only passing because of LLMs, and it's bad, like "leaving claude output in markdown files and finished source code on the faculty server in /tmp because opencode did so" bad. And our first year classes completely prohibit even sharing tests or talking about the solutions, which in my opinion a) makes people extremely asocial and atomized b) doesn't prepare students for real life c) promotes dishonesty.
Still, I think our university's thinking is in a stalemate, not wanting pure AI output and useless students, while also wanting to move with the times, and I doubt it's the only one.
[0]: https://web.archive.org/web/20081009111415/https://artefaktu... (absolutely amazing read, recommend it)
Sounds more like the score is 3/3 (100%)
Would you have accepted them cooy-pasting code from libraries together to build their project? If not, why is using LLM generated code different?
Yes, if they are "responsible" for the code delivered, where responsible means they understand the code, the architecture, the decisions made, etc.
In this case, the students had to invent multiple strategies to solve a specific problem. The "successful" groups did a mix of generated and hand-crafted code (don't know percentages), implemented different strategies and knew their plus and minuses, could change the code in a timely manner to accommodate some of my requests, etc. The "unsuccessful" group couldn't do any of that.
I'm not anti-AI (and really, what could I do if I were?) since I use it myself, I'm just anti-slop, especially from my students.
But in reality I've been slowly transitioning from group projects (for a subset of the grade) to "practical tests", where they must implement a significant subset of a larger project in a 2h class. Still experimenting though.
It was fine.
This is a good principle to maintain, I think.
I'm not a professor, but I manage a team of about a dozen people. The maxim I have is: "You're responsible for anything that hits git."
Don't care if the LLM generated it, or the LLM told you if it's a good idea. If you commit it, you are endorsing it as a good idea - so you're the one I'm going to ask about it. I see the same principle at work in your pedagogy.
> I'm not anti-AI (and really, what could I do if I were?) since I use it myself, I'm just anti-slop, especially from my students.
This hits. Especially this part:
> and really, what could I do if I were?
My completely unsolicited opinion: you're doing a responsible thing by teaching these students how to use AI as a reference, and keeping them honest about not using it as a substitute for their own critical thinking.
I have colleagues that are teaching for more than 30 years, few years away from retirement, who suddenly have been confronted with a new way of doing things. Those are the ones that are still insisting on doing practical projects, etc. I've only been doing this for 20 years, and I'm quite lazy (worked previously as software engineer), so I've moved to those practical tests. I guess that there should probably exist a class or workshop to teach these students how to use LLMs effectively, but as I said, this technology and its implications is quite new.
Personally, what I did was to give them the "lecture" in the line of that they do not understand what the machine has generated, that is not the way a true engineer does, try to do some parallel with things like an LLM designing a bridge and civil engineers building that bridge, and a fatal flaw collapsing the all thing, etc.
In other words, we do not have a formal system in place, it's all talking and convincing them. Obviously it's a big enough problem that should deserve more investment in solutions, but we are all overwhelmed by other tasks. Maybe LLM studios should be held responsible for all these "disruptions" and provide solutions to problems they created! :)