- 259comments
- 4comments
- 245comments
- 114comments
- 80comments
- 264comments
- 8comments
- 20comments
- 1comments
- 38comments
- 13comments
- 86comments
- 159comments
- 13comments
- 19comments
- 57comments
- 117comments
- 6comments
- 2comments
- 2comments
- 107comments
- 15comments
- 11comments
- 754comments
- 2comments
- 107comments
- 17comments
- 109comments
- 31comments
- 25comments
Just grade the process, not the result. It's literally that simple.
AI can talk the talk, but it can't walk the walk.
A simple "explain why this is the best solution" will make it shit the bed 10 out 10 times.
Thanks for sharing. Interesting to see how teachers are adapting to LLMs. I agree that trying to ban AI use is futile.
I have a fairly long experience of both teaching and grading, even though I stopped a couple of years ago. After a couple of years of trial-and-error, I think I found the "right" way:
* Alternate between theory and practice in the course of a single 2-hours slot. Theory is necessary, but the attention span of people is very limited. * Grade either a homework assignment in the form of a real project with specifications. You give the assignment half-semester, so that students who start early can ask questions. Otherwise, grade a in-session exam, with every resource available, including the course and the whole Internet. However, you don't assess the knowledge, but if the student is able to apply their knowledge to the different tasks at hand.
These approaches are now completely moot in the age of AI.
I was surprised the first time a colleague asked me to do an oral exam. I thought it was a burden on people with poor social interaction skills. It went surprisingly well: you could see in a couple of minutes if the student had integrated the concepts of the course, even if they were shy. Now I wonder if it's the way to go. There's one caveat, though: it doesn't scale.
I was thinking that one of the great ironies of artificial intelligence is that it will make good teaching even more labour intensive. Those kids that can afford the one to one tuition required to push past our natural inclination to be lazy will learn things, while everyone else will succumb to outsourcing all their thinking.
There’s no way I’d have learnt everything I know now about software in today’s environment. So much of software development - debugging techniques, architectural decisions, structuring data etc is learned through trial and error. As the models get better is becoming easier and easier to just not look too closely at their output. I suspect it’s human nature.
Oral exams don't scale at the beginning with hundreds of students per course, agreed. So make them in-person written exams, German universities managed that just fine a few decades ago. Oral exams were reserved to advance from Grundstudium (basic) to Hauptstudium (main).
In my very limited experience (I have German major degrees in Informatik and Philosophy, I was a TA for Logic in Philosophy) the more advanced the topic the thinner the attendance anyway.
The country of Argentina, despite its crippling debt, is able to provide free university education where the majority of classes for most majors are graded with an oral exam.
They've shown that it is entirely possible to scale a system like that, as long as the society sincerely values the role of an educator.
Maybe the Germans were right? One in-person test, preferably an interview, at the end of the semester should be enough to motivate the smarter part of the bunch to make sure they understand the issues at hand. The rest is chaff anyway. Don't optimize for chaff.
This makes me think of exams — I always did well on open-book exams, but I didn't do well on exams where I couldn't look at the book. Whether it's because of ADHD or just a lack of memorization ability.
Similarly, I did well in interviews where I could use the internet, but I didn't do well in interviews where I couldn't use the internet.
Everyone has a testing format they're good at. I'm curious about how this kind of thing is evaluated, and whether there are any papers related to this.
Isn't the answer the 'flipped classroom' - students are assigned reading/studying to do before a class and the class becomes more interactive, answering questions/discussing topics/solving problems etc depending on subject. Of course, this is more expensive than a hall with 400 people listening to a lecture with a couple of tests/essays.
https://fltmag.com/the-flipped-classroom/
https://en.wikipedia.org/wiki/Flipped_classroom
This only works if you either have very intrinsically motivated students or you verify that really everyone has actually read the assigned materials before class. Otherwise, some people will turn up unprepared, you will have to explain stuff they should have gotten from the reading assignment, other students will be annoyed, students will stop preparing for class, vicious cycle.
We still have developers because we can't outsource responsibility to AI. If shit breaks, we still need someone to fix it. Maybe you think that can also be automated. I have a counter-example. AI will rarely use the existing state when solving a bug. It will tack on it's own event queue, it will make up it's own messaging system, it is a cancerous growth. And this observation is based on both opus5.5 and gpt5.6-astra. The results are way past magical at this point, but you still have to weigh in pretty heavily for orderly code growth where solutions are succint, focused and preserve the existing structure. They are still noobs at fixing/adding stuff.
I am teaching some math courses, and I see how LLMs disrupted all standard approaches, and I don't know what to do.
I rely on written exams that are open book, but forbid any use of computers and smartphones in class.
The university insists that homework can't be optional, but it lost its meaning. I've tried to explain to my students that it's in their interest to think about home assignments on their own, but the majority will obviously use gen AI, and it's such a waste of time to check and grade LLM output.
I just had an experience where all students were given "sample problems" to try at home and prepare for the written test. Many of them just dumped the document into an LLM, asked it to produce some "exam guide", and showed up with that thing printed out, asking me during the test to explain what LLM output meant.
It seems like the university also has many people pushing for AI use for everything, but I teach basic stuff where the goal is to make students think on their own and digest some fundamental ideas, LLMs can produce perfect solutions, but relying on them is pointless.
Perhaps some people do not belong at university? It is just a waste of everyone time, to insist 50% of population studies until 25 years old.