I teach computer science / programming, and I don't know what a good AI policy is.
On the one hand, I use AI extensively for my own learning, and it's helping me a lot.
On the other hand, it gets work done quickly and poorly.
Students mistake mandatory assignments for something they have to overcome as effortlessly as possible. Once they're past this hurdle, they can mind their own business again. To them, AI is not a tutor, but a homework solver.
I can't ask them to not use computers.
I can't ask them to write in a language I made the compiler for that doesn't exist anywhere, since I teach at a (pre-university) level where that kind of skill transfer doesn't reliably occur.
So far we do project work and oral exams: Project work because it relies on cooperation and the assignment and evaluation is open-ended: There's no singular task description that can be plotted into an LLM. Oral exams because it becomes obvious how skilled they are, how deep their knowledge is.
But every year a small handful of dum-dums made it all the way to exam without having connected two dots, and I have to fail them and tell them that the three semesters they have wasted so far without any teachers calling their bullshit is a waste of life and won't lead them to a meaningful existence as a professional programmer.
Teaching Linux basics doesn't suffer the same because the exam-preparing exercise is typing things into a terminal, and LLMs still don't generally have API access to terminals.
Maybe providing the IDE online and observing copy-paste is a way forward. I just don't like the tendency that students can't run software on their own computers.
I'm not that old, and yet my university CS courses evaluated people with group projects, and in-person paper exams. We weren't allowed to bring computers or calculators into the exam room (or at least, not any calculators with programming or memory). It was fine.
I don't see why this is so hard, other than the usual intergenerational whining / a heaping pile of student entitlement.
If anything, the classes that required extensive paper-writing for evaluation are the ones that seem to be in trouble to me. I guess we're back to oral exams and blue books for those, but again...worked fine for prior generations.
> and in-person paper exams.
Yup. ~25 years ago competitions / NOI / leet_coding as they call it now were in a proctored room, computers with no internet access, just plain old borland c, a few problems and 3h of typing. All the uni exams were pen & paper. C++ OOP on paper was fun, but iirc the scoring was pretty lax (i.e. minor typos were usually ignored).
> I don't see why this is so hard, other than the usual intergenerational whining / a heaping pile of student entitlement.
You know that grading paper exams is a lot more hassle _for the teachers_?
Your overall point might or might not still stand. I'm just responding to your 'I don't see why this is so hard'. Show some imagination for why other people hold their positions.
(I'm sure there's lots of other factors that come into play that I am not thinking of here.)
...and yet, somehow we managed?
> Show some imagination for why other people hold their positions.
I say that as someone who has also graded piles of paper exams in graduate school (also not that long ago!)
I don't believe the argument you are making is true, but if the primary objection really is that teachers have to grade, then no, I don't have any sympathy.
It sorta depends on the material… I always thought paper programming tests were dumb: when I was taking them and when I was proctoring/grading them. It is not that similar to writing a program in an IDE where it will tell you if you make a little mistake, and often help you work your way through it.
We made it. But, that’s survivorship bias, right? We can’t really know how much potential has wasted.
Hear, hear!
Doing programming on paper seems to me like assessing someone's skills in acrobatics by watching them do the motions in a zero-gravity environment. Without the affordances given by the computer, it's just not the same activity.
Computer science, the academic discipline, is to programming as mechanics is to bowling.
You can very easily test CS concepts on paper, and programming is demonstrated via group projects.
Absolutely. It makes good sense to describe algorithms on paper via pseudo-code and diagrams, but they shouldn't be expected to write working code on paper.
I kinda had this sentiment until I actually started working - quite often an issue only manifests at an obscure customer system or is a race condition that it too rare to catch reliably, yet happens often enough so you can't just ignore it.
To solve those in a reasonable amount of time, you need to form a mental model of what is going on & how to fix it. Having access to a computer by itself won't really help for those.
In that context paper exams for computer science make much more sense to me now - they want you to understand the problem and provide a solution, with pen and paper being the output format.
> ...and yet, somehow we managed?
People in the past put up with all kinds of struggles. They had to.
> I don't believe the argument you are making is true, but if the primary objection really is that teachers have to grade, then no, I don't have any sympathy.
I have no clue what the primary objection really is. I was responding to "I don't see why this is so hard", which just shows a lack of imagination.
You’re making it seem those guys worked the fields 14 hours straight. It’s just some paperwork..
Why can't the teachers use LLMs to grade?
Ah, the eternal dream of offloading all human labor to machines. Why can't teachers just let an LLM grade? Because, of course, nothing says "educational integrity" like a glorified autocomplete deciding whether little Timmy's essay on Shakespeare adequately captures the existential dread of Hamlet. Sure, let's trust a model that hallucinates citations. But fine, if we're really committed to stripping all nuance from education, why stop there? Let's just plug students into Anki's FSRS algorithm and call it a day. Just assign grades based on how fast their retention decays, because nothing says "holistic assessment" like reducing a human being to a set of coefficients in a spaced repetition formula. Never mind that actual learning involves things like critical thinking or, heaven forbid, creativity. No, no, we'll just reduce the entire process to a forgetting curve. Because nothing inspires a love of knowledge like treating human minds as poorly optimized flashcard decks, mechanically processed and discarded the moment their retention scores dip below acceptable thresholds.
In general – why I'd put my effort into visiting (and paying for) a school and learning in such case? That's not what schools are for. I can get any amount of grades I want from LLM myself.
Grading is already mechanical, it's just a human does it. I'm not sure what you're objecting to here.
Might be interesting. You can at least use modern AI to turn scans of hand-scrawled-on paper into something readable.
Thing is, this hits the scaling problem in education and fucking hard.
There’s such a shortfall of teachers globally, and the role is a public good, so it’s constantly underpaid.
And if you are good - why would you teach ? You’d get paid to just take advantage of your skills.
And now we have a tool that makes it impossible to know if you have taught anyone because they can pass your exams.
I'm not too old either and in my university, CS was my major, we did group projects and in person paper exams as well.
We wrote c++ on paper for some questions and were graded on it. Ofcourse the tutors were lenient on the syntax they cared about the algorithm and the data structures not so much for the code. They did test syntax knowledge as well but more in code reasoning segments, i.e questions like what's the value of a after these two statements or after this loop is run.
We also had exams in the lab with computers disconnected from the internet. I don't remember the details of the grading but essentially the teaching team was in the room and pretty much scored us then and there.
> Students mistake mandatory assignments for something they have to overcome as effortlessly as possible.
It has been interesting to see this idea propagate throughout online spaces like Hacker News, too. Even before LLMs, the topic of cheating always drew a strangely large number of pro-cheating comments from people arguing that college is useless, a degree is just a piece of paper, knowledge learned in classes is worthless, and therefore cheating is a rational decision.
Meanwhile, whenever I’ve done hiring or internships screens for college students it’s trivial to see which students are actually learning the material and which ones treat every stage of their academic and career as a game they need to talk their way through while avoiding the hard questions.
I teach computer science / programming, and I know what a good AI policy is: No AI.
(Dramatic. AI is fine for upper-division courses, maybe. Absolutely no use for it in introductory courses.)
Our school converted a computer lab to a programming lab. Computers in the lab have editors/compilers/interpreters, and whitelist documentation, plus an internal server for grading and submission. No internet access otherwise. We've used it for one course so far with good results, and and extending it to more courses in the fall.
An upside: our exams are now auto-graded (professors are happy) and students get to compile/run/test code on exams (students are happy).
>Students mistake mandatory assignments for something they have to overcome as effortlessly as possible.
This is the real demon to vanquish. We're approaching course design differently now (a work in progress) to tie coding exams in the lab to the homework, so that solving the homework (worth a pittance of the grade) is direct preparation for the exam (the lion's share of the grade).
> Our school converted a computer lab to a programming lab. Computers in the lab have editors/compilers/interpreters, and whitelist documentation, plus an internal server for grading and submission. No internet access otherwise. We've used it for one course so far with good results, and and extending it to more courses in the fall.
Excellent approach. It requires a big buy-in from the school.
Thanks for suggesting it.
I'm doing something for one kind of assignment inspired by the game "bashcrawl" where you have to learn Linux commands through an adventure-style game. I'm bundling it in a container and letting you submit your progress via curl commands, so that you pass after having run a certain set of commands. Trying to make the levels unskippable by using tarballs. Essentially, if you can break the game instead of beating it honestly, you get a passing grade, too.
>Our school converted a computer lab to a programming lab. Computers in the lab have editors/compilers/interpreters, and whitelist documentation, plus an internal server for grading and submission. No internet access otherwise. We've used it for one course so far with good results, and extending it to more courses in the fall.
As an higher education (university) IT admin who is responsible for the CS program's computer labs and is also enrolled in this CS program, I would love to hear more about this setup, please & thank you. As recently as last semester, CS professors have been doing pen'n paper exams and group projects. This setup sounds great!
We've been doing this at Illinois for 10 years now. Here's the website with a description of the facility: https://cbtf.illinois.edu/. My colleagues have also published multiple papers on the testing center—operations, policies, results, and so on.
It's a complete game changer for assessment—anything, really, but basic programming skills in particular. At this point I wouldn't teach without it.
Isn't auto-grading cheating by the instructors? Isn't part of their job providing their expert feedback by actually reading the code the students have generating and providing feedback and suggestions for improvement even at for exams? A good educational program treats exams as learning opportunities, not just evaluations.
So if the professors can cheat and they're happy about having to do less teaching work, thereby giving the students a lower-quality educational experience, why shouldn't the students just get an LLM to write code that passes the auto-grader's checks? Then everyone's happy - the administration is getting the tuition, the professors don't have to grade or give feedback individually, and the students can finish their assignments in half an hour instead of having to stay up all night. Win win win!
Immediate feedback from a good autograder provides a much more interactive learning experience for students. They are able to face and correct their mistakes in real time until they arrive at a correct solution. That's a real learning opportunity.
The value of educational feedback drops rapidly as time passes. If a student receives immediate feedback and the opportunity to try again, they are much more likely to continue attempting to solve the problem. Autograders can support both; humans, neither. It typically takes hours or days to manually grade code just once. By that point students are unlikely to pay much attention to the feedback, and the considerable expense of human grading makes it unlikely that they are able to try again. That's just evaluation.
And the idea that instructors of computer science courses are in a position to provide "expert feedback" is very questionable. Most CS faculty don't create or maintain software. Grading is usually done by either research-focused Ph.D. students or undergraduates with barely more experience than the students they are evaluating.
> Isn't auto-grading cheating by the instructors?
Certainly not. There's a misconception at play here.
Once you have graded a few thousand assignments, you realize that people make the same mistakes. You think "I could do a really good write-up for the next student to make this mistake," and so you do and you save it as a snippet, and soon enough, 90% of your feedback are elaborate snippets. Once in a while you realize someone makes a new mistake, and it deserves another elaborate snippet. Some snippets don't generalise. That's called personal feedback. Other snippets generalise insanely. That's called being efficient.
Students don't care if their neighbors got the same feedback if the feedback applies well and is excellent. The difficult part is making that feedback apply well. A human robot will make that job better. And building a bot that gives the right feedback based on patterns is... actually a lot of work, even compared to copy-pasting snippets thousands of times.
But if you repeat an exercise enough times, it may be worth it.
Students are incentivised to put in the work in order to learn. Students cannot learn by copy-pasting from LLMs.
Instructors are incentivised to put in the work in order to provide authentic, valuable feedback. Instructors can provide that by repeating their best feedback when applicable. If instructors fed assignments to an LLM and said "give feedback", that'd be in the category of bullshit behavior we're criticising students for.
Is it in a Faraday cage too or do you just confiscate their phones. Or do you naively believe they aren't just using AI on their phones?
You should look at the cheating stories that come out of India, China, South Korea and other places that have been dealing with this dynamic for decades upon decades.
I know of a time where america didn’t have this problem and I could see it ramping up, because of my experience in India.
People will spend incredible efforts to cheat.
Like stories of parents or conspirators scaling buildings to whisper answers to students from windows.
I don't know what they do, but when we did it back in the 2000's there was a no phone policy and the exams were proctored.
People could try to cheat, but it would be pretty stupid to think they would not catch you.
> Oral exams because it becomes obvious how skilled they are, how deep their knowledge is.
Assuming you have access to a computer lab, have you considered requiring in-class programming exercises, regularly? Those could be a good way of checking actual skills.
> Maybe providing the IDE online and observing copy-paste is a way forward. I just don't like the tendency that students can't run software on their own computers.
And you'll frustrate the handful of students who know what they're doing and want to use a programmer's editor. I know that I wouldn't have wanted to type a large pile of code into a web anything.
> I know that I wouldn't have wanted to type a large pile of code into a web anything.
I might not have liked that, but I sure would have liked to see my useless classmates being forced to learn without cheating.
You can provide vscode, vim and emacs all in some web interface, and those are plenty good enough for those use cases. Choosing the plugin list for each would also be a good bikeshedding exercise for the department.
Even IntelliJ has gateway
> Even IntelliJ has gateway
By IntelliJ's own (on-machine) standards, Gateway is crap. I use the vi emulation mode (using ideavim) and the damn thing gets out of sync unless you type at like 20wpm or something. Then it tries to rollback whatever you type until you restart it and retry. I can't believe it is made by the same Jetbrains known for their excellent software.
> But every year a small handful of dum-dums made it all the way to exam without having connected two dots, and I have to fail them and tell them that the three semesters they have wasted so far without any teachers calling their bullshit is a waste of life
Wow.
Yeah, I've had teachers like that, who tell you that you're a "waste of life" and "what are you doing here?" and "you're dumb", so motivational.
I guess this "tough love" attitude helps for some people? But I think mostly it's just that people think it works for _other_ people, but rarely people think that this works when applied to themselves.
Like, imagine the school administration walking up to this teacher and saying "hey dum dum, you're failing too many students and the time you've spent teaching them is a waste of life."
Many teachers seem to think that students go to school/university because they're genuinely interested in motivated. But more often then not, they're there because of societal pressure, because they know they need a degree to have any kind of decent living standard, and because their parents told them to. Yeah you can call them names, call them lazy or whatever, but that's kinda like pointing at poor people and saying they should invest more.
> Yeah, I've had teachers like that, who tell you that you're a "waste of life" and "what are you doing here?" and "you're dumb", so motivational.
I'm sure GP isn't calling them dum-dum to their face. If they can't even do basic stuff, which seems to be their criteria here for the name calling, maybe a politely given reality-check isn't that bad. Some will wake up to the gravity of their situation and put in the hard work and surprise their teacher.
> Yeah you can call them names, call them lazy or whatever, but that's kinda like pointing at poor people and saying they should invest more.
They _should_ invest more because in this case, the "investment' is something that the curriculum simply demands - dedication and effort. I mean unless one is a genius, since when that demand is unreasonable? You want to work with people who got their degree without knowing their shit? (not saying that everyone who doesn't have a degree isn't knowledgeable - I've worked with very smart self-taught people).
> I'm sure GP isn't calling them dum-dum to their face. If they can't even do basic stuff, which seems to be their criteria here for the name calling, maybe a politely given reality-check isn't that bad. Some will wake up to the gravity of their situation and put in the hard work and surprise their teacher.
I certainly am not out to hurt anyone. I have a great deal of sympathy for someone who spent 18 months learning absolutely nothing, hiding behind their study group, making the LLM do the work, and apathetic teachers on prior semesters who could have caught this behavior.
But I will be blunt with them and say: "You have a very limited time before your studies end, and you have so far not learned basic programming. This means you will not be able to use your certificate for anything."
And then I will send an email to them, the teacher they will have on the next semester, and the study councillor, and say that this person urgently needs a better group and extra help. And I'll personally follow up after some weeks to see if they're actually preparing for their re-exam in a constructive way.
It usually came this far because giving a poor student the lowest grade rather than failing them resolves the teacher of paperwork and re-exams, which means they might get a day off.
Being blunt is being kind.
When I was a student, my teachers basically didn't care; I may have not liked it then due to immaturity, but a blunt teacher would have been more beneficial to me than the apathetic ones I got.
I have a hard time sympathizing with a student who cheated for 3 semesters then hit a brick wall when they finally can’t cheat. A student struggling with the material is one thing but a student finally getting caught after cheating through three semesters is another. “Dum dum” is being kind IMO.
I'm not using the word dum-dum to their face.
And when you see them several times a week for a semester, the sympathy grows on me at least. The people who don't show up before the exam and fail miserably, they're a little harder to sympathise with.
In response to the statement
> ...I have to fail them and tell them that the three semesters they have wasted so far without any teachers calling their bullshit is a waste of life
you said:
> Yeah, I've had teachers like that, who tell you that you're a "waste of life"...
You'd do well to slow down and re-read more carefully whenever you find something that offends you. You've failed to correctly identify the target of the "waste of life" commentary.
Moreover, were I paying tens of thousands of dollars a year for a self-improvement product, I'd be furious if the folks operating that product failed to notify me until three years into a four year program that I'd been getting next-to-nothing from the program. That's the sort of information you need right away, rather than thirty, sixty, (or more) thousand dollars in.
You are misrepresenting what the original poster said. He did not say that he actually called kids "dum-dums" or that the kids were, themselves, a waste of life. He said that using AI to blast through assignments without learning anything from them was a waste of life.
Frankly I applaud that approach. Classes are to convey knowledge, even if the student only gives a shit about the diploma at the end of the road. At least someone cares enough to tell these students the truth about where that approach is going to take them in life.
> I use AI extensively for my own learning, and it's helping me a lot. On the other hand, it gets work done quickly and poorly.
> small handful of dum-dums made it all the way to exam without having connected two dots, and I have to fail them ... won't lead them to a meaningful existence
I don't see a problem, the system is working.
The same group of people that are going to loose their job to an LLM arent getting smarter because of how they are using LLM's.
Ideally the system would encourage those dum-dums to realize they need to change their ways before they're screwed. Unless the system working is that people get screwed and cause problems for the rest of society.
I want to agree with your point, but also: someone who's middle-class enough to make it to uni in the first place won't cause much trouble for society.
Paternalism in the sense of 'we know what's better for you than you do' is perhaps justified for those people who really don't know better. But I don't think we should overextend that notion.
Well given that the article is about young people in schools, a little paternalism isn’t a bad thing.
Well, they also have actual parents.
I have to apologise, I was under this impression this thread was about university students, who should be old enough to fend for themselves (and enjoy respectively suffer from the consequences of their own actions). But I don't think anyone actually mentioned that age in the thread. I mixed it up with another one.
The students I teach are pre-university. It's called business school, and if a BSc is level 7, MSc is level 8 and PhD is level 9, then this is level 5. So they can become good programmers, but there's no math in the whole study programme.
> The same group of people that are going to loose their job to an LLM arent getting smarter because of how they are using LLM's.
Students who use LLMs and professional programmers who use LLMs: I wouldn't say it's necessarily the same group of people.
Sure, their incentives are the same, and they're equally unlikely to maintain jobs in the future.
But students can be told that their approach to become AI secretaries isn't going to pan out. They're not actively sacrificing a career because they're out of options. They can still learn valuable skills, because what they were taught has not been made redundant yet, unlike mediocre programmers who can only just compete with LLM gunk.
Programming with AI is the job now. That’s what you need to be teaching if you want your graduates to get a job programming.
What’s changed is that “some working code” is no longer proof that a student understands the material.
You’re going to need a new way to identify students that understand the material.
There really are two opposite policies at play:
- Just say no to AI
- Just embrace AI
I learned to program computers by reverse engineering. I don’t have a CS degree. I wonder if that’s what we need to teach kids now. The AI did this, but why? If the curriculum doesn’t change, then kids aren’t going to learn anything useful, because the AI can do the curriculum.
When I was studying games programming we used an in house framework developed by the lecturers for OGRE.
At the time it was optional, but I get the feeling that if they still use that framework, it just became mandatory, because it has no internet facing documentation.
That said, I imagine they might have chucked it in for Unity before AI hit, in which case they are largely out of luck.
>But every year a small handful of dum-dums made it all the way to exam without having connected two dots, and I have to fail them and tell them that the three semesters they have wasted so far without any teachers calling their bullshit is a waste of life and won't lead them to a meaningful existence as a professional programmer.
This happened to me with my 3d maths class, and I was able to power through a second run. But I am not sure I learned anything super meaningful, other than I should have been cramming better.
If there is another course where students design their own programming language, maybe you could use the best of the previous year's. That way LLMs are unlikely to be able to (easily) produce correct syntax. Just a thought from someone who teaches in a totally different neck of the mathematical/computational woods.
Modern LLMs can one-shot code in a totally new language, if you provide the language manual. And you have to provide the language manual, because otherwise how can the students learn the language.
I had numerous in person paper exams in CS (2009 - 2013) where we had to not only pseudo code an algorithm from a description, but also do the reverse of saying/describing what a chunk of pseudo code would do.
There you go. Actually that would be a great service, wouldn't it? Having them explain to an LLM what they are doing, out loud, while doing it, online. On a site that you trust to host it.
> Teaching Linux basics doesn't suffer the same because the exam-preparing exercise is typing things into a terminal, and LLMs still don't generally have API access to terminals.
Huh, fighting my way through a Linux CLI is exactly the kind of thing I use Chatgpt for professionally.
I did study it in compsci, but those commands are inherently not memorable.
Yes, LLMs have had API access to terminals for quite a while now. I've been using Windsurf and Claude Code to type terminal commands for me for a long while (and `gh copilot suggest` before that) and couldn't be happier. I still manually review most of them before approving, but I've seen that the chances of the AI getting an advanced incantation right on the first try are much higher than mine, and I haven't yet once had it make a disastrous one, while that's happened to me quite a few times with commands I typed on my own.
>and I haven't yet once had it make a disastrous one
I've had it make some pretty bad ones. Not directly hooked in to my terminal, just copy and paste. A couple of git doozies that lost my work, but I've done those too. Others more subtle, one of note is a ZFS ZPOOL creation script it gave me used classic linux style /dev/sda style drive identifiers instead of proper /dev/by-id paths which led to the disks being marked as failed every time I rebooted. Sure, that's on me for not verifying, but I was a little out of my depth with ZFS on Linux and thought that ZFS' own internal uuid scheme was handling it.
Oh, I just copy paste back and forth.
Often I just paste an error with some scroll back but no instructions and it works out what I need.
> their bullshit is a waste of life and won't lead them to a meaningful existence as a professional programmer
That's where you're wrong. Being a professional programmer is 10% programming, 40% office politics, and 50% project management. If your student managed to get halfway through college without any actual programming skills, they're perfect candidate, because they clearly own the 90% of skills needed to be a professional programmer.
> Being a professional programmer is 10% programming, 40% office politics, and 50% project management.
I'd say that really depends on your job.
At smaller companies, your job will likely be 60% programming at a minimum.
Only at ~100 employees do companies fall into lots of meetings and politics.
Hence my personal policy of never working for a company with more than ~100-150 people.
In my experience, it's 70% programming, 20% office politics, and 10% project management. People who realize late they're no good at programming, or don't enjoy it, will pivot towards other kinds of work, like project management. But people who think they'll have luck managing people without having any grasp of the skill set of the people they manage, they either need really good people skills, or they're obnoxiously incompetent in both humans and computers.
Do you find that thinking of your students as dum-dums makes you a better teacher?
Neither better nor worse.
Some of my students are naturally talented.
Others achieve great results through hard work.
Some half-assedly make it.
And some don't even try.
Those are the dum-dums.
They just play games and think everything is going to work out without effort.
> it gets work done quickly and poorly
This is only temporary. It will be able to code like anyone in time. The only way around this will be coding in-person, but only in elementary courses. Everyone in business will be using AI to code, so that will be the way in most university courses as well.
IMO no amount of AI should be used during an undergrad education, but I can see how people would react more strongly to its use in these intro to programming courses. I don't think there's as much of an issue with using it to churn out some C for an operating systems course or whatever. The main issue with it in programming education is when learning rudiments of programming IS the point of the course. Same with using to it crank out essays for freshman English courses. These courses are designed to introduce fundamental raw skills that everything else builds on. Someone's ability to write good code isn't as big a deal for classes in OS, algs, compilers, ML, etc., as the main concepts of those courses are.
It already can. Im flabbergasted how people haven't still figured out how good gemini 2.5 is.
Claude 3.7 and 4 are better for me than Gemini 2.5 for vibing with legacy code. Gemini 2.5 has some great solutions if you handhold it, but tends to make too many assumptions about what would be better which can tear things up as an agent, imo. In other words, Gemini is smarter, but less practical when working with existing code, from what I’ve experienced. To each their own, though.
The Claudes are a lot worse at even mildly challenging algorithmic problems than Gemini 2.5 Pro.
However, most legacy code is fairly primitive on that level, so my observation is in no way contradicting yours.