> I expect the system will only catch a small fraction of the cheating that occurs.
The main kind of cheating we need them to prevent is effective cheating - the kind that can meaningfully improve the cheater's score.
Requiring cheaters to put their belongings in a locker, using proctor-provided resources, and being monitored in a proctor-provided room puts substantial limits on effective cheating. That's pretty much the minimum that any proctor does.
It may not stop 100% of effective cheating 100% of the time, but it would make a tremendous impact in eliminating LLM-based cheating.
If you're worried about corrupt proctors, that's another matter. National brands that are both self- and externally-policed and depend on a good reputation to drive business from universities would help.
With this system, I expect that it would not take much to avoid almost all the important cheating that now occurs.
Remote proctoring programs at least are pretty rough these days. Their environment conditions are pretty exacting and then they expect you to just stare at the screen and think for basically the whole exam. Minor normal webcam problems can invalidate the entire exam through no fault of the examee or if you look around or fidget a lot it can trigger their cheat detection as well. I'm glad I finished my test taking time before it became the norm.
I had to retake a multi-hour proctored test (and only got to do so after a ridiculous amount of back and forth with the school) because my cat jumped up on my computer table while I was taking it, and I looked over at her and gave her a few pets before looking back at the screen. Not joking in the least. It was maddening.
What do they do if you don't have a webcam? Or if your webcam is broken? Or if you don't feel comfortable sharing your video?
They tell you to come back when you’re ready to take the test. This can’t be surprising…
Most institutions use a remote test proctoring service. Before the test begins, the service verify all the requirements are met.
Anything short of all the requirements and they do not administer the test. In the case something is broken, there is usually a window rather than a specific time when you can take the test, so you have an opportunity to resolve it.
If a webcam is required, students are informed about this requirement before enrolling in the institution, and are usually required to purchase hardware capable of doing this as a condition of enrollment.
You get a working web cam. It's a requirement for many remote proctoring services and if you don't have access to one you're screwed.
I get why they use it, without it there's no way to know you're not on your phone or another device cheating since they can only really see what's on the device you've installed the proctor software/rootkit on.
Sadly Linus Tech Tips video of him taking the CompTIA A+ exam has been taken down after threatening letters from CompTIA but they demanded a basically baren room, 360 photos and spotless web cams.
Webcam is broken is now pretty universally interpreted as I don’t want to be on video.
But it also only catches cheating on exams. For homework/projects, you can't really have that be in person.
My take:
- Make “homework” ungraded. Many college classes already do this, and it has been easy to cheat on it way before AI by sharing solutions. Knowledge is better measured in exams and competence in projects. My understanding is that homework is essentially just practice for exams, and it’s only graded so students don’t skip it then fail the exams; but presumably now students cheat on it then fail exams, and for students who don’t need as much practice it’s busywork.
- Make take-home projects complex and creative enough that they can’t be done by AI. Assign one large project with milestones throughout the semester. For example, in a web development class, have students build a website, importantly that is non-trivial and theoretically useful. If students can accomplish this in good quality with AI, then they can build professional websites so it doesn’t matter (the non-AI method is obsolete, like building a website without an IDE or in jQuery). Classes where a beyond-AI-quality project can’t be expected in reasonable time from students (e.g. in an intro course, students probably can’t make anything that AI couldn’t), don’t assign any take-home project.
- If exams (and maybe one large project) aren’t enough, make in-class assignments and projects, and put the lectures online to be watched outside class instead. There should be enough class time, since graded assignments are only to measure knowledge and competence; professors can still assign extra ungraded assignments and projects to help students learn.
In summary: undergraduate college’s purpose is to educate and measure knowledge and competence. Students’ knowledge and competence should be measured via in-class assignments/exams and, in later courses, advanced take-home projects. Students can be educated via ungraded out-of-class assignments/projects, as well as lectures, study sessions, tutoring, etc.
> My understanding is that homework is essentially just practice for exams
There are a LOT of people that don't take exams well. When you combine that with the fact that the real world doesn't work like exams in 90% of cases, it makes a lot of sense for grades to _not_ based on exams (as much as possible). Going the other direction (based on nothing _but_ exams) is going to be very painful to a lot of people; people that do learn the material but don't test well.
I made another comment on this thread about that. Exams should be test important knowledge (not computation or trick questions) so they should be easy for students who learned the material, even those who traditionally have trouble with exams. Most of the grade should be frequent in-class assignments or long take-home projects, which test almost if not the same skills students would use professionally (e.g. debug a simulated server failure in-class; develop a small filesystem with a novel feature at home).
The in-class assignments should also be easier than the take-home projects (although not as easy as the exams). In-class assignments and exams would be more common in earlier classes, and long projects would be more common in later classes.