I see that pressure as well. I find that a lot of the problems we have with AI are in fact AI exposing problems in other aspects of our society. In this case, one problem is that the people who do the teaching and know what needs to be learned are the faculty, but the decisions about how to teach are made by administrators. And another problem is that colleges are treating "make money" as a goal. These problems existed before AI, but AI is exacerbating them (and there are many, many more such cases).
I think things are going to have to get a lot worse before they get better. If we're lucky, things will get so bad that we finally fix some shaky foundations that our society has been trying to ignore for decades (or even centuries). If we're not lucky, things will still get that bad but we won't fix them.
Instructors and professors are required to be subject matter experts but many are not required to have a teaching certification or education-related degree.
So they know what students should be taught but I don't know that they necessarily know how any better than the administrators.
I've always found it weird that you need teaching certification to teach basic concepts to kindergartners but not to teach calculus to adults.
> Instructors and professors are required to be subject matter experts but many are not required to have a teaching certification or education-related degree.
This is universal. I’ve had largely the same experience. There’s several reasons for this.
1. Stupider people are better teachers. Smart people are too smart to have any empathic experience on what it’s like to not get something. They assume the world is smart like them so they glaze over topics they found trivial but most people found confusing.
2. They don’t need to teach. If the student body is so smart then the students themselves can learn without teaching.
3. Since students learn so well there’s no way to differentiate. So institutions make the material harder. They do this to differentiate students and give rankings. Inevitably this makes education worse.
It's simpler than that. "Prestigious" universities emphasize research prestige over all else on faculty. Faculty optimize for it and some even delight in being "hard" (bad) teachers because they see it as beneath them.
Less "prestigious" universities apply less of that pressure.
It can also be different within the same university, by department. I graduated from a university with a highly ranked and research oriented engineering department. I started in computer engineering which was in the college of engineering but ended up switching to computer science which was in the college of arts and sciences. The difference in the teachers and classroom experience was remarkable. It definitely seemed like the professors in the CS department actually wanted to teach and actually enjoyed teaching as compared to the engineering professors who treated it like it was wasting their time and expected you to learn everything from the book and their half-assed bullet point one way lectures. Unfortunately or fortunately, depending on your view, it also meant having to take more traditional liberal arts type electives in order to graduate.
I did once have a Physics lecturer say " When I took Quantum Mechanics back in my undergrad, I got an A but didn't actually understand anything" and then in the same lecture 20 minutes later: "What part of this do you not understand?" when the entire class was just blankly looking at the whiteboard.
At least at the undergrad level, it's not impossible to get an "A" without actually learning anything. Especially Freshman/Sophomore level classes. You just cram for the exams and regurgitate what you memorized. Within a few months time it's mostly gone.
Seriously, what so non-understandable in first 20 minutes of QM?
Probably depends on how it’s explained, no?
I could make arithmetic incomprehensible, let alone QM.
They never implied it was the first 20 minutes of the entire course
That's been my experience too, and I think it actually makes perfect sense from an evolutionary perspective - if the students are smart enough to learn well regardless of the level of the instruction, then the professors don't face any pressure to improve.
Taking this to the extreme, I think that a top-tier university could do very well for itself by only providing a highly selective admission system, good facilities and a rigorous assessment process, while leaving the actual learning to the students.
Universities don’t pick professors because they are good teachers, they pick them for their research publications. The fact that some professors end up being good teachers is almost coincidental.
For the most part, most universities, that is true. I was dissatisfied with the quality of my undergrad college education, and had the resources to try other universities. After two state schools, I figured out that Boston is The University City with 700,000 college students in the larger Boston area when I attended Boston University, MIT and Harvard. I found Boston's over sized undergraduate population created a credit sharing system for all the Boston area colleges, and if one wanted they could just walk onto anther campus and take their same class at your university. So, of course, I took at the classes I could at Harvard. I was formally an engineering student at BU, but as far as the professors at Harvard and MIT knew I was a student at their school. What I found was that at Harvard, and about 75% of the time at MIT, the professors are incredibly good, they are the educational best self actualizing as teachers. Every single Harvard professor took a personal interest in my learning their subject. I saw that no where else.
Yeah at that level you’re basically optimizing for all around excellence, and it’s hard to be a leader in your field without also being deeply interested in it at all levels — and being reasonably charismatic.
I’ve only taken classes at state schools, and my experience was that I’d often get a professor that was clearly brilliant at publishing but lacked even the most rudimentary teaching skills. Which is insightful in its own way…just not optimal for teaching.
This is true for research universities. There are many excellent teaching colleges where professors are hired to teach, and don't do research.
Sounds more like the unfortunate differences between teaching professors and research professors. Unfortunately some research schools force professors to teach N credits per semester even if that is not their speciality.
Your approach sounds too elitist for myself. I think you simply figure out the core skills of your professors. Maybe some teach undergrad well, others only advanced degrees. Maybe some should just be left to research with minimal classrooms etc.
I rather think it is a elitist concept of "I am a highly respected professor at a elite uni, how dare you bother me with your profane questions!"
I was at a Uni aiming for and then gaining "Elite" status in germany and I did not liked the concept and the changes.
I like high profile debates. As high as possible. But I don't like snobism. We all started as newbs.
>I've always found it weird that you need teaching certification to teach basic concepts to kindergartners but not to teach calculus to adults.
There is a lot more on the plate when you are kindergarten teacher - as the kids needs a lot of supervision and teaching outside the "subject" matters, basic life skills, learning to socialize.
Conversely, at a university the students should generally handle their life without your supervision, you can trust that all of them are able to communicate and to understand most of what you communicate to them.
So the subject matter expertise in kidnergartens is how to teach stuff to kids. Its not about holding a fork, or to not pull someones hair. Just as the subject matter expertise in an university can be maths. You rarely have both, and I don't understand how you suggest people get both a phd in maths, do enough research to get to be a professor and at the same time get a degree in education?
I was an instructor for a college credit eligible certification course. While I think that education degree is more than you need, providing effective and engaging instruction is a skill and is part of actual teaching at any level. Concepts like asking a few related open ended, no right answer questions at the beginning of a new topic to prime students’ thinking about that topic. Asking specific students “knowledge check” or “summarize/restate this topic” questions throughout keeps students from checking out. Alternating instruction with application type exercises help solidify concepts. Lesson plans/exercises/projects that build on each other and reincorporate previous topics. Consideration of how to assess students between testing and projects, for example a final vs a capstone project.
If you are just providing materials and testing, you aren’t actually teaching. Of course there are a ton of additional skills that go into childhood development, but just saying adults should figure it out and regurgitating material counts as “teaching” is BS.
Just watch out for who is certifying how things should be taught. It’s honestly one reason education is so bad and so slow to change.
Edit: and why perfectly capable professionals can’t be teachers without years of certification
> I've always found it weird that you need teaching certification to teach basic concepts to kindergartners but not to teach calculus to adults.
I think this is partially due to the age of the students, by the time you hit college the expectation is you can do a lot of the learning yourself outside of the classroom and will seek out additional assistance through office hours, self study, or tutors/classmates if you aren't able to understand from the lecture alone.
It's also down to cost cutting, instead of having entirely distinct teaching and research faculty universities require all professors to teach at least one class a semester. Usually though the large freshman and sophomore classes do get taught by quasi dedicated 'teaching' professors instead of a researcher ticking a box.
>don't know that they necessarily know how any better than the administrators.
If someone is doing something day in and day out, they do gain knowledge on what works and doesn't work. So just by doing that the professors typically know much more about how people should be taught than the administrators. Further, the administrators' incentives are not aligned towards insuring proper instruction. They are aligned with increasing student enrollment and then cashing out whenever they personally can.
This is very different in France. Studying to be a teacher at university level is a big deal.
Since the reform on University administration circa 2011, a big push was done towards 'evaluation continue' (basically regular tests), which now last until your third year in some Uni, to make public universities more like private schools, and against 'partiels' (two big batteries of standardized tests in person, with thousands in the same area, with only pen and papers, one early January, second in may, every year, over a week).
That push was accelerated because of COVID, but with the 'AI homework', it gave teachers a possibility to argue against that move and the trend seemed stopped last year (I don't now yet if it has reverted). In any case, I hope this AI trend will give more freedom to teachers, and maybe new ways of teaching.
And I'm not a big Llm fan in general, but in my country, in superior education, it seems good overall.
Ah, that's interesting, thanks. I lived in France in 2001-2 and was friends with someone who was studying for his partiels to become a chemistry teacher.
Once you're an adult some of the best lessons come from having bad teachers.
Nobody knows "how" things should be taught. Pedagogy is utter disaster.
I am pretty sure that early childhood education (until fifth grade) is a very active area of research in all highly developed nations. Almost, by definition, it you want to (a) become or (b) stay a highly developed nation, you need to have a high quality public education system.
My mother was a first grade teacher for 30+ years. In her school system, first grade is the year that students learn to read. Each year, she was also required to take professional training classes for a certain number of days. She told me that, in her career, there were many changes and improvements and new techniques developed to help children learn how to read. One thing that changed a lot: The techniques are way more inclusive, so non-normie kids can learn to read better at an earlier age.
A PhD was historically a teaching degree: that’s what the D stands for.
No?
PhD - Philosophy Doctor
Doctor is Latin for teacher; cf. "doctrine", "docent".
The instructors may not know the absolute best way to teach, but I think they do know more than the administrators. All my interaction with teacher training suggests to me that a large proportion of it is basically vacuous. On dimensions like the ones under discussion here (e.g., "should we use AI", "can we do this class online"), there is not really anything to "know": it's not like anyone is somehow a super expert on AI teaching. Teacher training in such cases is mostly just fads with little substantive basis.
Moreover, the same issues arise even outside a classroom setting. A person learning on their own from a book vs. a chatbot faces many of the same problems. People have to deal with the problem of AI slop in office emails and restaurant menus. The problem isn't really about teaching, it's about the difficulty of using AI to do anything involving substantive knowledge and the ease of using AI to do things involving superficial tasks.
I totally agree. I think the neo-liberal university model is the real culprit. Where I live, Universities get money for each student who graduates. This is up to 100k euros for a new doctorate. This means that the University and its admin want as many students to graduate as possible. The (BA&MA) students also want to graduate in target time: if they do, they get a huge part of their student loans forgiven.
What has AI done? I teach a BA thesis seminar. Last year, when AI wasn't used as much, around 30% of the students failed to turn in their BA thesises. 30% drop-out rate was normal. This year: only 5% dropped out, while the amount of ChatGPT generated text has skyrocketed. I think there is a correlation: ChatGPT helps students write their thesises, so they're not as likely to drop out.
The University and the admins are probably very happy that so many students are graduating. But also, some colleagues are seeing an upside to this: if more graduate, the University gets more money, which means less cuts to teaching budgets, which means that the teachers can actually do their job and improve their courses, for those students who are actually there to learn. But personally, as a teacher, I'm at loss of what to do. Some thesises had hallucinated sources, some had AI slop blogs as sources, the texts are robotic and boring. But should I fail them, out of principle on what the ideal University should be? Nobody else seems to care. Or should I pass them, let them graduate, and reserve my energy to teach those who are motivated and are willing to engage?
I think one of the outcomes might be a devaluation of the certifications offered in the public job marketplace.
I can say from some working experience in the United States that way too many jobs require a university degree. I remember being an intern or my first job after uni (which I struggled a great deal to complete), looking around and thinking: "There is no way that all of these people need a uni degree to do their jobs." I couldn't believe how easy work was compared to my uni studies (it was hell). I felt like I was playing at life with a cheat code (infinite lives, or whatever). I don't write that to brag; I am sure many people here feel the same. So many jobs at mega corps require little more than common sense: Come to work on time, dress well, say your pleases and thank yous, be compliant, do what is asked, etc. Repeat and you will have a reasonable middle class life.
Then there's Europe, where making it easy to get a master's degree just let to jobs requiring people to waste time getting yet another unneeded degree.
This entire situation is something that is predictable, and I have personally called it out years ago - not because of some unique ability, but because this is what happened in India and China decades upon decades ago.
There’s only so many jobs which have you a good salary.
So everyone had to become a doctor lawyer or engineer. Business degrees were seen as washouts.
Even for the job of a peon, you had to be educated.
So people followed incentives and got degrees - in any way or form they could.
This meant that degrees became a measure, and they were then ruthlessly optimized for, till they stopped having any ability to indicate that people were actually engineers.
So people then needed more degrees and so on - to distinguish their fitness amongst other candidates.
Education is what liberal arts colleges were meant to provide - but this worked only in an economy that could still provide employment for all the people who never wanted to be engineers, lawyers or doctors.
This mess will continue constantly, because we simply cannot match/sort humans, geographies, skills, and jobs well enough - and verifiably.
Not everyone is meant to be a startup founder. Or a doctor. Or a plumber, or a historian or an architect or an archaeologist.
It’s a jobs market problem, and has been this way ever since the American economy wasn’t able to match people with money for their skills.
Yep, it's a job market problem. Only degrees that are somehow limited in their supply will continue to hold value, the rest approach worthlessness. Neither the state nor universities have any interest to limit the supply.
In my country doctors earn huge salaries and have 100% job security, because their powerful interest groups have successfully lobbied to limit the number of grads below job market's demand. Other degrees don't come even close.
I agree. I tend to think though that the best way forward is to ignore all of these education issues and just focus on raising the floor. The difference between a "good-paying job" and a "not-so-good-paying job" should be small, and everyone should be able to have a good life regardless of what job they have. Then people can choose to go to college if they want to learn about things, and maybe to learn about subjects related to a job they want, but not because they think it's a way to make more money.
Well, see Germany. They do it pretty well. The expected lifetime earnings difference between university graduates and someone who took the trade/apprenticeship route is very similar. Does anyone know of other countries that are similar? Is it also true in Austria or Switzerland?
> Some thesises had hallucinated sources, some had AI slop blogs as sources, the texts are robotic and boring. But should I fail them, out of principle on what the ideal University should be?
No, you should fail them for turning in bad theses, just like you would before AI.
That's probably what should happen, but it's not what happens in reality. In grading I have to follow a very detailed grading matrix (made by some higher-ups) and the requirements for passing and getting the lowest grade are so incredibly low that it's almost impossible to fail, if the text even somewhat resembles a thesis. The only way I could fail a student, is if they cheated, plagiarised or fabricated stuff.
The person who used the AI slop blog for sources, we asked them to just remove them and resubmit. The person who hallucinated sources is however getting investigated for fabrication. But this is an incredibly long process to go through, which takes away time and energy from actual teaching / research / course prep. Most of the faculty is already overworked and on the verge of burnout (or are recovering post-burnout), so everybody tries to avoid it if they can. Besides, playing a cop is not what anybody wants to do, and its not what teaching should be about, as the original blog post mentioned. IF the University as an institution had some standards and actually valued education, it could be different. But it's not. The University only cares about some imaginary metrics, like international rankings and money. A few years ago they built a multi-million datacenter just for gathering data from everything that happens in the University, so they could make more convincing presentations for the ministry of education — to get more money and to "prove" that the money had a measurable impact. The University is a student-factory (this is a direct quote by a previous principal).
Yeah, our information and training systems are kinda failing at dealing with the reality of our actual information environment.
Take law for example and free speech - a central tenet to a functional democracy is effective ways to trade ideas.
A core response in our structure to falsehoods and rhetoric is counter speech.
But I can show you that counter speech fails. We have realms upon realms of data inside tech firms and online communities that shows us the mechanics of how our information economies actually work, and counter speech does diddly squat.
Education is also stuck in a bind. People need degrees to be employable today, but the idea of education is tied up with the idea of being a good educated thinking human being.
Meaning you are someone who is engaged with the ideas and concepts of your field, and have a mental model in your head, that takes calories, training and effort to use to do complex reasoning about the world.
This is often overkill for many jobs - the issue isn’t doing high level stats in a day science role, it’s doing boring data munging and actually getting the data in the first place. (Just an example).
High quality work is hard, and demanding, and in a market with unclear signals, people game the few systems that used to be signals.
Which eventually deteriorated signal till you get this mess.
We need jobs that give a living wage, or provide a pathway to achieving mastery while working, so that the pressure on the education lever can be reduced and spread elsewhere.
> A core response in our structure to falsehoods and rhetoric is counter speech.
> But I can show you that counter speech fails
Could you show me that? What's your definition of failure?
I get the feeling that you aren’t asking for the short version, because most people wouldn’t latch onto that point and create an account for it.
Hmmm.
An example - the inefficacy of Fact checking efforts. Fact checking is quintessentially counter speech, and we know that it has failed to stop the uptake and popularity of falsehoods. And I say this after speaking to people who work at fact checking orgs.
However, this is in itself too simple an example.
The mechanics of online forums are more interesting to illustrate the point - Truth is too expensive to compete with cheaper content.
Complex articles can be shared on a community, which debunk certain points, but the community doesn’t read it. They do engage heavily on emotional content, which ends up supporting their priors.
I struggle to make this point nicely, but The accuracy of your content is secondary to its value as an emotional and narrative utility for the audience.
People are not coming online to be scientists. They are coming online to be engaged. Counter speech solves the issue of inaccuracy, and is only valuable if inaccuracy is a negative force.
It is too expensive a good to produce, vs alternatives. People will coalesce around wounds and lacunae in their lives, and actively reject information that counters their beliefs. Cognitive dissonance results in mental strife and will result in people simply rejecting information rather than altering their priors.
Do note - this is a point about the efficacy of this intervention in upholding the effectiveness of the market where we exchange ideas. There will be many individual exchanges where counter speech does change minds.
But at a market level, it is ineffective as a guardian and tonic against the competitive advantage of falsehoods against facts.
——
Do forgive the disjointed quality in the response. It’s late here, and I wish I could have just linked you to a bunch of papers, but I dont think that would have been the response you are looking for.
I think this 3-part essay might be relevant to your argument: https://www.e-flux.com/journal/147/623330/society-of-the-psy...
I’ve been recommending network propaganda recently. The book has the data that makes the case better than I can about structural issues in the information ecosystem.
Also started going through this legal essay (paper?) recently, Lies, Counter-lies, and Disinformation in the Marketplace of Ideas
https://www.repository.law.indiana.edu/cgi/viewcontent.cgi?a...
The book "Nexus" by Yuval Noah Harari essentially makes this same point. The way he phrases it is that information's primary role throughout history hasn't necessarily been to convey objective truth but to connect people and enable large scale cooperation. So more information is not necessarily better.
Worth a read or you can check out one of his recent podcast appearances for a quicker download.
> The University is a student-factory
In The Netherlands, we have a three-tier tertiary system: MBO (practical job education / trades), HBO (college job education / applied college) and WO (scientific education / university).
A lot of the fancy jobs require WO. But in my opinion, WO is much too broad a program, because it tries to both create future high tier workers as well as researchers. The former would be served much better by a reduced, focused programme, which would leave more bandwidth for future researchers to get the 'true' university education they need.
> In grading I have to follow a very detailed grading matrix (made by some higher-ups) and the requirements for passing and getting the lowest grade are so incredibly low that it's almost impossible to fail, if the text even somewhat resembles a thesis. The only way I could fail a student, is if they cheated, plagiarised or fabricated stuff.
This is another example of "AI is exacerbating existing problems". :-) That kind of grading policy is absurd and should never have existed in the first place, but now AI is really making that obvious.
I've talked with professors at a major US research university. For Master's students, they are all paying a lot of money to get a credential. That's the transaction. They don't really care about cheating as long as they go through the motions of completing the assigned work. It's just a given, and like you say it takes more time than they have to go through the acacdemic dishonesty process for all the students who are getting outside help or (now) using AI.
> The person who used the AI slop blog for sources
You should fail them.
The larger work that the intellectual and academic forces of a liberal democracy is that of “verification”.
Part of the core part of the output, is showing that the output is actually what it claims to be.
The reproducibility crisis is a problem Precisely because a standard was missed.
In a larger perspective, we have mispriced facts and verification processes.
They are treated as public goods, when they are hard to produce and uphold.
Yet they compete with entertainment and “good enough” output, that is cheaper to produce.
The choice to fail or pass someone doesn’t address the mispricing of the output. We need new ways to address that issue.
Yet a major part of the job you do. is to hold up the result to a standard.
You and the institutions we depend on will continue to be crushed by these forces. Dealing with that is a separate discussion from the pass or fail discussion.
Fail them. Only let the ai generated text that has been verified and edited to be true to pass.
If they want to use AI make them use it right.
> Some thesises had hallucinated sources, some had AI slop blogs as sources, the texts are robotic and boring. But should I fail them, out of principle on what the ideal University should be?
I don't think you should fail them - instead, give them feedback on how to improve their thesis themselves, and how to make better use of tools like ChatGPT.
If instead of flat out failing to turn in their thesis, instead they are submitting work that needs more iteration due to bad use of AI, that sounds like a net win to me. The latter can be turned into something useful.