You're right, however I think we've already gone through this before. Most of us (probably) couldn't tell you exactly how an optimizing compiler picks optimizations or exactly how JavaScript maps to processor instructions, etc -- we hopefully understand enough at one level of abstraction to do our jobs. Maybe LLM driving will be another level of abstraction, when it gets better at (say) architecting projects.
> Most of us (probably) couldn't tell you exactly how an optimizing compiler picks optimizations or exactly how JavaScript maps to processor instructions,
That's because other people are making those working well. It's like how you don't care about how the bread is being made because you trust your baker (or the regulations). It's a chain of trust that is easily broken when LLMs are brought in.
Depends, if regulations are the cage that a baker has to work in to produce a product of agreed upon quality, then tests and types and LSPs etc. can be that cage for an LLM.
Regulations are not a cage. They don't constrains you for not doing stuff. They're a threshold for when behavior have destructive consequences for yourself. So you're very much incentivized for not doing them.
So tests may be the inspections, but what is the punitive action? Canceling the subscription?