A key thing that has been shown in research at Wharton is that LLMs elevate people with less experience a lot more than they elevate people a lots of experience.
If we take "operator skill" to mean "they know how to make prompts", there is some truth to it and we can see that by whether or not the operator is designing the context window or not.
But for the more important question, whether or not LLMs are useful has an inverse relationship with how skilled the person is in the context they are using for. This is why the best engineers mostly shrug at LLMs while those that arent the best feel a big lift.
So, LLMs are not mirrors of operator skill. This post is instead an argument that everyone should become prompt engineers.
Disagree. Poor engineers will go in circles with AI because they will under specify their tasks and fail to recognize improper solutions. Ultimately if you’re not thoughtful about your problem and critical about the solution you will fail. This is true with or without AI at the wheel.
> Poor engineers will go in circles with AI
But they move quickly around that circle making them feel much more productive. And if you don't need something outside of the circle it is good enough.
It's not an opinion, it's what research has shown many times. For example, they can ask the LLM about how to get started or what an experienced engineer might do, etc, as a research tool before writing code.
Experience, though, can definitely vary by domain. Trying to get one to code an algorithm recently that I had a pretty good idea of took longer and gave worse results than just doing it myself.
On the other hand, something like wrestling with the matplotlib API? I don't have too much experience there and an LLM was a great help in piecing things together.
Keen to read the research. Can you drop the link?
I was at Penn's first AI conference last year and heard Dr Lilach Mollick's keynote where she said this is shown to be true over and over. She doesnt seem to publish often, but her husband Ethan always has a lot to say about AI.