My answer is yes — at least for now. People tend to believe AI gives the great answers until it touches a field they actually have real expertise in. If you talk to true experts in humanities, you’ll find their depth of insight is on a totally different level compared to AI's surface-level summaries. AI mostly stitches together some published cliche like Max Planck’s driver in the old story.
That was true a few years ago, but now I'm getting better and better answers. Not saying I can totally trust AI, but the rate of progress is astonishing. Even if we see no more progress for the next 10 years, the effect of current models in our society will be profound.
You are right but these are two different questions. The effect in the society will be profound, yes. Replacing the true experts in the humanities? That's a different matter.