I’m not convinced either. If they assimilated into nearby native populations, wouldn’t someone have found a colonist or a descendant that could tell the story?
Oral history is not very good at long term tales. I know my grand parents stories, but even my great grand parents is something that I know little about and I have trouble passing that on to my kids (and that includes the great grandpa who was murdered and thus a more compelling story)