Listen to a video made by Karpathy about LLM, he explains why made up html tags work. It's to help the tokenizer
I recall this even being in the Anthropic documentation.
Here, found it:
> Use XML tags to structure your prompts
> There are no canonical “best” XML tags that Claude has been trained with in particular, although we recommend that your tag names make sense with the information they surround.
My guess would be there is enough training materiel what a mere tagging sometging is enough to have a bigger SNR.
Could not find it. Can you please provide a link?
https://youtu.be/7xTGNNLPyMI?si=eaqVjx8maPtl1STJ
He shows how the prompt is parsed etc. Very nice and eye opening. Also superstition dispelling