I didn't know that was the correct way to use them. It feels incorrect in a space delimited language. Interesting.
English is not actually a space-delimited language; that's an approximation which is, in this case, throwing you off.
Punctuation is usually set closed on at least one, if not both, sides, though there are exceptions.