I think people on both sides need to have more empathy, then. I'm generally one of the prompt people, and I'll try to start on time. If people are late, they'll arrive after we start, but that's fine.
And the late people need to understand that sometimes they will miss the beginnings of things, but that's ok too; their inability to be on time (for whatever reason) should not waste the time of those who get there on time.
My experience is that when you have habitually late people will enter a meeting after you start, their first question is, "what did I miss?"
So then you waste even more time when someone recaps for them.
It's almost like people need to think about their day when they're scheduling things instead of just accepting every single meeting.
You can request different times for things. That's an option.