I was surprised to find my old PC with a GTX1080 could transcribe/diarize about 10x faster than my m1 Mac. If anyone reading this is looking to transcribe 100s of hours of audio, do the extra work to get it set up on desktop with a dedicated graphics card.
Was the M1 using its GPU cores? It should have more power than a 1080.
Does macwhisper utilize metal GPUs by default or is it something you have to tick?