Valve guitar amps were actually the example I had in mind where high output impedance isn't clearly bad. There's a lot that the higher impedance designs in that space can do that the ones with nicer spec sheets can't, and the speakers are often set up to make high impedance work better. It is a bit of a weird space though, since arguably the best feedback is no feedback at all (single-ended with non-ridiculous gain).
It's definitely possible to design a good speaker for pretty much whatever source (or vice-versa), it's just a whole lot easier and more consistent when you can assume the source is low impedance. Apparently I thought that assumption was more standard in the modern day than it actually is.
I don't think the voltage rails are a big limitation in the first place. Looking at those datasheets I linked earlier, it's 205mV RMS to drive HD650s to 90 dB, which means 100 dB should be easy on a phone's Vbat. It'll be less under musical loads, but that's still a lot of noise from tough-to-drive cans. The Audio-Technicas should be able to ride that to more like 115 dB, which is just excessive.