Well, for 30 & 32 that makes sense, and as most 28 are actually taller than 28 it still works.
But if your army is 25mm and you add a 32mm figure, that figure is 28% taller than everyone else.
Given the average man is 5'10", the new guy would be 7'6".
Sure, people are different heights, and some are around 7'6" (I think that's Wilt Chamberlain's height) but it's extremely rare.
Going the other way, if most of your army is 32mm, then a 25mm mini would be like 4'6" - again, such people are out there, but other than children, they are rare (Danny DeVito is 5'2").