Straight up summoners arent really that much of a problem if I am playing competetively although the ones that draw cards and generally negate the negative affects of summoning, eg slow/drawing cards do seem to have no real downsides.
Masters who summon whilst killing your stuff are probably worse as they are doubling the activation advantage.