Yea, it's partly their spells, but it's also some of the wither effects. You'll take like 3-4 hits to death with iron armor, but then die from taint or another appearing behind your back.
Another thing that would be nice would be if the ones who have access to godmode (ET ones etc) make sure they are out of godmode before engaging in combat.