They sacrifice it almost entirely: it drops to 25. You turn that on, your Sherman is an AI tank only.
However, that HE toggle is a strict boost: the AP ammo has the same AI performance as the Cromwell and Comet. Maybe even slightly better given the superior scatter.
Yea I think the AP was actually contrast to the HE it would be a better "balanced" tank. The HE is great, and a huge bonus if you know there is no armour in the area, but the AP is plenty acceptable against infantry as well, and has higher than average pen iirc to boot.