[SCEV] Defer all work from ea12c2cb as late as possible

This is a second speculative compile time optimization to address a reported regression.  My actual suspicion is that availability of no-self-wrap is making some *other* bit of code trigger, but let's rule this out.