Revert "[InferAttributes] Materialize all infered attributes for declaration"
Breaks check-clang, see comments on D100400
Also revert follow-up "[NFC] Move a recently added utility into a location to enable reuse"
This reverts commit 3ce61fb6.
This reverts commit 61a85da8.