[Attributor] Only non-exact accesses require a uniform bit-pattern (=0)

If we only have exact accesses we should never require the bit-pattern
to be uniform (in this case 0). Only a non-exact access should force us
to require only 0 values.