Revert r72025. It is possible for clients to convert between signed types

and pointer types safely if they only do so when the sizes are the same.
llvm-gcc is such a client.

llvm-svn: 72029