[GPGPU] Only collect the access that belong to an array [NFC]

This avoid the construction of very large sets and in many cases also keeps the
number of parameters low. As a result, we see a compile time reduction from 5
minutes to only slightly above 1 minute for one of our larger test cases.

llvm-svn: 311127