Avoid overly large SmallPtrSet/SmallSet

These sets perform linear searching in small mode so it is never a good
idea to use SmallSize/N bigger than 32.

Differential Revision: http://reviews.llvm.org/D16705

llvm-svn: 259284