[NFC] Make FPClassTest a bitmask enumeration

With this change bitwise operations are allowed for FPClassTest
enumeration, it must simplify using this type. Also some functions
changed to get argument of type FPClassTest instead of unsigned.

Differential Revision: https://reviews.llvm.org/D144241