Revert "[InstCombine] Improve bswap + logic_op optimization"

The generic cast to `BinaryOperator` can break if `V` is not a
`BinaryOperator` (i.e a `ConstantExpr`). This occurs in things like
PPC linux build.

This reverts commit fe733f54.