Use default destructor and remove unnecessary virtual destructor

Only the virtual destructor in the base class is needed, and can use the
default.

llvm-svn: 263335