Removes the need to add and remove them manually depending on whether they are used in cvt*() functions. Also removes the compiler warnings about unused handlers when it happens to be the case. Part of <https://github.com/llvm/llvm-project/issues/62629>. Reviewed By: arsenm Differential Revision: https://reviews.llvm.org/D151688