Normal compilation doesn't care about tracking references, and shouldn't pay the compilation time cost.