Definition: TF32

(TensorFloat32) A proprietary Tensor floating point format supported in NVIDIA GPUs that was introduced with the Ampere architecture in 2020. Optimized for deep learning, TF32 is a variation of FP32 (32-bit floating point). It uses the same 8-bit exponent but only a 10-bit mantissa compared to 23 bits.

Tensor Cores vs. CUDA Cores
Tensor not only supports the TF32 mathematical data type, it also supports the FP16, BF16, INT8 and INT4. CUDA supports FP32 and INT32. See Bfloat16, Tensor core and deep learning.