Cppreference documents that stdfloat includes 5 new types: float16_t, float32_t, float64_t, float_128_t and bfloat16_t. While the first 4 types are self-explanatory (a float with 16, 32, 64, and 128 bits respectively), the last type bfloat16_t is not at all clear to me. What does this type represent? What does the b in its name mean?
>Solution :
"bfloat16" refers to a fairly recent 16-bit floating-point format that is not a valid IEEE-754/IEC 60559 defined format. But it is related to them.
BINARY16 is just BINARY32 with smaller numbers: smaller mantissa, smaller exponent, and even a smaller sign bit. Bfloat16 opts for a different idea. It elects to keep the exponent the same size as BINARY32 (8-bit), shrinks the mantissa down to 7-bits (explicit), and keeps the same number of sign bits (1). This makes transforming between bfloat16 and BINARY32 a faster operation.
It does have some special-case weirdness, but overall, it’s a truncated BINARY32. And while it’s widely supported in a surprising amount of GPU hardware, it’s not truly a standard.