I am calling vrndscalepd to round a zmm register. The register to round is:
3.9304247359239284, 0.85804618270500566, 1.899940908279022, 1.5554455222287524, 9.1150061014624413, 9.3562881423859334, 1.3206387781690712, 9.0367010342260201}
I want to round each of these to two digits past the decimal, but none of the rounding mode codes I have tried do that:
vrndscalepd zmm9{k1}{z},zmm8,32 ; 00100000
{4, 0.75, 2, 1.5, 9, 9.25, 1.25, 9}
vrndscalepd zmm9{k1}{z},zmm8,33 ; 00100001
{3.75, 0.75, 1.75, 1.5, 9, 9.25, 1.25, 9}
vrndscalepd zmm9{k1}{z},zmm8,34 ; 00100010
{4, 1, 2, 1.75, 9.25, 9.5, 1.5, 9.25}
vrndscalepd zmm9{k1}{z},zmm8,35 ; 00100011
{3.75, 0.75, 1.75, 1.5, 9, 9.25, 1.25, 9}
vrndscalepd zmm9{k1}{z},zmm8,41 ; 00110001
{3.75, 0.75, 1.75, 1.5, 9, 9.25, 1.25, 9}
vrndscalepd zmm9{k1}{z},zmm8,42 ; 00110010
{4, 1, 2, 1.75, 9.25, 9.5, 1.5, 9.25}
None of those does what I want. Is there another rounding mode that I have missed, that will round to 2 places past the decimal without rounding to the nearest even multiple of 0.25, on AVX512? If there another instruction that will do what I want on zmm registers?
>Solution :
No, AVX-512 uses binary floating point, not decimal.
(https://en.wikipedia.org/wiki/Double-precision_floating-point_format)
3.93 isn’t exactly representable as a float or double so it’s literally impossible for any sequence of instructions to create a bit-pattern that represents it. (https://www.h-schmidt.net/FloatConverter/IEEE754.html)
You could of course do something that gets a value that’s close, like 3.9300000667572021484375f, but it’s not as simple as zeroing some low bits of the mantissa like vrndscaleps/pd does. That will necessarily produce a value that’s representable as a fraction with a power-of-2 denominator (because all FP values are like that.) A "more round" binary FP value will have a smaller denominator; with 2 bits after the radix point, the denominator will be 4, so .0, .25, .5, or .75 are the only possibilites.
The normal way to round to 2 decimal digits is I think to multiply by 100.0, round to nearest integer, multiply by 1.0/100. I don’t know if there’s anything more efficient / clever. (Note that 1.0/100 isn’t exactly representable, so dividing by 100.0 could be more precise.)