Skip to content
Unverified Commit b23e518c authored by Thomas Preud'homme's avatar Thomas Preud'homme Committed by GitHub
Browse files

Fix TOSA FP16->INT16 CAST lowering (#79299)

Currently cast from FP to int is implemented by clamping on the min and
max
integer values in the floating-point domain and then converting to
integer. However, the max int values are often non representable in the
floating-point input type due to lack of mantissa bits.

This patch instead use a select acting on a compare against max int + 1
which is representable in floating-point. It also has a special lowering
for cases where the integer range is wider than the floating-point range
to clamp the infinite values.
parent 8a5bdd89
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment