upvote
Well, they don't produce the same result in floating point math, I'm afraid.

So you'd need to teach your compiler about what your formulas mean and what context you are using them in. (Ie are you actually doing geometry, or is your AI coding agent just trying arbitrary activation functions for your neuronal net experiments and some of them happen to look like geometry?)

reply
It's a mistake to care about equality of floating point numbers [1]. You must usually consider the lower bits of the number as random.

I assume you're saying something other than this though?

[1] https://en.wikipedia.org/wiki/Machine_epsilon

reply
I think the point is that, from a compiler's perspective, it's not obvious how much you should be allowed to optimise code at the cost of changing the outcomes of floating points maths - do you allow 1e-10, or 1e-6, or 1e-4 level changes? Does your compiler have to run some test calcs to bound the scale of the change introduced by rewriting fp maths? Some compilers will let you opt in to rewriting floating point maths, but that's opt in so users understand that their numeric outputs might change between optimisation levels.

For more, there's a good post on this kind of flag in Rust: https://pythonspeed.com/articles/faster-float-math-rust/

reply
This statement is a little too strong. It's a mistake to care about the equality of floating point numbers after subjecting them to irrational operations. On the other hand, the entire internet runs on the fact that doubles exactly represent the integers up to 2^53.
reply
Huh, what? Floating point numbers have a standard, you know. They aren't non-deterministic YOLO numbers.

By default, the compiler has to stick to what the standard requires, and can't just say add arbitrary imprecision.

Your epsilon is what you get when you try to analyse floating point numbers as approximations of real numbers. But they also have an independent life as bit patterns, and the compiler can't just willy-nilly muck around with these bit patterns.

reply
Please look at the link where context is clear here. Floating point has limited precision and the complete inability to exactly represent some numbers.

Example, this equality check is false:

0.1 + 0.2 == 0.3

Because the last bits of a floating point number, after any practical chain of operations, is practically random, do to errors from limited precision. Yes you can always know what the result will exactly be for any operation, if you know the exact number and operations used. Good luck with something like sin/cos though, where implementations can vary wildly depending on the platform/library.

reply
Your problem only occurs when you try to naively transport equality of real numbers into equality of floating point numbers.

https://www.netlib.org/fp/dtoa.c is how eg CPython parses literals like 0.3 into floating point numbers and how it converts floating point numbers back to strings. Lo and behold: these algorithm compare floating point numbers for equality, and would break catastrophically, if the compiler were allowed to willy-nilly fiddle with the bit patterns.

The authors of these algorithm did care about floating point equality, and that is not a mistake. (However it would be a mistake to assume that equality of mathematical real numbers translates to equality of floating point numbers.)

> Yes you can always know what the result will exactly be for any operation, if you know the exact number and operations used.

Yes, and for some algorithms like illustrated above this is exactly what people do, and have to do.

And the lower bits of 0.1 + 0.2 ain't random: they are the same on your computer as on mine, whether we run the code in 1999 or in 2029.

https://news.ycombinator.com/item?id=47767398 has a discussion.

reply
The short answer is you need fast-math flags to allow optimizations that may change floating-point results, and you also need to guarantee an implementation of sinpi/cospi (these were added in C23, so they're not all that common in host library implementations yet).

It's possible if you had the implementation of the math library visible to the compiler that it could do inlining and then simplify expressions, but honestly most math library function implementations are going to be the kind of function that doesn't get picked up by inline heuristics, as there's a pile of if statements (handling special cases and range reduction) that the compiler can't eliminate due to there not really existing a sufficiently powerful FP range analysis.

reply