upvote
note that similar concepts appear in mathematics. Generally the term for it is a "mollified" function.

applied to the step function, you would get a smooth cutoff function

https://en.wikipedia.org/wiki/Mollifier#Smooth_cutoff_functi...

this is also related somewhat to the notion of differentiable programming. RELU is (roughly) the same as x * step(x). In differentiable programming one can replace it with smooth approximations, cf "softplus"

https://arxiv.org/pdf/2403.14606

That book also has a chapter on control flow, which is very similar to what you're talking about.

Unrolling an if statement into x = b (result of one branch) + (1-b) (result of the other branch) is also incredibly common in cryptography. If `b` is a "secret" variable, an if statement may leak the value of it via the branch predictor/speculative execution. The way around this is to compute both branches, and then select them with the above arithmetic expression. This mostly works, though compilers are tediously smart, and so one often has to be careful how with how you precisely do it.

reply
Secret data flow should be directly controlled by programming language features. It's ugly that people are resorting to wrestling the compiler using such distant means as purity, applied in roundabout way.
reply
I don't know JAX, but can this trick be applied as a higher-level function to autodiff-capable languages like JAX?
reply