upvote
Nope. It'll be more efficient and faster.
reply
That's just semantics around the word "performance".

We're messaging each other right now inside a heavily sandboxed virtual environment that interprets XML-like markup and runs a text-based JITted scripting language from the 90s. It doesn't really matter that it's extremely inefficient, since it's still effectively instant.

reply
Actually not. Watt per task is a metric bothers more than LLM operators and users. We care about it a lot, and working a lot to measure and improve it, and we're not doing it in context of AI or LLMs.

Effectively instant means nothing, since everything is fast for small n. I love to give this example from a codebase I have worked on during my Ph.D. The problem which took half an hour under Matlab was done in ~40 seconds with mine, however, a thousand-step version of the same problem would take 11 hours. Saving a single second from that 40 second loop would shave my work by ~20 minutes, and this will compound for bigger problems still.

Just because we see the results instantly, doesn't mean it's fast or efficient by any means. It means we might forego optimizing it for sufficiently small n, if we don't care about that part of the code. That's all.

reply
Generally when I want to use a calculator, I use an app on my phone or computer. The watts per task is abysmal compared to a desk calculator.

The app is less efficient only in an abstract, impractical way.

reply
> The watts per task is abysmal compared to a desk calculator.

It depends on the calculator and task at hand. Consider graphing, integration or any programmable needs, and they might converge. ALU of a processor is pretty efficient at what it does.

> The app is less efficient only in an abstract, impractical way.

Again, depends, as I said above. You tend to brush efficiency away because you only mentally measure time to completion, only, which is understandable.

reply
Imagine performing literally trillions of matrix multiplications in order to figure out the answer to 1 + 2.
reply
Your computer/phone just performed millions of them in order to display the contents of my comment to you.
reply
Pretty much what our brain does.
reply
Sure, but what about performance per joule?
reply
If performance per joule is important in this loosely-defined hypothetical scenario, then it's important.

If it's not, then it's not.

reply