(journal.rafaelcosta.me)
The conclusion, that it was not the fault of the developer was correct, but assuming anything other than a problem at some point in the software stack is unreasonable.
You're being unfair here. The showpiece software that uses that hardware wouldn't install, and almost all software ignores it.
What has existed before is the Apple Neural Engine (ANE) which is very different from the newer Neural Accelerator support within the GPU blocks. In fact MLX does not even support ANE yet since at least in previous versions it was hardware-limited to computing FP16 and INT8 MADDs, and not even that fast.
"In fact MLX does not even support ANE yet"
I didn't say otherwise. The ANE is a fantastic unit for small, power-efficient models, like extracting text from images, doing depth modelling, etc. It's not made for LLMs, or the other sorts of experimental stuff MLX is intended for. Though note that MLX's author's reason for not supporting the ANE is that it has a "closed-source" API (https://github.com/ml-explore/mlx/issues/18#issuecomment-184...), making it unsuitable for an open-source project, and given that MLX didn't want to just lean on CoreML. But anyways, the ANE is fantastically fast at what it does, while sipping juice.
In any case, the code change shown should have zero impact on the running of MLX on an iPhone 16 Pro. MLX tries to really leverage platform optimizations so maybe another bifucation is making the wrong choice.
The MLX folks have various rationales for not supporting the ANE (at least as of yet), but one of them is that any real support requires implementing explicit splits in the graph of computations, where ANE-suitable portions are to be dispatched to the ANE and everything else goes back to the GPUs. That's not necessarily trivial.
Still, sad state of affairs that it seems like Apple is still fixing bugs based on what blog posts gets the most attention on the internet, but I guess once they started that approach, it's hard to stop and go back to figuring out priorities on their own.
I almost guarantee there is no way they can read this blogpost, escalate it internally, get the appropriate approval to the work item, actually work on the fix, get it through QA and get it live in production in 3 days. That would only happen on really critical issues, and this is definitely not critical enough for that.
I've seen a blog-post, authored a bug in Radar, assigned it to myself, and fixed it the same day. Whether it goes out in the next release is more a decision for the bug-review-board, but since the engineering manager (that would have been me) sits on that too, it's just a matter of timing and seeing if I can argue the case.
To be fair, the closer we are to a release, the less likely a change is to be accepted unless you can really sweet-talk the rest of the BRB, and there's usually a week of baking before the actual release goes out, but that has sometimes been shrunk for developer-preview releases...
If not, talk about coincident that someone reported an issue and all of that you mentioned was already done before that happened, and the only thing missing was merging the code to the repository which was done after the issue was reported. Not unheard of, but feels less unlikely than "Engineer decided to fix it".
I don't think that fix is specific to this, but it's absolutely true that MLX is trying to lever every advantage it can find on specific hardware, so it's possible it made a bad choice on a particular device.
But phenomenon is another thing. Apple's numerical APIs are producing inconsistent results on a minority of devices. This is something worth Apple's attention.
My mind instantly answered that with "bright", which is what you get when you combine the sun and moon radicals to make 明(https://en.wiktionary.org/wiki/%E6%98%8E)
Anyway, that question is not without reasonable answers. "Full Moon" might make sense too. No obvious deterministic answer, though, naturally.
Edit: Spoiler -
It's 'Eclipse'
Eclipse, obviously.
https://neal.fun/infinite-craft/
For the record, Sun+Moon is indeed eclipse.
Still think it was a good response :)
i just looked up mass of sun vs mass of moon (they differ by 10^30 vs 10^20), and the elemental composition of the sun: the moon would entirely disappear into the insignificant digits of trace elements which are in the range of .01 % of the sun. I could be off by orders of magnitude all over the place and it would still disappear.
It’s a reasonable Tarot question.
I'll just add that if you think this advice applies to you, it's the - https://en.wikipedia.org/wiki/Barnum_effect
"Monsoon," says ChatGPT.
An encrypted iTunes backup of a device was a perfect image. Take the backup, pull the SIM card, restore the backup to a new phone with the sim card installed, and it was like nothing had happened.
No reauthentication. No missing notifications. No lost data. Ever.
It was nice.
Isn’t this built in when transferring devices? Are backups different?
"Well, now it's Feb. 1st and I have an iPhone 17 Pro Max to test with and... everything works as expected. So it's pretty safe to say that THAT specific instance of iPhone 16 Pro Max was hardware-defective."
[1] as the author knows (“MLX uses Metal to compile tensor operations for this accelerator. Somewhere in that stack, the computations are going very wrong”) there’s lots of soft- and firmware in-between the code being run and the hardware of the neural engine. The issue might well be somewhere in those.
But it's still surprising that that LLM doesn't work on iPhone 16 at all. After all LLMs are known for their tolerance to quantization.
But, what got me about this is that:
* every other Apple device delivered the same results
* Apple's own LLM silently failed on this device
to me that behavior suggests an unexpected failure rather than a fundamental issue; it seems Bad (TM) that Apple would ship devices where their own LLM didn't work.
There's a C++26 paper about compile time math optimizations with a good overview and discussion about some of these issues [P1383]. The paper explicitly states:
1. It is acceptable for evaluation of mathematical functions to differ between translation time and runtime.
2. It is acceptable for constant evaluation of mathematical functions to differ between platforms.
So C++ has very much accepted the fact that floating point functions should not be presumed to give identical results in all circumstances.
Now, it is of course possible to ensure that floating point-related functions give identical results on all your target machines, but it's usually not worth the hassle.
[P1383]: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2023/p13...
Intel Compiler for e.g. uses less than IEEE764 precision for floating point ops by default, for example.
It is commutative (except for NaN). It isn't associative though.
Why? This is well specified by IEEE 754. Many runtimes (e.g. for Javascript) use NaN boxing. Treating floats as a semi-arbitrary selection of rational numbers plus a handful of special values is /more/ correct than treating them as real numbers, but treating them as actually specified does give more flexibility and power.
My understanding is the exact opposite - that it allows implementations to return any NaN value at all. It need not be any that were inputs.
It may be that JavaScript relies on it and that has become more binding than the actual spec, but I don't think the spec actually guarantees this.
Edit: actually it turns out nan-boxing does not involve arithmetic, which is why it works. I think my original point stands, if you are doing something that relies on how bit values of NaNs are propagated during arithmetic, you are on shaky ground.
> 6.2.3 NaN propagation
> An operation that propagates a NaN operand to its result and has a single NaN as an input should produce a NaN with the payload of the input NaN if representable in the destination format.
> If two or more inputs are NaN, then the payload of the resulting NaN should be identical to the payload of one of the input NaNs if representable in the destination format. This standard does not specify which of the input NaNs will provide the payload.
https://rust-lang.github.io/rfcs/3514-float-semantics.html
See also this section of wikipedia https://en.wikipedia.org/wiki/NaN#Canonical_NaN
"On RISC-V, most floating-point operations only ever generate the canonical NaN, even if a NaN is given as the operand (the payload is not propagated)."
And from the same article:
"IEEE 754-2008 recommends, but does not require, propagation of the NaN payload." (Emphasis mine)
I call bullshit on the statement "specifically binary operations combining two NaN inputs must result in one of the input NaNs." It is definitely not in the spec.
> For an operation with quiet NaN inputs, other than maximum and minimum operations, if a floating-point result is to be delivered the result shall be a quiet NaN which should be one of the input NaNs.
The same document say:
> shall -- indicates mandatory requirements strictly to be followed in order to conform to the standard and from which no deviation is permitted (“shall” means “is required to”)
> should -- indicates that among several possibilities, one is recommended as particularly suitable, without mentioning or excluding others; or that a certain course of action is preferred but not necessarily required; or that (in the negative form) a certain course of action is deprecated but not prohibited (“should” means “is recommended to”)
i.e. It required to be a quiet NaN, and recommended to use one of the input NaN.
I'm wondering if we couldn't re-think "bit" to the computer science usage instead of the thing that goes in the horse's mouth, and what it would mean for an AI agent to "champ at the bit"?
What new sayings will we want?
a * b = b * a for all "normal" floating point numbers.
The best way to do math on my phone I know of is the HP Prime emulator.
https://pcalc.com/mac/thirty.html
My other favorite calculator is free42, or its larger display version plus42
https://thomasokken.com/plus42/
For a CAS tool on a pocket mobile device, I haven't found anything better than MathStudio (formerly SpaceTime):
You can run that in your web browser, but they maintain a mobile app version. It's like a self-hosted Wolfram Alpha.
They do have some new AI math app that's regularly updated
Honestly, the main beef I have with Calculator.app is that on a screen this big, I ought to be able to see several previous calculations and scroll up if needed. I don't want an exact replica of a 1990s 4-function calculator like the default is (ok, it has more digits and the ability to paste, but besides that, adds almost nothing).
Also it does some level of symbolic evaluation: sin^-1(cos^-1(tan^-1(tan(cos(sin(9))))))== 9, which is a better result than many standalone calculators.
Also it has a library of built in unit conversations, including live updating currency conversions. You won’t see that on a TI-89!
And I just discovered it actually has a built in 2D/3D graphing ability. Now the question is it allows parametric graphing like the MacOS one…
All that said, obviously the TI-8X family hold a special place in my heart as TI-BASIC was my first language. I just don’t see a reason to use one any more day to day.
I use the NumWorks emulator app whenever I need something more advanced. It's pretty good https://www.numworks.com/simulator/
What I want is something like a repl. I want to be able to return to an earlier expression, modify it, assign it to a variable, use that variable in another expression, modify the variable and rerun and so on.
Typing on my iPhone in the last few months (~6 months?) has been absolutely atrocious. I've tried disabling/enabling every combination of keyboard setting I can thinkj of, but the predictive text just randomly breaks or it just gives up and stops correcting anything at all.
https://news.ycombinator.com/item?id=46232528 ("iPhone Typos? It's Not Just You - The iOS Keyboard is Broken")
At least the machine didn't say it was seven!
Did you file a radar? (silently laughing while writing this, but maybe there's someone left at Apple who reads those)
They noticed a discrepancy, then went back and wrote code to perform the same operations by hand, without the use of an LLM at all in the code production step. The results still diverged unpredictably from the baseline.
Normally, expecting floating-point MAC operations to produce deterministic results on modern hardware is a fool's errand; they usually operate asynchronously and so the non-commutative properties of floating-point addition rear their head and you get some divergence.
But an order of magnitude difference plus Apple's own LLM not working on this device suggests strongly to me that there is something wrong. Whether it's the silicon or the software would demand more investigation, but this is a well reasoned bug in my book.
https://ia800806.us.archive.org/20/items/TheFeelingOfPower/T...
I should think I'll probably see someone posting this on the front page of HN tomorrow, no doubt. I first read it when it was already enormously old, possibly nearly 30 years old, in the mid 1980s when I was about 11 or 12 and starting high school, and voraciously reading all the Golden Age Sci-Fi I could lay my grubby wee hands on. I still think about it, often.
(The idea being, a paragraph usually introduces a new thought.)
Whether you should do this on device is another story entirely.
What's to be gained, other than battery life, by offloading inference to someone else? To be lost, at least, is your data ownership and perhaps money.
Access to models that local hardware can't run. The kind of model that an iphone struggles to run is blown out of the water by most low end hosted models. Its the same reason that most devs opt for claude code, cursor, copilot, etc. instead of using hosted models for coding assistance.
You are claiming that if the price of the iPhone went down, apple would sell fewer phones?
Correspondingly, you are arguing that if they increased prices they could increase sales?
You are claiming that 100s of millions of people have all made the decision that the price of an iPhone is more than it is worth to them as a device, but is made up for by being seen with one in your hand?
Not all goods that signify status are Veblen goods.
Veblen goods aren't like this. If they were, everything would be priced at infinity. Veblen goods have to take into account the amount of spending money their target customers have, and how much they're willing to spend. Apple products are priced this way. They're not targeted just at people who can afford Rolls-Royce Silver Shadows, they're targeted at regular people who are willing to spend too much money on a phone when they can get an equivalent Android phone for half the price. Those people have limited money, but they're willing to overpay, but only so much.
>You are claiming that if the price of the iPhone went down, apple would sell fewer phones?
Quite likely, yes. If they adopted razor-thin profit margins on iPhones, their phones would be seen as "cheap" and wouldn't have the cachet they have now. More people would start looking at alternatives, and start buying Samsung Galaxies and other flagship Android phones.
Increasing demand with increasing prices is the very definition of a Veblen good. I never said anything like pricing them at infinity (an exceptionally stupid way of saying that something is not for sale).
I simply pointed out that there isn’t really any reason to believe that a mass produced easily available phone that holds a massive percentage of the entire global cell phone market would see increased demand from increased prices. It is an extraordinary claim with nothing resembling evidence. The most damning evidence is that the most expensive iPhone, the Pro Max, is outsold 2:1 by the base model for the last three generations, despite being visually distinguishable. (The 17 saw initial sales of Pro Maxes higher than base, but that appears to have corrected. Easily understandable that early adopters are more willing to pay for the best version of new tech)
There is an argument to be made that the Pro Max flirts with Veblen for small parts of the market, or that certain submarkets in poorer countries treat the iPhone that way, but that all looks more like conspicuous consumption. I still don’t believe that Pro Max sales increase if the price increases. A few individuals or submarket will not have the ability to invert a demand curve for an Apple device.
Again, I think that you are confusing conspicuous consumption with a Veblen good. This sentence is the giveaway:
> Those people have limited money, but they're willing to overpay, but only so much.
What you are describing is a normal demand curve. As price rises fewer people are willing to pay. People being unable to pay for something they still want does not make something a Veblen good (that would make insulin a Veblen good). You are describing a steep demand curve, not a reversed one.
Just because you perceive that an equivalent android can be purchased for half the price does not mean that everyone uses your criteria. I tried switching to a lower priced android made by google. In no way was it equivalent for my purposes. and I still wouldn’t want it. I am happy to pay the price, not because I care about being seen with an iPhone, but because it is the tool that I have determined to best suit my purposes. Many people refuse to believe this, but many people like the Apple ecosystem.
I don't think its unusual that a country with high median income and higher average income will tend to gravitate towards more expensive phones. Given that Apple doesn't make a cheap phone, it kind of follows that wealthier countries will buy more iPhones.
Of course the opposite is true as well, In a country where an iPhone is measured in months of salary, they won't sell well, but I'd be willing to bet that Androids in that price tier sell like shit in those countries too.
Is it a status symbol? arguably. But it also correlates pretty strongly with median income.
Function > Form.
I think its a Hero Complex, if Jung is correct.
There is no way a company could exist purely on marketing, Apple backs it up with tech.
But I agree Apple doesn't even though they've gone into a direction I couldn't follow them in.
They did this with the always-on screens for phones. My LGs had this many, many years ago. It was so bad that when Apple finally brought it out and acted like they had invented it, coworkers saw my LG and asked if I had gotten the latest iPhone, and I had to point out that it was a 5-year-old LG.
And then there's other stuff that Apple has which is just plain bad, but they present as new and wonderful, such as the "island" keyboard.
To wit, some people do value form over function. Some people do prefer a safe, curated walled garden.
I am not among them--I say this as someone who cannot stand using most Apple products for more than a minute. But I respect what they offer(ed) and for some people even recommended them. (Now I'm less sure because it seems like everything tech has gone to shit, but I can't tell if that's just "old man yells at cloud" or what)
Ideally there would be enough competition for us all to find what we're looking for. I think anticompetitive behavior is a worse sin
Not everyone cares for the most capable device on the planet. Sometimes people just want a pretty familiar and easy experience. I haven’t used my phone for anything more than browsing the web and texting in ages. I absolutely don’t care about whatever function you think I’m missing due to Apple, honestly.
As a side note, the fathers of Psychology were absolutely terrible scientists. The entire field almost failed because they took it so far into pseudo-science land. Of course Jung isn’t correct.
Just the other day I was reminded of the poor little "I am rich" iOS app (a thousand dollar ruby icon that performed diddly squat by design), which Apple deep-sixed from the app store PDQ.
If misery loves company, Veblen goods sure don't.
If I read most of the news on this very website, this is "way more efficient" and "it saves time" (and those who don’t do it will lose their job)
Then, when it produces wrong output AND it is obvious enough for you to notice, you blame the hardware.
It’s no different than someone testing a calculator with 2+2. If it gets that wrong, there’s a hardware issue. That doesn’t mean the only purpose of the calculator is to calculate 2+2. It is for debugging.
You could just as uncharitably complain that “these days no one does arithmetic anymore, they use a calculator for 2+2”.
The LLM that malfunctioned was there to slap categories on things. And something was going wrong in either the hardware or the compiler.
I don't get the snark about LLMs overall in this context; this author uses LLM to help write their code, but is also clearly competent enough to dig in and determine why things don't work when the LLM fails, and performed an LLM-out-of-the-loop debugging session once they decided it wasn't trustworthy. What else could you do in this situation?
> - MiniMax can't fit on an iPhone.
They asked MiniMax on their computer to make an iPhone app that didn't work.
It didn't work using the Apple Intelligence API. So then:
* They asked Minimax to use MLX instead. It didn't work.
* They Googled and found a thread where Apple Intelligence also didn't work for other people, but only sometimes.
* They HAND WROTE the MLX code. It didn't work. They isolated the step where the results diverged.
> Better to dig in a bit more.
The author already did 100% of the digging and then some.
Look, I am usually an AI rage-enthusiast. But in this case the author did every single bit of homework I would expect and more, and still found a bug. They rewrote the test harness code without an LLM. I don't find the results surprising insofar as that I wouldn't expect MAC to converge across platforms, but the fact that Apple's own LLM doesn't work on their hardware and their own is an order of magnitude off is a reasonable bug report, in my book.
Fascinating the claim is Apple Intelligence doesn't work altogether. Quite a scandal.
EDIT: If you wouldn't mind, could you edit out "AI rage enthusiast" you edited in? I understand it was in good humor, as you describe yourself that way as well. However, I don't want to eat downvotes on an empty comment that I immediately edited when you explained it wasn't minimax! People will assume I said something naughty :) I'm not sure it was possible to read rage into my comment.
No, the claim is their particular device has a hardware defect that causes MLX not to work (which includes Apple Intelligence).
> EDIT: If you wouldn't mind, could you edit out "AI rage enthusiast" you edited in? I understand it was in good humor, as you describe yourself that way as well. However, I don't want to eat downvotes on an empty comment that I immediately edited when you explained! People will assume I said something naughty :) I'm not sure it was possible to read rage into my comment.
Your comment originally read:
> This is blinkered.
> - MiniMax can't fit on an iPhone.
> - There's no reason to expect models to share OOMs for output.
> - It is likely this is a graceful failure mode for the model being far too large.
> No fan of Apple's NIH syndrome, or it manifested as MLX.
> I'm also no fan of "I told the robot [vibecoded] to hammer a banana into an apple. [do something impossible]. The result is inedible. Let me post to HN with the title 'My thousand dollars of fruits can't be food' [the result I have has ~nothing to do with the fruits]"
> Better to dig in a bit more.
Rather than erase it, and invite exactly the kind of misreading you don't want, you can leave it... honestly, transparently... with your admission in the replies below. And it won't be downvoted as much as when you're trying to manipulate / make requests of others to try to minimize your downvotes. Weird... voting... manipulating... stuff, like that, tends to be frowned upon on HN.
You have more HN karma than I do, even, so why care so much about downvotes...
If you really want to disown something you consider a terrible mistake, you can email the HN mods to ask for the comment to be dissociated from your account. Then future downvotes won't affect your karma. I did this once.
Who cares? The max amount of karma loss is 4 points, we can afford to eat our downvotes like adults.
nothing to see here.