It's crazy that Armv8/9-A permits speculative instruction fetches to device memory. Crazy enough that I had to look it up to believe it:
"Hardware does not prevent speculative instruction fetches from a memory location with any of the Device memory attributes unless the memory location is also marked as execute-never for all Exception levels." - ARM DDI 0487K.a § B2.15.2
The Armv7-M spec is less clear. It does say: "The architecture does not permit speculative accesses to memory marked as Device," in contrast to Armv8/9-A which qualifies a similar statement with "data accesses". But then it later says "To ensure correctness, read-sensitive locations must be marked as non-executable". (This is all from ARM DDI 0403E.e § A3.5.7)
I don't have v7-A older handy to compare what they say.
But then, a lot of correctness bugs like that are also security problems.
In the case that I know that has a 1:1 identity mapping of the address space, the physical address space doesn't actually have anything at 0 either, so it still doesn't need to be mapped.
https://support.arm.com/documentation/100941/0101/Memory-typ...
When it's done incompetently as on this ARM implementation, then you can't even run perfectly good and correct code, because the CPU will attempt speculative execution on a location that you never asked it to execute code at, and then bork itself when it realizes that can't possibly work.
Naturally, this is the sort of problem that requires tedious dissection of what exactly happened, and copious amounts of alcohol.
Interrupt(?) fires to trigger hypervisor, hypervisor figures out what it needs to do, jumps to that code, does its job, returns.
The “figured out what it needs to do” is the issue right? So what was actually happening was:
Same start… CPU predicts what hypervisor will do, speculatively loads instructions from mispredicted branch target, that wrong instruction reads memory(?) against the “no data prefetch” settings for that part of memory, CPU blows up/halts/whatever.
The fix is to mark the area the branch was mispredicted to in such a way that the CPU won’t prefetch instructions. Thus that won’t be run and prefetch data, thus no violation. CPU execution continues taking the correct branch and everything is fine.
The Armv8/9-A architecture reference manual is clear that speculative instruction fetches are permitted in Device memory unless that memory is also marked NX. So if your hardware has side-effects from a certain address, but it maps it as Device non-NX memory, then your code is not "perfectly good and correct". Assigning correct memory attributes is one of the many things needed for correct code.
So Arm did not add another means to disable this kind of speculative execution, after Spectre was discovered, but they just reused the existing NX flag, expanding its functionality.
It's simply speculative execution. Which doesn't look any different to regular execution. The fetcher has no idea that its predicted branch is about to invalidated and flushed, otherwise it would never have issued that fetch.
Actually, on a modern OoO core, [0] it's very rare for the instruction fetcher to not be doing speculative fetches. Even when it's not predicting a branch, the fact that it has "predicted" the lack of a branch is speculative in itself. It assumes it didn't fetch a branch in the last cycle, but it can't be sure until after instruction decoding, which takes at least 2 cycles (more on larger L1i caches).
About the only time the instruction fetcher is not doing speculative fetching is for a single cycle after each miss-predicted branch.
[0] Or even something technically in-order, like the Cortex A53 cores here. They might issue in-order, but because of how they implement dual issue, they look somewhat close to a simple OoO core... I suspect they actually do register renaming. And (most importantly) importantly they have a branch predictor.
But the original question was asking why disabling data prefetching to a memory region didn't automatically disable instruction prefetching at the same time.
And the answer is that speculative execution is a completely different mechanism that I'm not even sure can be disabled, at least not per memory region.
To the front end, one needs to pipe only appropriate logical combinations of the flags that describe the memory properties, not the flags as they are stored in some control register.
So a control register should contain flags that are meaningful for the programmer, while the kind of flags that hardware happens to need can be generated with a few logic gates from them and routed through hardware wherever they are needed.
It makes more sense to classify the memory in a few types, which must be specified by the programmer, including a "device memory" a.k.a. "memory-mapped peripherals" type, instead of having to specify for each memory area a long set of attributes about each kind of access that may be allowed, or not.
This is how it is done in x86-64.
A nitpick, but this is only true for some vendors, depending on their license, and is very much company-by-company. Many vendors, even big names like Meta, don't have the ability to roll their own. And even for the ones who do, 'customize at will' is a bit strong, as ARM very much does want to maintain uniformity across userspace implementations. E.g. Nvidia shouldn't add new traps for architecturally-legal behavior, since then code compiled for Apple hardware wouldn't work on Grace. Or worse, not trap for architecturally-illegal behavior, since then code compiled for Grace might not work for anyone else at all!
So by "vendors" it was indeed meant "some vendors" who can afford to not care much about compatibility with the specification.
iirc it's a documented feature now - FEAT_E2H0, https://support.arm.com/documentation/109697/2025_12/Feature...
And it was retroactively defined to be allowed starting from Armv8.0.
Apple designs pre-date the ID register bit for it being a thing so it takes a quirk there however.
Regardless what I mean to convey is that ARM would prefer such deviations to be rare, and especially for them to not be visible from userspace. Deviations will always exist if only due to hardware bugs, so contracts can only do so much.
JIT is important to Apple platforms, and they seem to manage to make it work well enough even with the need for explicit invalidation.
(Or did you confuse cache flushing with TLB flushing? The remap does the latter, not the former.)