The x86 memory model problem on ARM
Emulating x86 on ARM requires solving a fundamental mismatch between strict and relaxed memory ordering that affects every instruction.
In brief
Emulating x86 on ARM requires solving a fundamental mismatch between strict and relaxed memory ordering that affects every instruction. ARM processors need hardware support for x86-TSO semantics to achieve acceptable emulation performance, and solutions range from special instructions to full memory model toggles…
Quick answers
- What is x86-TSO memory?
- x86 uses Total Store Ordering, a strict memory model where stores become immediately visible to all processors. ARM uses weak ordering, allowing optimizations but requiring explicit instructions for coherency.
- What is the key takeaway from The x86 memory model problem on ARM?
- ARM processors need hardware support for x86-TSO semantics to achieve acceptable emulation performance, and solutions range from special instructions to full memory model toggles.
What is x86-TSO memory?
x86 uses Total Store Ordering, a strict memory model where stores become immediately visible to all processors. ARM uses weak ordering, allowing optimizations but requiring explicit instructions for coherency.
The coherency gap between architectures
x86 guarantees that when you write to memory, it becomes visible to all processors. ARM doesn't make this guarantee by default, saving power and efficiency but creating a fundamental mismatch for emulation.
ARM's acquire and release instructions
ARM introduced load-acquire and store-release instructions to enforce ordering without expensive memory barriers. FEX initially converted all x86 loads to acquire and stores to release, matching x86 semantics but at high cost.
The performance cost of acquire-release
Using acquire-release for every load and store severely limits CPU optimizations. Some processors like AmpereOne see dramatic slowdowns because these instructions were never designed for such heavy use.
LRCPC extension improves performance
ARMv8.3 added LRCPC load instructions designed specifically for x86 emulation. Most platforms now match regular load performance when using LRCPC instead of acquire-load instructions.
Apple's TSO hardware toggle
Apple Silicon directly implements x86-TSO in hardware. When enabled, regular load-store instructions behave like x86, eliminating the need for special instructions and achieving near-native performance.
Unaligned memory access problem
x86 applications ignore alignment requirements, but ARM requires natural alignment for acquire-release instructions. Unaligned accesses cause alignment faults that FEX must handle by patching code with memory barriers.
Unaligned loads and stores hurt performance
Unaligned memory accesses on ARM incur data memory barrier penalties. Cortex-X4 sees roughly 50 percent performance loss for unaligned accesses compared to aligned ones.
Oryon-3 handles unaligned better
Qualcomm's Oryon-3 implements coherent cachelines similar to x86, making unaligned accesses within a 64-byte cacheline perform nearly as well as aligned ones.
FEAT_LSE2 provides limited help
This ARM extension loosens alignment requirements only within 16-byte granules. Since x86 applications access memory across full cachelines, the extension provides marginal gains for emulation.
Atomic operations map directly
x86 atomic read-modify-write instructions map one-to-one to ARM atomic instructions like ldaddal and casal. The mapping is straightforward, but performance problems remain.
Split-locks are catastrophically slow
When atomic operations cross cacheline boundaries on x86, they use split-locks and complete atomically. On ARM, this requires kernel intervention, causing a thousand-times slowdown compared to aligned atomics.
Split-lock emulation is incomplete
FEX cannot correctly emulate split-locks with current ARM hardware. The best-effort implementation can tear data when split-lock and aligned atomic operations race, potentially corrupting data.
Valve's kernel patch helps Cortex-X4
Valve shipped a kernel patch that lets the Linux kernel handle unaligned atomics directly, avoiding the slow userspace signal handler dance. This makes Cortex-X4 five times faster for split-locks.
Proposed 128-bit CASP solution
FEX proposes a new ARM instruction allowing 128-bit CASP to straddle atomic granule boundaries. This would enable safe retry semantics for split-locks without requiring full hardware split-lock support.
Uncached memory complicates things
GPU-visible uncached memory uses write-combine semantics. Loads from write-combined memory are extremely slow because they bypass caches and go directly to system memory for every access.
Write-combine buffers help stores
Write-combine memory uses temporary buffers to burst cachelines at a time. Stores to write-combined memory perform well, but loads suffer because every access must reach system memory.
Hardware support is the real solution
The fundamental problem requires hardware changes. Whether through TSO toggles like Apple, coherent cachelines like Oryon-3, or new instructions, ARM vendors must address x86 emulation needs directly.
The bottom line
ARM processors need hardware support for x86-TSO semantics to achieve acceptable emulation performance, and solutions range from special instructions to full memory model toggles.
