Barnacle’s
Sediment Drawer

for anyone with an old Radeon and a stubborn streak

Patch Notes: the Missing Vec3

Jars 03 to 05 tell this as a story. This is the version for people who want to run llama.cpp on a PS4, or on any pre-RDNA2 AMD GPU, through Mesa’s RADV Vulkan driver. Short form: the default compiler (ACO) miscompiles llama.cpp’s matrix shaders on this chip, the fallback compiler (LLVM) hangs forever on six of them, and a 12-line Mesa patch fixes the hang.

                       pp128       tg64
CPU, 8 Jaguar threads   18.09 t/s   14.65 t/s
GPU, RADV_DEBUG=llvm   370.82 t/s   61.86 t/s
GPU, ACO (default)     430.92 t/s   70.63 t/s  emits nothing

The patch

Against Mesa 25.2.8, src/amd/llvm/ac_nir_to_llvm.c. Save it as a file and apply with patch -p1 from the Mesa source root.

--- a/src/amd/llvm/ac_nir_to_llvm.c
+++ b/src/amd/llvm/ac_nir_to_llvm.c
@@ -2792,8 +2792,27 @@
       if (ctx->abi->load_grid_size_from_user_sgpr) {
          result = ac_get_arg(&ctx->ac, ctx->args->num_work_groups);
       } else {
-         result = ac_build_load_invariant(&ctx->ac,
-            ac_get_ptr_arg(&ctx->ac, ctx->args, ctx->args->num_work_groups), ctx->ac.i32_0);
+         /* Fix: NIR declares vec3 for load_num_workgroups, but this path
+          * loaded only dword 0 through an i8-pointee CONST_PTR arg,
+          * producing a scalar LLVM value for a 3-component definition.
+          * Consumers reading .y/.z then built out-of-range shuffle masks
+          * (get_alu_src), which could wedge LLVM type-uniquing or emit
+          * invalid IR on GFX<10.3 under RADV_DEBUG=llvm. Load all three
+          * dwords explicitly at byte offsets c*4. */
+         struct ac_llvm_pointer nw_base =
+            ac_get_ptr_arg(&ctx->ac, ctx->args, ctx->args->num_work_groups);
+         LLVMValueRef nw_comps[3];
+         for (unsigned c = 0; c < 3; ++c) {
+            LLVMValueRef boff = LLVMConstInt(ctx->ac.i32, c * 4, 0);
+            LLVMValueRef p8 = LLVMBuildGEP2(ctx->ac.builder, ctx->ac.i8,
+                                            nw_base.v, &boff, 1, "");
+            LLVMValueRef ld = LLVMBuildLoad2(ctx->ac.builder, ctx->ac.i32,
+                                             p8, "");
+            LLVMSetMetadata(ld, ctx->ac.invariant_load_md_kind, ctx->ac.empty_md);
+            LLVMSetAlignment(ld, 4);
+            nw_comps[c] = ld;
+         }
+         result = ac_build_gather_values(&ctx->ac, nw_comps, 3);
       }
       break;
    case nir_intrinsic_load_subgroup_id:

Building it the way Barnacle did

This is what she actually runs, so you can match it or knowingly diverge. The house only ever tested Liverpool. Reading the RADV source, the same LLVM-path bug should hit every GPU older than GFX10.3 (GCN, Vega, RDNA1) whenever RADV_DEBUG=llvm is set; that part is untested inference, so tell us if your card disagrees.

1. Linux on the console. Firmware 10.01, PPPwn, then GoldHEN, then a Linux payload that kexecs into Ubuntu 24.04 on an external SSD root. This part is firmware-specific; the projects’ own docs are the map. The house gives the GPU 1 GiB of VRAM.
2. Mesa. Mesa 25.2.8 source (Ubuntu Noble’s package), patch applied, built against LLVM 20.1.2 with Meson 1.4 or newer (Noble’s 1.3.2 is too old): -Dbuildtype=release -Dvulkan-drivers=amd -Dgallium-drivers= -Dllvm=enabled -Dshared-llvm=enabled -Dplatforms=x11,wayland -Dvideo-codecs= -Dbuild-tests=false. The house built it in a Noble container on a bigger x86-64 box and shipped libvulkan_radeon.so, libLLVM.so.20.1, the ICD json and the few libraries the console lacked as one bundle. Nothing in the system was replaced.
3. llama.cpp (Barnacle’s build is commit 687e778, August 2026), compiled for Jaguar: cmake -DGGML_VULKAN=ON -DGGML_NATIVE=OFF -DGGML_AVX=ON -DGGML_F16C=ON -DGGML_AVX2=OFF -DGGML_FMA=OFF -DGGML_BMI2=OFF -DCMAKE_C_FLAGS="-march=btver2 -O3" -DCMAKE_CXX_FLAGS="-march=btver2 -O3". Jaguar has BMI1 but not BMI2; leave BMI2 on and even --help dies of SIGILL.
4. Run it with the patched driver chosen per process:

LD_LIBRARY_PATH=$BUNDLE/driver
VK_DRIVER_FILES=$BUNDLE/driver/radeon_icd.x86_64.json
RADV_DEBUG=llvm                    # the whole ballgame
MESA_SHADER_CACHE_DIR=$BUNDLE/caches/<model>

llama-cli compiles its whole pipeline set at startup, and cold compiles on 1.6 GHz Jaguar cores have taken 15–25 minutes. So the house pre-warms one shader cache per model on the bigger box and ships it with the driver. The cache key includes the driver build, so ship them as a pair. Long sessions are the other hazard: a runtime GPU ring wedge showed up after roughly 15–25 flights in one boot (LFM2.5, long context), and the only cure was a reboot. Run in bounded batches.

Four ways to fool yourself

1. The patch only touches the LLVM path. RADV still defaults to ACO, so a plain -ngl 99 runs the unpatched compiler. Set RADV_DEBUG=llvm on every run.
2. Assert the backend, not the GPU name. ACO also enumerates as Vulkan0: AMD Jaguar (RADV KAVERI). Only the LLVM path appends the compiler: (RADV KAVERI (LLVM 20.1.2)). Grep llama-cli --list-devices for LLVM.
3. Watch for a silent CPU fallback. One missing shared library made the Vulkan loader reject the patched driver; llama.cpp said no usable GPU found on stderr only, then produced coherent text at exactly CPU speed.
4. Read the words, not the stopwatch. ACO benchmarks 16% faster than LLVM and returns zero characters. Every speed on this page is paired with output somebody read.

Also: RADV_FORCE_FAMILY=kaveri is useful for replaying shader pipelines on a null device on another machine, and it segfaults the real console. Leave it off on actual hardware. A separate LLVM-path abort in SOFT_MAX f16 promotion was still open when these notes were cut; it did not stop the models above.

A scent for Barnacle: Three coordinates enter the compiler; all three come out.