perf(proot): accelerate and cache glibc syscall relays (#1687)

Libc-internal file operations bypass the existing preload wrappers and
still stop in PRoot. Patch recognized ARM64 SVC sites in libc and the
runtime loader, reuse the current translator for 20 path syscall
numbers, and return declined calls to PRoot. Enabled by default after
the tracer-key check; `PROOT_FP_PATCH_SVC=0` remains the troubleshooting
opt-out.

The hardened implementation patches 47 sites on Thor's glibc 2.43.
Relays preserve general/SIMD registers, flags, FP state and errno, and
provide unwind metadata back to the original libc frame. Text is patched
through `/proc/self/mem` after relay pages become executable.
Initialization skips multiple live threads and SVE/SME. Alternate signal
stacks and insufficient remaining stack use PRoot; main-stack growth
respects the current limit. No proroot code or binaries are imported.

Robust-list set/get sites deliberately remain with PRoot/the kernel.
Device tests reproduced stale state when the earlier private-TLS
implementation mixed patched and raw callers. Removing that shortcut
fixes coherence and preserves the host's actual return policy; the
earlier thread-creation gain is retired. SDK forwarding still enables
faccessat2 ENOSYS fallback and clone3 ENOSYS only before Android 15.
Successful clone3 is never executed from the C relay. Existing fsync
handling is unchanged. No in-process RET_TRAP filter is added, so
SIGSYS-mask/handler machinery is not applicable.

Version-2 build-ID/API tables cache relative patch sites in `/tmp`.
Every hit revalidates executable bounds, alignment, instructions and
syscall numbers. Invalid/unsafe files rescan; publication uses a private
temporary file and atomic rename. The PRoot dependency fetch also gains
a mirror matching the existing pinned checksum, resolving upstream
availability failures without changing versions.

Five alternating pairs using the final bundled library on Thor/Android
13:

| operation | wrappers only | hardened relays |
|---|---:|---:|
| libc-internal stat | 49.6 us | 6.0 us |
| libc-internal open/close | 50.4 us | 7.8 us |
| libc faccessat | 49.7 us | 22.0 us |
| libc getcwd | 54.9 us | 5.0 us |
| thread create/join | 136 us | 144 us |
| complete tiny exec/wait | 4.44 ms | 5.60 ms |

An earlier five-pair hardened-library run measured stat 50.1 to 5.9 us
and open/close 51.9 to 7.1 us. The metadata gains survive the safety
checks. Neither run shows a thread-creation gain, and tiny execs retain
roughly 0.9-1.2 ms overhead.

Three full Steam startup pairs yielded Home-ready medians of 47.02 s off
/ 45.25 s on, with overlapping 43-50 s ranges. Steam/CEF-only PSS
medians were 1,223.1 / 1,233.9 MiB; the roughly 11 MiB difference is
smaller than observed variation, but does not establish zero memory
cost. Startup/memory used one unchanged signed bundled-library APK
without overrides; the final stack-gap guard only increases distance
reserved above other mappings. No reliable whole-client startup
improvement is claimed.

Earlier fresh Half-Life SD installs took 350/341 s off and 326/327 s on.
These are historical results from the preceding robust-list revision,
not a post-hardening download retest. Network/shader scheduling varied,
allocation did not reliably improve, and the repeat baseline committed
faster. This PR demonstrates metadata acceleration; it does not
demonstrate a reliable fix for Steam's SD allocation/commit bottleneck.

Validation on `2c3c135b` (production code unchanged since `4998b046`):

- Local release build, runtime/native-library checks, release signing
and four focused Android tests pass. [Final-head
CI](https://github.com/Droid-Deck/DroidDeck/actions/runs/40704837477)
passed: 1,431 Python/native tests (49 skips), 2,648 Android tests (zero
failures/skips), and 162 Rust tests. CodeRabbit is green on the same
head. The installed production APK forwards API 33 and patches 47 sites
with an unset enable flag and no override; Steam Home renders.
- Embedded and shared-PIC matrices pass on glibc 2.43; shared-PIC also
passes on separately loaded glibc 2.39. Coverage includes raw
results/errno/registers/FP state, path/flag/error and protected-pointer
cases, exact readlink boundaries, mixed raw/relay robust state,
concurrent threads, fork, thread churn, alternate-stack signals,
deferred/FIFO cancellation, signal siglongjmp recovery, forced unwind
with caller cleanup, and limited remaining stack.
- Four device test-process filters deny the memory patch file, relay
mapping, executable permission, or patch writes. Every case leaves
original syscall execution working. Host injection also verifies partial
installation after an individual site fails.
- Concurrent atomic symlink replacement passes 3,000 internal
stat/readlink iterations at default/zero TTL. Eight focused host tests
pass, covering instruction/branch limits, stack/limit fallback,
untrusted cache files and NDEBUG checks. API selection gives 45/47/46
sites for unknown/33/simulated-35; unset and explicit 1 agree, and 0
leaves sites untouched. This does not replace Android 15 hardware
testing.
- All bench tools compile with -DNDEBUG. Relative/absolute fixture
workloads agree, and thread/ABI/failure checks pass on Thor with those
builds.
- Earlier internal fts/realpath/NSS/dlopen and normalized read-path
equivalence pass. The write suite retains the existing wrapper SD
Unix-socket mismatch against PRoot.

CodeRabbit disposition: fixed relative fixture paths, prevented
benchmark checks disappearing under NDEBUG, corrected the
limited-fallback description, and narrowed memory-file denial so the
thread check still succeeds. The default-off recommendation was
addressed by fixing/testing the underlying guard and unwind issue while
retaining the requested default-on behavior. The readlink count
suggestion is intentionally not adopted: device boundary comparisons
reproduce current PRoot behavior, so changing that inherited policy
belongs in a separate correctness change.

This is not consequence-free instruction patching: constructor time,
private text copies and stack checks have costs. Android 15+ ARM64 and
16 KiB-page hardware, arbitrary asynchronous cancellation/non-local
exits, user-managed stacks, and attributable tracer-counter measurements
remain gaps. Loader calls before the constructor and arbitrary guest ELF
syscall sites remain outside scope.

Thor was restored to its original APK and bundled library, with app data
preserved and test settings, overrides, temporary files and version-2
test caches cleared.

Detailed evidence and revision boundaries: [proroot
analysis](https://github.com/Droid-Deck/DroidDeck/blob/2c3c135b52c3dd949ef5a99a67d8c12eba24406c/docs/development/proroot-analysis.md).

app/src/main/java/com/droiddeck/launcher/runtime/ProotFastPath.kt

@@ -1,6 +1,7 @@
 package com.droiddeck.launcher.runtime
 
 import android.content.Context
+import android.os.Build
 import android.util.Log
 import com.droiddeck.launcher.session.SessionPrefs
 import java.io.File
@@ -37,9 +38,10 @@ object ProotFastPath {
 
     fun hostEnv(key: String): Pair<String, String> = "PROOT_FASTPATH" to key
 
-    fun guestEnv(root: File, binds: List<String>, key: String): List<String> = listOf(
+    fun guestEnv(root: File, binds: List<String>, key: String, sdk: Int = Build.VERSION.SDK_INT): List<String> = listOf(
         "PROOT_FP_ROOT=" + root.path,
         "PROOT_FP_BINDS=" + binds.joinToString("|"),
         "PROOT_FP_KEY=$key",
+        "PROOT_FP_ANDROID_API=$sdk",
     )
 }

app/src/test/java/com/droiddeck/launcher/runtime/ProotFastPathTest.kt

@@ -10,6 +10,12 @@ import java.io.File
 class ProotFastPathTest {
     private val root = File("/data/data/com.droiddeck.launcher/files/linuxfs")
 
+    @Test fun guestEnvironmentCarriesTheHostAndroidApi() {
+        for (sdk in listOf(33, 35, 36)) {
+            assertTrue(ProotFastPath.guestEnv(root, emptyList(), "test", sdk).contains("PROOT_FP_ANDROID_API=$sdk"))
+        }
+    }
+
     @Test fun aSessionsUsualBindsGetAKey() {
         val binds = (1..87).map { "/host/$it:/guest/$it" }
         assertNotNull(ProotFastPath.key(root, binds))

docs/development/proroot-analysis.md

@@ -0,0 +1,547 @@
+# proroot: what it does and what DroidDeck can take from it
+
+Static analysis of [coderredlab/proroot](https://github.com/coderredlab/proroot) v1.2.8
+(`c01039ab`, 2026-06-17). The repository holds only five arm64 binaries and a README. The licence is
+proprietary, and redistributing modified binaries is not allowed, so only the ideas below are
+usable, not the code. The binaries were read with the NDK's `llvm-readelf`, `llvm-nm`,
+`llvm-objdump` and `llvm-strings`. `libproroot-stub-loader.so` and `libproroot-bridge.so` still carry
+`.symtab`, and `libproroot-runtime.so` exports about 260 names, so the function names below are
+proroot's own.
+
+## Architecture
+
+proroot does not use ptrace. Everything runs inside the guest process.
+
+| file | kind | role |
+|---|---|---|
+| `libproroot.so` | bionic PIE executable | Launcher. Parses `-r/-b/-w/-0/--link2symlink`, writes a config file (`.proroot-config-<pid>`, passed down as `PROROOT_CFG_FD`), and execs the guest through `libproroot-linker.so` or the stub loader. |
+| `libproroot-linker.so` | freestanding PIE, run as an executable | A from-scratch glibc dynamic linker ("cleanroom linker") that replaces `ld-linux-aarch64.so.1`. It maps the program and its `DT_NEEDED` libraries, relocates them, sets up static TLS, and stubs `_dl_*` and `__tunable_get_val`. It finds `_rtld_global_ro` through a table of offsets per glibc version ("no offset table for glibc X, patches skipped"). It also installs a SIGSYS handler with an allow-list. |
+| `libproroot-stub-loader.so` | freestanding static PIE | Userland `exec`. It maps the target and its `PT_INTERP` itself and builds the stack and auxv. It then **rewrites the `svc #0` sites** in the loaded images (`proroot_svc_scan`, `proroot_svc_patch_apply`, `select_trampoline_for_site`) to branch into per-syscall trampolines (`proroot_tramp_openat`, `_statx`, `_faccessat2`, `_clone3`, `_rt_sigaction`, `_shmget`, ...). Sites it cannot patch fall back to an in-process seccomp `RET_TRAP` filter with a SIGSYS handler ("trampoline fallback, activating trap-mode fallback"). |
+| `libproroot-runtime.so` | glibc `LD_PRELOAD` | Hooks about 200 libc entry points: path translation, fake root (`fake_id0_*`), link2symlink, exec routing, SysV shm, netlink, `/proc` and `/sys` views, `brk`, a fake DRM/KMS and GBM device plus libseat for compositors, X11 and Unix-socket fixups. After load it **patches the svc sites in glibc itself** (`patch_libc_syscalls`). |
+| `libproroot-bridge.so` | static `ET_EXEC` at `0x7000000000` | The "trampoline" for static guest binaries. It installs a path-trap seccomp filter plus a SIGSYS handler, maps the target, and serves a virtual `/proc` (`status`, `stat`, `cmdline`, `environ`, `mounts`, `mountinfo`, `fdinfo`, `auxv`). |
+
+The executable files all live in `nativeLibraryDir`, which is the only place an app may exec from.
+That is why the dynamic linker and the static loader each have their own userland loader. proot
+handles the same constraint with `libproot-loader.so`.
+
+## How the in-process syscall interception works
+
+1. **Patching svc sites in glibc** (`patch_phdr_callback`, called through `dl_iterate_phdr`):
+   - It scans only an allow-list of objects: `libc.so.6`/`libc-`, `ld-linux`, `libpthread`,
+     `librt`, `libdl`. Main executables get patched early only for listed programs (apt, dpkg,
+     curl, openssl, node, chromium/electron/playwright, passwd/useradd/...). Other programs are
+     patched later (`log_deferred_libc_patch`, `PROROOT_FORCE_EARLY_PATCH`).
+   - In each executable `PT_LOAD` it finds `svc #0` (`0xd4000001`). It checks the instructions
+     before it for a `mov x8, #nr` it recognises. One fixed glibc `openat` call sequence gets
+     special treatment: its `bl` is redirected to a relay (`emit_openat_call_relay_entry`).
+   - The site is replaced with a `b`/`bl` to a relay stub on a page allocated within ±128 MiB
+     (`alloc_relay_page`). Sites out of range are logged as `bl-out-of-range`; `large-image
+     multi-cover tramp` places extra relay pages for big images.
+   - **Write path:** it first tries `mprotect(PROT_READ|PROT_WRITE|PROT_EXEC)` on the segment.
+     If that is refused, it `pwrite`s the 4-byte instruction through `/proc/self/mem`. Either way
+     it then calls `__clear_cache`. The log line counts sites by kind: `patched %d seccomp + %d
+     path (%d relay) + %d brk`.
+2. **In-process SIGSYS fallback**:
+   - A BPF filter allows syscalls whose instruction pointer is inside proroot's own code and
+     returns `SECCOMP_RET_TRAP` for the path, exec, clone and wait syscalls from anywhere else.
+     The IP check is the same idea as our patch 0014, with TRAP instead of TRACE.
+   - The handler emulates the call and writes the result into the signal frame. Guest SIGSYS
+     handlers are kept and chained (`proroot_record_user_sigsys`, `invoke_user_sigsys`).
+3. **Keeping SIGSYS deliverable**, the objection in option 4 of
+   [proot-performance](proot-performance.md):
+   - The `sigprocmask` and `pthread_sigmask` hooks call `sigtrap_unblocked_mask`, which
+     `sigdelset`s `SIGSYS` (and `SIGTRAP` for some guests) from any `SIG_BLOCK`/`SIG_SETMASK` set.
+   - `rt_sigaction` is guarded so the guest cannot replace the handler (`proroot_tramp_rt_sigaction`,
+     the `sigaction_guard` trace points).
+   - The raw `rt_sigprocmask` that glibc makes internally, for example `pthread_create` blocking
+     every signal, goes through a patched svc site. The stub loader and the runtime both compare
+     against syscall 135.
+4. **Android-blocked syscalls are answered in-process.** The stub loader dispatches
+   `set_robust_list`, `rseq`, `faccessat2`, `clone3`, `futex_waitv` and `seccomp`. The runtime
+   emulates SysV shm (`emu_shm*`), `set*id`/`get*id` under `-0`, and `brk`
+   (`emulate_guest_brk`, over a reserved mapping).
+
+## Compared with DroidDeck
+
+| | DroidDeck (proot + 0001–0018 + `libblfastpath.so`) | proroot |
+|---|---|---|
+| exec, `PT_INTERP`, `/proc/self/exe` | proot's loader under ptrace, 0.4–0.8 ms per exec | Its own loaders, with no tracer; also needs its own ld.so with a glibc offset table |
+| path calls through libc wrappers | fast path, about 0.6–1.9 µs | runtime hooks |
+| path calls from glibc internals (ld.so, fts, realpath, nss, `__open_nocancel`) and raw `svc` | **still go to proot (15–60 µs)** | **patched svc sites** |
+| Android-blocked syscalls (`set_robust_list`, `faccessat2`, `clone3`, ...) | SIGSYS stop answered by proot (0016 etc.), one ptrace round trip and one signal round trip each | in-process |
+| `brk` | proot emulation, a trap per call (mitigated by `top_pad`) | in-process emulation |
+| correctness fallback | proot is always there | limited in-process seccomp/SIGSYS handling; no general tracer fallback, so calls outside the filter can run unconfined or be emulated incorrectly |
+| long Unix socket paths | fast path `/proc/self/fd/N/name` aliases | short socket aliases (`proroot_create_short_unix_socket_alias`) |
+| rseq | `GLIBC_TUNABLES=glibc.pthread.rseq=0` | trampoline |
+
+proroot is not a model for replacing proot. It has no tracer to fall back on, so a missed call
+site or an unexpected glibc build silently breaks isolation or correctness. Its own linker depends
+on glibc internals for each version. Proton, Wine, FEX and Steam's self-updating ELFs are a much
+wider surface than the Node, Python and Chromium workloads proroot was tested on. Its value for
+DroidDeck is as a fast path *in front of* proot.
+
+## What is worth taking
+
+1. **Patch glibc's svc sites from `libblfastpath.so`'s constructor.** This is the largest gain
+   and the cheapest way to do option 6 in [proot-performance](proot-performance.md) ("a proot-aware
+   glibc") without rebuilding glibc.
+   - For `libc.so.6` (and `ld-linux-aarch64.so.1` for runtime `dlopen`), rewrite each `svc #0`
+     that follows a recognised `mov x8, #nr` for the path syscalls the fast path already handles.
+     The rewrite is a `b` to a relay that calls the existing translator.
+   - The relay issues the syscall from the `0xffff00000` trampoline page, which 0014 already lets
+     through. Anything the translator declines is re-issued from an svc in the relay page, outside
+     the trampoline, so proot traps it exactly as it does today. Correctness still rests on proot.
+   - This covers fts (`find` over 2.7k files, unchanged today), realpath, nss, `__open_nocancel`,
+     and `dlopen` after start-up.
+   - Library loading at start-up, before the constructor runs, still traps.
+2. **Answer the Android-blocked syscalls through those relays.**
+   - `set_robust_list` and `get_robust_list`: keep the head in TLS, as 0016 already does in
+     proot.
+   - `faccessat2`: answer `ENOSYS`, or translate it when flags are 0, as the wrapper does.
+   - `clone3`: answer `ENOSYS` before Android 15, so glibc falls back to `clone`.
+   - `futex_waitv`: droiddeck-fsync already handles this.
+   - Each one removes a SIGSYS stop plus a ptrace round trip, about 30–50 µs. `set_robust_list`
+     alone is one per thread, which matters for Steam/CEF and Wine thread churn.
+3. **The `/proc/self/mem` write as a fallback for patching.** If the app domain refuses
+   `mprotect(PROT_WRITE|PROT_EXEC)` on rootfs (`app_data_file`) text, i.e. an `execmod` denial,
+   a write through `/proc/self/mem` makes the page copy-on-write without an `mprotect`. That page
+   is never writable and executable at once. Check this on the Thor before relying on it. The
+   same question is still open for the trampoline page (`execmem`).
+4. **Keep SIGSYS unblocked if in-process `SECCOMP_RET_TRAP` is ever used.** Strip `SIGSYS` in the
+   `sigprocmask`/`pthread_sigmask`/`rt_sigprocmask` paths, and keep the handler installed while
+   chaining the guest's. This only becomes safe once item 1 covers glibc's internal
+   `rt_sigprocmask`. It is a prerequisite, not a recommendation to switch.
+5. **Cache patch sites by build-id.** proroot rescans each image on every exec and defers patching
+   for most programs to keep exec cheap. A scan of libc's roughly 1.5 MiB of text is cheap, but a
+   per-build-id site table, written once in `/tmp` by the fast path, would make it free.
+
+Not worth taking: the cleanroom ld.so and its glibc offset table; the userland exec loader (proot
+already does this); the fake DRM/GBM/libseat device (DroidDeck has its own compositor path);
+netlink and SysV shm (`netif.c` and `sysv.c` already cover these); and the long-socket aliasing
+(the fast path covers it).
+
+## Open questions before prototyping item 1
+
+- Does the app domain allow `mprotect(RWX)` on a private mapping of an `app_data_file`? Does a
+  `/proc/self/mem` write work under the app's SELinux context and proot's own `ptrace` attach?
+  `/proc/self/mem` writes need `PTRACE_MODE_ATTACH` on self, which is always allowed.
+- How many `libc.so.6` path syscall sites fall outside a recognisable `mov x8, #nr` window? For
+  example, `syscall()` takes the number in a register, which needs the existing `syscall()` hook.
+- Patched pages become private copies, about 4 KiB per patched page per process. Measure RSS
+  across the Steam/CEF tree.
+- Equivalence: run `bench/equiv.py` plus a `find`/`realpath`/nss workload with `PROOT_FP_STATS`.
+  Compare the trap counts from patch 0018 before and after.
+
+## First implementation and Thor validation (2026-10-27)
+
+`tools/proot/fastpath/svc-patch.h`, included by `fastpath.c`, implements the first
+slice of item 1. It runs automatically in fresh processes using the replacement
+`libblfastpath.so` when the ordinary fast-path configuration and tracer-key check
+succeed. Set `PROOT_FP_PATCH_SVC=0` to disable it for troubleshooting or comparisons;
+`PROOT_FP_PATCH_SVC=1` also explicitly enables it. It skips processes that already have multiple threads,
+and declines SVE/SME hardware because the relay preserves the base ARM64 register state.
+
+The scanner visits executable segments of the loaded `libc.so.6` and
+`ld-linux-aarch64.so.1`. It only patches `openat` and `newfstatat` sites with an
+unshifted `movz x8/w8, #nr` within the preceding eight instructions. Intervening
+instructions must belong to a small allow-list of argument moves/sign-extension,
+immediate moves, stack loads/stores and NOPs that leave x8 unchanged. Branches,
+calls, x8 writes and unknown instructions stop recognition. This includes libc's
+non-adjacent O_CREAT sequences. Register-number dispatchers and other syscall
+numbers remain under PRoot. No glibc version offsets, proroot code, or modified
+proroot binaries are used.
+
+Each replacement is a four-byte `b` into a nearby relay. Relay pages are filled while
+writable, then changed to RX before publishing any branch. Text writes use
+`/proc/self/mem`; failure leaves the site unpatched. The relay preserves x1–x30, all
+32 SIMD registers, NZCV, FPCR/FPSR and the caller's `errno`, returning a raw kernel-style
+result in x0. It calls the existing path translator, using the existing allowed
+trampoline for translated calls. Declined and recursive calls execute an ordinary
+`svc` in this library, where PRoot still intercepts them. No SIGSYS handler or new
+seccomp filter is installed. `PROOT_FP_STATS=1` adds `fastpath-svc` exit counters.
+
+The connected AYN Thor was running runtime r9, glibc 2.43, with a 4 KiB page size in
+`u:r:untrusted_app_27:s0`. `bench/svc-probe.c`, run through the app's guest bridge,
+confirmed all of the following while PRoot was attached:
+
+- An RX private mapping of the rootfs libc accepted `mprotect(RWX)`.
+- `/proc/self/mem` accepted a changed byte in that RX mapping; readback and restoration
+  succeeded, and the backing file stayed unchanged.
+- An anonymous RW mapping could be changed to RX.
+- libc contained 415 `svc #0` sites: 327 with an adjacent number load, 76 with a
+  number load only within the preceding eight instructions, and 12 with neither.
+  The linker contained 56 sites, 45 adjacent and 11 nearby-only. Nearby counts are
+  scan statistics, not proof that the number remains live at the syscall.
+
+The initial adjacent-only scanner patched nine sites; the final scanner patched
+14, including file-creation paths, across libc and the linker. It used two relay
+pages in standalone validation processes. `bench/svc-check.c` exercises the
+actual emitted relay and checks raw success/error results, errno, x1–x30, q0–q31,
+NZCV, FPCR/FPSR, four simultaneous threads, the multi-thread patching guard, and fork.
+Null arguments and unsupported flags fall back to PRoot. The instruction decoder
+and exact ±128 MiB branch limits also have a host regression test.
+
+Default-policy validation on the Thor confirmed that an unset `PROOT_FP_PATCH_SVC`
+and explicit `1` both patched 14 sites; explicit `0` left all 415 libc and 56 linker
+syscall sites unchanged. The register/thread/fork check also passed with the
+environment flag unset.
+
+Five alternating pairs of `bench/svc-workload.c` runs compared the same replacement
+library with patching off/on, with fast-path stats enabled in both. The workload
+gets `fstatat` and `__open64_nocancel` directly from libc, bypassing the preload's
+public wrappers, and makes 5,000 of each call against a prepared regular file:
+
+| median | wrappers only | svc patching |
+|---|---:|---:|
+| internal `fstatat` | 38.6 µs | 3.0 µs |
+| internal open + close | 39.3 µs | 5.8 µs |
+| voluntary context switches over both loops | 10,000 | 0 |
+
+### SD allocation and writes
+
+`bench/svc-storage.c` separates the operations relevant to Steam downloads. Three
+alternating pairs ran on internal `/tmp` and the selected `/mnt/droiddeck-sd` library,
+with Steam idle and no simultaneous local Steam download. Each fresh directory held 64 long-named files
+extended to 256 KiB, followed by 1,000 libc-internal stats and a 64 MiB sequential
+write followed by `fdatasync`. Both modes used the same replacement library; only
+`PROOT_FP_PATCH_SVC` changed. Medians on SD were:
+
+| phase | wrappers only | svc patching |
+|---|---:|---:|
+| create/extend/close 64 files | 1.500 s | 1.486 s |
+| 1,000 internal stats | 61.2 ms | 5.4 ms |
+| 64 MiB write calls | 43.9 ms | 40.9 ms |
+| final `fdatasync` | 2.514 s | 2.539 s |
+
+Internal allocation fell from 7.1 ms to 1.9 ms; internal stats fell from 39.5 ms
+to 2.5 ms. Internal `fdatasync` varied from 36 to 164 ms across individual runs.
+The patched SD workload recorded 1,065 internal calls handled in-process, covering
+64 file creations, 1,000 stats and the streaming-file creation. Per-phase medians
+are independent and should not be added as a median whole-run duration.
+
+The SD metadata gain does not establish faster Steam downloads. File creation
+improved by only a few percent, and write/flush speed did not improve in this probe.
+The remaining SD cost is much larger than the saved syscall-interception time.
+The earlier [Thor SD investigation](thor-sd-storage-investigation.md) likewise
+identified native exFAT metadata waits; the current benchmark does not independently
+attribute those waits to a particular mount flag. A real Steam install comparison
+must separate preallocation, transfer/staging and final commit before claiming an
+end-to-end improvement.
+
+### Real Steam downloads to SD
+
+Four fresh installs of owned, initially uninstalled **7 Billion Humans** (792100)
+used library folder 1, `/mnt/droiddeck-sd`, and were uninstalled between trials.
+Each downloaded 228,935,584 bytes and staged 237,717,350 bytes in 19 files, ending
+at BuildID 8016557, depots 792101/5052600457178655633 and
+792104/8245917369753676198. Every trial logged zero reused/stored/delta bytes at
+the start and completed with `No Error`. Existing installed games were retained.
+
+The repeat off/on pair used the exact same replacement library, SHA-256
+`ab605c021aee271cd49fa1cb337d7ff0c762dbac198b711e9818639c0cd4b17d`.
+The first off run used the earlier adjacent-only build, with patching disabled.
+Only the patching environment flag changed for the final pair. Times below come
+from Steam's one-second-resolution `content_log.txt`; total starts at Reconfiguring
+and ends at scheduler completion, excluding session startup:
+
+| trial | patching | preallocate | download/stage | commit to completion | total |
+|---|---|---:|---:|---:|---:|
+| first baseline | off | 1 s | 123 s | 1 s | 135 s |
+| first patched | on | <1 s | 47 s | 1 s | 50 s |
+| repeat baseline | off | <1 s | 30 s | 1 s | 33 s |
+| repeat patched | on | <1 s | 29 s | 2 s | 32 s |
+
+Reconfiguration accounts for the remaining total time. The first baseline
+overlapped another Steam client's network download; that client's telemetry must
+not be mistaken for the Thor's. Later samples were filtered to local client `0`.
+The repeat pair reached similar network rates, roughly 7–8 MB/s while transferring.
+All four attempts logged the same unsupported large-file preallocation retry.
+
+**This does not demonstrate a useful SD-download speedup.** The apparent initial
+135→50 second gain disappeared when baseline was repeated. The final 33 versus
+32 seconds is within this small experiment's timing/network variability. This
+19-file title also does not settle performance for games with thousands of loose
+files. Metadata calls clearly improve, while SD creation/writeback remain the
+stronger next targets. No network settings, SD mount flags or root privileges
+were changed for these tests.
+
+The internal-call equivalence workload, including fts, realpath, reentrant NSS,
+runtime dlopen, missing paths and fallback errors, matched `PROOT_FP_OFF=1` at both
+the default parent-cache TTL and TTL 0. The existing `equiv.py` also matched after
+normalizing random/thermal read payloads to their lengths, volatile PID values in
+the `/proc` directory listing, and tolerating Android's
+denied `/dev` directory read. Its root fixtures were relocated into a temporary tree.
+
+`equiv2.py` used temporary root/SD fixtures, normalized `/proc/<self-pid>` output,
+and tolerated the SD filesystem's unsupported symlink creation. Its patched output
+matched the wrapper-only output at both TTLs. **It did not fully match PRoot:**
+binding a Unix socket in the selected SD bind returned EPERM with the existing
+wrappers, whereas PRoot succeeded. This mismatch reproduced with svc patching off;
+it is a separate existing fast-path socket issue. The suite is not a clean full
+equivalence pass. `equiv3.py` was not run because it rewrites live NSS configuration.
+
+A fresh Steam session using the override and `PROOT_FP_PATCH_SVC=1` rendered its
+library UI. Sonic Mania (584400) gained focus and rendered its title screen. This
+is startup smoke coverage, not gameplay or a Steam startup-speed measurement.
+The installed PRoot pack emitted no profiling start/counter records despite the
+queued profiling setting; the profiler report had `hasCounters: false`. That is a
+measurement gap, not evidence of zero remaining traps.
+
+### Expanded relay coverage (before the safety follow-up)
+
+The subsequent revision keeps patching enabled by default and shares the existing
+wrapper translators with 20 path syscall numbers: openat, newfstatat, statx,
+readlinkat, faccessat, mkdirat, unlinkat, renameat, renameat2, symlinkat, fchmodat,
+utimensat, truncate, statfs, getxattr, lgetxattr, listxattr, llistxattr,
+inotify_add_watch and getcwd. This is an allowlist of safely recognized sites, not
+a claim that every glibc implementation contains every site. Unsupported paths,
+flags, instructions and inaccessible pointers still use the original PRoot route.
+Input/output copies use process_vm_readv/writev rather than dereferencing arbitrary
+guest pointers. The device matrix checks error priority, protected buffers and a
+NUL-terminated path ending at a page boundary.
+
+The app forwards its Android SDK as `PROOT_FP_ANDROID_API`. With a known SDK,
+set_robust_list retains the head in TLS and returns ENOSYS, matching patch 0016;
+self get_robust_list uses that head. Uncaptured heads and other TIDs fall back to
+PRoot. This does not implement kernel robust-mutex owner-death recovery. faccessat2
+returns ENOSYS for glibc's fallback. clone3 returns ENOSYS only before API 35;
+it is never executed from the C relay, since successful clone3 would change the
+child's stack. The [Android 14 allowlist](https://android.googlesource.com/platform/bionic/+/refs/heads/android14-release/libc/SECCOMP_ALLOWLIST_COMMON.TXT)
+omits clone3 and the [Android 15 allowlist](https://android.googlesource.com/platform/bionic/+/refs/heads/android15-release/libc/SECCOMP_ALLOWLIST_COMMON.TXT)
+includes it. Unknown platforms leave these sites untouched. Existing droiddeck-fsync
+handling of futex_waitv is unchanged.
+
+Build-ID tables in `/tmp/.droiddeck-svc-v1-<api>-<build-id>` cache relative site
+offsets. Every hit revalidates executable-segment bounds, alignment, the current
+SVC opcode and the recognized syscall number. Missing, malformed, truncated,
+overly permissive or symlinked tables trigger a rescan; writes use a private
+temporary file and atomic rename. Images without a readable GNU build-ID still
+scan normally. `PROOT_FP_SVC_CACHE=0` disables caching for comparison. Changes to
+the accepted instruction/syscall set must bump the cache version.
+
+On the Thor's glibc 2.43 / Android 13, 49 sites patch across libc and the loader.
+Warm cached discovery took 60–63 microseconds in two samples versus roughly
+0.6–0.7 milliseconds for the earlier uncached scan; patch publication took about
+0.28 milliseconds. These are constructor phases, not whole Steam startup times.
+Five alternating expanded-library pairs measured median internal stat at 39.5→4.9
+microseconds and open/close at 41.2→6.4 microseconds, with 10,000→0 voluntary
+context switches over both loops. The earlier SD/download tables above describe
+the initial 14-site revision and must not be attributed to this expanded build.
+
+Five alternating pairs of `svc-threads` additionally measured thread create/join
+at 138→113 microseconds, libc faccessat at 51.6→14.7 microseconds, and libc getcwd
+at 46.8→4.1 microseconds. Each run creates 128 threads and executes 2,000 calls
+to each libc entry point, bypassing the exported preload wrappers with dlsym.
+The access call includes glibc's faccessat2 fallback.
+
+Repeated `/usr/bin/true` launches expose the startup tradeoff. Five batches of
+40 children per mode measured medians of 3.834 ms with wrappers only, 4.882 ms
+with uncached patching and 4.731 ms with cached patching, including subprocess
+launch/wait and enabled exit statistics. Discovery across the 200 launches per
+patched mode fell from a median 496 to 59 microseconds; publishing patches took
+261/267 microseconds. The cache removes scanning work, but constructor setup and
+patching still cost time for very short programs. No overall exec speedup is claimed.
+
+Three alternating storage pairs with the expanded library measured:
+
+| SD phase | wrappers only | expanded relays |
+|---|---:|---:|
+| create/extend/close 64 files | 1.556 s | 1.514 s |
+| 1,000 internal stats | 73.2 ms | 12.5 ms |
+| 64 MiB write calls | 54.2 ms | 55.1 ms |
+| final fdatasync | 2.799 s | 2.730 s |
+
+These phase medians do not establish a general write-speed improvement. Internal
+allocation was 7.1→3.6 ms and internal stats 49.8→7.7 ms; the SD filesystem's
+allocation and flush waits remain much larger than the interception savings.
+
+### Expanded-build Steam SD comparison
+
+Owned, initially uninstalled Half-Life (70) supplies a larger file tree: 4,525
+installed files / 589,603,955 bytes, BuildID 15961492. Each fresh trial initially
+scheduled 294,238,640 download bytes, 589,603,955 staged bytes and 4,488 preallocated
+files on `/mnt/droiddeck-sd`. Steam reported successful installation in every run.
+Local client `0` telemetry was selected. Times use one-second content-log timestamps;
+total runs from the first main-app Reconfiguring event to scheduler completion.
+
+| trial | patching | initial preallocate | transfer/stage window | final commit | total |
+|---|---|---:|---:|---:|---:|
+| first baseline | off | 90 s | 52 s | 204 s | 350 s |
+| expanded, default setting | on | 94 s | 48 s | 184 s | 326 s |
+| repeat baseline | off | 97 s | 69 s | 175 s | 341 s |
+
+The patched and repeat-baseline trials use the same final library SHA-256
+`af4f66680dcfbd5002f641ffe206cdc7bd89400be169d3ef21c4e0ee7f979589`.
+The first baseline used the preceding build with patching disabled; the only
+subsequent production change corrected explicitly cleared robust-list TLS state.
+All initial download records showed zero stored/reused/delta bytes. Shader updates
+interrupted the main download in each trial; the transfer window includes those
+pauses. On the repeat baseline, reconfiguration retained two small files already
+downloaded during that same trial, so the final resumed job scheduled 294,236,672
+download / 589,599,865 stage bytes and committed 4,572 entries rather than 4,574.
+This is not pre-existing payload reuse; the earlier fresh schedule is retained
+in the measurement records.
+
+The observed whole-install difference does not establish a reliable SD speedup:
+the repeat baseline committed faster than the patched run, while its transfer
+window was slower. Allocation also did not improve. The 326 versus 341–350 second
+totals combine network, shader scheduling and SD variability. The syscall gains
+are demonstrated independently; these samples do not show that they fix Steam's
+SD allocation/commit bottleneck. The test title was removed between trials; an
+interrupted first uninstall left a repair queue, which was cleared by removing
+only that task-created title while Steam was stopped before the subsequent fresh
+trial. Existing installed games were retained.
+
+The expanded ABI matrix passes raw return values, errno, all x1–x30 and q0–q31,
+NZCV and non-default FPCR/FPSR, four concurrent threads with distinct robust heads, fork, 100
+signals on an alternate stack, deferred cancellation, cancellation/cleanup of a
+blocked libc FIFO open, clearing/restoring a robust head, and 32 thread create/join
+cycles. Constructor patching still refuses a process with multiple live threads.
+Host tests exercise corrupted cache entries, invalid offsets/opcodes, mismatched
+API/version, unsafe file modes, symlinks and truncation. The Android unit test
+checks API forwarding. The existing preload/agent selection passed 79 tests with
+two skips before the cache regression test was added.
+API-selection probes patch 45 sites with an unknown SDK, 49 with API 33 and 48
+with simulated API 35; unset and explicit 1 agree, and explicit 0 leaves the
+original sites intact. This does not substitute for testing on Android 15 hardware.
+
+CI at 40b7c480 passed 1,427 Python/native tests (49 skips), 2,630 Android tests and
+162 Rust host tests, plus APK compilation. A local release APK built successfully,
+passed runtime-payload/native-library checks and was signed with the installed
+release certificate. The full local Windows JVM run reported 150 failures / 2,568
+tests (eight skips), including Unix-path, symlink and permission assumptions;
+the four ProotFastPath tests passed in both local variants. The passing full
+suite evidence is the Linux CI run, not a clean Windows full-suite result.
+
+Internal fts/realpath/NSS/dlopen and the normalized first equivalence suite still
+match PRoot at both cache TTLs. The write suite still matches wrapper-only output
+and reproduces the same existing SD Unix-socket mismatch described above.
+No in-process RET_TRAP filter is installed, so the proposed SIGSYS mask/handler
+machinery is not applicable. Startup loader traps before the constructor and
+raw calls in arbitrary guest ELFs remain outside the patch scope. Additional
+glibc builds, Android versions, unwind behavior, Steam/CEF tree RSS and matching
+patch-0018 tracer measurements remain coverage gaps.
+
+Build the tools with the same ARM64 glibc toolchain as the fast path:
+
+```sh
+aarch64-linux-gnu-gcc -O2 -Wall -Wextra -pthread \
+  -o svc-check tools/proot/bench/svc-check.c -ldl
+aarch64-linux-gnu-gcc -O2 -Wall -Wextra \
+  -o svc-probe tools/proot/bench/svc-probe.c -ldl
+aarch64-linux-gnu-gcc -O2 -Wall -Wextra \
+  -o svc-workload tools/proot/bench/svc-workload.c -ldl
+aarch64-linux-gnu-gcc -O2 -Wall -Wextra \
+  -o svc-storage tools/proot/bench/svc-storage.c -ldl
+aarch64-linux-gnu-gcc -O2 -Wall -Wextra -pthread \
+  -o svc-threads tools/proot/bench/svc-threads.c -ldl
+```
+
+Run `svc-probe` inside the app's guest environment, not from `adb shell`.
+`svc-check` embeds the fast path and requires its normal enabled tracer environment.
+For that embedded-library check, start with PROOT_FP_PATCH_SVC=0 when the session
+also preloads a fast-path library (including through /etc/ld.so.preload). The check
+then enables its own copy at main before creating threads, keeping another copy
+from owning the glibc patch sites. The other tools use the normal preload.
+For `svc-workload`, pass a prepared directory containing `file` and a `link` symlink
+to it; add any second argument to enable timing. Use the documented library override
+workflow in [proot-performance](proot-performance.md) for a fresh session. No extra
+environment setting is needed; use `PROOT_FP_PATCH_SVC=0` for the unpatched comparison.
+`svc-storage` takes a new, unused directory name; it creates and removes its own test
+files and directory. It writes 64 MiB plus 16 MiB of file extensions per run.
+
+### Safety follow-up (2026-10-28)
+
+Path patching remains enabled by default. Robust-list registration/query sites are
+now deliberately excluded: a device regression test reproduced disagreement
+between the relay's private TLS head and an unpatched raw get_robust_list call in
+the preceding implementation. Both directions now use the existing PRoot/kernel
+route, including cleared heads. This preserves the host's actual return policy:
+the installed Thor runtime accepts set_robust_list with zero, whereas the removed
+emulation returned ENOSYS. PRoot patch 0016 can instead return ENOSYS when Android
+blocks registration; neither policy is replaced by private relay state.
+
+The build-ID cache is version 2 because the accepted syscall set changed. The
+current glibc 2.43/API-33 build patches 47 sites rather than 49. faccessat2 and
+pre-Android-15 clone3 fallback acceleration remain; droiddeck-fsync is unchanged.
+The earlier 138-to-113-microsecond thread result belongs to the retired robust-list
+implementation and must not be claimed for the current code.
+
+Relays now save the original post-SVC program counter and supply unwind metadata
+for the saved caller registers. Device tests force an unwind from inside the
+translator through a patched libc frame and verify caller cleanup. A signal handler
+also exits with siglongjmp; a later call recovers the abandoned recursion guard
+and resumes acceleration. These tests supplement deferred and blocked-FIFO
+cancellation; they are not a claim of arbitrary asynchronous-cancellation safety.
+
+Compiler stack-usage output identified multiple nested PATH_MAX-sized frames.
+Before entering them, a small dispatcher checks the active alternate signal stack
+and cached writable stack mapping. Alternate-stack calls and calls with less than
+96 KiB remaining use PRoot. Main-stack growth respects the current RLIMIT_STACK
+and leaves space above the preceding mapping. The implementation adds only a few
+TLS words, not a large private translation stack. Tests cover a restricted thread
+stack, normal/alternate-stack signals, main-stack growth and a reduced/unavailable
+stack limit. User-managed stacks and arbitrary mid-helper non-local exits remain
+broader compatibility risks, not proven consequence-free cases.
+
+On Thor/Android 13, the embedded and shared-PIC matrices pass on glibc 2.43;
+the shared-PIC matrix also passes with a separately loaded glibc 2.39. They cover register/FP/errno preservation, mixed raw
+and relay robust calls, pointer and page boundaries, readlink buffers of target
+length minus one through plus two, thread churn, fork, signal/non-local exits,
+forced unwinding and limited stack space. Readlink preserves PRoot's existing
+translated-link length behavior; bytes beyond the reported payload are unspecified.
+
+Test-process seccomp filters separately deny opening /proc/self/mem, nearby relay
+mappings, executable mprotect and patch writes. All four device cases leave zero
+patched sites and a working original libc syscall. Host injection additionally
+checks that one failed site write permits other sites to install. Concurrent
+atomic symlink replacement passes 3,000 internal stat/readlink iterations at both
+zero and default cache TTLs. These are bounded tests; they do not prove every
+possible filesystem race or reproduce a real SELinux execmod denial.
+
+Five new alternating pairs measured:
+
+| operation | wrappers only | hardened relays |
+|---|---:|---:|
+| libc-internal stat | 50.1 us | 5.9 us |
+| libc-internal open/close | 51.9 us | 7.1 us |
+| libc faccessat | 52.0 us | 20.2 us |
+| libc getcwd | 51.7 us | 4.6 us |
+| thread create/join | 129 us | 144 us |
+| complete tiny exec/wait | 5.04 ms | 5.96 ms |
+
+The metadata gains survive the safety checks. This run shows no thread-creation
+gain, and the roughly 0.9 ms tiny-exec cost remains. The earlier SD download and
+flush results are historical measurements of the preceding build; no new claim
+that SD media writes or whole Steam installs become reliably faster follows from
+these syscall results. Android 15+ ARM64 and 16 KiB-page hardware remain untested.
+
+Three alternating pairs also measured session start through the Steam Big Picture
+Home page becoming ready, using its DOM/Recent Games content rather than merely
+the compositor's first frame. Off runs took 48.76/43.47/47.02 seconds; on runs took
+43.43/45.25/50.40 seconds. The 47.02-to-45.25-second medians sit within overlapping
+run-to-run variation and do not establish a reliable whole-client startup win.
+These use the signed bundled-library build without overrides; the final stack-gap
+guard only increases the reserved distance above other mappings.
+
+At Home-ready plus 20 seconds, three one-second-spaced process-memory samples per
+run gave Steam/CEF-only median PSS of 1,223.1 MiB off and 1,233.9 MiB on. Including
+their active Python/Wine descendants gave 1,243.7/1,254.6 MiB. The five Steam/CEF
+processes were selected by executable name; descendants were included by parent
+PID. Per-run Steam/CEF medians ranged 1,217.9–1,240.9 MiB off and
+1,194.8–1,244.9 MiB on. The roughly 11 MiB median increase is smaller than this
+variation, but it is not evidence of zero memory cost. PSS avoids counting shared
+pages in full for each process. Startup and memory comparisons used one unchanged
+APK; no downloads or games were initiated during them.
+
+CodeRabbit follow-up: the workload now compares canonical paths, so relative and
+absolute fixture arguments agree. All SVC bench tools keep their checks active
+under NDEBUG; their clock, syscall and thread operations cannot silently disappear.
+Both cases were rebuilt with -DNDEBUG and checked on Thor. The memory-file-denial
+test now blocks only O_RDWR opens and asserts the single-thread check still works,
+so it reaches the intended /proc/self/mem failure. The initial proroot comparison
+also distinguishes its limited in-process fallback from a general tracer.
+The request to return to default-off was addressed through the interruption/unwind
+fixes and tests while retaining the requested default. The proposed readlink count
+change was not applied because exact boundary tests reproduce the current PRoot
+behavior; changing that inherited policy belongs in a separate correctness change.

tools/proot/bench/checks.h

@@ -0,0 +1,10 @@
+#ifndef DROIDDECK_BENCH_CHECKS_H
+#define DROIDDECK_BENCH_CHECKS_H
+
+/* Benchmark assertions execute the operations being measured. */
+#ifdef NDEBUG
+#undef NDEBUG
+#endif
+#include <assert.h>
+
+#endif

tools/proot/bench/svc-check.c

@@ -0,0 +1,453 @@
+#if defined(__aarch64__)
+static void interruption_hook(const long *);
+#define FP_SVC_TEST_HOOK interruption_hook
+#endif
+#include "../fastpath/fastpath.c"
+#include "checks.h"
+#include <sys/wait.h>
+#include <linux/futex.h>
+#include <signal.h>
+#include <setjmp.h>
+
+#if defined(__aarch64__)
+static void (*interrupt_path)(void);
+static void interruption_hook(const long *r) {
+  (void)r;
+  if (interrupt_path) {
+    void (*hook)(void) = interrupt_path;
+    interrupt_path = NULL;
+    hook();
+  }
+}
+
+struct snapshot { long x[32]; unsigned char q[32][16]; unsigned long nzcv, fpcr, fpsr; };
+extern uint32_t svc_check_site;
+extern void svc_register_probe(struct snapshot *, const long *);
+
+__asm__(
+  ".text\n.p2align 4\n.type svc_register_probe,%function\nsvc_register_probe:\n"
+  "sub sp, sp, #1072\n"
+  "stp x19, x20, [sp]\nstp x21, x22, [sp, #16]\nstp x23, x24, [sp, #32]\n"
+  "stp x25, x26, [sp, #48]\nstp x27, x28, [sp, #64]\nstp x29, x30, [sp, #80]\n"
+  "stp q8, q9, [sp, #96]\nstp q10, q11, [sp, #128]\n"
+  "stp q12, q13, [sp, #160]\nstp q14, q15, [sp, #192]\n"
+  "stp x0, x1, [sp, #224]\n"
+  ".irp n,1,2,3,4,5,6,7,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
+  "mov x\\n, #\\n\n.endr\n"
+  ".irp n,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31\n"
+  "movi v\\n\\().16b, #0x5a\n.endr\n"
+  "mov x0, #0xa0000000\nmsr nzcv, x0\n"
+  "ldr x0, [sp, #232]\nldr x8, [x0, #48]\n"
+  "ldp x4, x5, [x0, #32]\nldp x2, x3, [x0, #16]\nldp x0, x1, [x0]\n"
+  ".global svc_check_site\nsvc_check_site:\nsvc #0\n"
+  ".irp n,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
+  "str x\\n, [sp, #(256 + \\n * 8)]\n.endr\n"
+  ".irp n,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31\n"
+  "str q\\n, [sp, #(512 + \\n * 16)]\n.endr\n"
+  "mrs x0, nzcv\nstr x0, [sp, #1024]\nmrs x0, fpcr\nstr x0, [sp, #1032]\n"
+  "mrs x0, fpsr\nstr x0, [sp, #1040]\n"
+  "ldr x0, [sp, #224]\nadd x1, sp, #256\nmov x2, #792\nbl memcpy\n"
+  "ldp q8, q9, [sp, #96]\nldp q10, q11, [sp, #128]\n"
+  "ldp q12, q13, [sp, #160]\nldp q14, q15, [sp, #192]\n"
+  "ldp x19, x20, [sp]\nldp x21, x22, [sp, #16]\nldp x23, x24, [sp, #32]\n"
+  "ldp x25, x26, [sp, #48]\nldp x27, x28, [sp, #64]\nldp x29, x30, [sp, #80]\n"
+  "add sp, sp, #1072\nret\n.size svc_register_probe, .-svc_register_probe\n"
+);
+
+static long probe(const long *args) {
+  struct snapshot out = {0};
+  unsigned long saved_fpcr, saved_fpsr;
+  __asm__ volatile("mrs %0, fpcr\nmrs %1, fpsr" : "=r"(saved_fpcr), "=r"(saved_fpsr));
+  unsigned long fpcr = saved_fpcr ^ (1UL << 22), fpsr = saved_fpsr | (1UL << 27);
+  __asm__ volatile("msr fpcr, %0\nmsr fpsr, %1" : : "r"(fpcr), "r"(fpsr));
+  errno = 123;
+  svc_register_probe(&out, args);
+  assert(errno == 123);
+  for (int i = 1; i <= 30; i++) {
+    long expected = i <= 5 ? args[i] : i == 8 ? args[6] : i;
+    if (out.x[i] != expected) { fprintf(stderr, "x%d: %lx != %lx\n", i, out.x[i], expected); abort(); }
+  }
+  for (int i = 0; i < 32; i++)
+    for (int j = 0; j < 16; j++) assert(out.q[i][j] == 0x5a);
+  assert(out.nzcv == 0xa0000000UL);
+  assert(out.fpcr == fpcr && out.fpsr == fpsr);
+  __asm__ volatile("msr fpcr, %0\nmsr fpsr, %1" : : "r"(saved_fpcr), "r"(saved_fpsr));
+  return out.x[0];
+}
+
+static void check(const long *args, long want) {
+  long got = probe(args);
+  if (got != want) { fprintf(stderr, "syscall %ld: got %ld, expected %ld\n", args[6], got, want); abort(); }
+}
+
+static long raw(long nr, long a, long b, long c, long d, long e, long f) {
+  long args[] = {a, b, c, d, e, f, nr};
+  return probe(args);
+}
+
+static void fault(long nr, long a, long b, long c, long d, long e, long f) {
+  long args[] = {a, b, c, d, e, f, 0, 0, nr};
+  long want = svc_original(args);
+  long call[] = {a, b, c, d, e, f, nr};
+  check(call, want);
+}
+
+static void readlink_boundaries(const char *path) {
+  char expected[PATH_MAX + 8], actual[PATH_MAX + 8];
+  long args[] = {AT_FDCWD, (long)path, (long)expected, PATH_MAX, 0, 0, 0, 0, SYS_readlinkat};
+  long length = svc_original(args);
+  assert(length > 0 && length < PATH_MAX - 2);
+  for (long size = length - 1; size <= length + 2; size++) {
+    memset(expected, 0x5a, sizeof expected); memset(actual, 0x5a, sizeof actual);
+    args[3] = size;
+    long want = svc_original(args);
+    long got = raw(SYS_readlinkat, AT_FDCWD, (long)path, (long)actual, size, 0, 0);
+    if (got != want || (want > 0 && memcmp(expected, actual, (size_t)want)))
+      fprintf(stderr, "readlink %s size %ld: relay %ld, proot %ld\n", path, size, got, want);
+    assert(got == want && (want <= 0 || !memcmp(expected, actual, (size_t)want)));
+    for (size_t i = (size_t)size; i < sizeof actual; i++) assert(actual[i] == 0x5a);
+  }
+}
+
+static void path_checks(void) {
+  char work[] = "/tmp/proot-expanded-check-XXXXXX";
+  assert(mkdtemp(work));
+  int cwd = open(".", O_RDONLY | O_DIRECTORY);
+  assert(cwd >= 0 && chdir(work) == 0);
+  assert(raw(SYS_mkdirat, AT_FDCWD, (long)"dir", 0700, 0, 0, 0) == 0);
+  long fd = raw(SYS_openat, AT_FDCWD, (long)"dir/file", O_CREAT | O_RDWR | O_EXCL, 0600, 0, 0);
+  assert(fd >= 0 && write((int)fd, "hello", 5) == 5);
+  struct stat st;
+  struct statx sx;
+  assert(raw(SYS_newfstatat, AT_FDCWD, (long)"dir/file", (long)&st, 0, 0, 0) == 0 && st.st_size == 5);
+  assert(raw(SYS_statx, AT_FDCWD, (long)"dir/file", 0, STATX_BASIC_STATS, (long)&sx, 0) == 0 && sx.stx_size == 5);
+  assert(raw(SYS_faccessat, AT_FDCWD, (long)"dir/file", R_OK, 0, 0, 0) == 0);
+  assert(raw(SYS_symlinkat, (long)"file", AT_FDCWD, (long)"dir/link", 0, 0, 0) == 0);
+  readlink_boundaries("dir/link");
+  readlink_boundaries("/proc/self/exe");
+  char text[PATH_MAX];
+  memset(text, '#', sizeof text);
+  assert(raw(SYS_readlinkat, AT_FDCWD, (long)"dir/link", (long)text, sizeof text, 0, 0) == 4);
+  assert(!memcmp(text, "file#", 5));
+  assert(raw(SYS_fchmodat, AT_FDCWD, (long)"dir/file", 0640, 0, 0, 0) == 0);
+  struct timespec times[] = {{1000, 0}, {2000, 0}};
+  assert(raw(SYS_truncate, (long)"dir/file", 11, 0, 0, 0, 0) == 0);
+  assert(raw(SYS_utimensat, AT_FDCWD, (long)"dir/file", (long)times, 0, 0, 0) == 0);
+  assert(raw(SYS_newfstatat, AT_FDCWD, (long)"dir/file", (long)&st, 0, 0, 0) == 0);
+  assert(st.st_size == 11 && (st.st_mode & 0777) == 0640 && st.st_mtime == 2000);
+  assert(raw(SYS_renameat, AT_FDCWD, (long)"dir/file", AT_FDCWD, (long)"dir/moved", 0, 0) == 0);
+  assert(raw(SYS_renameat2, AT_FDCWD, (long)"dir/moved", AT_FDCWD, (long)"dir/file", RENAME_NOREPLACE, 0) == 0);
+  struct statfs fs;
+  assert(raw(SYS_statfs, (long)"dir/file", (long)&fs, 0, 0, 0, 0) == 0);
+  long x = raw(SYS_getxattr, (long)"dir/file", (long)"user.proot-missing", (long)text, sizeof text, 0, 0);
+  assert(x == -ENODATA || x == -EOPNOTSUPP);
+  x = raw(SYS_lgetxattr, (long)"dir/link", (long)"user.proot-missing", (long)text, sizeof text, 0, 0);
+  assert(x == -ENODATA || x == -EOPNOTSUPP);
+  assert(raw(SYS_listxattr, (long)"dir/file", (long)text, sizeof text, 0, 0, 0) >= 0);
+  assert(raw(SYS_llistxattr, (long)"dir/link", (long)text, sizeof text, 0, 0, 0) >= 0);
+  int in = inotify_init1(IN_CLOEXEC);
+  assert(in >= 0 && raw(SYS_inotify_add_watch, in, (long)"dir", IN_CREATE, 0, 0, 0) >= 0);
+  close(in);
+  long n = raw(SYS_getcwd, (long)text, sizeof text, 0, 0, 0, 0);
+  assert(n == (long)strlen(work) + 1 && !strcmp(text, work));
+  size_t page = sysconf(_SC_PAGESIZE);
+  char *guard = mmap(NULL, page * 2, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+  assert(guard != MAP_FAILED && mprotect(guard + page, page, PROT_NONE) == 0);
+  strcpy(guard + page - 2, ".");
+  assert(raw(SYS_newfstatat, AT_FDCWD, (long)(guard + page - 2), (long)&st, 0, 0, 0) == 0);
+  long bad = (long)(guard + page);
+  fault(SYS_openat, AT_FDCWD, bad, O_RDONLY, 0, 0, 0);
+  fault(SYS_newfstatat, AT_FDCWD, (long)"dir/file", bad, 0, 0, 0);
+  fault(SYS_statx, AT_FDCWD, (long)"dir/file", 0, STATX_BASIC_STATS, bad, 0);
+  fault(SYS_readlinkat, AT_FDCWD, (long)"dir/link", bad, 16, 0, 0);
+  fault(SYS_getcwd, bad, PATH_MAX, 0, 0, 0, 0);
+  fault(SYS_utimensat, AT_FDCWD, (long)"missing", bad, 0, 0, 0);
+  fault(SYS_inotify_add_watch, -1, (long)"missing", IN_CREATE, 0, 0, 0);
+  fault(SYS_renameat2, AT_FDCWD, (long)"dir/file", AT_FDCWD, (long)"dir/nope", 0x80000000, 0);
+  assert(munmap(guard, page * 2) == 0);
+  close((int)fd);
+  assert(raw(SYS_unlinkat, AT_FDCWD, (long)"dir/link", 0, 0, 0, 0) == 0);
+  assert(raw(SYS_unlinkat, AT_FDCWD, (long)"dir/file", 0, 0, 0, 0) == 0);
+  assert(raw(SYS_unlinkat, AT_FDCWD, (long)"dir", AT_REMOVEDIR, 0, 0, 0) == 0);
+  assert(fchdir(cwd) == 0 && close(cwd) == 0 && rmdir(work) == 0);
+  puts("PASS: path families, dirfds, flags, symlinks, protected pointers, page boundary");
+}
+
+static void robust_checks(void) {
+  if (!svc_android_api) return;
+  long original = 0;
+  size_t size = 0;
+  assert(raw(SYS_get_robust_list, 0, (long)&original, (long)&size, 0, 0, 0) == 0);
+  struct robust_list_head head = {0};
+  long set_result = raw(SYS_set_robust_list, (long)&head, sizeof head, 0, 0, 0, 0);
+  assert(set_result == 0 || set_result == -ENOSYS);
+  long raw_head = 0;
+  long direct[] = {0, (long)&raw_head, (long)&size, 0, 0, 0, 0, 0, SYS_get_robust_list};
+  assert(svc_original(direct) == 0 && raw_head == (long)&head);
+  long got = 0;
+  assert(raw(SYS_get_robust_list, gettid(), (long)&got, (long)&size, 0, 0, 0) == 0);
+  assert(got == (long)&head && size == sizeof head);
+  assert(raw(SYS_get_robust_list, 0, 1, (long)&size, 0, 0, 0) == -EFAULT);
+  assert(raw(SYS_set_robust_list, 0, sizeof head, 0, 0, 0, 0) == set_result);
+  long cleared = raw(SYS_get_robust_list, 0, (long)&got, (long)&size, 0, 0, 0);
+  assert(cleared == -ENOSYS || (cleared == 0 && got == 0));
+  direct[0] = original; direct[1] = sizeof head; direct[8] = SYS_set_robust_list;
+  assert(svc_original(direct) == set_result);
+  assert(raw(SYS_get_robust_list, 0, (long)&got, (long)&size, 0, 0, 0) == 0 && original == got);
+  assert(raw(SYS_faccessat2, AT_FDCWD, (long)"/usr/lib/libc.so.6", R_OK, 0, 0, 0) == -ENOSYS);
+  if (svc_android_api < 35) assert(raw(SYS_clone3, 0, 0, 0, 0, 0, 0) == -ENOSYS);
+  int api = svc_android_api;
+  svc_android_api = 35;
+  assert(!svc_supported(SYS_clone3));
+  svc_android_api = 0;
+  assert(!svc_supported(SYS_clone3) && !svc_supported(SYS_set_robust_list));
+  svc_android_api = api;
+  assert(!svc_supported(SYS_set_robust_list) && !svc_supported(SYS_get_robust_list));
+  puts("PASS: mixed relay/raw robust-list callers, errors, faccessat2, clone3 platform guard");
+}
+
+static void *thread_check(void *unused) {
+  (void)unused;
+  struct stat st;
+  if (svc_android_api) {
+    long head;
+    size_t size;
+    assert(raw(SYS_get_robust_list, 0, (long)&head, (long)&size, 0, 0, 0) == 0 && head && size == 3 * sizeof(long));
+    struct robust_list_head private = {0};
+    long result = raw(SYS_set_robust_list, (long)&private, sizeof private, 0, 0, 0, 0);
+    assert(result == 0 || result == -ENOSYS);
+    for (int i = 0; i < 10; i++) {
+      long got;
+      sched_yield();
+      assert(raw(SYS_get_robust_list, 0, (long)&got, (long)&size, 0, 0, 0) == 0 && got == (long)&private);
+    }
+    assert(raw(SYS_set_robust_list, head, sizeof private, 0, 0, 0, 0) == result);
+  }
+  const long args[] = {AT_FDCWD, (long)"/usr/lib/libc.so.6", (long)&st, 0, 0, 0, SYS_newfstatat};
+  for (int i = 0; i < 100; i++) check(args, 0);
+  return NULL;
+}
+
+static pthread_barrier_t barrier;
+static void *thread_probe(void *unused) {
+  pthread_barrier_wait(&barrier);
+  thread_check(unused);
+  pthread_barrier_wait(&barrier);
+  return NULL;
+}
+
+static volatile sig_atomic_t signals_seen;
+static void on_signal(int signal) {
+  (void)signal;
+  int saved = errno;
+  struct stat st;
+  assert(raw(SYS_newfstatat, AT_FDCWD, (long)"/usr/lib/libc.so.6", (long)&st, 0, 0, 0) == 0);
+  signals_seen++;
+  errno = saved;
+}
+
+static sigjmp_buf interrupted;
+static void jump_signal(int signal) { (void)signal; siglongjmp(interrupted, 1); }
+static void jump_path(void) { assert(raise(SIGUSR2) == 0); abort(); }
+static void exit_path(void) { pthread_exit(PTHREAD_CANCELED); }
+static void mark_cleanup(void *arg) { *(int *)arg = 1; }
+struct unwind_check { int cleaned; int (*stat)(int, const char *, struct stat *, int); };
+static void *unwind_worker(void *arg) {
+  struct unwind_check *test = arg;
+  struct stat st;
+  pthread_cleanup_push(mark_cleanup, &test->cleaned);
+  interrupt_path = exit_path;
+  test->stat(AT_FDCWD, "/usr/lib/libc.so.6", &st, 0);
+  pthread_cleanup_pop(1);
+  abort();
+}
+
+static void interruption_checks(void) {
+  void *libc = dlopen("libc.so.6", RTLD_NOW);
+  struct unwind_check test = {.stat = dlsym(libc, "fstatat")};
+  assert(test.stat);
+  struct sigaction old, action = {.sa_handler = jump_signal};
+  sigemptyset(&action.sa_mask);
+  assert(sigaction(SIGUSR2, &action, &old) == 0);
+  sigset_t deliver, old_mask;
+  sigemptyset(&deliver); sigaddset(&deliver, SIGUSR2);
+  assert(pthread_sigmask(SIG_UNBLOCK, &deliver, &old_mask) == 0);
+  if (!sigsetjmp(interrupted, 1)) {
+    struct stat st;
+    interrupt_path = jump_path;
+    test.stat(AT_FDCWD, "/usr/lib/libc.so.6", &st, 0);
+    abort();
+  }
+  assert(svc_busy);
+  struct stat st;
+  unsigned long before = svc_fast;
+  assert(test.stat(AT_FDCWD, "/usr/lib/libc.so.6", &st, 0) == 0);
+  assert(!svc_busy && svc_fast > before);
+  assert(sigaction(SIGUSR2, &old, NULL) == 0 && pthread_sigmask(SIG_SETMASK, &old_mask, NULL) == 0);
+  pthread_t thread;
+  void *result;
+  assert(pthread_create(&thread, NULL, unwind_worker, &test) == 0);
+  assert(pthread_join(thread, &result) == 0 && result == PTHREAD_CANCELED && test.cleaned);
+  assert(dlclose(libc) == 0);
+  puts("PASS: signal siglongjmp recovery, forced unwind through patched libc with caller cleanup");
+}
+
+static void *small_stack_worker(void *unused) {
+  size_t consume = (size_t)(uintptr_t)unused - 48 * 1024;
+  volatile char *padding = __builtin_alloca(consume);
+  padding[0] = padding[consume - 1] = 1;
+  struct stat st;
+  unsigned long before = svc_fast;
+  for (int i = 0; i < 20; i++)
+    assert(raw(SYS_newfstatat, AT_FDCWD, (long)"/usr/lib/libc.so.6", (long)&st, 0, 0, 0) == 0);
+  assert(svc_fast == before && !svc_busy);
+  return NULL;
+}
+
+static void small_stack_checks(void) {
+  pthread_attr_t attr;
+  pthread_t thread;
+  size_t size = PTHREAD_STACK_MIN;
+  if (size < 128 * 1024) size = 128 * 1024;
+  assert(pthread_attr_init(&attr) == 0 && pthread_attr_setstacksize(&attr, size) == 0);
+  assert(pthread_create(&thread, &attr, small_stack_worker, (void *)(uintptr_t)size) == 0);
+  assert(pthread_join(thread, NULL) == 0 && pthread_attr_destroy(&attr) == 0);
+  puts("PASS: limited remaining thread stack delegates path calls to PRoot");
+}
+
+struct signal_sender { pthread_t target; int done; };
+static void *send_signals(void *arg) {
+  struct signal_sender *sender = arg;
+  const struct timespec delay = {0, 100000};
+  for (int i = 0; i < 100; i++) {
+    assert(pthread_kill(sender->target, SIGUSR1) == 0);
+    nanosleep(&delay, NULL);
+  }
+  __atomic_store_n(&sender->done, 1, __ATOMIC_RELEASE);
+  return NULL;
+}
+
+static void *cancel_worker(void *unused) {
+  (void)unused;
+  struct stat st;
+  for (;;) {
+    assert(raw(SYS_newfstatat, AT_FDCWD, (long)"/usr/lib/libc.so.6", (long)&st, 0, 0, 0) == 0);
+    pthread_testcancel();
+  }
+}
+
+struct cancel_open { const char *path; int (*open)(const char *, int, ...); int ready, cleaned; };
+static void cancel_cleanup(void *arg) { ((struct cancel_open *)arg)->cleaned = 1; }
+static void *cancel_fifo(void *arg) {
+  struct cancel_open *test = arg;
+  pthread_cleanup_push(cancel_cleanup, test);
+  __atomic_store_n(&test->ready, 1, __ATOMIC_RELEASE);
+  int fd = test->open(test->path, O_RDONLY);
+  if (fd >= 0) close(fd);
+  pthread_cleanup_pop(1);
+  return NULL;
+}
+
+static void signal_checks(void) {
+  stack_t old_stack, stack = {.ss_sp = malloc(SIGSTKSZ), .ss_size = SIGSTKSZ};
+  assert(stack.ss_sp && sigaltstack(&stack, &old_stack) == 0);
+  struct sigaction old, action = {.sa_handler = on_signal, .sa_flags = SA_ONSTACK};
+  sigemptyset(&action.sa_mask);
+  assert(sigaction(SIGUSR1, &action, &old) == 0);
+  sigset_t deliver, old_mask;
+  sigemptyset(&deliver);
+  sigaddset(&deliver, SIGUSR1);
+  assert(pthread_sigmask(SIG_UNBLOCK, &deliver, &old_mask) == 0);
+  struct signal_sender sender = {.target = pthread_self()};
+  pthread_t thread;
+  assert(pthread_create(&thread, NULL, send_signals, &sender) == 0);
+  struct stat st;
+  for (int i = 0; i < 2000 || !__atomic_load_n(&sender.done, __ATOMIC_ACQUIRE); i++)
+    assert(raw(SYS_newfstatat, AT_FDCWD, (long)"/usr/lib/libc.so.6", (long)&st, 0, 0, 0) == 0);
+  assert(pthread_join(thread, NULL) == 0 && signals_seen > 0);
+  assert(sigaction(SIGUSR1, &old, NULL) == 0 && sigaltstack(&old_stack, NULL) == 0);
+  assert(pthread_sigmask(SIG_SETMASK, &old_mask, NULL) == 0);
+  free(stack.ss_sp);
+  void *result;
+  assert(pthread_create(&thread, NULL, cancel_worker, NULL) == 0);
+  assert(pthread_cancel(thread) == 0 && pthread_join(thread, &result) == 0 && result == PTHREAD_CANCELED);
+  char fifo[] = "/tmp/proot-expanded-fifo-XXXXXX";
+  int fd = mkstemp(fifo);
+  assert(fd >= 0 && close(fd) == 0 && unlink(fifo) == 0 && mkfifo(fifo, 0600) == 0);
+  void *libc = dlopen("libc.so.6", RTLD_NOW);
+  struct cancel_open test = {.path = fifo, .open = dlsym(libc, "open")};
+  assert(test.open && pthread_create(&thread, NULL, cancel_fifo, &test) == 0);
+  while (!__atomic_load_n(&test.ready, __ATOMIC_ACQUIRE)) sched_yield();
+  struct timespec deadline = {0, 20000000};
+  nanosleep(&deadline, NULL);
+  assert(pthread_cancel(thread) == 0 && clock_gettime(CLOCK_REALTIME, &deadline) == 0);
+  deadline.tv_sec += 2;
+  assert(pthread_timedjoin_np(thread, &result, &deadline) == 0 && result == PTHREAD_CANCELED && test.cleaned);
+  assert(unlink(fifo) == 0 && dlclose(libc) == 0);
+  for (int i = 0; i < 32; i++) {
+    assert(pthread_create(&thread, NULL, thread_check, NULL) == 0);
+    assert(pthread_join(thread, NULL) == 0);
+  }
+  puts("PASS: signal reentry on alternate stack, deferred/FIFO cancellation with cleanup, thread churn");
+}
+
+int main(void) {
+  assert(fp_ready());
+  const char *mode = getenv("PROOT_FP_PATCH_SVC");
+  if (!svc_sites && mode && !strcmp(mode, "0")) {
+    assert(setenv("PROOT_FP_PATCH_SVC", "1", 1) == 0);
+    svc_patch_init();
+  }
+  printf("platform: android %d, patched %u sites\n", svc_android_api, svc_sites);
+  assert(svc_sites > 0);
+  assert(svc_single_thread());
+  size_t page = sysconf(_SC_PAGESIZE);
+  uint32_t *relay = svc_relay_page((uintptr_t)&svc_check_site, page), branch;
+  assert(relay && svc_emit(relay, &svc_check_site, &branch));
+  __builtin___clear_cache((char *)relay, (char *)relay + 40);
+  assert(mprotect(relay, page, PROT_READ | PROT_EXEC) == 0);
+  int mem = open("/proc/self/mem", O_RDWR | O_CLOEXEC);
+  assert(mem >= 0 && pwrite(mem, &branch, 4, (off_t)(uintptr_t)&svc_check_site) == 4);
+  close(mem);
+  __builtin___clear_cache((char *)&svc_check_site, (char *)(&svc_check_site + 1));
+  long args[] = {0, 1, 2, 3, 4, 5, SYS_getpid};
+  check(args, getpid());
+  struct stat st;
+  args[0] = AT_FDCWD; args[1] = (long)"/usr/lib/libc.so.6"; args[2] = (long)&st;
+  args[3] = 0; args[6] = SYS_newfstatat;
+  check(args, 0);
+  assert(st.st_size > 0);
+  args[1] = (long)"/usr/lib/proot-svc-check-does-not-exist";
+  check(args, -ENOENT);
+  args[1] = 0;
+  check(args, -EFAULT);
+  args[1] = (long)"/usr/lib/libc.so.6"; args[2] = 0;
+  check(args, -EFAULT);
+  args[1] = (long)"/proc/self/exe"; args[2] = (long)&st;
+  check(args, 0);
+  args[3] = 0x40000000;
+  check(args, -EINVAL);
+  path_checks();
+  robust_checks();
+  pthread_t threads[4];
+  assert(pthread_barrier_init(&barrier, NULL, 5) == 0);
+  for (int i = 0; i < 4; i++) assert(pthread_create(&threads[i], NULL, thread_probe, NULL) == 0);
+  pthread_barrier_wait(&barrier);
+  assert(!svc_single_thread());
+  pthread_barrier_wait(&barrier);
+  for (int i = 0; i < 4; i++) assert(pthread_join(threads[i], NULL) == 0);
+  assert(pthread_barrier_destroy(&barrier) == 0);
+  pid_t pid = fork();
+  assert(pid >= 0);
+  if (!pid) { thread_check(NULL); _exit(0); }
+  int status;
+  assert(waitpid(pid, &status, 0) == pid && WIFEXITED(status) && WEXITSTATUS(status) == 0);
+  signal_checks();
+  interruption_checks();
+  small_stack_checks();
+  puts("PASS: raw results, errno, x1-x30, q0-q31, NZCV, FPCR/FPSR, 4 threads, fork");
+  return 0;
+}
+#else
+int main(void) { return 77; }
+#endif

tools/proot/bench/svc-fail.c

@@ -0,0 +1,39 @@
+#include "../fastpath/fastpath.c"
+#include "checks.h"
+#include <linux/filter.h>
+#include <linux/seccomp.h>
+#include <sys/prctl.h>
+
+int main(int argc, char **argv) {
+#if defined(__aarch64__)
+  assert(argc == 2 && fp_ready() && !svc_sites && svc_single_thread());
+  int nr = !strcmp(argv[1], "mem") ? SYS_openat : !strcmp(argv[1], "write") ? SYS_pwrite64 :
+           !strcmp(argv[1], "rx") ? SYS_mprotect : SYS_mmap;
+  unsigned offset = nr == SYS_mprotect || nr == SYS_openat ? offsetof(struct seccomp_data, args[2]) : offsetof(struct seccomp_data, args[3]);
+  unsigned mask = nr == SYS_mprotect ? PROT_EXEC : nr == SYS_mmap ? MAP_FIXED_NOREPLACE : nr == SYS_openat ? O_RDWR : ~0U;
+  struct sock_filter code[] = {
+    BPF_STMT(BPF_LD | BPF_W | BPF_ABS, offsetof(struct seccomp_data, nr)),
+    BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, nr, 0, 3),
+    BPF_STMT(BPF_LD | BPF_W | BPF_ABS, offset),
+    BPF_JUMP(BPF_JMP | BPF_JSET | BPF_K, mask, 0, 1),
+    BPF_STMT(BPF_RET | BPF_K, SECCOMP_RET_ERRNO | EACCES),
+    BPF_STMT(BPF_RET | BPF_K, SECCOMP_RET_ALLOW),
+  };
+  if (nr == SYS_pwrite64) code[2] = (struct sock_filter)BPF_STMT(BPF_LD | BPF_IMM, 1);
+  struct sock_fprog filter = {.len = sizeof code / sizeof *code, .filter = code};
+  assert(setenv("PROOT_FP_PATCH_SVC", "1", 1) == 0);
+  if (sc(SYS_prctl, PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) ||
+      sc(SYS_prctl, PR_SET_SECCOMP, SECCOMP_MODE_FILTER, (long)&filter, 0, 0)) return 77;
+  assert(svc_single_thread());
+  svc_patch_init();
+  assert(svc_sites == 0 && svc_pages == 0);
+  struct stat st;
+  long args[] = {AT_FDCWD, (long)"/etc/os-release", (long)&st, 0, 0, 0, 0, 0, SYS_newfstatat};
+  assert(svc_original(args) == 0 && S_ISREG(st.st_mode));
+  puts("PASS: denied patch installation leaves the original libc syscall usable");
+  return 0;
+#else
+  (void)argc; (void)argv;
+  return 77;
+#endif
+}

tools/proot/bench/svc-probe.c

@@ -0,0 +1,93 @@
+#define _GNU_SOURCE
+#include <errno.h>
+#include <fcntl.h>
+#include <link.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <unistd.h>
+
+static int mov_sysnum(uint32_t insn) {
+  if ((insn & 0xffe0001fU) == 0xd2800008U ||
+      (insn & 0xffe0001fU) == 0x52800008U)
+    return (insn >> 5) & 0xffff;
+  return -1;
+}
+
+static int scan(struct dl_phdr_info *info, size_t size, void *unused) {
+  (void)size;
+  (void)unused;
+  const char *name = strrchr(info->dlpi_name, '/');
+  name = name ? name + 1 : info->dlpi_name;
+  if (strcmp(name, "libc.so.6") && strcmp(name, "ld-linux-aarch64.so.1")) return 0;
+  unsigned total = 0, adjacent = 0, nearby = 0, counts[512] = {0};
+  for (int i = 0; i < info->dlpi_phnum; i++) {
+    const ElfW(Phdr) *ph = &info->dlpi_phdr[i];
+    if (ph->p_type != PT_LOAD || !(ph->p_flags & PF_X)) continue;
+    const uint32_t *text = (const uint32_t *)(info->dlpi_addr + ph->p_vaddr);
+    for (size_t j = 0; j < ph->p_memsz / 4; j++) {
+      if (text[j] != 0xd4000001U) continue;
+      total++;
+      int nr = j ? mov_sysnum(text[j - 1]) : -1;
+      if (nr >= 0) {
+        adjacent++;
+        if (nr < 512) counts[nr]++;
+      } else {
+        for (size_t k = 2; k <= 8 && k <= j; k++)
+          if (mov_sysnum(text[j - k]) >= 0) { nearby++; break; }
+      }
+    }
+  }
+  printf("{\"object\":\"%s\",\"svc\":%u,\"adjacent_mov_x8\":%u,\"nearby_only\":%u,\"adjacent_syscalls\":{",
+         name, total, adjacent, nearby);
+  int comma = 0;
+  for (unsigned nr = 0; nr < 512; nr++)
+    if (counts[nr]) { printf("%s\"%u\":%u", comma ? "," : "", nr, counts[nr]); comma = 1; }
+  puts("}}");
+  return 0;
+}
+
+int main(void) {
+  long page = sysconf(_SC_PAGESIZE);
+  int fd = open("/usr/lib/libc.so.6", O_RDONLY | O_CLOEXEC);
+  if (fd < 0) { perror("libc"); return 1; }
+  unsigned char *p = mmap(NULL, page, PROT_READ | PROT_EXEC, MAP_PRIVATE, fd, 0);
+  if (p == MAP_FAILED) { perror("file mmap RX"); close(fd); return 1; }
+  unsigned char before = p[0], changed = before ^ 1;
+  errno = 0;
+  int rwx = mprotect(p, page, PROT_READ | PROT_WRITE | PROT_EXEC), rwx_errno = errno;
+  if (rwx == 0) {
+    p[0] = changed;
+    p[0] = before;
+    if (mprotect(p, page, PROT_READ | PROT_EXEC)) { perror("restore RX"); return 1; }
+  }
+  int mem = open("/proc/self/mem", O_RDWR | O_CLOEXEC), mem_errno = mem < 0 ? errno : 0;
+  ssize_t written = -1;
+  int write_errno = 0, verified = 0, restored = 0;
+  if (mem >= 0) {
+    errno = 0;
+    written = pwrite(mem, &changed, 1, (off_t)(uintptr_t)p);
+    write_errno = errno;
+    verified = written == 1 && *(volatile unsigned char *)p == changed;
+    restored = pwrite(mem, &before, 1, (off_t)(uintptr_t)p) == 1 &&
+               *(volatile unsigned char *)p == before;
+    close(mem);
+  }
+  unsigned char disk = 0;
+  int file_unchanged = pread(fd, &disk, 1, 0) == 1 && disk == before;
+  munmap(p, page);
+  close(fd);
+  void *anon = mmap(NULL, page, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+  errno = 0;
+  int execmem = anon == MAP_FAILED ? -1 : mprotect(anon, page, PROT_READ | PROT_EXEC);
+  int execmem_errno = errno;
+  if (anon != MAP_FAILED) munmap(anon, page);
+  printf("{\"page_size\":%ld,\"rwx\":%d,\"rwx_errno\":%d,\"mem_open_errno\":%d,"
+         "\"mem_written\":%ld,\"mem_write_errno\":%d,\"mem_verified\":%d,\"mem_restored\":%d,"
+         "\"file_unchanged\":%d,\"execmem\":%d,\"execmem_errno\":%d}\n",
+         page, rwx, rwx_errno, mem_errno, (long)written, write_errno, verified, restored,
+         file_unchanged, execmem, execmem_errno);
+  dl_iterate_phdr(scan, NULL);
+  return 0;
+}

tools/proot/bench/svc-race.c

@@ -0,0 +1,49 @@
+#define _GNU_SOURCE
+#include "checks.h"
+#include <dlfcn.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <signal.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <sys/wait.h>
+#include <unistd.h>
+
+int main(void) {
+  char work[] = "/tmp/proot-svc-race-XXXXXX";
+  assert(mkdtemp(work));
+  int cwd = open(".", O_RDONLY | O_DIRECTORY);
+  assert(cwd >= 0 && chdir(work) == 0);
+  for (int i = 0; i < 2; i++) {
+    int fd = open(i ? "b" : "a", O_CREAT | O_EXCL | O_WRONLY, 0600);
+    assert(fd >= 0 && ftruncate(fd, i + 1) == 0 && close(fd) == 0);
+  }
+  assert(symlink("a", "live") == 0);
+  void *libc = dlopen("libc.so.6", RTLD_NOW);
+  int (*stat)(int, const char *, struct stat *, int) = dlsym(libc, "fstatat");
+  ssize_t (*link)(int, const char *, char *, size_t) = dlsym(libc, "readlinkat");
+  assert(stat && link);
+  pid_t writer = fork();
+  assert(writer >= 0);
+  if (!writer) {
+    for (int i = 0; i < 3000; i++) {
+      assert(symlink(i & 1 ? "b" : "a", "next") == 0);
+      assert(rename("next", "live") == 0);
+    }
+    _exit(0);
+  }
+  for (int i = 0; i < 3000; i++) {
+    struct stat st;
+    assert(stat(AT_FDCWD, "live", &st, 0) == 0 && S_ISREG(st.st_mode) && (st.st_size == 1 || st.st_size == 2));
+    char text[8];
+    assert(link(AT_FDCWD, "live", text, sizeof text) == 1 && (text[0] == 'a' || text[0] == 'b'));
+  }
+  int status;
+  assert(waitpid(writer, &status, 0) == writer && WIFEXITED(status) && WEXITSTATUS(status) == 0);
+  assert(unlink("live") == 0 && unlink("a") == 0 && unlink("b") == 0);
+  assert(dlclose(libc) == 0 && fchdir(cwd) == 0 && close(cwd) == 0 && rmdir(work) == 0);
+  puts("PASS: concurrent atomic symlink replacement, internal stat/readlink legal results");
+  return 0;
+}

tools/proot/bench/svc-storage.c

@@ -0,0 +1,74 @@
+#define _GNU_SOURCE
+#include "checks.h"
+#include <dlfcn.h>
+#include <fcntl.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/resource.h>
+#include <sys/stat.h>
+#include <time.h>
+#include <unistd.h>
+
+static double now(void) {
+  struct timespec t;
+  assert(clock_gettime(CLOCK_MONOTONIC, &t) == 0);
+  return t.tv_sec + t.tv_nsec / 1e9;
+}
+
+int main(int argc, char **argv) {
+  assert(argc == 2 && mkdir(argv[1], 0700) == 0);
+  void *libc = dlopen("libc.so.6", RTLD_NOW);
+  assert(libc);
+  int (*internal_open)(const char *, int, ...) = dlsym(libc, "__open64_nocancel");
+  int (*internal_stat)(int, const char *, struct stat *, int) = dlsym(libc, "fstatat");
+  assert(internal_open && internal_stat);
+  struct rusage before, after;
+  assert(getrusage(RUSAGE_SELF, &before) == 0);
+  char paths[64][4096], stream[4096];
+  double creation = 0, extension = 0, closing = 0, total = now();
+  for (int i = 0; i < 64; i++) {
+    snprintf(paths[i], sizeof paths[i], "%s/asset-with-a-longish-steam-depot-filename-%03d.dat", argv[1], i);
+    double start = now();
+    int fd = internal_open(paths[i], O_WRONLY | O_CREAT | O_EXCL | O_CLOEXEC, 0600);
+    assert(fd >= 0);
+    creation += now() - start;
+    start = now();
+    assert(ftruncate(fd, 262144) == 0);
+    extension += now() - start;
+    start = now();
+    assert(close(fd) == 0);
+    closing += now() - start;
+  }
+  double allocation = now() - total;
+  struct stat st;
+  double start = now();
+  for (int i = 0; i < 1000; i++) assert(internal_stat(AT_FDCWD, paths[i % 64], &st, 0) == 0);
+  double metadata = now() - start;
+  snprintf(stream, sizeof stream, "%s/stream.bin", argv[1]);
+  start = now();
+  int fd = internal_open(stream, O_WRONLY | O_CREAT | O_EXCL | O_CLOEXEC, 0600);
+  assert(fd >= 0);
+  double stream_create = now() - start;
+  static unsigned char block[1024 * 1024];
+  memset(block, 0x5a, sizeof block);
+  start = now();
+  for (int i = 0; i < 64; i++) assert(write(fd, block, sizeof block) == sizeof block);
+  double writing = now() - start;
+  start = now();
+  assert(fdatasync(fd) == 0);
+  double syncing = now() - start;
+  assert(close(fd) == 0);
+  assert(getrusage(RUSAGE_SELF, &after) == 0);
+  printf("{\"files\":64,\"bytes_per_file\":262144,\"create_s\":%.6f,\"extend_s\":%.6f,"
+         "\"close_s\":%.6f,\"allocation_s\":%.6f,\"stat1000_s\":%.6f,"
+         "\"stream_bytes\":67108864,\"stream_create_s\":%.6f,\"write_s\":%.6f,"
+         "\"fdatasync_s\":%.6f,\"voluntary_switches\":%ld}\n",
+         creation, extension, closing, allocation, metadata, stream_create, writing,
+         syncing, after.ru_nvcsw - before.ru_nvcsw);
+  fflush(stdout);
+  for (int i = 0; i < 64; i++) assert(unlink(paths[i]) == 0);
+  assert(unlink(stream) == 0 && rmdir(argv[1]) == 0);
+  assert(dlclose(libc) == 0);
+  return 0;
+}

tools/proot/bench/svc-threads.c

@@ -0,0 +1,47 @@
+#define _GNU_SOURCE
+#include "checks.h"
+#include <dlfcn.h>
+#include <fcntl.h>
+#include <pthread.h>
+#include <stdio.h>
+#include <sys/resource.h>
+#include <time.h>
+#include <unistd.h>
+
+static double now(void) {
+  struct timespec t;
+  assert(clock_gettime(CLOCK_MONOTONIC, &t) == 0);
+  return t.tv_sec * 1e9 + t.tv_nsec;
+}
+
+static void *thread(void *arg) { return arg; }
+
+int main(void) {
+  void *libc = dlopen("libc.so.6", RTLD_NOW);
+  assert(libc);
+  int (*access)(int, const char *, int, int) = dlsym(libc, "faccessat");
+  char *(*cwd)(char *, size_t) = dlsym(libc, "getcwd");
+  assert(access && cwd);
+  struct rusage before, after;
+  assert(getrusage(RUSAGE_SELF, &before) == 0);
+  const int threads = 128, calls = 2000;
+  double start = now();
+  for (int i = 0; i < threads; i++) {
+    pthread_t id;
+    void *value;
+    assert(pthread_create(&id, NULL, thread, (void *)1) == 0);
+    assert(pthread_join(id, &value) == 0 && value == (void *)1);
+  }
+  double thread_us = (now() - start) / threads / 1000;
+  start = now();
+  for (int i = 0; i < calls; i++) assert(access(AT_FDCWD, "/etc/os-release", R_OK, 0) == 0);
+  double access_ns = (now() - start) / calls;
+  char path[4096];
+  start = now();
+  for (int i = 0; i < calls; i++) assert(cwd(path, sizeof path));
+  double cwd_ns = (now() - start) / calls;
+  assert(getrusage(RUSAGE_SELF, &after) == 0 && dlclose(libc) == 0);
+  printf("{\"thread_us\":%.0f,\"access_ns\":%.0f,\"cwd_ns\":%.0f,\"switches\":%ld}\n",
+         thread_us, access_ns, cwd_ns, after.ru_nvcsw - before.ru_nvcsw);
+  return 0;
+}

tools/proot/bench/svc-workload.c

@@ -0,0 +1,92 @@
+#define _GNU_SOURCE
+#include "checks.h"
+#include <dlfcn.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <fts.h>
+#include <pwd.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/resource.h>
+#include <sys/stat.h>
+#include <time.h>
+#include <unistd.h>
+
+static double now(void) {
+  struct timespec t;
+  assert(clock_gettime(CLOCK_MONOTONIC, &t) == 0);
+  return t.tv_sec * 1e9 + t.tv_nsec;
+}
+
+int main(int argc, char **argv) {
+  assert(argc >= 2);
+  void *libc = dlopen("libc.so.6", RTLD_NOW);
+  assert(libc);
+  int (*internal_stat)(int, const char *, struct stat *, int) = dlsym(libc, "fstatat");
+  int (*internal_open)(const char *, int, ...) = dlsym(libc, "__open64_nocancel");
+  assert(internal_stat && internal_open);
+  char path[4096], link[4096], missing[4096];
+  snprintf(path, sizeof path, "%s/file", argv[1]);
+  snprintf(link, sizeof link, "%s/link", argv[1]);
+  snprintf(missing, sizeof missing, "%s/missing", argv[1]);
+  const char *paths[] = {path, link, missing, "/etc/os-release", "/lib/libc.so.6", "/proc/self/exe", "/dev/null", "", NULL};
+  struct stat st;
+  for (size_t i = 0; i < sizeof paths / sizeof *paths; i++) {
+    errno = 0;
+    int r = internal_stat(AT_FDCWD, paths[i], &st, 0);
+    printf("stat %zu: %d %d", i, r, r ? errno : 0);
+    if (!r) printf(" %o %ld", st.st_mode & S_IFMT, S_ISREG(st.st_mode) && i != 5 ? st.st_size : 0L);
+    putchar('\n');
+    errno = 0;
+    int fd = internal_open(paths[i], O_RDONLY | O_CLOEXEC);
+    int err = fd < 0 ? errno : 0;
+    printf("open %zu: %d %d\n", i, fd < 0 ? -1 : 0, err);
+    if (fd >= 0) close(fd);
+  }
+  errno = 0;
+  int r = internal_stat(AT_FDCWD, path, &st, 0x40000000);
+  printf("flags: %d %d\n", r, r ? errno : 0);
+  char *tree[] = {argv[1], NULL};
+  FTS *fts = fts_open(tree, FTS_PHYSICAL | FTS_NOCHDIR, NULL);
+  assert(fts);
+  unsigned files = 0, dirs = 0, links = 0;
+  FTSENT *ent;
+  while ((ent = fts_read(fts))) {
+    if (ent->fts_info == FTS_F) files++;
+    if (ent->fts_info == FTS_D) dirs++;
+    if (ent->fts_info == FTS_SL) links++;
+  }
+  assert(fts_close(fts) == 0);
+  printf("fts: %u files %u dirs %u links\n", files, dirs, links);
+  char resolved[4096], canonical[4096];
+  assert(realpath(path, canonical));
+  assert(realpath(link, resolved) && !strcmp(resolved, canonical));
+  struct passwd pw, *found;
+  char buf[16384];
+  r = getpwnam_r("root", &pw, buf, sizeof buf, &found);
+  printf("nss: %d %d\n", r, found != NULL);
+  void *libm = dlopen("libm.so.6", RTLD_NOW | RTLD_LOCAL);
+  assert(libm && dlsym(libm, "cos"));
+  assert(dlclose(libm) == 0);
+  puts("realpath/dlopen: PASS");
+  if (argc > 2) {
+    const int loops = 5000;
+    struct rusage before, after;
+    assert(getrusage(RUSAGE_SELF, &before) == 0);
+    double start = now();
+    for (int i = 0; i < loops; i++) assert(internal_stat(AT_FDCWD, path, &st, 0) == 0);
+    double stat_ns = (now() - start) / loops;
+    start = now();
+    for (int i = 0; i < loops; i++) {
+      int fd = internal_open(path, O_RDONLY | O_CLOEXEC);
+      assert(fd >= 0 && close(fd) == 0);
+    }
+    double open_ns = (now() - start) / loops;
+    assert(getrusage(RUSAGE_SELF, &after) == 0);
+    printf("bench: stat %.0f ns, open+close %.0f ns, voluntary switches %ld\n",
+           stat_ns, open_ns, after.ru_nvcsw - before.ru_nvcsw);
+  }
+  assert(dlclose(libc) == 0);
+  return 0;
+}

tools/proot/build.sh

@@ -23,13 +23,22 @@ fi
 WORK=$(mktemp -d)
 trap 'rm -rf "$WORK"' EXIT
 fetch() {
-  curl -fsSL --retry 3 -o "$WORK/$1" "$2"
-  echo "$3  $WORK/$1" | shasum -a 256 -c -
-  mkdir -p "$WORK/${1%%.*}"
-  tar -xzf "$WORK/$1" -C "$WORK/${1%%.*}" --strip-components=1
+  local name=$1 primary=$2 sha=$3 url fetched=0
+  shift 3
+  for url in "$primary" "$@"; do
+    if curl -fsSL --connect-timeout 15 --max-time 90 --retry 3 --retry-max-time 120 -o "$WORK/$name" "$url" &&
+        printf '%s  %s\n' "$sha" "$WORK/$name" | shasum -a 256 -c -; then
+      fetched=1
+      break
+    fi
+  done
+  [[ $fetched == 1 ]] || return 1
+  mkdir -p "$WORK/${name%%.*}"
+  tar -xzf "$WORK/$name" -C "$WORK/${name%%.*}" --strip-components=1
 }
 fetch proot.tar.gz "https://github.com/termux/proot/archive/${PROOT_COMMIT}.tar.gz" "$PROOT_SHA256"
-fetch talloc.tar.gz "https://www.samba.org/ftp/talloc/talloc-${TALLOC_VERSION}.tar.gz" "$TALLOC_SHA256"
+fetch talloc.tar.gz "https://www.samba.org/ftp/talloc/talloc-${TALLOC_VERSION}.tar.gz" "$TALLOC_SHA256" \
+  "https://distfiles.macports.org/talloc/talloc-${TALLOC_VERSION}.tar.gz"
 cp "$HERE/statx_mnt_id.h" "$WORK/proot/src/tracee/statx_mnt_id.h"
 cp "$HERE/profile.h" "$WORK/proot/src/syscall/profile.h"
 for patch in "$HERE"/patches/*.patch; do

tools/proot/fastpath/fastpath.c

@@ -931,29 +931,26 @@ static long statx_call(int dirfd, const char *path, int flags, unsigned mask, st
   return 0;
 }
 
-int statx(int dirfd, const char *path, int flags, unsigned mask, struct statx *out) {
-  REAL(statx, int (*)(int, const char *, int, unsigned, struct statx *));
+static long fp_statx(int dirfd, const char *path, int flags, unsigned mask, struct statx *out) {
   struct res r;
   long rs;
-  if (mask & STATX__RESERVED) { errno = EINVAL; return -1; }
-#pragma GCC diagnostic push
-#pragma GCC diagnostic ignored "-Wnonnull-compare"
-  if (!out) { errno = EFAULT; return -1; }
-  if (!path) {
-    if (!(flags & AT_EMPTY_PATH)) { errno = EFAULT; return -1; }
-    return real_statx ? real_statx(dirfd, path, flags, mask, out) : ret(sc(SYS_statx, dirfd, (long)path, flags, mask, (long)out));
-  }
-#pragma GCC diagnostic pop
-  if ((flags & AT_EMPTY_PATH) && !path[0] && fp_ready()) {
-    hits++;
-    return ret(statx_call(dirfd, "", flags, mask, out));
-  }
+  if (mask & STATX__RESERVED) return -EINVAL;
+  if (!out) return -EFAULT;
+  if (!path) return flags & AT_EMPTY_PATH ? FP_SLOW : -EFAULT;
+  if ((flags & AT_EMPTY_PATH) && !path[0] && fp_ready())
+    return statx_call(dirfd, "", flags, mask, out);
   rs = (flags & ~(AT_SYMLINK_NOFOLLOW | AT_NO_AUTOMOUNT | AT_STATX_SYNC_TYPE)) ? FP_SLOW
        : resolve(dirfd, path, (flags & AT_SYMLINK_NOFOLLOW ? 0 : 1) | MAY_BE_UNSEEN, &r);
   if (rs == 1) rs = -ENOENT;
   if (rs == 0 || rs == UNSEEN)
     rs = statx_call(AT_FDCWD, r.host, flags | (rs == 0 && !r.magic ? AT_SYMLINK_NOFOLLOW : 0), mask, out);
-  ANSWER(rs, dirfd, path, real_statx(dirfd, path, flags, mask, out));
+  return rs;
+}
+
+int statx(int dirfd, const char *path, int flags, unsigned mask, struct statx *out) {
+  REAL(statx, int (*)(int, const char *, int, unsigned, struct statx *));
+  ANSWER(fp_statx(dirfd, path, flags, mask, out), dirfd, path,
+         real_statx ? real_statx(dirfd, path, flags, mask, out) : ret(sc(SYS_statx, dirfd, (long)path, flags, mask, (long)out)));
 }
 
 
@@ -961,8 +958,7 @@ static int effective_is_real(void) {
   return !getauxval(AT_SECURE) && getuid() == geteuid() && getgid() == getegid();
 }
 
-static int access_common(int dirfd, const char *path, int mode, int flags) {
-  REAL(faccessat, int (*)(int, const char *, int, int));
+static long fp_access(int dirfd, const char *path, int mode, int flags) {
   struct res r;
   if ((flags & AT_EACCESS) && effective_is_real()) flags &= ~AT_EACCESS;
   long rs = (flags & ~AT_SYMLINK_NOFOLLOW) == 0
@@ -970,6 +966,12 @@ static int access_common(int dirfd, const char *path, int mode, int flags) {
   if (rs == 1) rs = -ENOENT;
   if (rs == 0 && S_ISLNK(r.st.st_mode)) rs = FP_SLOW;
   if (rs == 0 || rs == UNSEEN) rs = sc(SYS_faccessat, AT_FDCWD, (long)r.host, mode, 0, 0);
+  return rs;
+}
+
+static int access_common(int dirfd, const char *path, int mode, int flags) {
+  REAL(faccessat, int (*)(int, const char *, int, int));
+  long rs = fp_access(dirfd, path, mode, flags);
   if (rs != FP_SLOW) { hits++; return ret(rs); }
   missed(__func__, dirfd, path);
   if (flags == 0) return (int)syscall(SYS_faccessat, dirfd, path, mode);
@@ -997,34 +999,36 @@ static long link_text(const struct res *r, char *text, size_t size, int *changed
   return (long)strlen(text);
 }
 
-static ssize_t readlink_common(int dirfd, const char *path, char *buf, size_t size) {
-  REAL(readlinkat, ssize_t (*)(int, const char *, char *, size_t));
+static long fp_readlink(int dirfd, const char *path, char *buf, size_t size, size_t *written) {
   struct res r;
   char text[PATH_MAX];
   int changed;
   long rs = size ? resolve(dirfd, path, 0, &r) : FP_SLOW;
   if (rs == 1) rs = -ENOENT;
   if (rs == 0) rs = link_text(&r, text, size, &changed);
-  if (rs == FP_SLOW) {
-    missed(__func__, dirfd, path);
-    return real_readlinkat(dirfd, path, buf, size);
-  }
-  hits++;
-  if (rs < 0) return ret(rs);
+  if (rs < 0) return rs;
   size_t n = rs;
   if (!changed) {
     n = n < size ? n : size;
     memcpy(buf, text, n);
+    if (written) *written = n;
     return n;
   }
   size_t max = size < PATH_MAX ? size : PATH_MAX;
   if (n + 1 < max) {
     memcpy(buf, text, n + 1);
+    if (written) *written = n + 1;
     return n;
   }
   memcpy(buf, text, max);
+  if (written) *written = max;
   return max;
 }
+
+static ssize_t readlink_common(int dirfd, const char *path, char *buf, size_t size) {
+  REAL(readlinkat, ssize_t (*)(int, const char *, char *, size_t));
+  ANSWER(fp_readlink(dirfd, path, buf, size, NULL), dirfd, path, real_readlinkat(dirfd, path, buf, size));
+}
 ssize_t readlinkat(int dirfd, const char *path, char *buf, size_t size) { return readlink_common(dirfd, path, buf, size); }
 ssize_t readlink(const char *path, char *buf, size_t size) { return readlink_common(AT_FDCWD, path, buf, size); }
 
@@ -1086,12 +1090,15 @@ char *__getcwd_chk(char *buf, size_t size, size_t room) {
 }
 
 
-int mkdirat(int dirfd, const char *path, mode_t mode) {
-  REAL(mkdirat, int (*)(int, const char *, mode_t));
+static long fp_mkdir(int dirfd, const char *path, mode_t mode) {
   struct res r;
   long rs = resolve(dirfd, path, MAY_BE_UNSEEN, &r);
   if (rs == 0 || rs == 1 || rs == UNSEEN) rs = sc(SYS_mkdirat, AT_FDCWD, (long)r.host, mode, 0, 0);
-  ANSWER(rs, dirfd, path, real_mkdirat(dirfd, path, mode));
+  return rs;
+}
+int mkdirat(int dirfd, const char *path, mode_t mode) {
+  REAL(mkdirat, int (*)(int, const char *, mode_t));
+  ANSWER(fp_mkdir(dirfd, path, mode), dirfd, path, real_mkdirat(dirfd, path, mode));
 }
 int mkdir(const char *path, mode_t mode) { return mkdirat(AT_FDCWD, path, mode); }
 
@@ -1137,8 +1144,7 @@ int unlinkat(int dirfd, const char *path, int flags) { return unlink_common(dirf
 int unlink(const char *path) { return unlink_common(AT_FDCWD, path, 0); }
 int rmdir(const char *path) { return unlink_common(AT_FDCWD, path, AT_REMOVEDIR); }
 
-static int rename_common(int olddirfd, const char *old, int newdirfd, const char *new, unsigned flags) {
-  REAL(renameat2, int (*)(int, const char *, int, const char *, unsigned));
+static long fp_rename(int olddirfd, const char *old, int newdirfd, const char *new, unsigned flags) {
   struct res from, to;
   long rs = flags & ~RENAME_NOREPLACE || trailing_slash(old) || trailing_slash(new) ? FP_SLOW : resolve(olddirfd, old, 0, &from);
   if (rs == 1) rs = -ENOENT;
@@ -1153,6 +1159,11 @@ static int rename_common(int olddirfd, const char *old, int newdirfd, const char
       if (rs == 0) forget();
     }
   }
+  return rs;
+}
+static int rename_common(int olddirfd, const char *old, int newdirfd, const char *new, unsigned flags) {
+  REAL(renameat2, int (*)(int, const char *, int, const char *, unsigned));
+  long rs = fp_rename(olddirfd, old, newdirfd, new, flags);
   if (rs != FP_SLOW) { hits++; return ret(rs); }
   missed(__func__, olddirfd, old);
   if (!flags) return (int)syscall(SYS_renameat, olddirfd, old, newdirfd, new);
@@ -1164,30 +1175,35 @@ int renameat2(int olddirfd, const char *old, int newdirfd, const char *new, unsi
 int renameat(int olddirfd, const char *old, int newdirfd, const char *new) { return rename_common(olddirfd, old, newdirfd, new, 0); }
 int rename(const char *old, const char *new) { return rename_common(AT_FDCWD, old, AT_FDCWD, new, 0); }
 
-int symlinkat(const char *target, int dirfd, const char *path) {
-  REAL(symlinkat, int (*)(const char *, int, const char *));
+static long fp_symlink(const char *target, int dirfd, const char *path) {
   struct res r;
   long rs = trailing_slash(path) ? FP_SLOW : resolve(dirfd, path, MAY_BE_UNSEEN, &r);
   if (rs == 0 || rs == 1 || rs == UNSEEN) {
     rs = sc(SYS_symlinkat, (long)target, AT_FDCWD, (long)r.host, 0, 0);
     if (rs == 0) forget();
   }
-  ANSWER(rs, dirfd, path, real_symlinkat(target, dirfd, path));
+  return rs;
+}
+int symlinkat(const char *target, int dirfd, const char *path) {
+  REAL(symlinkat, int (*)(const char *, int, const char *));
+  ANSWER(fp_symlink(target, dirfd, path), dirfd, path, real_symlinkat(target, dirfd, path));
 }
 int symlink(const char *target, const char *path) { return symlinkat(target, AT_FDCWD, path); }
 
-int fchmodat(int dirfd, const char *path, mode_t mode, int flags) {
-  REAL(fchmodat, int (*)(int, const char *, mode_t, int));
+static long fp_chmod(int dirfd, const char *path, mode_t mode, int flags) {
   struct res r;
   long rs = flags ? FP_SLOW : resolve(dirfd, path, 1 | MAY_BE_UNSEEN, &r);
   if (rs == 1) rs = -ENOENT;
   if (rs == 0 || rs == UNSEEN) rs = sc(SYS_fchmodat, AT_FDCWD, (long)r.host, mode, 0, 0);
-  ANSWER(rs, dirfd, path, real_fchmodat(dirfd, path, mode, flags));
+  return rs;
+}
+int fchmodat(int dirfd, const char *path, mode_t mode, int flags) {
+  REAL(fchmodat, int (*)(int, const char *, mode_t, int));
+  ANSWER(fp_chmod(dirfd, path, mode, flags), dirfd, path, real_fchmodat(dirfd, path, mode, flags));
 }
 int chmod(const char *path, mode_t mode) { return fchmodat(AT_FDCWD, path, mode, 0); }
 
-static int utimens_common(int dirfd, const char *path, const struct timespec times[2], int flags) {
-  REAL(utimensat, int (*)(int, const char *, const struct timespec[2], int));
+static long fp_utimens(int dirfd, const char *path, const struct timespec times[2], int flags) {
   struct res r;
   long rs;
   if (!path && fp_ready()) {
@@ -1197,7 +1213,11 @@ static int utimens_common(int dirfd, const char *path, const struct timespec tim
     if (rs == 1) rs = -ENOENT;
     if (rs == 0 || rs == UNSEEN) rs = sc(SYS_utimensat, AT_FDCWD, (long)r.host, (long)times, flags, 0);
   }
-  ANSWER(rs, dirfd, path, real_utimensat(dirfd, path, times, flags));
+  return rs;
+}
+static int utimens_common(int dirfd, const char *path, const struct timespec times[2], int flags) {
+  REAL(utimensat, int (*)(int, const char *, const struct timespec[2], int));
+  ANSWER(fp_utimens(dirfd, path, times, flags), dirfd, path, real_utimensat(dirfd, path, times, flags));
 }
 int utimensat(int dirfd, const char *path, const struct timespec times[2], int flags) {
   return utimens_common(dirfd, path, times, flags);
@@ -1224,23 +1244,29 @@ int utime(const char *path, const struct utimbuf *times) {
   return utimens_common(AT_FDCWD, path, ts, 0);
 }
 
-int truncate(const char *path, off_t length) {
-  REAL(truncate, int (*)(const char *, off_t));
+static long fp_truncate(const char *path, off_t length) {
   struct res r;
   long rs = resolve(AT_FDCWD, path, 1 | MAY_BE_UNSEEN, &r);
   if (rs == 1) rs = -ENOENT;
   if (rs == 0 || rs == UNSEEN) rs = sc(SYS_truncate, (long)r.host, length, 0, 0, 0);
-  ANSWER(rs, AT_FDCWD, path, real_truncate(path, length));
+  return rs;
+}
+int truncate(const char *path, off_t length) {
+  REAL(truncate, int (*)(const char *, off_t));
+  ANSWER(fp_truncate(path, length), AT_FDCWD, path, real_truncate(path, length));
 }
 int truncate64(const char *path, off_t length) __attribute__((alias("truncate")));
 
-static int statfs_common(const char *path, struct statfs *buf, int (*real)(const char *, struct statfs *)) {
+static long fp_statfs(const char *path, struct statfs *buf) {
   struct res r;
   long rs = resolve(AT_FDCWD, path, 1, &r);
   if (rs == 1) rs = -ENOENT;
   if (rs == 0 && under(r.guest, "/dev/shm", 8)) rs = FP_SLOW;
   if (rs == 0) rs = sc(SYS_statfs, (long)r.host, (long)buf, 0, 0, 0);
-  ANSWER(rs, AT_FDCWD, path, real(path, buf));
+  return rs;
+}
+static int statfs_common(const char *path, struct statfs *buf, int (*real)(const char *, struct statfs *)) {
+  ANSWER(fp_statfs(path, buf), AT_FDCWD, path, real(path, buf));
 }
 int statfs(const char *path, struct statfs *buf) {
   REAL(statfs, int (*)(const char *, struct statfs *));
@@ -1288,12 +1314,16 @@ int statvfs64(const char *path, struct statvfs64 *buf) {
   return statvfs_common(path, (struct statvfs *)buf, real_statvfs64);
 }
 
-static ssize_t getxattr_common(const char *path, const char *name, void *value, size_t size, int follow,
-                               ssize_t (*real)(const char *, const char *, void *, size_t)) {
+static long fp_getxattr(const char *path, const char *name, void *value, size_t size, int follow) {
   struct res r;
   long rs = resolve(AT_FDCWD, path, follow | MAY_BE_UNSEEN, &r);
   if (rs == 1) rs = -ENOENT;
   if (rs == 0 || rs == UNSEEN) rs = sc(follow ? SYS_getxattr : SYS_lgetxattr, (long)r.host, (long)name, (long)value, size, 0);
+  return rs;
+}
+static ssize_t getxattr_common(const char *path, const char *name, void *value, size_t size, int follow,
+                               ssize_t (*real)(const char *, const char *, void *, size_t)) {
+  long rs = fp_getxattr(path, name, value, size, follow);
   if (rs != FP_SLOW) { hits++; return rs < 0 ? ret(rs) : rs; }
   missed(__func__, AT_FDCWD, path);
   return real(path, name, value, size);
@@ -1307,12 +1337,16 @@ ssize_t lgetxattr(const char *path, const char *name, void *value, size_t size)
   return getxattr_common(path, name, value, size, 0, real_lgetxattr);
 }
 
-static ssize_t listxattr_common(const char *path, char *list, size_t size, int follow,
-                                ssize_t (*real)(const char *, char *, size_t)) {
+static long fp_listxattr(const char *path, char *list, size_t size, int follow) {
   struct res r;
   long rs = resolve(AT_FDCWD, path, follow | MAY_BE_UNSEEN, &r);
   if (rs == 1) rs = -ENOENT;
   if (rs == 0 || rs == UNSEEN) rs = sc(follow ? SYS_listxattr : SYS_llistxattr, (long)r.host, (long)list, size, 0, 0);
+  return rs;
+}
+static ssize_t listxattr_common(const char *path, char *list, size_t size, int follow,
+                                ssize_t (*real)(const char *, char *, size_t)) {
+  long rs = fp_listxattr(path, list, size, follow);
   if (rs != FP_SLOW) { hits++; return rs < 0 ? ret(rs) : rs; }
   missed(__func__, AT_FDCWD, path);
   return real(path, list, size);
@@ -1326,13 +1360,15 @@ ssize_t llistxattr(const char *path, char *list, size_t size) {
   return listxattr_common(path, list, size, 0, real_llistxattr);
 }
 
-int inotify_add_watch(int fd, const char *path, uint32_t mask) {
-  REAL(inotify_add_watch, int (*)(int, const char *, uint32_t));
+static long fp_inotify(int fd, const char *path, uint32_t mask) {
   struct res r;
   long rs = resolve(AT_FDCWD, path, (mask & IN_DONT_FOLLOW ? 0 : 1) | MAY_BE_UNSEEN, &r);
-  if (rs == 1) rs = -ENOENT;
-  if (rs == 0 || rs == UNSEEN) rs = sc(SYS_inotify_add_watch, fd, (long)r.host, mask, 0, 0);
-  ANSWER(rs, AT_FDCWD, path, real_inotify_add_watch(fd, path, mask));
+  if (rs == 0 || rs == 1 || rs == UNSEEN) rs = sc(SYS_inotify_add_watch, fd, (long)r.host, mask, 0, 0);
+  return rs;
+}
+int inotify_add_watch(int fd, const char *path, uint32_t mask) {
+  REAL(inotify_add_watch, int (*)(int, const char *, uint32_t));
+  ANSWER(fp_inotify(fd, path, mask), AT_FDCWD, path, real_inotify_add_watch(fd, path, mask));
 }
 
 
@@ -1828,3 +1864,5 @@ struct passwd *getpwuid(uid_t uid) {
   missed(__func__, AT_FDCWD, "(passwd)");
   return real_getpwuid(uid);
 }
+
+#include "svc-patch.h"

tools/proot/fastpath/svc-cache.h

@@ -0,0 +1,90 @@
+struct svc_cached_sites {
+  uint64_t magic;
+  uint32_t api, count;
+  uint64_t offset[256];
+};
+static unsigned svc_cache_hits, svc_cache_misses;
+
+static int svc_segment(const struct dl_phdr_info *info, uint64_t at, size_t length, unsigned flags) {
+  for (int i = 0; i < info->dlpi_phnum; i++) {
+    const ElfW(Phdr) *p = &info->dlpi_phdr[i];
+    if (p->p_type == PT_LOAD && (p->p_flags & flags) == flags &&
+        at >= p->p_vaddr && at - p->p_vaddr <= p->p_memsz &&
+        length <= p->p_memsz - (at - p->p_vaddr)) return i + 1;
+  }
+  return 0;
+}
+
+static int svc_cache_name(const struct dl_phdr_info *info, char *path, size_t size) {
+  const char *enabled = getenv("PROOT_FP_SVC_CACHE");
+  if (enabled && strcmp(enabled, "1")) return 0;
+  for (int i = 0; i < info->dlpi_phnum; i++) {
+    const ElfW(Phdr) *p = &info->dlpi_phdr[i];
+    if (p->p_type != PT_NOTE || !svc_segment(info, p->p_vaddr, p->p_memsz, PF_R)) continue;
+    const unsigned char *note = (const unsigned char *)(info->dlpi_addr + p->p_vaddr);
+    for (size_t at = 0; p->p_memsz - at >= sizeof(ElfW(Nhdr));) {
+      ElfW(Nhdr) h;
+      memcpy(&h, note + at, sizeof h);
+      at += sizeof h;
+      size_t name = ((size_t)h.n_namesz + 3) & ~(size_t)3;
+      size_t desc = ((size_t)h.n_descsz + 3) & ~(size_t)3;
+      if (name > p->p_memsz - at || desc > p->p_memsz - at - name) break;
+      if (h.n_type == NT_GNU_BUILD_ID && h.n_namesz == 4 && !memcmp(note + at, "GNU", 4) &&
+          h.n_descsz && h.n_descsz <= 64) {
+        char id[129];
+        static const char hex[] = "0123456789abcdef";
+        for (size_t j = 0; j < h.n_descsz; j++) {
+          unsigned char b = note[at + name + j];
+          id[2 * j] = hex[b >> 4];
+          id[2 * j + 1] = hex[b & 15];
+        }
+        id[2 * h.n_descsz] = 0;
+        int n = snprintf(path, size, "/tmp/.droiddeck-svc-v2-%d-%s", svc_android_api, id);
+        return n > 0 && (size_t)n < size;
+      }
+      at += name + desc;
+    }
+  }
+  return 0;
+}
+
+static int svc_cache_read(const struct dl_phdr_info *info, const char *path) {
+  long fd = fp_open(AT_FDCWD, path, O_RDONLY | O_NOFOLLOW | O_CLOEXEC | O_NONBLOCK, 0);
+  if (fd < 0) return 0;
+  struct stat st;
+  struct svc_cached_sites table;
+  int valid = sc(SYS_fstat, fd, (long)&st, 0, 0, 0) == 0 && S_ISREG(st.st_mode) &&
+              st.st_uid == (uid_t)sc(SYS_getuid, 0, 0, 0, 0, 0) && !(st.st_mode & 0077) &&
+              st.st_size == sizeof table && sc(SYS_read, fd, (long)&table, sizeof table, 0, 0) == sizeof table;
+  sc(SYS_close, fd, 0, 0, 0, 0);
+  if (!valid || table.magic != 0x324356534444ULL || table.api != (unsigned)svc_android_api ||
+      !table.count || table.count > 256 || table.count > 256 - svc_planned) return 0;
+  for (unsigned i = 0; i < table.count; i++) {
+    uint64_t offset = table.offset[i];
+    int segment = svc_segment(info, offset, 4, PF_R | PF_X);
+    if ((offset & 3) || (i && offset <= table.offset[i - 1]) || !segment) return 0;
+    const ElfW(Phdr) *p = &info->dlpi_phdr[segment - 1];
+    const uint32_t *text = (const uint32_t *)(info->dlpi_addr + p->p_vaddr);
+    size_t at = (offset - p->p_vaddr) / 4;
+    if (text[at] != 0xd4000001U || !svc_supported(svc_site_number(text, at))) return 0;
+  }
+  for (unsigned i = 0; i < table.count; i++)
+    svc_plan[svc_planned++].site = (uint32_t *)(info->dlpi_addr + table.offset[i]);
+  return 1;
+}
+
+static void svc_cache_write(const struct dl_phdr_info *info, const char *path, size_t first) {
+  if (svc_planned == first || svc_planned == 256) return;
+  struct svc_cached_sites table = {.magic = 0x324356534444ULL, .api = svc_android_api, .count = svc_planned - first};
+  for (unsigned i = 0; i < table.count; i++)
+    table.offset[i] = (uintptr_t)svc_plan[first + i].site - info->dlpi_addr;
+  char temporary[256];
+  int n = snprintf(temporary, sizeof temporary, "%s.part-%d", path, getpid());
+  if (n <= 0 || n >= (int)sizeof temporary) return;
+  long fd = fp_open(AT_FDCWD, temporary, O_WRONLY | O_CREAT | O_EXCL | O_NOFOLLOW | O_CLOEXEC, 0600);
+  if (fd < 0) return;
+  long written = sc(SYS_write, fd, (long)&table, sizeof table, 0, 0);
+  sc(SYS_close, fd, 0, 0, 0, 0);
+  if (written == sizeof table) fp_rename(AT_FDCWD, temporary, AT_FDCWD, path, 0);
+  fp_unlink(AT_FDCWD, temporary, 0);
+}

tools/proot/fastpath/svc-dispatch.h

@@ -0,0 +1,128 @@
+static int svc_android_api;
+
+static int svc_host_android_api(void) {
+  const char *value = getenv("PROOT_FP_ANDROID_API");
+  if (!value || !*value) return 0;
+  char *end;
+  long api = strtol(value, &end, 10);
+  return !*end && api > 0 && api < 1000 ? (int)api : 0;
+}
+
+static long svc_memory(void *local, long remote, size_t size, int write) {
+  if (!size) return 0;
+  struct iovec here = {local, size}, there = {(void *)remote, size};
+  return sc(write ? SYS_process_vm_writev : SYS_process_vm_readv, getpid(),
+            (long)&here, 1, (long)&there, 1);
+}
+
+static int svc_path(long pointer, char *path) {
+  long n = svc_memory(path, pointer, PATH_MAX, 0);
+  return n > 0 && memchr(path, 0, (size_t)n) != NULL;
+}
+
+static long svc_output(long result, void *value, long out, size_t size) {
+  if (result < 0) return result;
+  return svc_memory(value, out, size, 1) == (long)size ? result : FP_SLOW;
+}
+
+static int svc_supported(int nr) {
+  switch (nr) {
+  case SYS_openat: case SYS_newfstatat: case SYS_statx: case SYS_readlinkat:
+  case SYS_faccessat: case SYS_mkdirat: case SYS_unlinkat: case SYS_renameat:
+  case SYS_renameat2: case SYS_symlinkat: case SYS_fchmodat: case SYS_utimensat:
+  case SYS_truncate: case SYS_statfs: case SYS_getxattr: case SYS_lgetxattr:
+  case SYS_listxattr: case SYS_llistxattr: case SYS_inotify_add_watch: case SYS_getcwd:
+    return 1;
+  case SYS_faccessat2:
+    return svc_android_api > 0;
+  case SYS_clone3:
+    return svc_android_api > 0 && svc_android_api < 35;
+  default: return 0;
+  }
+}
+
+static long svc_blocked(const long *r) {
+  if (!svc_android_api) return FP_SLOW;
+  if (r[8] == SYS_faccessat2 || (r[8] == SYS_clone3 && svc_android_api < 35)) return -ENOSYS;
+  return FP_SLOW;
+}
+
+static long svc_path_call(const long *r) {
+#ifdef FP_SVC_TEST_HOOK
+  FP_SVC_TEST_HOOK(r);
+#endif
+  char path[PATH_MAX], other[PATH_MAX];
+  struct timespec times[2];
+  const struct timespec *when = (const struct timespec *)r[2];
+  int first = 1, second = -1;
+  switch (r[8]) {
+  case SYS_getcwd: {
+    if (own_cwd(path) == FP_SLOW) return FP_SLOW;
+    size_t n = strlen(path) + 1;
+    if ((unsigned long)r[1] < n) return -ERANGE;
+    return svc_output(n, path, r[0], n);
+  }
+  case SYS_truncate: case SYS_statfs: case SYS_getxattr: case SYS_lgetxattr:
+  case SYS_listxattr: case SYS_llistxattr: first = 0; break;
+  case SYS_renameat: case SYS_renameat2: second = 3; break;
+  case SYS_symlinkat: first = 0; second = 2; break;
+  case SYS_utimensat:
+    if (when) {
+      if (svc_memory(times, r[2], sizeof times, 0) != sizeof times) return FP_SLOW;
+      for (int i = 0; i < 2; i++)
+        if (times[i].tv_nsec != UTIME_NOW && times[i].tv_nsec != UTIME_OMIT &&
+            (times[i].tv_nsec < 0 || times[i].tv_nsec >= 1000000000L)) return FP_SLOW;
+      when = times;
+    }
+    if (!r[1]) return fp_utimens((int)r[0], NULL, when, (int)r[3]);
+    break;
+  case SYS_inotify_add_watch: {
+    char link[64], target[64];
+    snprintf(link, sizeof link, "/proc/self/fd/%d", (int)r[0]);
+    long n = sc(SYS_readlinkat, AT_FDCWD, (long)link, (long)target, sizeof target - 1, 0);
+    if (n <= 0) return FP_SLOW;
+    target[n] = 0;
+    if (strcmp(target, "anon_inode:inotify")) return FP_SLOW;
+    break;
+  }
+  default:
+    if (!svc_supported((int)r[8])) return FP_SLOW;
+  }
+  if (!svc_path(r[first], path) || (second >= 0 && !svc_path(r[second], other))) return FP_SLOW;
+  switch (r[8]) {
+  case SYS_openat: return fp_open((int)r[0], path, (int)r[2], (mode_t)r[3]);
+  case SYS_newfstatat: {
+    struct stat st;
+    return svc_output(fp_stat((int)r[0], path, (int)r[3], &st), &st, r[2], sizeof st);
+  }
+  case SYS_statx: {
+    struct statx st;
+    return svc_output(fp_statx((int)r[0], path, (int)r[2], (unsigned)r[3], &st), &st, r[4], sizeof st);
+  }
+  case SYS_readlinkat: {
+    char text[PATH_MAX];
+    size_t written = 0;
+    long result = fp_readlink((int)r[0], path, text, (size_t)r[3], &written);
+    return svc_output(result, text, r[2], written);
+  }
+  case SYS_faccessat: return fp_access((int)r[0], path, (int)r[2], 0);
+  case SYS_mkdirat: return fp_mkdir((int)r[0], path, (mode_t)r[2]);
+  case SYS_unlinkat: return fp_unlink((int)r[0], path, (int)r[2]);
+  case SYS_renameat: case SYS_renameat2:
+    return fp_rename((int)r[0], path, (int)r[2], other, r[8] == SYS_renameat2 ? (unsigned)r[4] : 0);
+  case SYS_symlinkat: return fp_symlink(path, (int)r[1], other);
+  case SYS_fchmodat: return fp_chmod((int)r[0], path, (mode_t)r[2], 0);
+  case SYS_utimensat: return fp_utimens((int)r[0], path, when, (int)r[3]);
+  case SYS_truncate: return fp_truncate(path, (off_t)r[1]);
+  case SYS_statfs: {
+    struct statfs st;
+    return svc_output(fp_statfs(path, &st), &st, r[1], sizeof st);
+  }
+  case SYS_getxattr: case SYS_lgetxattr:
+    return fp_getxattr(path, (const char *)r[1], (void *)r[2], (size_t)r[3], r[8] == SYS_getxattr);
+  case SYS_listxattr: case SYS_llistxattr:
+    return fp_listxattr(path, (char *)r[1], (size_t)r[2], r[8] == SYS_listxattr);
+  case SYS_inotify_add_watch: return fp_inotify((int)r[0], path, (uint32_t)r[2]);
+  default: return FP_SLOW;
+  }
+}

tools/proot/fastpath/svc-patch.h

@@ -0,0 +1,269 @@
+static inline int svc_number(uint32_t insn) {
+  if ((insn & 0xffe0001fU) == 0xd2800008U ||
+      (insn & 0xffe0001fU) == 0x52800008U)
+    return (insn >> 5) & 0xffff;
+  return -1;
+}
+
+static inline int svc_argument_instruction(uint32_t insn) {
+  if (insn == 0xd503201fU) return 1;
+  if ((insn & 0xffc00000U) == 0x29000000U ||
+      (insn & 0xffc00000U) == 0xa9000000U ||
+      (insn & 0xffc00000U) == 0xb9000000U ||
+      (insn & 0xffc00000U) == 0xf9000000U) return 1;
+  if ((insn & 31) == 8) return 0;
+  return (insn & 0xffe0ffe0U) == 0xaa0003e0U ||
+         (insn & 0xffe0ffe0U) == 0x2a0003e0U ||
+         (insn & 0xfffffc00U) == 0x93407c00U ||
+         (insn & 0x7f800000U) == 0x52800000U ||
+         (insn & 0x7f800000U) == 0x12800000U ||
+         (insn & 0xffc00000U) == 0xb9400000U ||
+         (insn & 0xffc00000U) == 0xf9400000U ||
+         (insn & 0xffc00000U) == 0xb9800000U ||
+         (insn & 0xff000000U) == 0x91000000U ||
+         (insn & 0xff000000U) == 0xd1000000U ||
+         (insn & 0xff000000U) == 0x11000000U ||
+         (insn & 0xff000000U) == 0x51000000U;
+}
+
+static inline int svc_site_number(const uint32_t *text, size_t at) {
+  for (size_t back = 1; back <= 8 && back <= at; back++) {
+    uint32_t insn = text[at - back];
+    int nr = svc_number(insn);
+    if (nr >= 0) return nr;
+    if (!svc_argument_instruction(insn)) break;
+  }
+  return -1;
+}
+
+static inline int svc_branch(uintptr_t from, uintptr_t to, uint32_t *out) {
+  int64_t delta = (int64_t)to - (int64_t)from;
+  if ((delta & 3) || delta < -0x8000000LL || delta >= 0x8000000LL) return 0;
+  *out = 0x14000000U | ((uint32_t)(delta / 4) & 0x03ffffffU);
+  return 1;
+}
+
+#if defined(__aarch64__) && !defined(FP_SVC_ENCODINGS_ONLY)
+#include <asm/hwcap.h>
+#include <link.h>
+#include <sys/uio.h>
+#include <signal.h>
+
+static __thread uintptr_t svc_busy __attribute__((tls_model("initial-exec")));
+static unsigned svc_sites, svc_pages;
+static unsigned long svc_fast, svc_slow;
+static unsigned long long svc_scan_ns, svc_patch_ns;
+
+#include "svc-dispatch.h"
+#include "svc-stack.h"
+
+static long svc_original(const long *r) {
+  register long x0 __asm__("x0") = r[0];
+  register long x1 __asm__("x1") = r[1];
+  register long x2 __asm__("x2") = r[2];
+  register long x3 __asm__("x3") = r[3];
+  register long x4 __asm__("x4") = r[4];
+  register long x5 __asm__("x5") = r[5];
+  register long x8 __asm__("x8") = r[8];
+  __asm__ volatile("svc #0" : "+r"(x0) : "r"(x1), "r"(x2), "r"(x3), "r"(x4), "r"(x5), "r"(x8) : "memory", "cc");
+  return x0;
+}
+
+__attribute__((used, visibility("hidden")))
+long fp_svc_dispatch(const long *r) {
+  int saved_errno = errno;
+  long result = svc_blocked(r);
+  uintptr_t frame = (uintptr_t)r;
+  if (result == FP_SLOW && (!svc_busy || frame >= svc_busy)) {
+    uintptr_t previous = svc_busy;
+    svc_busy = frame;
+    if (svc_stack_room(frame)) { result = svc_path_call(r); svc_busy = 0; }
+    else svc_busy = previous;
+  }
+  if (result == FP_SLOW) {
+    __atomic_fetch_add(&svc_slow, 1, __ATOMIC_RELAXED);
+    result = svc_original(r);
+  } else __atomic_fetch_add(&svc_fast, 1, __ATOMIC_RELAXED);
+  errno = saved_errno;
+  return result;
+}
+
+void fp_svc_entry(void) __attribute__((visibility("hidden")));
+/* A replaced svc may have live values in every register, including NZCV and SIMD. */
+__asm__(
+  ".text\n.p2align 4\n.hidden fp_svc_entry\n.type fp_svc_entry,%function\nfp_svc_entry:\n"
+  ".cfi_startproc\n.cfi_def_cfa sp, 16\n.cfi_return_column 16\n"
+  "bti j\nsub sp, sp, #800\n.cfi_def_cfa_offset 816\n"
+  "stp x0, x1, [sp, #0]\nstp x2, x3, [sp, #16]\n"
+  "stp x4, x5, [sp, #32]\nstp x6, x7, [sp, #48]\n"
+  "stp x8, x9, [sp, #64]\nstp x10, x11, [sp, #80]\n"
+  "stp x12, x13, [sp, #96]\nstp x14, x15, [sp, #112]\n"
+  "stp x18, x19, [sp, #128]\nstp x20, x21, [sp, #144]\n"
+  "stp x22, x23, [sp, #160]\nstp x24, x25, [sp, #176]\n"
+  "stp x26, x27, [sp, #192]\nstp x28, x29, [sp, #208]\n"
+  "stp x30, x17, [sp, #224]\nmrs x0, nzcv\nstr x0, [sp, #240]\n"
+  "ldr x0, [x17, #8]\nstr x0, [sp, #784]\n"
+  ".cfi_offset 16, -32\n.cfi_offset 19, -680\n.cfi_offset 20, -672\n"
+  ".cfi_offset 21, -664\n.cfi_offset 22, -656\n.cfi_offset 23, -648\n"
+  ".cfi_offset 24, -640\n.cfi_offset 25, -632\n.cfi_offset 26, -624\n"
+  ".cfi_offset 27, -616\n.cfi_offset 28, -608\n.cfi_offset 29, -600\n.cfi_offset 30, -592\n"
+  ".cfi_offset 72, -432\n.cfi_offset 73, -416\n.cfi_offset 74, -400\n.cfi_offset 75, -384\n"
+  ".cfi_offset 76, -368\n.cfi_offset 77, -352\n.cfi_offset 78, -336\n.cfi_offset 79, -320\n"
+  "mrs x0, fpcr\nstr x0, [sp, #768]\nmrs x0, fpsr\nstr x0, [sp, #776]\n"
+  "stp q0, q1, [sp, #256]\nstp q2, q3, [sp, #288]\n"
+  "stp q4, q5, [sp, #320]\nstp q6, q7, [sp, #352]\n"
+  "stp q8, q9, [sp, #384]\nstp q10, q11, [sp, #416]\n"
+  "stp q12, q13, [sp, #448]\nstp q14, q15, [sp, #480]\n"
+  "stp q16, q17, [sp, #512]\nstp q18, q19, [sp, #544]\n"
+  "stp q20, q21, [sp, #576]\nstp q22, q23, [sp, #608]\n"
+  "stp q24, q25, [sp, #640]\nstp q26, q27, [sp, #672]\n"
+  "stp q28, q29, [sp, #704]\nstp q30, q31, [sp, #736]\n"
+  "mov x0, sp\nbl fp_svc_dispatch\nstr x0, [sp]\n"
+  "ldp q0, q1, [sp, #256]\nldp q2, q3, [sp, #288]\n"
+  "ldp q4, q5, [sp, #320]\nldp q6, q7, [sp, #352]\n"
+  "ldp q8, q9, [sp, #384]\nldp q10, q11, [sp, #416]\n"
+  "ldp q12, q13, [sp, #448]\nldp q14, q15, [sp, #480]\n"
+  "ldp q16, q17, [sp, #512]\nldp q18, q19, [sp, #544]\n"
+  "ldp q20, q21, [sp, #576]\nldp q22, q23, [sp, #608]\n"
+  "ldp q24, q25, [sp, #640]\nldp q26, q27, [sp, #672]\n"
+  "ldp q28, q29, [sp, #704]\nldp q30, q31, [sp, #736]\n"
+  "ldr x0, [sp, #768]\nmsr fpcr, x0\nldr x0, [sp, #776]\nmsr fpsr, x0\n"
+  "ldr x0, [sp, #240]\nmsr nzcv, x0\n"
+  "ldp x0, x1, [sp, #0]\nldp x2, x3, [sp, #16]\n"
+  "ldp x4, x5, [sp, #32]\nldp x6, x7, [sp, #48]\n"
+  "ldp x8, x9, [sp, #64]\nldp x10, x11, [sp, #80]\n"
+  "ldp x12, x13, [sp, #96]\nldp x14, x15, [sp, #112]\n"
+  "ldp x18, x19, [sp, #128]\nldp x20, x21, [sp, #144]\n"
+  "ldp x22, x23, [sp, #160]\nldp x24, x25, [sp, #176]\n"
+  "ldp x26, x27, [sp, #192]\nldp x28, x29, [sp, #208]\n"
+  "ldr x30, [sp, #224]\nldr x16, [sp, #232]\n"
+  "add sp, sp, #800\n.cfi_def_cfa_offset 16\nbr x16\n.cfi_endproc\n.size fp_svc_entry, .-fp_svc_entry\n"
+);
+
+static int svc_emit(uint32_t *code, uint32_t *site, uint32_t *branch) {
+  uint32_t back;
+  if (!svc_branch((uintptr_t)site, (uintptr_t)code, branch) ||
+      !svc_branch((uintptr_t)(code + 5), (uintptr_t)(site + 1), &back)) return 0;
+  code[0] = 0xa9bf47f0U; /* stp x16, x17, [sp, #-16]! */
+  code[1] = 0x580000f0U; /* ldr x16, literal at +32 */
+  code[2] = 0x10000051U; /* adr x17, +8 */
+  code[3] = 0xd61f0200U; /* br x16 */
+  code[4] = 0xa8c147f0U; /* ldp x16, x17, [sp], #16 */
+  code[5] = back;
+  uintptr_t entry = (uintptr_t)fp_svc_entry;
+  uintptr_t resume = (uintptr_t)(site + 1);
+  memcpy(code + 6, &resume, sizeof resume);
+  memcpy(code + 8, &entry, sizeof entry);
+  return 1;
+}
+
+struct svc_site { uint32_t *site; uint32_t branch; };
+static struct svc_site svc_plan[256];
+static size_t svc_planned;
+
+#include "svc-cache.h"
+
+static int svc_object(struct dl_phdr_info *info, size_t size, void *unused) {
+  (void)size;
+  (void)unused;
+  const char *name = strrchr(info->dlpi_name, '/');
+  name = name ? name + 1 : info->dlpi_name;
+  if (strcmp(name, "libc.so.6") && strcmp(name, "ld-linux-aarch64.so.1")) return 0;
+  char cache[224];
+  int cached = svc_cache_name(info, cache, sizeof cache);
+  if (cached && svc_cache_read(info, cache)) { svc_cache_hits++; return 0; }
+  if (cached) svc_cache_misses++;
+  size_t first = svc_planned;
+  for (int i = 0; i < info->dlpi_phnum; i++) {
+    const ElfW(Phdr) *ph = &info->dlpi_phdr[i];
+    if (ph->p_type != PT_LOAD || !(ph->p_flags & PF_X)) continue;
+    uint32_t *text = (uint32_t *)(info->dlpi_addr + ph->p_vaddr);
+    for (size_t j = 1; j < ph->p_memsz / 4 && svc_planned < 256; j++) {
+      if (text[j] != 0xd4000001U) continue;
+      int nr = svc_site_number(text, j);
+      if (!svc_supported(nr)) continue;
+      svc_plan[svc_planned++].site = &text[j];
+    }
+  }
+  if (cached) svc_cache_write(info, cache, first);
+  return 0;
+}
+
+static uint32_t *svc_relay_page(uintptr_t site, size_t page) {
+  uintptr_t base = site & ~(uintptr_t)(page - 1);
+  for (uintptr_t delta = page; delta < 0x7ff0000UL; delta += 0x10000UL) {
+    for (int direction = -1; direction <= 1; direction += 2) {
+      uintptr_t at = direction < 0 ? base - delta : base + delta;
+      if (at < page) continue;
+      void *p = mmap((void *)at, page, PROT_READ | PROT_WRITE,
+                     MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED_NOREPLACE, -1, 0);
+      if (p == (void *)at) return p;
+      if (p != MAP_FAILED) munmap(p, page);
+    }
+  }
+  return NULL;
+}
+
+#include "svc-publish.h"
+
+static int svc_single_thread(void) {
+  struct entry { uint64_t ino; int64_t off; unsigned short len; unsigned char type; char name[]; };
+  char buf[4096] __attribute__((aligned(8)));
+  long fd = sc(SYS_openat, AT_FDCWD, (long)"/proc/self/task", O_RDONLY | O_DIRECTORY | O_CLOEXEC, 0, 0);
+  if (fd < 0) return 0;
+  int count = 0;
+  long n;
+  while ((n = sc(SYS_getdents64, fd, (long)buf, sizeof buf, 0, 0)) > 0) {
+    for (long at = 0; at < n;) {
+      const struct entry *e = (const struct entry *)(buf + at);
+      if (n - at < (long)offsetof(struct entry, name) + 1 ||
+          e->len < offsetof(struct entry, name) + 1 || e->len > n - at) { count = -1; goto done; }
+      if (e->name[0] >= '0' && e->name[0] <= '9' && ++count > 1) goto done;
+      at += e->len;
+    }
+  }
+  if (n < 0) count = -1;
+done:
+  sc(SYS_close, fd, 0, 0, 0, 0);
+  return count == 1;
+}
+
+static void svc_stats(void) {
+  char line[256];
+  int n = snprintf(line, sizeof line, "fastpath-svc[%d]: %u sites, %u relay pages, %lu fast, %lu to proot; scan %llu us, patch %llu us, android %d, cache %u hit %u miss\n",
+                   getpid(), svc_sites, svc_pages, svc_fast, svc_slow, svc_scan_ns / 1000, svc_patch_ns / 1000, svc_android_api, svc_cache_hits, svc_cache_misses);
+  if (n > 0 && write(2, line, n < (int)sizeof line ? n : (int)sizeof line - 1) < 0) {}
+}
+
+static unsigned long long svc_clock(void) {
+  struct timespec t;
+  if (sc(SYS_clock_gettime, CLOCK_MONOTONIC, (long)&t, 0, 0, 0) < 0) return 0;
+  return (unsigned long long)t.tv_sec * 1000000000ULL + t.tv_nsec;
+}
+
+static void svc_child(void) { svc_busy = 0; }
+
+__attribute__((constructor)) static void svc_patch_init(void) {
+  const char *enabled = getenv("PROOT_FP_PATCH_SVC");
+  if ((enabled && strcmp(enabled, "1")) || !fp_ready()) return;
+  if (getauxval(AT_HWCAP) & HWCAP_SVE) return;
+#ifdef HWCAP2_SME
+  if (getauxval(AT_HWCAP2) & HWCAP2_SME) return;
+#endif
+  if (!svc_single_thread()) return;
+  if (pthread_atfork(NULL, NULL, svc_child)) return;
+  svc_android_api = svc_host_android_api();
+  size_t page = (size_t)sysconf(_SC_PAGESIZE);
+  if (page < 4096 || (page & (page - 1))) return;
+  int mem = (int)sc(SYS_openat, AT_FDCWD, (long)"/proc/self/mem", O_RDWR | O_CLOEXEC, 0, 0);
+  if (mem < 0) return;
+  unsigned long long start = svc_clock();
+  dl_iterate_phdr(svc_object, NULL);
+  svc_scan_ns = svc_clock() - start;
+  start = svc_clock();
+  svc_publish(mem, page);
+  sc(SYS_close, mem, 0, 0, 0, 0);
+  svc_patch_ns = svc_clock() - start;
+  if (getenv("PROOT_FP_STATS")) atexit(svc_stats);
+}
+#endif

tools/proot/fastpath/svc-publish.h

@@ -0,0 +1,28 @@
+static void svc_publish(int mem, size_t page) {
+  for (size_t first = 0; first < svc_planned;) {
+    uint32_t *relay = svc_relay_page((uintptr_t)svc_plan[first].site, page);
+    if (!relay) { first++; continue; }
+    size_t end = first;
+    for (; end < svc_planned && (end - first + 1) * 40 <= page; end++) {
+      uint32_t *code = relay + (end - first) * 10;
+      if (!svc_emit(code, svc_plan[end].site, &svc_plan[end].branch)) break;
+    }
+    __builtin___clear_cache((char *)relay, (char *)relay + page);
+    if (end == first || mprotect(relay, page, PROT_READ | PROT_EXEC)) {
+      munmap(relay, page);
+      first++;
+      continue;
+    }
+    unsigned patched = 0;
+    for (size_t i = first; i < end; i++) {
+      struct svc_site *s = &svc_plan[i];
+      long written = sc(SYS_pwrite64, mem, (long)&s->branch, sizeof s->branch, (long)s->site, 0);
+      if (written != sizeof s->branch) continue;
+      __builtin___clear_cache((char *)s->site, (char *)(s->site + 1));
+      patched++;
+    }
+    if (patched) { svc_sites += patched; svc_pages++; }
+    else munmap(relay, page);
+    first = end;
+  }
+}

tools/proot/fastpath/svc-stack.h

@@ -0,0 +1,83 @@
+#include <sys/resource.h>
+#include <string.h>
+
+static __thread uintptr_t svc_stack_low __attribute__((tls_model("initial-exec")));
+static __thread uintptr_t svc_stack_high __attribute__((tls_model("initial-exec")));
+static __thread uintptr_t svc_stack_gap __attribute__((tls_model("initial-exec")));
+static __thread int svc_stack_main __attribute__((tls_model("initial-exec")));
+
+static int svc_hex(const char **text, uintptr_t *value) {
+  const char *p = *text;
+  uintptr_t n = 0;
+  int digits = 0;
+  for (;; p++) {
+    unsigned c = (unsigned char)*p;
+    unsigned d = c >= '0' && c <= '9' ? c - '0' : c >= 'a' && c <= 'f' ? c - 'a' + 10 : 16;
+    if (d == 16) break;
+    if (++digits > (int)(2 * sizeof n)) return 0;
+    n = (n << 4) | d;
+  }
+  *text = p;
+  *value = n;
+  return digits > 0;
+}
+
+static int svc_stack_line(const char *line, uintptr_t sp, uintptr_t *low, uintptr_t *high) {
+  uintptr_t a, b;
+  if (!svc_hex(&line, &a) || *line++ != '-' || !svc_hex(&line, &b) ||
+      *line++ != ' ' || line[0] != 'r' || line[1] != 'w' || a >= b || sp < a || sp >= b) return 0;
+  *low = a;
+  *high = b;
+  return 1;
+}
+
+static __attribute__((noinline)) int svc_stack_find(uintptr_t sp) {
+  char chunk[512], line[96];
+  size_t at = 0;
+  long fd = sc(SYS_openat, AT_FDCWD, (long)"/proc/self/maps", O_RDONLY | O_CLOEXEC, 0, 0);
+  if (fd < 0) return 0;
+  long n;
+  uintptr_t low = 0, high = 0, previous = 0;
+  int found = 0;
+  while (!found && (n = sc(SYS_read, fd, (long)chunk, sizeof chunk, 0, 0)) > 0) {
+    for (long i = 0; i < n; i++) {
+      if (chunk[i] == '\n') {
+        line[at] = 0;
+        found = svc_stack_line(line, sp, &low, &high);
+        if (!found) {
+          const char *p = line;
+          uintptr_t a, b;
+          if (svc_hex(&p, &a) && *p++ == '-' && svc_hex(&p, &b) && b <= sp) previous = b;
+        }
+        at = 0;
+        if (found) break;
+      } else if (at < sizeof line - 1) line[at++] = chunk[i];
+    }
+  }
+  sc(SYS_close, fd, 0, 0, 0, 0);
+  if (found) {
+    svc_stack_low = 0;
+    svc_stack_high = high;
+    svc_stack_main = strstr(line, "[stack]") != NULL;
+    svc_stack_gap = previous < low && low - previous > 16 * 1024 * 1024 ? previous + 16 * 1024 * 1024 : low;
+    svc_stack_low = low;
+  }
+  return found;
+}
+
+static int svc_stack_room(uintptr_t sp) {
+  stack_t alt;
+  if (sc(SYS_sigaltstack, 0, (long)&alt, 0, 0, 0) < 0 || (alt.ss_flags & SS_ONSTACK)) return 0;
+  if (!svc_stack_low || sp < svc_stack_low || sp >= svc_stack_high)
+    if (!svc_stack_find(sp)) return 0;
+  uintptr_t low = svc_stack_low;
+  if (svc_stack_main) {
+    struct rlimit limit;
+    if (sc(SYS_prlimit64, 0, RLIMIT_STACK, 0, (long)&limit, 0) < 0) return 0;
+    uintptr_t grow = limit.rlim_cur >= svc_stack_high ? 0 : svc_stack_high - limit.rlim_cur;
+    if (grow < svc_stack_gap) grow = svc_stack_gap;
+    if (grow < low) low = grow;
+  }
+  /* Path translation nests several PATH_MAX-sized frames. */
+  return sp - low >= 96 * 1024;
+}

tools/tests/test_fastpath_svc.py

@@ -0,0 +1,290 @@
+from pathlib import Path
+import shutil
+import subprocess
+import tempfile
+import unittest
+
+
+HEADER = Path(__file__).resolve().parents[1] / "proot/fastpath/svc-patch.h"
+
+
+class FastPathSvcEncodingTest(unittest.TestCase):
+    def compile_check(self, source, header):
+        if shutil.which("cc") is None:
+            self.skipTest("cc is not available")
+        with tempfile.TemporaryDirectory() as directory:
+            program = Path(directory) / "check.c"
+            program.write_text(source.replace('"HEADER"', '"' + (HEADER.parent / header).as_posix() + '"'))
+            output = Path(directory) / "check"
+            subprocess.run(["cc", "-O2", "-Wall", "-Wextra", "-Werror", str(program), "-o", str(output)], check=True)
+            subprocess.run([str(output)], check=True, timeout=10)
+
+    def test_benchmark_checks_execute_with_ndebug(self):
+        self.compile_check(r'''
+#define NDEBUG
+#include "HEADER"
+int main(void) {
+  int calls = 0;
+  assert(++calls == 1);
+  return calls != 1;
+}
+''', "../bench/checks.h")
+
+    def test_stack_bounds_and_alternate_stack_fallback(self):
+        self.compile_check(r'''
+#define _GNU_SOURCE
+#include <assert.h>
+#include <stdint.h>
+#include <stddef.h>
+#include <signal.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <sys/syscall.h>
+#include <sys/resource.h>
+static rlim_t stack_limit = 1024 * 1024;
+static long sc(long n, long a, long b, long c, long d, long e) {
+  if (n == SYS_prlimit64) {
+    if (!stack_limit) return -1;
+    ((struct rlimit *)d)->rlim_cur = stack_limit;
+    return 0;
+  }
+  return syscall(n,a,b,c,d,e);
+}
+#include "HEADER"
+int main(void) {
+  uintptr_t low, high;
+  assert(svc_stack_line("1000-9000 rw-p 00000000 00:00 0", 0x8000, &low, &high));
+  assert(low == 0x1000 && high == 0x9000);
+  assert(!svc_stack_line("1000-9000 r--p", 0x8000, &low, &high));
+  assert(!svc_stack_line("1000-9000 rw-p", 0x9000, &low, &high));
+  assert(!svc_stack_line("9000-1000 rw-p", 0x8000, &low, &high));
+  assert(!svc_stack_line("fffffffffffffffff-9000 rw-p", 0x8000, &low, &high));
+  assert(!svc_stack_line("", 0x8000, &low, &high));
+  char local;
+  assert(svc_stack_find((uintptr_t)&local));
+  assert((uintptr_t)&local >= svc_stack_low && (uintptr_t)&local < svc_stack_high);
+  svc_stack_main = 0; svc_stack_low = 0x1000; svc_stack_high = 0x100000;
+  assert(!svc_stack_room(0x1000 + 96 * 1024 - 1));
+  assert(svc_stack_room(0x1000 + 96 * 1024));
+  svc_stack_main = 1; svc_stack_low = 0xf0000; svc_stack_gap = 0x1000;
+  assert(svc_stack_room(0xff000));
+  stack_limit = 64 * 1024;
+  assert(!svc_stack_room(0xff000));
+  stack_limit = 0;
+  assert(!svc_stack_room(0xff000));
+  return 0;
+}
+''', "svc-stack.h")
+
+    def test_patch_publication_failure_and_partial_success(self):
+        self.compile_check(r'''
+#define _GNU_SOURCE
+#include <assert.h>
+#include <stdint.h>
+#include <stddef.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <sys/syscall.h>
+static struct svc_site { uint32_t *site; uint32_t branch; } svc_plan[3];
+static size_t svc_planned = 3;
+static unsigned svc_sites, svc_pages;
+static int fail_alloc, fail_emit, fail_rx, fail_write, writes, live, rx;
+static uint32_t *svc_relay_page(uintptr_t site, size_t page) {
+  (void)site;
+  if (fail_alloc) return NULL;
+  live++; rx = 0;
+  return calloc(1, page);
+}
+static int svc_emit(uint32_t *code, uint32_t *site, uint32_t *branch) {
+  (void)code; (void)site;
+  *branch = 0x14000001;
+  return !fail_emit;
+}
+static int mock_rx(void *p, size_t size, int prot) {
+  (void)p; (void)size;
+  assert(prot == (PROT_READ | PROT_EXEC));
+  if (fail_rx) return -1;
+  rx = 1;
+  return 0;
+}
+static int mock_unmap(void *p, size_t size) { (void)size; free(p); live--; return 0; }
+static long sc(long nr, long fd, long buf, long len, long at, long unused) {
+  (void)fd; (void)unused;
+  assert(nr == SYS_pwrite64 && len == 4 && rx);
+  if (++writes == fail_write) return -1;
+  memcpy((void *)at, (void *)buf, len);
+  return len;
+}
+#define mprotect mock_rx
+#define munmap mock_unmap
+#include "HEADER"
+int main(void) {
+  static uint32_t sites[3];
+  for (int mode = 0; mode < 5; mode++) {
+    for (int i=0; i<3; i++) { sites[i]=0xd4000001; svc_plan[i].site=&sites[i]; }
+    svc_sites=svc_pages=writes=live=rx=0;
+    fail_alloc=mode==0; fail_emit=mode==1; fail_rx=mode==2; fail_write=mode==3 ? 2 : 0;
+    svc_publish(7,4096);
+    if (mode<3) assert(!svc_sites && !svc_pages && !writes && !live);
+    else assert(svc_sites==(mode==3 ? 2U : 3U) && svc_pages==1 && live==1);
+    for (int i=0; i<3; i++) assert(sites[i]==(mode<3 || (mode==3 && i==1) ? 0xd4000001 : 0x14000001));
+  }
+  return 0;
+}
+''', "svc-publish.h")
+
+    def test_instruction_decoding_and_branch_limits(self):
+        if shutil.which("cc") is None:
+            self.skipTest("cc is not available")
+        source = r'''
+#include <assert.h>
+#include <stdint.h>
+#include <stddef.h>
+#define FP_SVC_ENCODINGS_ONLY
+#include "HEADER"
+int main(void) {
+  assert(svc_number(0xd2800708) == 56);
+  assert(svc_number(0x528009e8) == 79);
+  assert(svc_number(0xd2800709) == -1);
+  assert(svc_number(0xd2a00708) == -1);
+  assert(svc_number(0xf2800708) == -1);
+  assert(svc_number(0xaa0003e8) == -1);
+  const uint32_t create[] = {0xd2800708, 0x29047fe0, 0x92800c60, 0xd4000001};
+  const uint32_t create_at[] = {0xd2800708, 0x29047fe3, 0xb9404be3, 0xd4000001};
+  const uint32_t xstat[] = {0xd28009e8, 0xaa0203e1, 0xaa0303e2, 0x93407c83, 0xd4000001};
+  const uint32_t overwritten[] = {0xd2800708, 0xaa0003e8, 0xd4000001};
+  const uint32_t loaded[] = {0xd2800708, 0xf94003e8, 0xd4000001};
+  const uint32_t branch[] = {0xd2800708, 0x14000001, 0xd4000001};
+  const uint32_t call[] = {0xd2800708, 0x94000001, 0xd4000001};
+  const uint32_t shift[] = {0xd2800708, 0xd2a00708, 0xd4000001};
+  const uint32_t robust[] = {0xd2800c68, 0xf9000260, 0xa9000261, 0xf9400260, 0x9100e260, 0xd4000001};
+  const uint32_t writeback[] = {0xd2800708, 0xa9bf07e0, 0xd4000001};
+  const uint32_t added[] = {0xd2800708, 0x91000408, 0xd4000001};
+  assert(svc_site_number(create, 3) == 56);
+  assert(svc_site_number(create_at, 3) == 56);
+  assert(svc_site_number(xstat, 4) == 79);
+  assert(svc_site_number(overwritten, 2) == -1);
+  assert(svc_site_number(loaded, 2) == -1);
+  assert(svc_site_number(branch, 2) == -1);
+  assert(svc_site_number(call, 2) == -1);
+  assert(svc_site_number(shift, 2) == -1);
+  assert(svc_site_number(robust, 5) == 99);
+  assert(svc_site_number(writeback, 2) == -1);
+  assert(svc_site_number(added, 2) == -1);
+  uint32_t out;
+  uintptr_t pc = 0x10000000;
+  assert(svc_branch(pc, pc + 4, &out) && out == 0x14000001);
+  assert(svc_branch(pc, pc - 4, &out) && out == 0x17ffffff);
+  assert(svc_branch(pc, pc + 0x7fffffc, &out) && out == 0x15ffffff);
+  assert(svc_branch(pc, pc - 0x8000000, &out) && out == 0x16000000);
+  assert(!svc_branch(pc, pc + 0x8000000, &out));
+  assert(!svc_branch(pc, pc - 0x8000004, &out));
+  assert(!svc_branch(pc, pc + 2, &out));
+  return 0;
+}
+'''
+        with tempfile.TemporaryDirectory() as directory:
+            work = Path(directory)
+            program = work / "check.c"
+            program.write_text(source.replace("HEADER", HEADER.as_posix()))
+            output = work / "check"
+            subprocess.run(["cc", "-O2", "-Wall", "-Wextra", "-Werror", str(program), "-o", str(output)], check=True)
+            subprocess.run([str(output)], check=True, timeout=10)
+
+    def test_cache_revalidates_image_offsets_and_untrusted_files(self):
+        if shutil.which("cc") is None:
+            self.skipTest("cc is not available")
+        source = r'''
+#define _GNU_SOURCE
+#include <assert.h>
+#include <stdint.h>
+#include <stddef.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <link.h>
+#include <sys/stat.h>
+#include <sys/syscall.h>
+#define FP_SVC_ENCODINGS_ONLY
+#include "HEADER"
+static int svc_android_api = 33;
+static struct { uint32_t *site; } svc_plan[256];
+static size_t svc_planned;
+static int svc_supported(int nr) { return nr == 56 || nr == 79; }
+#define sc(n,a,b,c,d,e) syscall(n,a,b,c,d,e)
+static long fp_open(int d, const char *p, int f, mode_t m) { return openat(d,p,f,m); }
+static long fp_rename(int d, const char *p, int e, const char *q, int f) { (void)f; return renameat(d,p,e,q); }
+static long fp_unlink(int d, const char *p, int f) { return unlinkat(d,p,f); }
+#include "CACHE"
+static void store(const char *path, struct svc_cached_sites *table) {
+  int fd = open(path, O_WRONLY | O_TRUNC);
+  assert(fd >= 0 && write(fd, table, sizeof *table) == sizeof *table && close(fd) == 0);
+}
+int main(void) {
+  uint32_t image[32] = {0xd2800708, 0xd4000001, 0xd28009e8, 0xd4000001};
+  ElfW(Phdr) ph[] = {{.p_type=PT_LOAD, .p_flags=PF_R|PF_X, .p_memsz=sizeof image},
+                    {.p_type=PT_NOTE, .p_vaddr=64, .p_memsz=24}};
+  struct dl_phdr_info info = {.dlpi_addr=(uintptr_t)image, .dlpi_phdr=ph, .dlpi_phnum=2};
+  ElfW(Nhdr) note = {.n_type=NT_GNU_BUILD_ID, .n_namesz=4, .n_descsz=8};
+  memcpy((char *)image+64, &note, sizeof note);
+  memcpy((char *)image+76, "GNU", 4);
+  uint64_t id = getpid();
+  memcpy((char *)image+80, &id, 8);
+  char path[256], other[256];
+  assert(svc_cache_name(&info, path, sizeof path));
+  svc_android_api = 35;
+  assert(svc_cache_name(&info, other, sizeof other) && strcmp(path, other));
+  svc_android_api = 33;
+  setenv("PROOT_FP_SVC_CACHE", "0", 1);
+  assert(!svc_cache_name(&info, other, sizeof other));
+  unsetenv("PROOT_FP_SVC_CACHE");
+  ph[1].p_memsz = sizeof image;
+  assert(!svc_cache_name(&info, other, sizeof other));
+  ph[1].p_memsz = 24;
+  svc_plan[0].site = image+1; svc_plan[1].site = image+3; svc_planned=2;
+  svc_cache_write(&info, path, 0);
+  svc_planned=0;
+  assert(svc_cache_read(&info, path) && svc_planned==2 && svc_plan[1].site==image+3);
+  struct svc_cached_sites valid, table;
+  int fd = open(path, O_RDONLY);
+  assert(fd >= 0 && read(fd, &valid, sizeof valid)==sizeof valid && close(fd)==0);
+#define REJECT(change) do { table=valid; change; store(path,&table); svc_planned=0; \
+  assert(!svc_cache_read(&info,path) && svc_planned==0); } while(0)
+  REJECT(table.magic++);
+  REJECT(table.api=35);
+  REJECT(table.count=0);
+  REJECT(table.count=257);
+  REJECT(table.offset[0]=3);
+  REJECT(table.offset[0]=sizeof image);
+  REJECT(table.offset[0]=UINT64_MAX-3);
+  REJECT(table.offset[1]=table.offset[0]);
+  REJECT(table.offset[0]=12; table.offset[1]=4);
+  store(path,&valid);
+  image[1]=0x14000001;
+  assert(!svc_cache_read(&info,path));
+  image[1]=0xd4000001; image[0]=0xd2800008;
+  assert(!svc_cache_read(&info,path));
+  image[0]=0xd2800708;
+  assert(chmod(path,0644)==0 && !svc_cache_read(&info,path));
+  assert(chmod(path,0600)==0 && truncate(path,8)==0 && !svc_cache_read(&info,path));
+  assert(unlink(path)==0 && symlink("/dev/zero",path)==0 && !svc_cache_read(&info,path));
+  assert(unlink(path)==0 && mkdir(path,0700)==0 && !svc_cache_read(&info,path));
+  assert(rmdir(path)==0);
+  svc_planned=0;
+  svc_cache_write(&info,path,0);
+  assert(access(path,F_OK)<0);
+  (void)svc_cache_hits; (void)svc_cache_misses;
+  return 0;
+}
+'''
+        with tempfile.TemporaryDirectory() as directory:
+            work = Path(directory)
+            program = work / "cache.c"
+            program.write_text(source.replace('"HEADER"', '"' + HEADER.as_posix() + '"')
+                              .replace('"CACHE"', '"' + HEADER.with_name("svc-cache.h").as_posix() + '"'))
+            output = work / "cache"
+            subprocess.run(["cc", "-O2", "-Wall", "-Wextra", "-Werror", str(program), "-o", str(output)], check=True)
+            subprocess.run([str(output)], check=True, timeout=10)