mirror of
https://github.com/Droid-Deck/DroidDeck.git
synced 2026-11-18 20:36:04 +00:00
perf(proot): add opt-in glibc syscall patching
This commit is contained in:
1 parent
ef9866e423
commit
4b230cbd0f
8 files changed
+1004
No files matched your search
@@ -0,0 +1,311 @@
|
||||
# proroot: what it does and what DroidDeck can take from it
|
||||
|
||||
Static analysis of [coderredlab/proroot](https://github.com/coderredlab/proroot) v1.2.8
|
||||
(`c01039ab`, 2026-06-17). The repository holds only five arm64 binaries and a README. The licence is
|
||||
proprietary, and redistributing modified binaries is not allowed, so only the ideas below are
|
||||
usable, not the code. The binaries were read with the NDK's `llvm-readelf`, `llvm-nm`,
|
||||
`llvm-objdump` and `llvm-strings`. `libproroot-stub-loader.so` and `libproroot-bridge.so` still carry
|
||||
`.symtab`, and `libproroot-runtime.so` exports about 260 names, so the function names below are
|
||||
proroot's own.
|
||||
|
||||
## Architecture
|
||||
|
||||
proroot does not use ptrace. Everything runs inside the guest process.
|
||||
|
||||
| file | kind | role |
|
||||
|---|---|---|
|
||||
| `libproroot.so` | bionic PIE executable | Launcher. Parses `-r/-b/-w/-0/--link2symlink`, writes a config file (`.proroot-config-<pid>`, passed down as `PROROOT_CFG_FD`), and execs the guest through `libproroot-linker.so` or the stub loader. |
|
||||
| `libproroot-linker.so` | freestanding PIE, run as an executable | A from-scratch glibc dynamic linker ("cleanroom linker") that replaces `ld-linux-aarch64.so.1`. It maps the program and its `DT_NEEDED` libraries, relocates them, sets up static TLS, and stubs `_dl_*` and `__tunable_get_val`. It finds `_rtld_global_ro` through a table of offsets per glibc version ("no offset table for glibc X, patches skipped"). It also installs a SIGSYS handler with an allow-list. |
|
||||
| `libproroot-stub-loader.so` | freestanding static PIE | Userland `exec`. It maps the target and its `PT_INTERP` itself and builds the stack and auxv. It then **rewrites the `svc #0` sites** in the loaded images (`proroot_svc_scan`, `proroot_svc_patch_apply`, `select_trampoline_for_site`) to branch into per-syscall trampolines (`proroot_tramp_openat`, `_statx`, `_faccessat2`, `_clone3`, `_rt_sigaction`, `_shmget`, ...). Sites it cannot patch fall back to an in-process seccomp `RET_TRAP` filter with a SIGSYS handler ("trampoline fallback, activating trap-mode fallback"). |
|
||||
| `libproroot-runtime.so` | glibc `LD_PRELOAD` | Hooks about 200 libc entry points: path translation, fake root (`fake_id0_*`), link2symlink, exec routing, SysV shm, netlink, `/proc` and `/sys` views, `brk`, a fake DRM/KMS and GBM device plus libseat for compositors, X11 and Unix-socket fixups. After load it **patches the svc sites in glibc itself** (`patch_libc_syscalls`). |
|
||||
| `libproroot-bridge.so` | static `ET_EXEC` at `0x7000000000` | The "trampoline" for static guest binaries. It installs a path-trap seccomp filter plus a SIGSYS handler, maps the target, and serves a virtual `/proc` (`status`, `stat`, `cmdline`, `environ`, `mounts`, `mountinfo`, `fdinfo`, `auxv`). |
|
||||
|
||||
The executable files all live in `nativeLibraryDir`, which is the only place an app may exec from.
|
||||
That is why the dynamic linker and the static loader each have their own userland loader. proot
|
||||
handles the same constraint with `libproot-loader.so`.
|
||||
|
||||
## How the in-process syscall interception works
|
||||
|
||||
1. **Patching svc sites in glibc** (`patch_phdr_callback`, called through `dl_iterate_phdr`):
|
||||
- It scans only an allow-list of objects: `libc.so.6`/`libc-`, `ld-linux`, `libpthread`,
|
||||
`librt`, `libdl`. Main executables get patched early only for listed programs (apt, dpkg,
|
||||
curl, openssl, node, chromium/electron/playwright, passwd/useradd/...). Other programs are
|
||||
patched later (`log_deferred_libc_patch`, `PROROOT_FORCE_EARLY_PATCH`).
|
||||
- In each executable `PT_LOAD` it finds `svc #0` (`0xd4000001`). It checks the instructions
|
||||
before it for a `mov x8, #nr` it recognises. One fixed glibc `openat` call sequence gets
|
||||
special treatment: its `bl` is redirected to a relay (`emit_openat_call_relay_entry`).
|
||||
- The site is replaced with a `b`/`bl` to a relay stub on a page allocated within ±128 MiB
|
||||
(`alloc_relay_page`). Sites out of range are logged as `bl-out-of-range`; `large-image
|
||||
multi-cover tramp` places extra relay pages for big images.
|
||||
- **Write path:** it first tries `mprotect(PROT_READ|PROT_WRITE|PROT_EXEC)` on the segment.
|
||||
If that is refused, it `pwrite`s the 4-byte instruction through `/proc/self/mem`. Either way
|
||||
it then calls `__clear_cache`. The log line counts sites by kind: `patched %d seccomp + %d
|
||||
path (%d relay) + %d brk`.
|
||||
2. **In-process SIGSYS fallback**:
|
||||
- A BPF filter allows syscalls whose instruction pointer is inside proroot's own code and
|
||||
returns `SECCOMP_RET_TRAP` for the path, exec, clone and wait syscalls from anywhere else.
|
||||
The IP check is the same idea as our patch 0014, with TRAP instead of TRACE.
|
||||
- The handler emulates the call and writes the result into the signal frame. Guest SIGSYS
|
||||
handlers are kept and chained (`proroot_record_user_sigsys`, `invoke_user_sigsys`).
|
||||
3. **Keeping SIGSYS deliverable**, the objection in option 4 of
|
||||
[proot-performance](proot-performance.md):
|
||||
- The `sigprocmask` and `pthread_sigmask` hooks call `sigtrap_unblocked_mask`, which
|
||||
`sigdelset`s `SIGSYS` (and `SIGTRAP` for some guests) from any `SIG_BLOCK`/`SIG_SETMASK` set.
|
||||
- `rt_sigaction` is guarded so the guest cannot replace the handler (`proroot_tramp_rt_sigaction`,
|
||||
the `sigaction_guard` trace points).
|
||||
- The raw `rt_sigprocmask` that glibc makes internally, for example `pthread_create` blocking
|
||||
every signal, goes through a patched svc site. The stub loader and the runtime both compare
|
||||
against syscall 135.
|
||||
4. **Android-blocked syscalls are answered in-process.** The stub loader dispatches
|
||||
`set_robust_list`, `rseq`, `faccessat2`, `clone3`, `futex_waitv` and `seccomp`. The runtime
|
||||
emulates SysV shm (`emu_shm*`), `set*id`/`get*id` under `-0`, and `brk`
|
||||
(`emulate_guest_brk`, over a reserved mapping).
|
||||
|
||||
## Compared with DroidDeck
|
||||
|
||||
| | DroidDeck (proot + 0001–0018 + `libblfastpath.so`) | proroot |
|
||||
|---|---|---|
|
||||
| exec, `PT_INTERP`, `/proc/self/exe` | proot's loader under ptrace, 0.4–0.8 ms per exec | Its own loaders, with no tracer; also needs its own ld.so with a glibc offset table |
|
||||
| path calls through libc wrappers | fast path, about 0.6–1.9 µs | runtime hooks |
|
||||
| path calls from glibc internals (ld.so, fts, realpath, nss, `__open_nocancel`) and raw `svc` | **still go to proot (15–60 µs)** | **patched svc sites** |
|
||||
| Android-blocked syscalls (`set_robust_list`, `faccessat2`, `clone3`, ...) | SIGSYS stop answered by proot (0016 etc.), one ptrace round trip and one signal round trip each | in-process |
|
||||
| `brk` | proot emulation, a trap per call (mitigated by `top_pad`) | in-process emulation |
|
||||
| correctness fallback | proot is always there | none: anything missed runs unconfined or is emulated wrongly |
|
||||
| long Unix socket paths | fast path `/proc/self/fd/N/name` aliases | short socket aliases (`proroot_create_short_unix_socket_alias`) |
|
||||
| rseq | `GLIBC_TUNABLES=glibc.pthread.rseq=0` | trampoline |
|
||||
|
||||
proroot is not a model for replacing proot. It has no tracer to fall back on, so a missed call
|
||||
site or an unexpected glibc build silently breaks isolation or correctness. Its own linker depends
|
||||
on glibc internals for each version. Proton, Wine, FEX and Steam's self-updating ELFs are a much
|
||||
wider surface than the Node, Python and Chromium workloads proroot was tested on. Its value for
|
||||
DroidDeck is as a fast path *in front of* proot.
|
||||
|
||||
## What is worth taking
|
||||
|
||||
1. **Patch glibc's svc sites from `libblfastpath.so`'s constructor.** This is the largest gain
|
||||
and the cheapest way to do option 6 in [proot-performance](proot-performance.md) ("a proot-aware
|
||||
glibc") without rebuilding glibc.
|
||||
- For `libc.so.6` (and `ld-linux-aarch64.so.1` for runtime `dlopen`), rewrite each `svc #0`
|
||||
that follows a recognised `mov x8, #nr` for the path syscalls the fast path already handles.
|
||||
The rewrite is a `b` to a relay that calls the existing translator.
|
||||
- The relay issues the syscall from the `0xffff00000` trampoline page, which 0014 already lets
|
||||
through. Anything the translator declines is re-issued from an svc in the relay page, outside
|
||||
the trampoline, so proot traps it exactly as it does today. Correctness still rests on proot.
|
||||
- This covers fts (`find` over 2.7k files, unchanged today), realpath, nss, `__open_nocancel`,
|
||||
and `dlopen` after start-up.
|
||||
- Library loading at start-up, before the constructor runs, still traps.
|
||||
2. **Answer the Android-blocked syscalls through those relays.**
|
||||
- `set_robust_list` and `get_robust_list`: keep the head in TLS, as 0016 already does in
|
||||
proot.
|
||||
- `faccessat2`: answer `ENOSYS`, or translate it when flags are 0, as the wrapper does.
|
||||
- `clone3`: answer `ENOSYS` before Android 15, so glibc falls back to `clone`.
|
||||
- `futex_waitv`: droiddeck-fsync already handles this.
|
||||
- Each one removes a SIGSYS stop plus a ptrace round trip, about 30–50 µs. `set_robust_list`
|
||||
alone is one per thread, which matters for Steam/CEF and Wine thread churn.
|
||||
3. **The `/proc/self/mem` write as a fallback for patching.** If the app domain refuses
|
||||
`mprotect(PROT_WRITE|PROT_EXEC)` on rootfs (`app_data_file`) text, i.e. an `execmod` denial,
|
||||
a write through `/proc/self/mem` makes the page copy-on-write without an `mprotect`. That page
|
||||
is never writable and executable at once. Check this on the Thor before relying on it. The
|
||||
same question is still open for the trampoline page (`execmem`).
|
||||
4. **Keep SIGSYS unblocked if in-process `SECCOMP_RET_TRAP` is ever used.** Strip `SIGSYS` in the
|
||||
`sigprocmask`/`pthread_sigmask`/`rt_sigprocmask` paths, and keep the handler installed while
|
||||
chaining the guest's. This only becomes safe once item 1 covers glibc's internal
|
||||
`rt_sigprocmask`. It is a prerequisite, not a recommendation to switch.
|
||||
5. **Cache patch sites by build-id.** proroot rescans each image on every exec and defers patching
|
||||
for most programs to keep exec cheap. A scan of libc's roughly 1.5 MiB of text is cheap, but a
|
||||
per-build-id site table, written once in `/tmp` by the fast path, would make it free.
|
||||
|
||||
Not worth taking: the cleanroom ld.so and its glibc offset table; the userland exec loader (proot
|
||||
already does this); the fake DRM/GBM/libseat device (DroidDeck has its own compositor path);
|
||||
netlink and SysV shm (`netif.c` and `sysv.c` already cover these); and the long-socket aliasing
|
||||
(the fast path covers it).
|
||||
|
||||
## Open questions before prototyping item 1
|
||||
|
||||
- Does the app domain allow `mprotect(RWX)` on a private mapping of an `app_data_file`? Does a
|
||||
`/proc/self/mem` write work under the app's SELinux context and proot's own `ptrace` attach?
|
||||
`/proc/self/mem` writes need `PTRACE_MODE_ATTACH` on self, which is always allowed.
|
||||
- How many `libc.so.6` path syscall sites fall outside a recognisable `mov x8, #nr` window? For
|
||||
example, `syscall()` takes the number in a register, which needs the existing `syscall()` hook.
|
||||
- Patched pages become private copies, about 4 KiB per patched page per process. Measure RSS
|
||||
across the Steam/CEF tree.
|
||||
- Equivalence: run `bench/equiv.py` plus a `find`/`realpath`/nss workload with `PROOT_FP_STATS`.
|
||||
Compare the trap counts from patch 0018 before and after.
|
||||
|
||||
## First implementation and Thor validation (2026-10-27)
|
||||
|
||||
`tools/proot/fastpath/svc-patch.h`, included by `fastpath.c`, implements an opt-in first
|
||||
slice of item 1. Set `PROOT_FP_PATCH_SVC=1` in a fresh process using the replacement
|
||||
`libblfastpath.so`. The ordinary fast-path configuration and tracer-key check must also
|
||||
succeed. It is disabled by default, skips processes that already have multiple threads,
|
||||
and declines SVE/SME hardware because the relay preserves the base ARM64 register state.
|
||||
|
||||
The scanner visits executable segments of the loaded `libc.so.6` and
|
||||
`ld-linux-aarch64.so.1`. It only patches `openat` and `newfstatat` sites with an
|
||||
unshifted `movz x8/w8, #nr` within the preceding eight instructions. Intervening
|
||||
instructions must belong to a small allow-list of argument moves/sign-extension,
|
||||
immediate moves, stack loads/stores and NOPs that leave x8 unchanged. Branches,
|
||||
calls, x8 writes and unknown instructions stop recognition. This includes libc's
|
||||
non-adjacent O_CREAT sequences. Register-number dispatchers and other syscall
|
||||
numbers remain under PRoot. No glibc version offsets, proroot code, or modified
|
||||
proroot binaries are used.
|
||||
|
||||
Each replacement is a four-byte `b` into a nearby relay. Relay pages are filled while
|
||||
writable, then changed to RX before publishing any branch. Text writes use
|
||||
`/proc/self/mem`; failure leaves the site unpatched. The relay preserves x1–x30, all
|
||||
32 SIMD registers, NZCV, FPCR/FPSR and the caller's `errno`, returning a raw kernel-style
|
||||
result in x0. It calls the existing path translator, using the existing allowed
|
||||
trampoline for translated calls. Declined and recursive calls execute an ordinary
|
||||
`svc` in this library, where PRoot still intercepts them. No SIGSYS handler or new
|
||||
seccomp filter is installed. `PROOT_FP_STATS=1` adds `fastpath-svc` exit counters.
|
||||
|
||||
The connected AYN Thor was running runtime r9, glibc 2.43, with a 4 KiB page size in
|
||||
`u:r:untrusted_app_27:s0`. `bench/svc-probe.c`, run through the app's guest bridge,
|
||||
confirmed all of the following while PRoot was attached:
|
||||
|
||||
- An RX private mapping of the rootfs libc accepted `mprotect(RWX)`.
|
||||
- `/proc/self/mem` accepted a changed byte in that RX mapping; readback and restoration
|
||||
succeeded, and the backing file stayed unchanged.
|
||||
- An anonymous RW mapping could be changed to RX.
|
||||
- libc contained 415 `svc #0` sites: 327 with an adjacent number load, 76 with a
|
||||
number load only within the preceding eight instructions, and 12 with neither.
|
||||
The linker contained 56 sites, 45 adjacent and 11 nearby-only. Nearby counts are
|
||||
scan statistics, not proof that the number remains live at the syscall.
|
||||
|
||||
The initial adjacent-only scanner patched nine sites; the final scanner patched
|
||||
14, including file-creation paths, across libc and the linker. It used two relay
|
||||
pages in standalone validation processes. `bench/svc-check.c` exercises the
|
||||
actual emitted relay and checks raw success/error results, errno, x1–x30, q0–q31,
|
||||
NZCV, FPCR/FPSR, four simultaneous threads, the multi-thread patching guard, and fork.
|
||||
Null arguments and unsupported flags fall back to PRoot. The instruction decoder
|
||||
and exact ±128 MiB branch limits also have a host regression test.
|
||||
|
||||
Five alternating pairs of `bench/svc-workload.c` runs compared the same replacement
|
||||
library with patching off/on, with fast-path stats enabled in both. The workload
|
||||
gets `fstatat` and `__open64_nocancel` directly from libc, bypassing the preload's
|
||||
public wrappers, and makes 5,000 of each call against a prepared regular file:
|
||||
|
||||
| median | wrappers only | svc patching |
|
||||
|---|---:|---:|
|
||||
| internal `fstatat` | 38.6 µs | 3.0 µs |
|
||||
| internal open + close | 39.3 µs | 5.8 µs |
|
||||
| voluntary context switches over both loops | 10,000 | 0 |
|
||||
|
||||
### SD allocation and writes
|
||||
|
||||
`bench/svc-storage.c` separates the operations relevant to Steam downloads. Three
|
||||
alternating pairs ran on internal `/tmp` and the selected `/mnt/droiddeck-sd` library,
|
||||
with Steam idle and no simultaneous local Steam download. Each fresh directory held 64 long-named files
|
||||
extended to 256 KiB, followed by 1,000 libc-internal stats and a 64 MiB sequential
|
||||
write followed by `fdatasync`. Both modes used the same replacement library; only
|
||||
`PROOT_FP_PATCH_SVC` changed. Medians on SD were:
|
||||
|
||||
| phase | wrappers only | svc patching |
|
||||
|---|---:|---:|
|
||||
| create/extend/close 64 files | 1.500 s | 1.486 s |
|
||||
| 1,000 internal stats | 61.2 ms | 5.4 ms |
|
||||
| 64 MiB write calls | 43.9 ms | 40.9 ms |
|
||||
| final `fdatasync` | 2.514 s | 2.539 s |
|
||||
|
||||
Internal allocation fell from 7.1 ms to 1.9 ms; internal stats fell from 39.5 ms
|
||||
to 2.5 ms. Internal `fdatasync` varied from 36 to 164 ms across individual runs.
|
||||
The patched SD workload recorded 1,065 internal calls handled in-process, covering
|
||||
64 file creations, 1,000 stats and the streaming-file creation. Per-phase medians
|
||||
are independent and should not be added as a median whole-run duration.
|
||||
|
||||
The SD metadata gain does not establish faster Steam downloads. File creation
|
||||
improved by only a few percent, and write/flush speed did not improve in this probe.
|
||||
The remaining SD cost is much larger than the saved syscall-interception time.
|
||||
The earlier [Thor SD investigation](thor-sd-storage-investigation.md) likewise
|
||||
identified native exFAT metadata waits; the current benchmark does not independently
|
||||
attribute those waits to a particular mount flag. A real Steam install comparison
|
||||
must separate preallocation, transfer/staging and final commit before claiming an
|
||||
end-to-end improvement.
|
||||
|
||||
### Real Steam downloads to SD
|
||||
|
||||
Four fresh installs of owned, initially uninstalled **7 Billion Humans** (792100)
|
||||
used library folder 1, `/mnt/droiddeck-sd`, and were uninstalled between trials.
|
||||
Each downloaded 228,935,584 bytes and staged 237,717,350 bytes in 19 files, ending
|
||||
at BuildID 8016557, depots 792101/5052600457178655633 and
|
||||
792104/8245917369753676198. Every trial logged zero reused/stored/delta bytes at
|
||||
the start and completed with `No Error`. Existing installed games were retained.
|
||||
|
||||
The repeat off/on pair used the exact same replacement library, SHA-256
|
||||
`ab605c021aee271cd49fa1cb337d7ff0c762dbac198b711e9818639c0cd4b17d`.
|
||||
The first off run used the earlier adjacent-only build, with patching disabled.
|
||||
Only the opt-in environment flag changed for the final pair. Times below come
|
||||
from Steam's one-second-resolution `content_log.txt`; total starts at Reconfiguring
|
||||
and ends at scheduler completion, excluding session startup:
|
||||
|
||||
| trial | patching | preallocate | download/stage | commit to completion | total |
|
||||
|---|---|---:|---:|---:|---:|
|
||||
| first baseline | off | 1 s | 123 s | 1 s | 135 s |
|
||||
| first patched | on | <1 s | 47 s | 1 s | 50 s |
|
||||
| repeat baseline | off | <1 s | 30 s | 1 s | 33 s |
|
||||
| repeat patched | on | <1 s | 29 s | 2 s | 32 s |
|
||||
|
||||
Reconfiguration accounts for the remaining total time. The first baseline
|
||||
overlapped another Steam client's network download; that client's telemetry must
|
||||
not be mistaken for the Thor's. Later samples were filtered to local client `0`.
|
||||
The repeat pair reached similar network rates, roughly 7–8 MB/s while transferring.
|
||||
All four attempts logged the same unsupported large-file preallocation retry.
|
||||
|
||||
**This does not demonstrate a useful SD-download speedup.** The apparent initial
|
||||
135→50 second gain disappeared when baseline was repeated. The final 33 versus
|
||||
32 seconds is within this small experiment's timing/network variability. This
|
||||
19-file title also does not settle performance for games with thousands of loose
|
||||
files. Metadata calls clearly improve, while SD creation/writeback remain the
|
||||
stronger next targets. No network settings, SD mount flags or root privileges
|
||||
were changed for these tests.
|
||||
|
||||
The internal-call equivalence workload, including fts, realpath, reentrant NSS,
|
||||
runtime dlopen, missing paths and fallback errors, matched `PROOT_FP_OFF=1` at both
|
||||
the default parent-cache TTL and TTL 0. The existing `equiv.py` also matched after
|
||||
normalizing random/thermal read payloads to their lengths, volatile PID values in
|
||||
the `/proc` directory listing, and tolerating Android's
|
||||
denied `/dev` directory read. Its root fixtures were relocated into a temporary tree.
|
||||
|
||||
`equiv2.py` used temporary root/SD fixtures, normalized `/proc/<self-pid>` output,
|
||||
and tolerated the SD filesystem's unsupported symlink creation. Its patched output
|
||||
matched the wrapper-only output at both TTLs. **It did not fully match PRoot:**
|
||||
binding a Unix socket in the selected SD bind returned EPERM with the existing
|
||||
wrappers, whereas PRoot succeeded. This mismatch reproduced with svc patching off;
|
||||
it is a separate existing fast-path socket issue. The suite is not a clean full
|
||||
equivalence pass. `equiv3.py` was not run because it rewrites live NSS configuration.
|
||||
|
||||
A fresh Steam session using the override and `PROOT_FP_PATCH_SVC=1` rendered its
|
||||
library UI. Sonic Mania (584400) gained focus and rendered its title screen. This
|
||||
is startup smoke coverage, not gameplay or a Steam startup-speed measurement.
|
||||
The installed PRoot pack emitted no profiling start/counter records despite the
|
||||
queued profiling setting; the profiler report had `hasCounters: false`. That is a
|
||||
measurement gap, not evidence of zero remaining traps.
|
||||
|
||||
Before enabling this by default: validate signal/cancellation/unwind behavior,
|
||||
additional glibc builds and workloads, startup cost and Steam/CEF tree RSS; obtain
|
||||
matching patch-0018 tracer measurements; and resolve the existing SD socket mismatch.
|
||||
Additional syscall families, Android-blocked syscall emulation and build-id caching
|
||||
remain follow-ups. In particular this implementation does not remove startup loader
|
||||
traps before the preload constructor, nor patch raw calls in arbitrary guest ELFs.
|
||||
|
||||
Build the tools with the same ARM64 glibc toolchain as the fast path:
|
||||
|
||||
```sh
|
||||
aarch64-linux-gnu-gcc -O2 -Wall -Wextra -pthread \
|
||||
-o svc-check tools/proot/bench/svc-check.c -ldl
|
||||
aarch64-linux-gnu-gcc -O2 -Wall -Wextra \
|
||||
-o svc-probe tools/proot/bench/svc-probe.c -ldl
|
||||
aarch64-linux-gnu-gcc -O2 -Wall -Wextra \
|
||||
-o svc-workload tools/proot/bench/svc-workload.c -ldl
|
||||
aarch64-linux-gnu-gcc -O2 -Wall -Wextra \
|
||||
-o svc-storage tools/proot/bench/svc-storage.c -ldl
|
||||
```
|
||||
|
||||
Run `svc-probe` inside the app's guest environment, not from `adb shell`.
|
||||
`svc-check` embeds the fast path and requires its normal enabled tracer environment.
|
||||
For `svc-workload`, pass a prepared directory containing `file` and a `link` symlink
|
||||
to it; add any second argument to enable timing. Use the documented library override
|
||||
workflow in [proot-performance](proot-performance.md) for a fresh session, adding
|
||||
`PROOT_FP_PATCH_SVC=1` through `droiddeckctl env set` before starting it.
|
||||
`svc-storage` takes a new, unused directory name; it creates and removes its own test
|
||||
files and directory. It writes 64 MiB plus 16 MiB of file extensions per run.
|
||||
@@ -0,0 +1,121 @@
|
||||
#include "../fastpath/fastpath.c"
|
||||
#include <assert.h>
|
||||
#include <sys/wait.h>
|
||||
|
||||
#if defined(__aarch64__)
|
||||
struct snapshot { long x[32]; unsigned char q[32][16]; unsigned long nzcv, fpcr, fpsr; };
|
||||
extern uint32_t svc_check_site;
|
||||
extern void svc_register_probe(struct snapshot *, const long *);
|
||||
|
||||
__asm__(
|
||||
".text\n.p2align 4\n.type svc_register_probe,%function\nsvc_register_probe:\n"
|
||||
"sub sp, sp, #1072\n"
|
||||
"stp x19, x20, [sp]\nstp x21, x22, [sp, #16]\nstp x23, x24, [sp, #32]\n"
|
||||
"stp x25, x26, [sp, #48]\nstp x27, x28, [sp, #64]\nstp x29, x30, [sp, #80]\n"
|
||||
"stp q8, q9, [sp, #96]\nstp q10, q11, [sp, #128]\n"
|
||||
"stp q12, q13, [sp, #160]\nstp q14, q15, [sp, #192]\n"
|
||||
"stp x0, x1, [sp, #224]\n"
|
||||
".irp n,1,2,3,4,5,6,7,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
|
||||
"mov x\\n, #\\n\n.endr\n"
|
||||
".irp n,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31\n"
|
||||
"movi v\\n\\().16b, #0x5a\n.endr\n"
|
||||
"mov x0, #0xa0000000\nmsr nzcv, x0\n"
|
||||
"ldr x0, [sp, #232]\nldr x8, [x0, #48]\n"
|
||||
"ldp x4, x5, [x0, #32]\nldp x2, x3, [x0, #16]\nldp x0, x1, [x0]\n"
|
||||
".global svc_check_site\nsvc_check_site:\nsvc #0\n"
|
||||
".irp n,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
|
||||
"str x\\n, [sp, #(256 + \\n * 8)]\n.endr\n"
|
||||
".irp n,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31\n"
|
||||
"str q\\n, [sp, #(512 + \\n * 16)]\n.endr\n"
|
||||
"mrs x0, nzcv\nstr x0, [sp, #1024]\nmrs x0, fpcr\nstr x0, [sp, #1032]\n"
|
||||
"mrs x0, fpsr\nstr x0, [sp, #1040]\n"
|
||||
"ldr x0, [sp, #224]\nadd x1, sp, #256\nmov x2, #792\nbl memcpy\n"
|
||||
"ldp q8, q9, [sp, #96]\nldp q10, q11, [sp, #128]\n"
|
||||
"ldp q12, q13, [sp, #160]\nldp q14, q15, [sp, #192]\n"
|
||||
"ldp x19, x20, [sp]\nldp x21, x22, [sp, #16]\nldp x23, x24, [sp, #32]\n"
|
||||
"ldp x25, x26, [sp, #48]\nldp x27, x28, [sp, #64]\nldp x29, x30, [sp, #80]\n"
|
||||
"add sp, sp, #1072\nret\n.size svc_register_probe, .-svc_register_probe\n"
|
||||
);
|
||||
|
||||
static void check(const long *args, long want) {
|
||||
struct snapshot out = {0};
|
||||
unsigned long fpcr, fpsr;
|
||||
__asm__ volatile("mrs %0, fpcr\nmrs %1, fpsr" : "=r"(fpcr), "=r"(fpsr));
|
||||
errno = 123;
|
||||
svc_register_probe(&out, args);
|
||||
assert(errno == 123);
|
||||
assert(out.x[0] == want);
|
||||
for (int i = 1; i <= 30; i++) {
|
||||
long expected = i <= 5 ? args[i] : i == 8 ? args[6] : i;
|
||||
if (out.x[i] != expected) { fprintf(stderr, "x%d: %lx != %lx\n", i, out.x[i], expected); abort(); }
|
||||
}
|
||||
for (int i = 0; i < 32; i++)
|
||||
for (int j = 0; j < 16; j++) assert(out.q[i][j] == 0x5a);
|
||||
assert(out.nzcv == 0xa0000000UL);
|
||||
assert(out.fpcr == fpcr && out.fpsr == fpsr);
|
||||
}
|
||||
|
||||
static void *thread_check(void *unused) {
|
||||
(void)unused;
|
||||
struct stat st;
|
||||
const long args[] = {AT_FDCWD, (long)"/usr/lib/libc.so.6", (long)&st, 0, 0, 0, SYS_newfstatat};
|
||||
for (int i = 0; i < 100; i++) check(args, 0);
|
||||
return NULL;
|
||||
}
|
||||
|
||||
static pthread_barrier_t barrier;
|
||||
static void *thread_probe(void *unused) {
|
||||
pthread_barrier_wait(&barrier);
|
||||
thread_check(unused);
|
||||
pthread_barrier_wait(&barrier);
|
||||
return NULL;
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
assert(fp_ready());
|
||||
assert(svc_single_thread());
|
||||
size_t page = sysconf(_SC_PAGESIZE);
|
||||
uint32_t *relay = svc_relay_page((uintptr_t)&svc_check_site, page), branch;
|
||||
assert(relay && svc_emit(relay, &svc_check_site, &branch));
|
||||
__builtin___clear_cache((char *)relay, (char *)relay + 32);
|
||||
assert(mprotect(relay, page, PROT_READ | PROT_EXEC) == 0);
|
||||
int mem = open("/proc/self/mem", O_RDWR | O_CLOEXEC);
|
||||
assert(mem >= 0 && pwrite(mem, &branch, 4, (off_t)(uintptr_t)&svc_check_site) == 4);
|
||||
close(mem);
|
||||
__builtin___clear_cache((char *)&svc_check_site, (char *)(&svc_check_site + 1));
|
||||
long args[] = {0, 1, 2, 3, 4, 5, SYS_getpid};
|
||||
check(args, getpid());
|
||||
struct stat st;
|
||||
args[0] = AT_FDCWD; args[1] = (long)"/usr/lib/libc.so.6"; args[2] = (long)&st;
|
||||
args[3] = 0; args[6] = SYS_newfstatat;
|
||||
check(args, 0);
|
||||
assert(st.st_size > 0);
|
||||
args[1] = (long)"/usr/lib/proot-svc-check-does-not-exist";
|
||||
check(args, -ENOENT);
|
||||
args[1] = 0;
|
||||
check(args, -EFAULT);
|
||||
args[1] = (long)"/usr/lib/libc.so.6"; args[2] = 0;
|
||||
check(args, -EFAULT);
|
||||
args[1] = (long)"/proc/self/exe"; args[2] = (long)&st;
|
||||
check(args, 0);
|
||||
args[3] = 0x40000000;
|
||||
check(args, -EINVAL);
|
||||
pthread_t threads[4];
|
||||
assert(pthread_barrier_init(&barrier, NULL, 5) == 0);
|
||||
for (int i = 0; i < 4; i++) assert(pthread_create(&threads[i], NULL, thread_probe, NULL) == 0);
|
||||
pthread_barrier_wait(&barrier);
|
||||
assert(!svc_single_thread());
|
||||
pthread_barrier_wait(&barrier);
|
||||
for (int i = 0; i < 4; i++) assert(pthread_join(threads[i], NULL) == 0);
|
||||
assert(pthread_barrier_destroy(&barrier) == 0);
|
||||
pid_t pid = fork();
|
||||
assert(pid >= 0);
|
||||
if (!pid) { thread_check(NULL); _exit(0); }
|
||||
int status;
|
||||
assert(waitpid(pid, &status, 0) == pid && WIFEXITED(status) && WEXITSTATUS(status) == 0);
|
||||
puts("PASS: raw results, errno, x1-x30, q0-q31, NZCV, FPCR/FPSR, 4 threads, fork");
|
||||
return 0;
|
||||
}
|
||||
#else
|
||||
int main(void) { return 77; }
|
||||
#endif
|
||||
@@ -0,0 +1,93 @@
|
||||
#define _GNU_SOURCE
|
||||
#include <errno.h>
|
||||
#include <fcntl.h>
|
||||
#include <link.h>
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <string.h>
|
||||
#include <sys/mman.h>
|
||||
#include <unistd.h>
|
||||
|
||||
static int mov_sysnum(uint32_t insn) {
|
||||
if ((insn & 0xffe0001fU) == 0xd2800008U ||
|
||||
(insn & 0xffe0001fU) == 0x52800008U)
|
||||
return (insn >> 5) & 0xffff;
|
||||
return -1;
|
||||
}
|
||||
|
||||
static int scan(struct dl_phdr_info *info, size_t size, void *unused) {
|
||||
(void)size;
|
||||
(void)unused;
|
||||
const char *name = strrchr(info->dlpi_name, '/');
|
||||
name = name ? name + 1 : info->dlpi_name;
|
||||
if (strcmp(name, "libc.so.6") && strcmp(name, "ld-linux-aarch64.so.1")) return 0;
|
||||
unsigned total = 0, adjacent = 0, nearby = 0, counts[512] = {0};
|
||||
for (int i = 0; i < info->dlpi_phnum; i++) {
|
||||
const ElfW(Phdr) *ph = &info->dlpi_phdr[i];
|
||||
if (ph->p_type != PT_LOAD || !(ph->p_flags & PF_X)) continue;
|
||||
const uint32_t *text = (const uint32_t *)(info->dlpi_addr + ph->p_vaddr);
|
||||
for (size_t j = 0; j < ph->p_memsz / 4; j++) {
|
||||
if (text[j] != 0xd4000001U) continue;
|
||||
total++;
|
||||
int nr = j ? mov_sysnum(text[j - 1]) : -1;
|
||||
if (nr >= 0) {
|
||||
adjacent++;
|
||||
if (nr < 512) counts[nr]++;
|
||||
} else {
|
||||
for (size_t k = 2; k <= 8 && k <= j; k++)
|
||||
if (mov_sysnum(text[j - k]) >= 0) { nearby++; break; }
|
||||
}
|
||||
}
|
||||
}
|
||||
printf("{\"object\":\"%s\",\"svc\":%u,\"adjacent_mov_x8\":%u,\"nearby_only\":%u,\"adjacent_syscalls\":{",
|
||||
name, total, adjacent, nearby);
|
||||
int comma = 0;
|
||||
for (unsigned nr = 0; nr < 512; nr++)
|
||||
if (counts[nr]) { printf("%s\"%u\":%u", comma ? "," : "", nr, counts[nr]); comma = 1; }
|
||||
puts("}}");
|
||||
return 0;
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
long page = sysconf(_SC_PAGESIZE);
|
||||
int fd = open("/usr/lib/libc.so.6", O_RDONLY | O_CLOEXEC);
|
||||
if (fd < 0) { perror("libc"); return 1; }
|
||||
unsigned char *p = mmap(NULL, page, PROT_READ | PROT_EXEC, MAP_PRIVATE, fd, 0);
|
||||
if (p == MAP_FAILED) { perror("file mmap RX"); close(fd); return 1; }
|
||||
unsigned char before = p[0], changed = before ^ 1;
|
||||
errno = 0;
|
||||
int rwx = mprotect(p, page, PROT_READ | PROT_WRITE | PROT_EXEC), rwx_errno = errno;
|
||||
if (rwx == 0) {
|
||||
p[0] = changed;
|
||||
p[0] = before;
|
||||
if (mprotect(p, page, PROT_READ | PROT_EXEC)) { perror("restore RX"); return 1; }
|
||||
}
|
||||
int mem = open("/proc/self/mem", O_RDWR | O_CLOEXEC), mem_errno = mem < 0 ? errno : 0;
|
||||
ssize_t written = -1;
|
||||
int write_errno = 0, verified = 0, restored = 0;
|
||||
if (mem >= 0) {
|
||||
errno = 0;
|
||||
written = pwrite(mem, &changed, 1, (off_t)(uintptr_t)p);
|
||||
write_errno = errno;
|
||||
verified = written == 1 && *(volatile unsigned char *)p == changed;
|
||||
restored = pwrite(mem, &before, 1, (off_t)(uintptr_t)p) == 1 &&
|
||||
*(volatile unsigned char *)p == before;
|
||||
close(mem);
|
||||
}
|
||||
unsigned char disk = 0;
|
||||
int file_unchanged = pread(fd, &disk, 1, 0) == 1 && disk == before;
|
||||
munmap(p, page);
|
||||
close(fd);
|
||||
void *anon = mmap(NULL, page, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
|
||||
errno = 0;
|
||||
int execmem = anon == MAP_FAILED ? -1 : mprotect(anon, page, PROT_READ | PROT_EXEC);
|
||||
int execmem_errno = errno;
|
||||
if (anon != MAP_FAILED) munmap(anon, page);
|
||||
printf("{\"page_size\":%ld,\"rwx\":%d,\"rwx_errno\":%d,\"mem_open_errno\":%d,"
|
||||
"\"mem_written\":%ld,\"mem_write_errno\":%d,\"mem_verified\":%d,\"mem_restored\":%d,"
|
||||
"\"file_unchanged\":%d,\"execmem\":%d,\"execmem_errno\":%d}\n",
|
||||
page, rwx, rwx_errno, mem_errno, (long)written, write_errno, verified, restored,
|
||||
file_unchanged, execmem, execmem_errno);
|
||||
dl_iterate_phdr(scan, NULL);
|
||||
return 0;
|
||||
}
|
||||
@@ -0,0 +1,74 @@
|
||||
#define _GNU_SOURCE
|
||||
#include <assert.h>
|
||||
#include <dlfcn.h>
|
||||
#include <fcntl.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <sys/resource.h>
|
||||
#include <sys/stat.h>
|
||||
#include <time.h>
|
||||
#include <unistd.h>
|
||||
|
||||
static double now(void) {
|
||||
struct timespec t;
|
||||
assert(clock_gettime(CLOCK_MONOTONIC, &t) == 0);
|
||||
return t.tv_sec + t.tv_nsec / 1e9;
|
||||
}
|
||||
|
||||
int main(int argc, char **argv) {
|
||||
assert(argc == 2 && mkdir(argv[1], 0700) == 0);
|
||||
void *libc = dlopen("libc.so.6", RTLD_NOW);
|
||||
assert(libc);
|
||||
int (*internal_open)(const char *, int, ...) = dlsym(libc, "__open64_nocancel");
|
||||
int (*internal_stat)(int, const char *, struct stat *, int) = dlsym(libc, "fstatat");
|
||||
assert(internal_open && internal_stat);
|
||||
struct rusage before, after;
|
||||
assert(getrusage(RUSAGE_SELF, &before) == 0);
|
||||
char paths[64][4096], stream[4096];
|
||||
double creation = 0, extension = 0, closing = 0, total = now();
|
||||
for (int i = 0; i < 64; i++) {
|
||||
snprintf(paths[i], sizeof paths[i], "%s/asset-with-a-longish-steam-depot-filename-%03d.dat", argv[1], i);
|
||||
double start = now();
|
||||
int fd = internal_open(paths[i], O_WRONLY | O_CREAT | O_EXCL | O_CLOEXEC, 0600);
|
||||
assert(fd >= 0);
|
||||
creation += now() - start;
|
||||
start = now();
|
||||
assert(ftruncate(fd, 262144) == 0);
|
||||
extension += now() - start;
|
||||
start = now();
|
||||
assert(close(fd) == 0);
|
||||
closing += now() - start;
|
||||
}
|
||||
double allocation = now() - total;
|
||||
struct stat st;
|
||||
double start = now();
|
||||
for (int i = 0; i < 1000; i++) assert(internal_stat(AT_FDCWD, paths[i % 64], &st, 0) == 0);
|
||||
double metadata = now() - start;
|
||||
snprintf(stream, sizeof stream, "%s/stream.bin", argv[1]);
|
||||
start = now();
|
||||
int fd = internal_open(stream, O_WRONLY | O_CREAT | O_EXCL | O_CLOEXEC, 0600);
|
||||
assert(fd >= 0);
|
||||
double stream_create = now() - start;
|
||||
static unsigned char block[1024 * 1024];
|
||||
memset(block, 0x5a, sizeof block);
|
||||
start = now();
|
||||
for (int i = 0; i < 64; i++) assert(write(fd, block, sizeof block) == sizeof block);
|
||||
double writing = now() - start;
|
||||
start = now();
|
||||
assert(fdatasync(fd) == 0);
|
||||
double syncing = now() - start;
|
||||
assert(close(fd) == 0);
|
||||
assert(getrusage(RUSAGE_SELF, &after) == 0);
|
||||
printf("{\"files\":64,\"bytes_per_file\":262144,\"create_s\":%.6f,\"extend_s\":%.6f,"
|
||||
"\"close_s\":%.6f,\"allocation_s\":%.6f,\"stat1000_s\":%.6f,"
|
||||
"\"stream_bytes\":67108864,\"stream_create_s\":%.6f,\"write_s\":%.6f,"
|
||||
"\"fdatasync_s\":%.6f,\"voluntary_switches\":%ld}\n",
|
||||
creation, extension, closing, allocation, metadata, stream_create, writing,
|
||||
syncing, after.ru_nvcsw - before.ru_nvcsw);
|
||||
fflush(stdout);
|
||||
for (int i = 0; i < 64; i++) assert(unlink(paths[i]) == 0);
|
||||
assert(unlink(stream) == 0 && rmdir(argv[1]) == 0);
|
||||
assert(dlclose(libc) == 0);
|
||||
return 0;
|
||||
}
|
||||
@@ -0,0 +1,91 @@
|
||||
#define _GNU_SOURCE
|
||||
#include <assert.h>
|
||||
#include <dlfcn.h>
|
||||
#include <errno.h>
|
||||
#include <fcntl.h>
|
||||
#include <fts.h>
|
||||
#include <pwd.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <sys/resource.h>
|
||||
#include <sys/stat.h>
|
||||
#include <time.h>
|
||||
#include <unistd.h>
|
||||
|
||||
static double now(void) {
|
||||
struct timespec t;
|
||||
assert(clock_gettime(CLOCK_MONOTONIC, &t) == 0);
|
||||
return t.tv_sec * 1e9 + t.tv_nsec;
|
||||
}
|
||||
|
||||
int main(int argc, char **argv) {
|
||||
assert(argc >= 2);
|
||||
void *libc = dlopen("libc.so.6", RTLD_NOW);
|
||||
assert(libc);
|
||||
int (*internal_stat)(int, const char *, struct stat *, int) = dlsym(libc, "fstatat");
|
||||
int (*internal_open)(const char *, int, ...) = dlsym(libc, "__open64_nocancel");
|
||||
assert(internal_stat && internal_open);
|
||||
char path[4096], link[4096], missing[4096];
|
||||
snprintf(path, sizeof path, "%s/file", argv[1]);
|
||||
snprintf(link, sizeof link, "%s/link", argv[1]);
|
||||
snprintf(missing, sizeof missing, "%s/missing", argv[1]);
|
||||
const char *paths[] = {path, link, missing, "/etc/os-release", "/lib/libc.so.6", "/proc/self/exe", "/dev/null", "", NULL};
|
||||
struct stat st;
|
||||
for (size_t i = 0; i < sizeof paths / sizeof *paths; i++) {
|
||||
errno = 0;
|
||||
int r = internal_stat(AT_FDCWD, paths[i], &st, 0);
|
||||
printf("stat %zu: %d %d", i, r, r ? errno : 0);
|
||||
if (!r) printf(" %o %ld", st.st_mode & S_IFMT, S_ISREG(st.st_mode) && i != 5 ? st.st_size : 0L);
|
||||
putchar('\n');
|
||||
errno = 0;
|
||||
int fd = internal_open(paths[i], O_RDONLY | O_CLOEXEC);
|
||||
int err = fd < 0 ? errno : 0;
|
||||
printf("open %zu: %d %d\n", i, fd < 0 ? -1 : 0, err);
|
||||
if (fd >= 0) close(fd);
|
||||
}
|
||||
errno = 0;
|
||||
int r = internal_stat(AT_FDCWD, path, &st, 0x40000000);
|
||||
printf("flags: %d %d\n", r, r ? errno : 0);
|
||||
char *tree[] = {argv[1], NULL};
|
||||
FTS *fts = fts_open(tree, FTS_PHYSICAL | FTS_NOCHDIR, NULL);
|
||||
assert(fts);
|
||||
unsigned files = 0, dirs = 0, links = 0;
|
||||
FTSENT *ent;
|
||||
while ((ent = fts_read(fts))) {
|
||||
if (ent->fts_info == FTS_F) files++;
|
||||
if (ent->fts_info == FTS_D) dirs++;
|
||||
if (ent->fts_info == FTS_SL) links++;
|
||||
}
|
||||
assert(fts_close(fts) == 0);
|
||||
printf("fts: %u files %u dirs %u links\n", files, dirs, links);
|
||||
char resolved[4096];
|
||||
assert(realpath(link, resolved) && !strcmp(resolved, path));
|
||||
struct passwd pw, *found;
|
||||
char buf[16384];
|
||||
r = getpwnam_r("root", &pw, buf, sizeof buf, &found);
|
||||
printf("nss: %d %d\n", r, found != NULL);
|
||||
void *libm = dlopen("libm.so.6", RTLD_NOW | RTLD_LOCAL);
|
||||
assert(libm && dlsym(libm, "cos"));
|
||||
assert(dlclose(libm) == 0);
|
||||
puts("realpath/dlopen: PASS");
|
||||
if (argc > 2) {
|
||||
const int loops = 5000;
|
||||
struct rusage before, after;
|
||||
assert(getrusage(RUSAGE_SELF, &before) == 0);
|
||||
double start = now();
|
||||
for (int i = 0; i < loops; i++) assert(internal_stat(AT_FDCWD, path, &st, 0) == 0);
|
||||
double stat_ns = (now() - start) / loops;
|
||||
start = now();
|
||||
for (int i = 0; i < loops; i++) {
|
||||
int fd = internal_open(path, O_RDONLY | O_CLOEXEC);
|
||||
assert(fd >= 0 && close(fd) == 0);
|
||||
}
|
||||
double open_ns = (now() - start) / loops;
|
||||
assert(getrusage(RUSAGE_SELF, &after) == 0);
|
||||
printf("bench: stat %.0f ns, open+close %.0f ns, voluntary switches %ld\n",
|
||||
stat_ns, open_ns, after.ru_nvcsw - before.ru_nvcsw);
|
||||
}
|
||||
assert(dlclose(libc) == 0);
|
||||
return 0;
|
||||
}
|
||||
@@ -1828,3 +1828,5 @@ struct passwd *getpwuid(uid_t uid) {
|
||||
missed(__func__, AT_FDCWD, "(passwd)");
|
||||
return real_getpwuid(uid);
|
||||
}
|
||||
|
||||
#include "svc-patch.h"
|
||||
@@ -0,0 +1,250 @@
|
||||
static inline int svc_number(uint32_t insn) {
|
||||
if ((insn & 0xffe0001fU) == 0xd2800008U ||
|
||||
(insn & 0xffe0001fU) == 0x52800008U)
|
||||
return (insn >> 5) & 0xffff;
|
||||
return -1;
|
||||
}
|
||||
|
||||
static inline int svc_argument_instruction(uint32_t insn) {
|
||||
if (insn == 0xd503201fU) return 1;
|
||||
if ((insn & 0xffc003e0U) == 0x290003e0U ||
|
||||
(insn & 0xffc003e0U) == 0xa90003e0U) return 1;
|
||||
if ((insn & 31) == 8) return 0;
|
||||
return (insn & 0xffe0ffe0U) == 0xaa0003e0U ||
|
||||
(insn & 0xffe0ffe0U) == 0x2a0003e0U ||
|
||||
(insn & 0xfffffc00U) == 0x93407c00U ||
|
||||
(insn & 0x7f800000U) == 0x52800000U ||
|
||||
(insn & 0x7f800000U) == 0x12800000U ||
|
||||
(insn & 0xffc003e0U) == 0xb94003e0U ||
|
||||
(insn & 0xffc003e0U) == 0xf94003e0U ||
|
||||
(insn & 0xffc003e0U) == 0xb98003e0U;
|
||||
}
|
||||
|
||||
static inline int svc_site_number(const uint32_t *text, size_t at) {
|
||||
for (size_t back = 1; back <= 8 && back <= at; back++) {
|
||||
uint32_t insn = text[at - back];
|
||||
int nr = svc_number(insn);
|
||||
if (nr >= 0) return nr;
|
||||
if (!svc_argument_instruction(insn)) break;
|
||||
}
|
||||
return -1;
|
||||
}
|
||||
|
||||
static inline int svc_branch(uintptr_t from, uintptr_t to, uint32_t *out) {
|
||||
int64_t delta = (int64_t)to - (int64_t)from;
|
||||
if ((delta & 3) || delta < -0x8000000LL || delta >= 0x8000000LL) return 0;
|
||||
*out = 0x14000000U | ((uint32_t)(delta / 4) & 0x03ffffffU);
|
||||
return 1;
|
||||
}
|
||||
|
||||
#if defined(__aarch64__) && !defined(FP_SVC_ENCODINGS_ONLY)
|
||||
#include <asm/hwcap.h>
|
||||
#include <link.h>
|
||||
|
||||
static __thread int svc_busy __attribute__((tls_model("initial-exec")));
|
||||
static unsigned svc_sites, svc_pages;
|
||||
static unsigned long svc_fast, svc_slow;
|
||||
|
||||
static long svc_original(const long *r) {
|
||||
register long x0 __asm__("x0") = r[0];
|
||||
register long x1 __asm__("x1") = r[1];
|
||||
register long x2 __asm__("x2") = r[2];
|
||||
register long x3 __asm__("x3") = r[3];
|
||||
register long x4 __asm__("x4") = r[4];
|
||||
register long x5 __asm__("x5") = r[5];
|
||||
register long x8 __asm__("x8") = r[8];
|
||||
__asm__ volatile("svc #0" : "+r"(x0) : "r"(x1), "r"(x2), "r"(x3), "r"(x4), "r"(x5), "r"(x8) : "memory", "cc");
|
||||
return x0;
|
||||
}
|
||||
|
||||
__attribute__((used, visibility("hidden")))
|
||||
long fp_svc_dispatch(const long *r) {
|
||||
int saved_errno = errno;
|
||||
long result = FP_SLOW;
|
||||
if (!svc_busy) {
|
||||
svc_busy = 1;
|
||||
if (r[8] == SYS_openat)
|
||||
result = fp_open((int)r[0], (const char *)r[1], (int)r[2], (mode_t)r[3]);
|
||||
else if (r[8] == SYS_newfstatat && r[2])
|
||||
result = fp_stat((int)r[0], (const char *)r[1], (int)r[3], (struct stat *)r[2]);
|
||||
if (result != FP_SLOW) __atomic_fetch_add(&svc_fast, 1, __ATOMIC_RELAXED);
|
||||
svc_busy = 0;
|
||||
}
|
||||
if (result == FP_SLOW) {
|
||||
__atomic_fetch_add(&svc_slow, 1, __ATOMIC_RELAXED);
|
||||
result = svc_original(r);
|
||||
}
|
||||
errno = saved_errno;
|
||||
return result;
|
||||
}
|
||||
|
||||
void fp_svc_entry(void) __attribute__((visibility("hidden")));
|
||||
/* A replaced svc may have live values in every register, including NZCV and SIMD. */
|
||||
__asm__(
|
||||
".text\n.p2align 4\n.hidden fp_svc_entry\n.type fp_svc_entry,%function\nfp_svc_entry:\n"
|
||||
"bti j\nsub sp, sp, #800\n"
|
||||
"stp x0, x1, [sp, #0]\nstp x2, x3, [sp, #16]\n"
|
||||
"stp x4, x5, [sp, #32]\nstp x6, x7, [sp, #48]\n"
|
||||
"stp x8, x9, [sp, #64]\nstp x10, x11, [sp, #80]\n"
|
||||
"stp x12, x13, [sp, #96]\nstp x14, x15, [sp, #112]\n"
|
||||
"stp x18, x19, [sp, #128]\nstp x20, x21, [sp, #144]\n"
|
||||
"stp x22, x23, [sp, #160]\nstp x24, x25, [sp, #176]\n"
|
||||
"stp x26, x27, [sp, #192]\nstp x28, x29, [sp, #208]\n"
|
||||
"stp x30, x17, [sp, #224]\nmrs x0, nzcv\nstr x0, [sp, #240]\n"
|
||||
"mrs x0, fpcr\nstr x0, [sp, #768]\nmrs x0, fpsr\nstr x0, [sp, #776]\n"
|
||||
"stp q0, q1, [sp, #256]\nstp q2, q3, [sp, #288]\n"
|
||||
"stp q4, q5, [sp, #320]\nstp q6, q7, [sp, #352]\n"
|
||||
"stp q8, q9, [sp, #384]\nstp q10, q11, [sp, #416]\n"
|
||||
"stp q12, q13, [sp, #448]\nstp q14, q15, [sp, #480]\n"
|
||||
"stp q16, q17, [sp, #512]\nstp q18, q19, [sp, #544]\n"
|
||||
"stp q20, q21, [sp, #576]\nstp q22, q23, [sp, #608]\n"
|
||||
"stp q24, q25, [sp, #640]\nstp q26, q27, [sp, #672]\n"
|
||||
"stp q28, q29, [sp, #704]\nstp q30, q31, [sp, #736]\n"
|
||||
"mov x0, sp\nbl fp_svc_dispatch\nstr x0, [sp]\n"
|
||||
"ldp q0, q1, [sp, #256]\nldp q2, q3, [sp, #288]\n"
|
||||
"ldp q4, q5, [sp, #320]\nldp q6, q7, [sp, #352]\n"
|
||||
"ldp q8, q9, [sp, #384]\nldp q10, q11, [sp, #416]\n"
|
||||
"ldp q12, q13, [sp, #448]\nldp q14, q15, [sp, #480]\n"
|
||||
"ldp q16, q17, [sp, #512]\nldp q18, q19, [sp, #544]\n"
|
||||
"ldp q20, q21, [sp, #576]\nldp q22, q23, [sp, #608]\n"
|
||||
"ldp q24, q25, [sp, #640]\nldp q26, q27, [sp, #672]\n"
|
||||
"ldp q28, q29, [sp, #704]\nldp q30, q31, [sp, #736]\n"
|
||||
"ldr x0, [sp, #768]\nmsr fpcr, x0\nldr x0, [sp, #776]\nmsr fpsr, x0\n"
|
||||
"ldr x0, [sp, #240]\nmsr nzcv, x0\n"
|
||||
"ldp x0, x1, [sp, #0]\nldp x2, x3, [sp, #16]\n"
|
||||
"ldp x4, x5, [sp, #32]\nldp x6, x7, [sp, #48]\n"
|
||||
"ldp x8, x9, [sp, #64]\nldp x10, x11, [sp, #80]\n"
|
||||
"ldp x12, x13, [sp, #96]\nldp x14, x15, [sp, #112]\n"
|
||||
"ldp x18, x19, [sp, #128]\nldp x20, x21, [sp, #144]\n"
|
||||
"ldp x22, x23, [sp, #160]\nldp x24, x25, [sp, #176]\n"
|
||||
"ldp x26, x27, [sp, #192]\nldp x28, x29, [sp, #208]\n"
|
||||
"ldr x30, [sp, #224]\nldr x16, [sp, #232]\n"
|
||||
"add sp, sp, #800\nbr x16\n.size fp_svc_entry, .-fp_svc_entry\n"
|
||||
);
|
||||
|
||||
static int svc_emit(uint32_t *code, uint32_t *site, uint32_t *branch) {
|
||||
uint32_t back;
|
||||
if (!svc_branch((uintptr_t)site, (uintptr_t)code, branch) ||
|
||||
!svc_branch((uintptr_t)(code + 5), (uintptr_t)(site + 1), &back)) return 0;
|
||||
code[0] = 0xa9bf47f0U; /* stp x16, x17, [sp, #-16]! */
|
||||
code[1] = 0x580000b0U; /* ldr x16, literal at +24 */
|
||||
code[2] = 0x10000051U; /* adr x17, +8 */
|
||||
code[3] = 0xd61f0200U; /* br x16 */
|
||||
code[4] = 0xa8c147f0U; /* ldp x16, x17, [sp], #16 */
|
||||
code[5] = back;
|
||||
uintptr_t entry = (uintptr_t)fp_svc_entry;
|
||||
memcpy(code + 6, &entry, sizeof entry);
|
||||
return 1;
|
||||
}
|
||||
|
||||
struct svc_site { uint32_t *site; uint32_t branch; };
|
||||
static struct svc_site svc_plan[256];
|
||||
static size_t svc_planned;
|
||||
|
||||
static int svc_object(struct dl_phdr_info *info, size_t size, void *unused) {
|
||||
(void)size;
|
||||
(void)unused;
|
||||
const char *name = strrchr(info->dlpi_name, '/');
|
||||
name = name ? name + 1 : info->dlpi_name;
|
||||
if (strcmp(name, "libc.so.6") && strcmp(name, "ld-linux-aarch64.so.1")) return 0;
|
||||
for (int i = 0; i < info->dlpi_phnum; i++) {
|
||||
const ElfW(Phdr) *ph = &info->dlpi_phdr[i];
|
||||
if (ph->p_type != PT_LOAD || !(ph->p_flags & PF_X)) continue;
|
||||
uint32_t *text = (uint32_t *)(info->dlpi_addr + ph->p_vaddr);
|
||||
for (size_t j = 1; j < ph->p_memsz / 4 && svc_planned < 256; j++) {
|
||||
if (text[j] != 0xd4000001U) continue;
|
||||
int nr = svc_site_number(text, j);
|
||||
if (nr != SYS_openat && nr != SYS_newfstatat) continue;
|
||||
svc_plan[svc_planned++].site = &text[j];
|
||||
}
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
static uint32_t *svc_relay_page(uintptr_t site, size_t page) {
|
||||
uintptr_t base = site & ~(uintptr_t)(page - 1);
|
||||
for (uintptr_t delta = page; delta < 0x7ff0000UL; delta += 0x10000UL) {
|
||||
for (int direction = -1; direction <= 1; direction += 2) {
|
||||
uintptr_t at = direction < 0 ? base - delta : base + delta;
|
||||
if (at < page) continue;
|
||||
void *p = mmap((void *)at, page, PROT_READ | PROT_WRITE,
|
||||
MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED_NOREPLACE, -1, 0);
|
||||
if (p == (void *)at) return p;
|
||||
if (p != MAP_FAILED) munmap(p, page);
|
||||
}
|
||||
}
|
||||
return NULL;
|
||||
}
|
||||
|
||||
static int svc_single_thread(void) {
|
||||
struct entry { uint64_t ino; int64_t off; unsigned short len; unsigned char type; char name[]; };
|
||||
char buf[4096] __attribute__((aligned(8)));
|
||||
long fd = sc(SYS_openat, AT_FDCWD, (long)"/proc/self/task", O_RDONLY | O_DIRECTORY | O_CLOEXEC, 0, 0);
|
||||
if (fd < 0) return 0;
|
||||
int count = 0;
|
||||
long n;
|
||||
while ((n = sc(SYS_getdents64, fd, (long)buf, sizeof buf, 0, 0)) > 0) {
|
||||
for (long at = 0; at < n;) {
|
||||
const struct entry *e = (const struct entry *)(buf + at);
|
||||
if (n - at < (long)offsetof(struct entry, name) + 1 ||
|
||||
e->len < offsetof(struct entry, name) + 1 || e->len > n - at) { count = -1; goto done; }
|
||||
if (e->name[0] >= '0' && e->name[0] <= '9' && ++count > 1) goto done;
|
||||
at += e->len;
|
||||
}
|
||||
}
|
||||
if (n < 0) count = -1;
|
||||
done:
|
||||
sc(SYS_close, fd, 0, 0, 0, 0);
|
||||
return count == 1;
|
||||
}
|
||||
|
||||
static void svc_stats(void) {
|
||||
char line[160];
|
||||
int n = snprintf(line, sizeof line, "fastpath-svc[%d]: %u sites, %u relay pages, %lu fast, %lu to proot\n",
|
||||
getpid(), svc_sites, svc_pages, svc_fast, svc_slow);
|
||||
if (write(2, line, n) < 0) {}
|
||||
}
|
||||
|
||||
__attribute__((constructor)) static void svc_patch_init(void) {
|
||||
const char *enabled = getenv("PROOT_FP_PATCH_SVC");
|
||||
if (!enabled || strcmp(enabled, "1") || !fp_ready()) return;
|
||||
if (getauxval(AT_HWCAP) & HWCAP_SVE) return;
|
||||
#ifdef HWCAP2_SME
|
||||
if (getauxval(AT_HWCAP2) & HWCAP2_SME) return;
|
||||
#endif
|
||||
if (!svc_single_thread()) return;
|
||||
size_t page = (size_t)sysconf(_SC_PAGESIZE);
|
||||
if (page < 4096 || (page & (page - 1))) return;
|
||||
int mem = (int)sc(SYS_openat, AT_FDCWD, (long)"/proc/self/mem", O_RDWR | O_CLOEXEC, 0, 0);
|
||||
if (mem < 0) return;
|
||||
dl_iterate_phdr(svc_object, NULL);
|
||||
for (size_t first = 0; first < svc_planned;) {
|
||||
uint32_t *relay = svc_relay_page((uintptr_t)svc_plan[first].site, page);
|
||||
if (!relay) { first++; continue; }
|
||||
size_t end = first;
|
||||
for (; end < svc_planned && (end - first + 1) * 32 <= page; end++) {
|
||||
uint32_t *code = relay + (end - first) * 8;
|
||||
if (!svc_emit(code, svc_plan[end].site, &svc_plan[end].branch)) break;
|
||||
}
|
||||
__builtin___clear_cache((char *)relay, (char *)relay + page);
|
||||
if (end == first || mprotect(relay, page, PROT_READ | PROT_EXEC)) {
|
||||
munmap(relay, page);
|
||||
first++;
|
||||
continue;
|
||||
}
|
||||
unsigned patched = 0;
|
||||
for (size_t i = first; i < end; i++) {
|
||||
struct svc_site *s = &svc_plan[i];
|
||||
long written = sc(SYS_pwrite64, mem, (long)&s->branch, sizeof s->branch, (long)s->site, 0);
|
||||
if (written != sizeof s->branch) continue;
|
||||
__builtin___clear_cache((char *)s->site, (char *)(s->site + 1));
|
||||
patched++;
|
||||
}
|
||||
if (patched) { svc_sites += patched; svc_pages++; }
|
||||
else munmap(relay, page);
|
||||
first = end;
|
||||
}
|
||||
sc(SYS_close, mem, 0, 0, 0, 0);
|
||||
if (getenv("PROOT_FP_STATS")) atexit(svc_stats);
|
||||
}
|
||||
#endif
|
||||
@@ -0,0 +1,62 @@
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
|
||||
HEADER = Path(__file__).resolve().parents[1] / "proot/fastpath/svc-patch.h"
|
||||
|
||||
|
||||
class FastPathSvcEncodingTest(unittest.TestCase):
|
||||
def test_instruction_decoding_and_branch_limits(self):
|
||||
if shutil.which("cc") is None:
|
||||
self.skipTest("cc is not available")
|
||||
source = r'''
|
||||
#include <assert.h>
|
||||
#include <stdint.h>
|
||||
#include <stddef.h>
|
||||
#define FP_SVC_ENCODINGS_ONLY
|
||||
#include "HEADER"
|
||||
int main(void) {
|
||||
assert(svc_number(0xd2800708) == 56);
|
||||
assert(svc_number(0x528009e8) == 79);
|
||||
assert(svc_number(0xd2800709) == -1);
|
||||
assert(svc_number(0xd2a00708) == -1);
|
||||
assert(svc_number(0xf2800708) == -1);
|
||||
assert(svc_number(0xaa0003e8) == -1);
|
||||
const uint32_t create[] = {0xd2800708, 0x29047fe0, 0x92800c60, 0xd4000001};
|
||||
const uint32_t create_at[] = {0xd2800708, 0x29047fe3, 0xb9404be3, 0xd4000001};
|
||||
const uint32_t xstat[] = {0xd28009e8, 0xaa0203e1, 0xaa0303e2, 0x93407c83, 0xd4000001};
|
||||
const uint32_t overwritten[] = {0xd2800708, 0xaa0003e8, 0xd4000001};
|
||||
const uint32_t loaded[] = {0xd2800708, 0xf94003e8, 0xd4000001};
|
||||
const uint32_t branch[] = {0xd2800708, 0x14000001, 0xd4000001};
|
||||
const uint32_t call[] = {0xd2800708, 0x94000001, 0xd4000001};
|
||||
const uint32_t shift[] = {0xd2800708, 0xd2a00708, 0xd4000001};
|
||||
assert(svc_site_number(create, 3) == 56);
|
||||
assert(svc_site_number(create_at, 3) == 56);
|
||||
assert(svc_site_number(xstat, 4) == 79);
|
||||
assert(svc_site_number(overwritten, 2) == -1);
|
||||
assert(svc_site_number(loaded, 2) == -1);
|
||||
assert(svc_site_number(branch, 2) == -1);
|
||||
assert(svc_site_number(call, 2) == -1);
|
||||
assert(svc_site_number(shift, 2) == -1);
|
||||
uint32_t out;
|
||||
uintptr_t pc = 0x10000000;
|
||||
assert(svc_branch(pc, pc + 4, &out) && out == 0x14000001);
|
||||
assert(svc_branch(pc, pc - 4, &out) && out == 0x17ffffff);
|
||||
assert(svc_branch(pc, pc + 0x7fffffc, &out) && out == 0x15ffffff);
|
||||
assert(svc_branch(pc, pc - 0x8000000, &out) && out == 0x16000000);
|
||||
assert(!svc_branch(pc, pc + 0x8000000, &out));
|
||||
assert(!svc_branch(pc, pc - 0x8000004, &out));
|
||||
assert(!svc_branch(pc, pc + 2, &out));
|
||||
return 0;
|
||||
}
|
||||
'''
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
work = Path(directory)
|
||||
program = work / "check.c"
|
||||
program.write_text(source.replace("HEADER", HEADER.as_posix()))
|
||||
output = work / "check"
|
||||
subprocess.run(["cc", "-O2", "-Wall", "-Wextra", "-Werror", str(program), "-o", str(output)], check=True)
|
||||
subprocess.run([str(output)], check=True, timeout=10)
|
||||
Reference in new issue
Block a user